2023-06-23T18:01:09.380 INFO:root:teuthology version: 0.0.1.dev128+g076bbeb 2023-06-23T18:01:09.381 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873 --verbose --owner scheduled_rishabh@teuthology --name rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} -- /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/orig.config.yaml 2023-06-23T18:01:09.442 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T18:01:09.540 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873 branch: wip-rishabh-improvements-authmon description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} email: null first_in_suite: false job_id: '7313873' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-improvements-authmon ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-improvements-authmon sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-improvements-authmon suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi017.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi082.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 076bbebc42a14f7d568aaa78eabb0038327bcb23 timestamp: 2023-06-23_17:37:30 tube: smithi user: rishabh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-06-23T18:01:09.541 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa; will attempt to use it 2023-06-23T18:01:09.542 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks 2023-06-23T18:01:09.542 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-06-23T18:01:09.544 INFO:teuthology.task.internal:Checking packages... 2023-06-23T18:01:09.572 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '8627af3e0adcb765a3c249fcc209cba9f4873e1b' 2023-06-23T18:01:09.572 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-06-23T18:01:09.573 INFO:teuthology.packaging:ref: None 2023-06-23T18:01:09.573 INFO:teuthology.packaging:tag: None 2023-06-23T18:01:09.573 INFO:teuthology.packaging:branch: wip-rishabh-improvements-authmon 2023-06-23T18:01:09.573 INFO:teuthology.packaging:sha1: 8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:01:09.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-improvements-authmon 2023-06-23T18:01:09.817 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4479.g8627af3e 2023-06-23T18:01:09.817 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-06-23T18:01:09.824 INFO:teuthology.task.internal:no buildpackages task found 2023-06-23T18:01:09.824 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-06-23T18:01:09.832 INFO:teuthology.task.internal:Saving configuration 2023-06-23T18:01:09.852 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-06-23T18:01:09.857 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T18:01:09.911 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 17:55:52.388548', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-23T18:01:09.933 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 17:55:52.390669', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-23T18:01:09.962 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 17:55:52.389731', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-06-23T18:01:09.962 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-06-23T18:01:09.968 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-06-23T18:01:09.968 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-06-23T18:01:09.968 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-06-23T18:01:09.968 INFO:teuthology.run_tasks:Running task console_log... 2023-06-23T18:01:10.078 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8cc2add160>, signals=[15]) 2023-06-23T18:01:10.080 INFO:teuthology.run_tasks:Running task internal.connect... 2023-06-23T18:01:10.085 INFO:teuthology.task.internal:Opening connections... 2023-06-23T18:01:10.085 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2023-06-23T18:01:10.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:01:10.159 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2023-06-23T18:01:10.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:01:10.228 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2023-06-23T18:01:10.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:01:10.295 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-06-23T18:01:10.302 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2023-06-23T18:01:10.323 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2023-06-23T18:01:10.323 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2023-06-23T18:01:10.383 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T18:01:10.383 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8.6 (Ootpa)" 2023-06-23T18:01:10.383 INFO:teuthology.orchestra.run.smithi002.stdout:ID="rhel" 2023-06-23T18:01:10.383 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="fedora" 2023-06-23T18:01:10.383 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8.6" 2023-06-23T18:01:10.384 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2023-06-23T18:01:10.384 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-23T18:01:10.384 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2023-06-23T18:01:10.384 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-23T18:01:10.384 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T18:01:10.384 INFO:teuthology.orchestra.run.smithi002.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T18:01:10.384 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T18:01:10.385 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:01:10.385 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T18:01:10.385 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-23T18:01:10.385 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T18:01:10.385 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-23T18:01:10.386 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2023-06-23T18:01:10.408 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2023-06-23T18:01:10.429 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2023-06-23T18:01:10.430 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2023-06-23T18:01:10.489 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T18:01:10.490 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="8.6 (Ootpa)" 2023-06-23T18:01:10.490 INFO:teuthology.orchestra.run.smithi017.stdout:ID="rhel" 2023-06-23T18:01:10.490 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE="fedora" 2023-06-23T18:01:10.490 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="8.6" 2023-06-23T18:01:10.490 INFO:teuthology.orchestra.run.smithi017.stdout:PLATFORM_ID="platform:el8" 2023-06-23T18:01:10.490 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-23T18:01:10.490 INFO:teuthology.orchestra.run.smithi017.stdout:ANSI_COLOR="0;31" 2023-06-23T18:01:10.490 INFO:teuthology.orchestra.run.smithi017.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-23T18:01:10.491 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T18:01:10.491 INFO:teuthology.orchestra.run.smithi017.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T18:01:10.491 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T18:01:10.491 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:01:10.491 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T18:01:10.491 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-23T18:01:10.491 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T18:01:10.492 INFO:teuthology.orchestra.run.smithi017.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-23T18:01:10.492 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2023-06-23T18:01:10.516 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2023-06-23T18:01:10.536 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2023-06-23T18:01:10.536 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2023-06-23T18:01:10.595 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Red Hat Enterprise Linux" 2023-06-23T18:01:10.595 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="8.6 (Ootpa)" 2023-06-23T18:01:10.595 INFO:teuthology.orchestra.run.smithi082.stdout:ID="rhel" 2023-06-23T18:01:10.595 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE="fedora" 2023-06-23T18:01:10.595 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="8.6" 2023-06-23T18:01:10.595 INFO:teuthology.orchestra.run.smithi082.stdout:PLATFORM_ID="platform:el8" 2023-06-23T18:01:10.596 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-06-23T18:01:10.596 INFO:teuthology.orchestra.run.smithi082.stdout:ANSI_COLOR="0;31" 2023-06-23T18:01:10.596 INFO:teuthology.orchestra.run.smithi082.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-06-23T18:01:10.596 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.redhat.com/" 2023-06-23T18:01:10.596 INFO:teuthology.orchestra.run.smithi082.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-06-23T18:01:10.596 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-06-23T18:01:10.596 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:01:10.597 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-06-23T18:01:10.597 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-06-23T18:01:10.597 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-06-23T18:01:10.597 INFO:teuthology.orchestra.run.smithi082.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-06-23T18:01:10.597 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2023-06-23T18:01:10.616 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-06-23T18:01:10.625 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-06-23T18:01:10.630 INFO:teuthology.task.internal:Checking for old test directory... 2023-06-23T18:01:10.630 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2023-06-23T18:01:10.632 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2023-06-23T18:01:10.634 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2023-06-23T18:01:10.652 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-06-23T18:01:10.658 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-06-23T18:01:10.658 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2023-06-23T18:01:10.692 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2023-06-23T18:01:10.694 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2023-06-23T18:01:10.717 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-06-23T18:01:10.799 INFO:teuthology.run_tasks:Running task kernel... 2023-06-23T18:01:10.814 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-06-23T18:01:10.814 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-06-23T18:01:10.815 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T18:01:10.816 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T18:01:10.816 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2023-06-23T18:01:10.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:01:10.834 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2023-06-23T18:01:10.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:01:10.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:01:10.836 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2023-06-23T18:01:10.837 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2023-06-23T18:01:10.892 INFO:teuthology.orchestra.run.smithi082.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:10.894 INFO:teuthology.task.kernel:Running kernel on smithi082: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:10.894 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum install -y kernel 2023-06-23T18:01:10.896 INFO:teuthology.orchestra.run.smithi002.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:10.896 INFO:teuthology.task.kernel:Running kernel on smithi002: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:10.896 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2023-06-23T18:01:10.897 INFO:teuthology.orchestra.run.smithi017.stdout:4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:10.897 INFO:teuthology.task.kernel:Running kernel on smithi017: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:10.897 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum install -y kernel 2023-06-23T18:01:12.072 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:12.072 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-06-23T18:01:12.155 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:12.155 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2023-06-23T18:01:12.181 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:12.181 INFO:teuthology.orchestra.run.smithi017.stdout:Unable to read consumer identity 2023-06-23T18:01:15.765 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2023-06-23T18:01:16.027 INFO:teuthology.orchestra.run.smithi017.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-23T18:01:16.034 INFO:teuthology.orchestra.run.smithi002.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-23T18:01:18.770 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 16 MB 00:02 2023-06-23T18:01:18.793 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 16 MB 00:02 2023-06-23T18:01:18.840 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 5.1 MB/s | 16 MB 00:03 2023-06-23T18:01:22.356 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 307 kB/s | 24 kB 00:00 2023-06-23T18:01:22.476 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 448 kB/s | 24 kB 00:00 2023-06-23T18:01:22.520 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 259 kB/s | 24 kB 00:00 2023-06-23T18:01:23.059 INFO:teuthology.orchestra.run.smithi082.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-23T18:01:23.108 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-06-23T18:01:23.109 INFO:teuthology.orchestra.run.smithi082.stdout:Nothing to do. 2023-06-23T18:01:23.109 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-06-23T18:01:23.146 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-23T18:01:23.181 INFO:teuthology.orchestra.run.smithi017.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-06-23T18:01:23.193 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-06-23T18:01:23.194 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2023-06-23T18:01:23.194 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-06-23T18:01:23.220 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-06-23T18:01:23.222 INFO:teuthology.orchestra.run.smithi017.stdout:Nothing to do. 2023-06-23T18:01:23.222 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-06-23T18:01:23.243 DEBUG:teuthology.orchestra.run.smithi082:> echo no | sudo yum reinstall kernel || true 2023-06-23T18:01:23.319 DEBUG:teuthology.orchestra.run.smithi002:> echo no | sudo yum reinstall kernel || true 2023-06-23T18:01:23.341 DEBUG:teuthology.orchestra.run.smithi017:> echo no | sudo yum reinstall kernel || true 2023-06-23T18:01:23.612 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:23.613 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-06-23T18:01:23.691 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:23.691 INFO:teuthology.orchestra.run.smithi017.stdout:Unable to read consumer identity 2023-06-23T18:01:23.711 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:23.711 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2023-06-23T18:01:23.874 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:01 ago on Fri 23 Jun 2023 06:01:22 PM UTC. 2023-06-23T18:01:23.922 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:01:23.958 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2023-06-23T18:01:23.966 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:01 ago on Fri 23 Jun 2023 06:01:22 PM UTC. 2023-06-23T18:01:23.988 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:01 ago on Fri 23 Jun 2023 06:01:22 PM UTC. 2023-06-23T18:01:24.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum reinstall -y kernel || true 2023-06-23T18:01:24.009 INFO:teuthology.orchestra.run.smithi017.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:01:24.031 INFO:teuthology.orchestra.run.smithi002.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:01:24.037 INFO:teuthology.orchestra.run.smithi017.stderr:Error: No packages marked for reinstall. 2023-06-23T18:01:24.073 INFO:teuthology.orchestra.run.smithi002.stderr:Error: No packages marked for reinstall. 2023-06-23T18:01:24.083 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum reinstall -y kernel || true 2023-06-23T18:01:24.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum reinstall -y kernel || true 2023-06-23T18:01:24.331 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:24.331 INFO:teuthology.orchestra.run.smithi082.stdout:Unable to read consumer identity 2023-06-23T18:01:24.432 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:24.432 INFO:teuthology.orchestra.run.smithi017.stdout:Unable to read consumer identity 2023-06-23T18:01:24.470 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-06-23T18:01:24.471 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2023-06-23T18:01:24.586 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 06:01:22 PM UTC. 2023-06-23T18:01:24.626 INFO:teuthology.orchestra.run.smithi082.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:01:24.658 INFO:teuthology.orchestra.run.smithi082.stderr:Error: No packages marked for reinstall. 2023-06-23T18:01:24.696 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 06:01:22 PM UTC. 2023-06-23T18:01:24.703 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T18:01:24.734 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:02 ago on Fri 23 Jun 2023 06:01:22 PM UTC. 2023-06-23T18:01:24.738 INFO:teuthology.orchestra.run.smithi017.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:01:24.779 INFO:teuthology.orchestra.run.smithi002.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-06-23T18:01:24.782 INFO:teuthology.orchestra.run.smithi082.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:24.783 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:24.784 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-23T18:01:24.784 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2023-06-23T18:01:24.784 DEBUG:teuthology.orchestra.run.smithi082:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T18:01:24.809 INFO:teuthology.orchestra.run.smithi002.stderr:Error: No packages marked for reinstall. 2023-06-23T18:01:24.810 INFO:teuthology.orchestra.run.smithi082.stdout:ttyS1 2023-06-23T18:01:24.825 DEBUG:teuthology.parallel:result is None 2023-06-23T18:01:24.864 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T18:01:24.885 INFO:teuthology.orchestra.run.smithi017.stderr:Error: No packages marked for reinstall. 2023-06-23T18:01:24.933 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q kernel | sort -rV | head -n 1 2023-06-23T18:01:24.946 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:24.947 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:24.947 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-23T18:01:24.947 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-06-23T18:01:24.948 DEBUG:teuthology.orchestra.run.smithi002:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T18:01:24.995 INFO:teuthology.orchestra.run.smithi002.stdout:ttyS1 2023-06-23T18:01:25.007 DEBUG:teuthology.parallel:result is None 2023-06-23T18:01:25.015 INFO:teuthology.orchestra.run.smithi017.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:25.015 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-06-23T18:01:25.016 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-06-23T18:01:25.016 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-06-23T18:01:25.016 DEBUG:teuthology.orchestra.run.smithi017:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-06-23T18:01:25.048 INFO:teuthology.orchestra.run.smithi017.stdout:ttyS1 2023-06-23T18:01:25.064 DEBUG:teuthology.parallel:result is None 2023-06-23T18:01:25.064 INFO:teuthology.run_tasks:Running task internal.base... 2023-06-23T18:01:25.072 INFO:teuthology.task.internal:Creating test directory... 2023-06-23T18:01:25.072 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T18:01:25.074 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T18:01:25.106 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-06-23T18:01:25.123 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-06-23T18:01:25.129 INFO:teuthology.run_tasks:Running task internal.archive... 2023-06-23T18:01:25.136 INFO:teuthology.task.internal:Creating archive directory... 2023-06-23T18:01:25.137 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T18:01:25.139 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T18:01:25.164 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-06-23T18:01:25.187 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-06-23T18:01:25.194 INFO:teuthology.task.internal:Enabling coredump saving... 2023-06-23T18:01:25.194 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T18:01:25.197 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T18:01:25.222 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-06-23T18:01:25.237 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:01:25.248 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:01:25.249 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:01:25.257 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:01:25.260 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:01:25.268 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-06-23T18:01:25.269 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-06-23T18:01:25.276 INFO:teuthology.task.internal:Configuring sudo... 2023-06-23T18:01:25.276 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T18:01:25.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T18:01:25.312 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-06-23T18:01:25.337 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-06-23T18:01:25.346 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-06-23T18:01:25.346 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T18:01:25.359 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T18:01:25.378 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-06-23T18:01:25.395 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:01:25.460 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:01:25.505 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:01:25.591 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:01:25.630 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:01:25.658 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T18:01:25.699 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:01:25.747 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:01:25.792 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:01:25.877 DEBUG:teuthology.orchestra.run.smithi017:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:01:25.918 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:01:25.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T18:01:25.985 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:01:26.032 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-06-23T18:01:26.091 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:01:26.136 DEBUG:teuthology.orchestra.run.smithi082:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-06-23T18:01:26.177 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:01:26.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-06-23T18:01:26.244 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2023-06-23T18:01:26.246 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2023-06-23T18:01:26.248 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2023-06-23T18:01:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T18:01:26.326 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T18:01:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T18:01:26.862 INFO:teuthology.run_tasks:Running task internal.timer... 2023-06-23T18:01:26.870 INFO:teuthology.task.internal:Starting timer... 2023-06-23T18:01:26.870 INFO:teuthology.run_tasks:Running task pcp... 2023-06-23T18:01:26.889 INFO:teuthology.run_tasks:Running task selinux... 2023-06-23T18:01:26.898 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-06-23T18:01:26.898 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2023-06-23T18:01:26.960 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2023-06-23T18:01:26.962 DEBUG:teuthology.orchestra.run.smithi017:> sudo service auditd rotate 2023-06-23T18:01:27.026 INFO:teuthology.orchestra.run.smithi017.stdout:Rotating logs: 2023-06-23T18:01:27.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo service auditd rotate 2023-06-23T18:01:27.089 INFO:teuthology.orchestra.run.smithi082.stdout:Rotating logs: 2023-06-23T18:01:27.091 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-06-23T18:01:27.091 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2023-06-23T18:01:27.111 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2023-06-23T18:01:27.111 DEBUG:teuthology.orchestra.run.smithi017:> /usr/sbin/getenforce 2023-06-23T18:01:27.132 INFO:teuthology.orchestra.run.smithi017.stdout:Permissive 2023-06-23T18:01:27.133 DEBUG:teuthology.orchestra.run.smithi082:> /usr/sbin/getenforce 2023-06-23T18:01:27.154 INFO:teuthology.orchestra.run.smithi082.stdout:Permissive 2023-06-23T18:01:27.155 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi017.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi082.front.sepia.ceph.com': 'permissive'} 2023-06-23T18:01:27.155 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T18:01:27.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:01:27.185 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T18:01:27.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:01:27.216 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T18:01:27.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:01:27.245 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-06-23T18:01:27.245 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2023-06-23T18:01:27.274 DEBUG:teuthology.orchestra.run.smithi017:> sudo /usr/sbin/setenforce permissive 2023-06-23T18:01:27.302 DEBUG:teuthology.orchestra.run.smithi082:> sudo /usr/sbin/setenforce permissive 2023-06-23T18:01:27.330 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-06-23T18:01:27.339 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-06-23T18:01:27.339 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-06-23T18:01:27.359 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-06-23T18:01:27.360 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi017.front.sepia.ceph.com,smithi082.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-06-23T18:10:02.217 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi082.front.sepia.ceph.com')] 2023-06-23T18:10:02.219 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-06-23T18:10:02.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:10:02.289 DEBUG:teuthology.orchestra.run.smithi002:> true 2023-06-23T18:10:02.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2023-06-23T18:10:02.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:10:02.429 DEBUG:teuthology.orchestra.run.smithi017:> true 2023-06-23T18:10:02.457 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2023-06-23T18:10:02.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-06-23T18:10:02.524 DEBUG:teuthology.orchestra.run.smithi082:> true 2023-06-23T18:10:02.593 INFO:teuthology.run_tasks:Running task clock... 2023-06-23T18:10:02.604 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-06-23T18:10:02.605 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T18:10:02.605 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T18:10:02.608 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T18:10:02.608 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T18:10:02.610 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-06-23T18:10:02.610 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T18:10:02.642 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T18:10:02.643 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T18:10:02.658 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T18:10:02.659 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T18:10:02.663 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-06-23T18:10:02.677 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-06-23T18:10:02.731 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: ntpd: command not found 2023-06-23T18:10:02.731 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2023-06-23T18:10:02.744 INFO:teuthology.orchestra.run.smithi017.stdout:506 Cannot talk to daemon 2023-06-23T18:10:02.745 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2023-06-23T18:10:02.745 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: ntpd: command not found 2023-06-23T18:10:02.756 INFO:teuthology.orchestra.run.smithi082.stdout:506 Cannot talk to daemon 2023-06-23T18:10:02.759 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T18:10:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T18:10:02.771 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-06-23T18:10:02.773 INFO:teuthology.orchestra.run.smithi017.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T18:10:02.776 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T18:10:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-06-23T18:10:02.845 INFO:teuthology.orchestra.run.smithi002.stderr:bash: ntpq: command not found 2023-06-23T18:10:02.848 INFO:teuthology.orchestra.run.smithi017.stderr:bash: ntpq: command not found 2023-06-23T18:10:02.848 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T18:10:02.849 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2023-06-23T18:10:02.849 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.849 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.849 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.849 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.851 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T18:10:02.851 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2023-06-23T18:10:02.852 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.852 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.852 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.852 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.869 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2023-06-23T18:10:02.872 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T18:10:02.873 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2023-06-23T18:10:02.873 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.873 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.873 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.873 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T18:10:02.874 INFO:teuthology.run_tasks:Running task pexec... 2023-06-23T18:10:02.883 INFO:teuthology.task.pexec:Executing custom commands... 2023-06-23T18:10:02.884 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi002.front.sepia.ceph.com 2023-06-23T18:10:02.884 DEBUG:teuthology.orchestra.run.smithi002:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-23T18:10:02.885 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi017.front.sepia.ceph.com 2023-06-23T18:10:02.885 DEBUG:teuthology.orchestra.run.smithi017:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-23T18:10:02.885 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi082.front.sepia.ceph.com 2023-06-23T18:10:02.885 DEBUG:teuthology.orchestra.run.smithi082:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-06-23T18:10:03.283 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:03.284 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:03.284 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:04.117 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:40 ago on Fri 23 Jun 2023 06:08:23 PM UTC. 2023-06-23T18:10:04.164 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:25 ago on Fri 23 Jun 2023 06:08:38 PM UTC. 2023-06-23T18:10:04.187 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:27 ago on Fri 23 Jun 2023 06:08:36 PM UTC. 2023-06-23T18:10:05.992 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-06-23T18:10:05.994 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-06-23T18:10:05.994 INFO:teuthology.orchestra.run.smithi082.stdout: Package Architecture Version Repository Size 2023-06-23T18:10:05.994 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-06-23T18:10:05.994 INFO:teuthology.orchestra.run.smithi082.stdout:Resetting modules: 2023-06-23T18:10:05.994 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools 2023-06-23T18:10:05.994 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:05.994 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-06-23T18:10:05.995 INFO:teuthology.orchestra.run.smithi082.stdout:================================================================================ 2023-06-23T18:10:05.995 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:06.176 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-06-23T18:10:06.177 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-06-23T18:10:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: Package Architecture Version Repository Size 2023-06-23T18:10:06.178 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-06-23T18:10:06.178 INFO:teuthology.orchestra.run.smithi017.stdout:Resetting modules: 2023-06-23T18:10:06.178 INFO:teuthology.orchestra.run.smithi017.stdout: container-tools 2023-06-23T18:10:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:10:06.179 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-06-23T18:10:06.179 INFO:teuthology.orchestra.run.smithi017.stdout:================================================================================ 2023-06-23T18:10:06.179 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:10:06.189 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-06-23T18:10:06.191 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2023-06-23T18:10:06.191 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2023-06-23T18:10:06.191 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2023-06-23T18:10:06.191 INFO:teuthology.orchestra.run.smithi002.stdout:Resetting modules: 2023-06-23T18:10:06.191 INFO:teuthology.orchestra.run.smithi002.stdout: container-tools 2023-06-23T18:10:06.192 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:06.192 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2023-06-23T18:10:06.192 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2023-06-23T18:10:06.192 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:06.958 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:43 ago on Fri 23 Jun 2023 06:08:23 PM UTC. 2023-06-23T18:10:07.187 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:28 ago on Fri 23 Jun 2023 06:08:38 PM UTC. 2023-06-23T18:10:07.197 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:30 ago on Fri 23 Jun 2023 06:08:36 PM UTC. 2023-06-23T18:10:10.253 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-06-23T18:10:10.269 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-06-23T18:10:10.527 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-06-23T18:10:10.699 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:10.736 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:11.017 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:11.565 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:48 ago on Fri 23 Jun 2023 06:08:23 PM UTC. 2023-06-23T18:10:11.695 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:33 ago on Fri 23 Jun 2023 06:08:38 PM UTC. 2023-06-23T18:10:11.998 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:35 ago on Fri 23 Jun 2023 06:08:36 PM UTC. 2023-06-23T18:10:13.289 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-06-23T18:10:13.383 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-06-23T18:10:13.383 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-06-23T18:10:13.384 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-06-23T18:10:13.384 INFO:teuthology.orchestra.run.smithi082.stdout:Installing group/module packages: 2023-06-23T18:10:13.384 INFO:teuthology.orchestra.run.smithi082.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-06-23T18:10:13.384 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-06-23T18:10:13.384 INFO:teuthology.orchestra.run.smithi082.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-06-23T18:10:13.384 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-06-23T18:10:13.385 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-23T18:10:13.385 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-06-23T18:10:13.385 INFO:teuthology.orchestra.run.smithi082.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-06-23T18:10:13.385 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-06-23T18:10:13.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-06-23T18:10:13.385 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-06-23T18:10:13.385 INFO:teuthology.orchestra.run.smithi082.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-23T18:10:13.385 INFO:teuthology.orchestra.run.smithi082.stdout:Installing module profiles: 2023-06-23T18:10:13.386 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools/common 2023-06-23T18:10:13.386 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:13.386 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2023-06-23T18:10:13.386 INFO:teuthology.orchestra.run.smithi082.stdout: container-tools rhel8 2023-06-23T18:10:13.386 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:13.386 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-06-23T18:10:13.386 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================================================ 2023-06-23T18:10:13.387 INFO:teuthology.orchestra.run.smithi082.stdout:Install 10 Packages 2023-06-23T18:10:13.387 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:13.387 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 18 M 2023-06-23T18:10:13.388 INFO:teuthology.orchestra.run.smithi082.stdout:Installed size: 65 M 2023-06-23T18:10:13.388 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-06-23T18:10:13.476 INFO:teuthology.orchestra.run.smithi082.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 276 kB/s | 25 kB 00:00 2023-06-23T18:10:13.493 INFO:teuthology.orchestra.run.smithi082.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 382 kB/s | 41 kB 00:00 2023-06-23T18:10:13.518 INFO:teuthology.orchestra.run.smithi082.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 711 kB/s | 94 kB 00:00 2023-06-23T18:10:13.535 INFO:teuthology.orchestra.run.smithi082.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 840 kB/s | 49 kB 00:00 2023-06-23T18:10:13.568 INFO:teuthology.orchestra.run.smithi082.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.8 MB/s | 524 kB 00:00 2023-06-23T18:10:13.602 INFO:teuthology.orchestra.run.smithi082.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-06-23T18:10:13.627 INFO:teuthology.orchestra.run.smithi082.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 3.5 MB/s | 213 kB 00:00 2023-06-23T18:10:13.664 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-06-23T18:10:13.736 INFO:teuthology.orchestra.run.smithi082.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-06-23T18:10:13.765 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================================ 2023-06-23T18:10:13.766 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-06-23T18:10:13.766 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================================ 2023-06-23T18:10:13.766 INFO:teuthology.orchestra.run.smithi017.stdout:Installing group/module packages: 2023-06-23T18:10:13.766 INFO:teuthology.orchestra.run.smithi017.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-06-23T18:10:13.766 INFO:teuthology.orchestra.run.smithi017.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-06-23T18:10:13.766 INFO:teuthology.orchestra.run.smithi017.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-06-23T18:10:13.766 INFO:teuthology.orchestra.run.smithi017.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-06-23T18:10:13.767 INFO:teuthology.orchestra.run.smithi017.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-23T18:10:13.767 INFO:teuthology.orchestra.run.smithi017.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-06-23T18:10:13.767 INFO:teuthology.orchestra.run.smithi017.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-06-23T18:10:13.767 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-06-23T18:10:13.767 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-06-23T18:10:13.768 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-06-23T18:10:13.768 INFO:teuthology.orchestra.run.smithi017.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-23T18:10:13.768 INFO:teuthology.orchestra.run.smithi017.stdout:Installing module profiles: 2023-06-23T18:10:13.768 INFO:teuthology.orchestra.run.smithi017.stdout: container-tools/common 2023-06-23T18:10:13.768 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:10:13.768 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling module streams: 2023-06-23T18:10:13.769 INFO:teuthology.orchestra.run.smithi017.stdout: container-tools rhel8 2023-06-23T18:10:13.769 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:10:13.769 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-06-23T18:10:13.769 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================================================ 2023-06-23T18:10:13.769 INFO:teuthology.orchestra.run.smithi017.stdout:Install 10 Packages 2023-06-23T18:10:13.769 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:10:13.770 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 18 M 2023-06-23T18:10:13.770 INFO:teuthology.orchestra.run.smithi017.stdout:Installed size: 65 M 2023-06-23T18:10:13.770 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-06-23T18:10:13.840 INFO:teuthology.orchestra.run.smithi017.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-06-23T18:10:13.853 INFO:teuthology.orchestra.run.smithi082.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 24 MB/s | 8.0 MB 00:00 2023-06-23T18:10:13.866 INFO:teuthology.orchestra.run.smithi017.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 254 kB/s | 25 kB 00:00 2023-06-23T18:10:13.891 INFO:teuthology.orchestra.run.smithi017.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 332 kB/s | 41 kB 00:00 2023-06-23T18:10:13.916 INFO:teuthology.orchestra.run.smithi017.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-06-23T18:10:13.944 INFO:teuthology.orchestra.run.smithi082.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 21 MB/s | 7.0 MB 00:00 2023-06-23T18:10:13.945 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:10:13.945 INFO:teuthology.orchestra.run.smithi082.stdout:Total 33 MB/s | 18 MB 00:00 2023-06-23T18:10:13.950 INFO:teuthology.orchestra.run.smithi017.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-06-23T18:10:13.965 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-06-23T18:10:13.975 INFO:teuthology.orchestra.run.smithi017.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-06-23T18:10:14.000 INFO:teuthology.orchestra.run.smithi017.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.2 MB/s | 213 kB 00:00 2023-06-23T18:10:14.065 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================================ 2023-06-23T18:10:14.065 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2023-06-23T18:10:14.065 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================================ 2023-06-23T18:10:14.066 INFO:teuthology.orchestra.run.smithi002.stdout:Installing group/module packages: 2023-06-23T18:10:14.066 INFO:teuthology.orchestra.run.smithi002.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-06-23T18:10:14.066 INFO:teuthology.orchestra.run.smithi002.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-06-23T18:10:14.066 INFO:teuthology.orchestra.run.smithi002.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-06-23T18:10:14.066 INFO:teuthology.orchestra.run.smithi002.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-06-23T18:10:14.066 INFO:teuthology.orchestra.run.smithi002.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-06-23T18:10:14.067 INFO:teuthology.orchestra.run.smithi002.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-06-23T18:10:14.067 INFO:teuthology.orchestra.run.smithi002.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-06-23T18:10:14.067 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2023-06-23T18:10:14.067 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-06-23T18:10:14.067 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-06-23T18:10:14.067 INFO:teuthology.orchestra.run.smithi002.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-06-23T18:10:14.067 INFO:teuthology.orchestra.run.smithi002.stdout:Installing module profiles: 2023-06-23T18:10:14.067 INFO:teuthology.orchestra.run.smithi002.stdout: container-tools/common 2023-06-23T18:10:14.068 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:14.068 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling module streams: 2023-06-23T18:10:14.068 INFO:teuthology.orchestra.run.smithi002.stdout: container-tools rhel8 2023-06-23T18:10:14.068 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:14.068 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2023-06-23T18:10:14.068 INFO:teuthology.orchestra.run.smithi002.stdout:============================================================================================================ 2023-06-23T18:10:14.068 INFO:teuthology.orchestra.run.smithi002.stdout:Install 10 Packages 2023-06-23T18:10:14.069 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:14.069 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 18 M 2023-06-23T18:10:14.069 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 65 M 2023-06-23T18:10:14.070 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2023-06-23T18:10:14.101 INFO:teuthology.orchestra.run.smithi017.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-06-23T18:10:14.140 INFO:teuthology.orchestra.run.smithi002.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.3 MB/s | 94 kB 00:00 2023-06-23T18:10:14.144 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-06-23T18:10:14.163 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-06-23T18:10:14.163 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-06-23T18:10:14.165 INFO:teuthology.orchestra.run.smithi002.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 255 kB/s | 25 kB 00:00 2023-06-23T18:10:14.190 INFO:teuthology.orchestra.run.smithi002.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 333 kB/s | 41 kB 00:00 2023-06-23T18:10:14.216 INFO:teuthology.orchestra.run.smithi002.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 655 kB/s | 49 kB 00:00 2023-06-23T18:10:14.226 INFO:teuthology.orchestra.run.smithi017.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-06-23T18:10:14.257 INFO:teuthology.orchestra.run.smithi002.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.6 MB/s | 524 kB 00:00 2023-06-23T18:10:14.283 INFO:teuthology.orchestra.run.smithi002.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-06-23T18:10:14.308 INFO:teuthology.orchestra.run.smithi002.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-06-23T18:10:14.318 INFO:teuthology.orchestra.run.smithi017.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-06-23T18:10:14.318 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:10:14.318 INFO:teuthology.orchestra.run.smithi017.stdout:Total 33 MB/s | 18 MB 00:00 2023-06-23T18:10:14.345 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-06-23T18:10:14.349 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-06-23T18:10:14.408 INFO:teuthology.orchestra.run.smithi002.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-06-23T18:10:14.526 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-06-23T18:10:14.542 INFO:teuthology.orchestra.run.smithi002.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 27 MB/s | 7.0 MB 00:00 2023-06-23T18:10:14.546 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-06-23T18:10:14.546 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-06-23T18:10:14.634 INFO:teuthology.orchestra.run.smithi002.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-06-23T18:10:14.634 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:10:14.635 INFO:teuthology.orchestra.run.smithi002.stdout:Total 32 MB/s | 18 MB 00:00 2023-06-23T18:10:14.741 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-06-23T18:10:14.745 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-06-23T18:10:14.842 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2023-06-23T18:10:14.862 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2023-06-23T18:10:14.863 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2023-06-23T18:10:14.893 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-06-23T18:10:15.056 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2023-06-23T18:10:15.061 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2023-06-23T18:10:15.085 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-06-23T18:10:15.250 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-06-23T18:10:15.266 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-06-23T18:10:15.436 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-06-23T18:10:15.455 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-06-23T18:10:15.622 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2023-06-23T18:10:15.648 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-06-23T18:10:15.655 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-06-23T18:10:15.776 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-06-23T18:10:15.844 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-06-23T18:10:15.930 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-06-23T18:10:15.996 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-06-23T18:10:16.040 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-06-23T18:10:16.160 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-06-23T18:10:16.364 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-06-23T18:10:16.406 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-06-23T18:10:16.720 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-06-23T18:10:16.797 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-06-23T18:10:17.242 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-06-23T18:10:17.544 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-06-23T18:10:17.687 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-06-23T18:10:17.892 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-06-23T18:10:18.073 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-06-23T18:10:18.149 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-06-23T18:10:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-06-23T18:10:18.345 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-06-23T18:10:18.354 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-06-23T18:10:18.354 INFO:teuthology.orchestra.run.smithi082.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T18:10:18.355 INFO:teuthology.orchestra.run.smithi082.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T18:10:18.355 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-06-23T18:10:18.502 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-06-23T18:10:18.503 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-06-23T18:10:18.504 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-06-23T18:10:18.504 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-06-23T18:10:18.504 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-06-23T18:10:18.504 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-06-23T18:10:18.504 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-06-23T18:10:18.505 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-06-23T18:10:18.505 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-06-23T18:10:18.505 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-06-23T18:10:18.693 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-06-23T18:10:18.696 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-06-23T18:10:18.798 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-06-23T18:10:18.799 INFO:teuthology.orchestra.run.smithi017.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T18:10:18.799 INFO:teuthology.orchestra.run.smithi017.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T18:10:18.799 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:10:18.836 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-06-23T18:10:18.956 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-06-23T18:10:18.956 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-06-23T18:10:18.957 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-06-23T18:10:18.957 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-06-23T18:10:18.957 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-06-23T18:10:18.957 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-06-23T18:10:18.957 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-06-23T18:10:18.957 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-06-23T18:10:18.957 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-06-23T18:10:18.996 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-06-23T18:10:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-06-23T18:10:19.116 INFO:teuthology.orchestra.run.smithi002.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T18:10:19.116 INFO:teuthology.orchestra.run.smithi002.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T18:10:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:19.285 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-06-23T18:10:19.285 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-06-23T18:10:19.285 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-06-23T18:10:19.285 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-06-23T18:10:19.286 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-06-23T18:10:19.286 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-06-23T18:10:19.286 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-06-23T18:10:19.286 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-06-23T18:10:19.286 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-06-23T18:10:19.669 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-06-23T18:10:19.669 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:56 ago on Fri 23 Jun 2023 06:08:23 PM UTC. 2023-06-23T18:10:20.175 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-06-23T18:10:20.175 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:41 ago on Fri 23 Jun 2023 06:08:38 PM UTC. 2023-06-23T18:10:20.434 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-06-23T18:10:20.434 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:43 ago on Fri 23 Jun 2023 06:08:36 PM UTC. 2023-06-23T18:10:22.496 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-06-23T18:10:22.684 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:22.684 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-06-23T18:10:22.685 INFO:teuthology.orchestra.run.smithi082.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:22.685 INFO:teuthology.orchestra.run.smithi082.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-06-23T18:10:22.685 INFO:teuthology.orchestra.run.smithi082.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:22.685 INFO:teuthology.orchestra.run.smithi082.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-06-23T18:10:22.685 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-06-23T18:10:22.685 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-06-23T18:10:22.685 INFO:teuthology.orchestra.run.smithi082.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:22.686 INFO:teuthology.orchestra.run.smithi082.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:22.686 INFO:teuthology.orchestra.run.smithi082.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-06-23T18:10:22.686 INFO:teuthology.orchestra.run.smithi082.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-23T18:10:22.686 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:22.686 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-06-23T18:10:22.826 DEBUG:teuthology.parallel:result is None 2023-06-23T18:10:23.435 INFO:teuthology.orchestra.run.smithi017.stdout:Installed products updated. 2023-06-23T18:10:23.498 INFO:teuthology.orchestra.run.smithi002.stdout:Installed products updated. 2023-06-23T18:10:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:10:23.646 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-06-23T18:10:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-06-23T18:10:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-06-23T18:10:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-06-23T18:10:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-06-23T18:10:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-06-23T18:10:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-23T18:10:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:10:23.648 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-06-23T18:10:23.705 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:23.706 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2023-06-23T18:10:23.706 INFO:teuthology.orchestra.run.smithi002.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:23.706 INFO:teuthology.orchestra.run.smithi002.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-06-23T18:10:23.706 INFO:teuthology.orchestra.run.smithi002.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:23.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-06-23T18:10:23.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-06-23T18:10:23.706 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-06-23T18:10:23.707 INFO:teuthology.orchestra.run.smithi002.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:23.707 INFO:teuthology.orchestra.run.smithi002.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-06-23T18:10:23.707 INFO:teuthology.orchestra.run.smithi002.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-06-23T18:10:23.707 INFO:teuthology.orchestra.run.smithi002.stdout: yajl-2.1.0-10.el8.x86_64 2023-06-23T18:10:23.707 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:23.707 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-06-23T18:10:23.801 DEBUG:teuthology.parallel:result is None 2023-06-23T18:10:23.879 DEBUG:teuthology.parallel:result is None 2023-06-23T18:10:23.880 INFO:teuthology.run_tasks:Running task install... 2023-06-23T18:10:23.889 DEBUG:teuthology.task.install:project ceph 2023-06-23T18:10:23.889 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'}} 2023-06-23T18:10:23.889 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b'} 2023-06-23T18:10:23.890 INFO:teuthology.task.install:Using flavor: default 2023-06-23T18:10:23.896 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-06-23T18:10:23.896 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-06-23T18:10:23.897 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'tag': None, 'wait_for_package': False} 2023-06-23T18:10:23.898 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:10:23.900 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'tag': None, 'wait_for_package': False} 2023-06-23T18:10:23.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:10:23.901 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'tag': None, 'wait_for_package': False} 2023-06-23T18:10:23.902 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:10:24.039 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/ 2023-06-23T18:10:24.040 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4479.g8627af3e 2023-06-23T18:10:24.044 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/ 2023-06-23T18:10:24.044 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4479.g8627af3e 2023-06-23T18:10:24.117 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/ 2023-06-23T18:10:24.118 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4479.g8627af3e 2023-06-23T18:10:24.149 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T18:10:24.149 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:10:24.149 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T18:10:24.151 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T18:10:24.152 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:10:24.152 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T18:10:24.180 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-23T18:10:24.180 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8627af3e0adcb765a3c249fcc209cba9f4873e1b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T18:10:24.184 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-23T18:10:24.185 DEBUG:teuthology.orchestra.run.smithi082:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8627af3e0adcb765a3c249fcc209cba9f4873e1b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T18:10:24.219 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-06-23T18:10:24.219 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:10:24.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-06-23T18:10:24.248 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T18:10:24.249 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-06-23T18:10:24.250 DEBUG:teuthology.orchestra.run.smithi017:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8627af3e0adcb765a3c249fcc209cba9f4873e1b/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-06-23T18:10:24.256 DEBUG:teuthology.orchestra.run.smithi082:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T18:10:24.319 DEBUG:teuthology.orchestra.run.smithi017:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-06-23T18:10:24.326 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T18:10:24.336 DEBUG:teuthology.orchestra.run.smithi082:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T18:10:24.393 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2023-06-23T18:10:24.394 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2023-06-23T18:10:24.397 DEBUG:teuthology.orchestra.run.smithi017:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-06-23T18:10:24.403 INFO:teuthology.orchestra.run.smithi082.stdout:check_obsoletes = 1 2023-06-23T18:10:24.404 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum clean all 2023-06-23T18:10:24.463 INFO:teuthology.orchestra.run.smithi017.stdout:check_obsoletes = 1 2023-06-23T18:10:24.465 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum clean all 2023-06-23T18:10:24.778 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:24.786 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:24.856 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:25.053 INFO:teuthology.orchestra.run.smithi082.stdout:44 files removed 2023-06-23T18:10:25.081 INFO:teuthology.orchestra.run.smithi002.stdout:44 files removed 2023-06-23T18:10:25.100 DEBUG:teuthology.orchestra.run.smithi082:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-23T18:10:25.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-23T18:10:25.139 INFO:teuthology.orchestra.run.smithi017.stdout:44 files removed 2023-06-23T18:10:25.188 DEBUG:teuthology.orchestra.run.smithi017:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-06-23T18:10:25.428 INFO:teuthology.orchestra.run.smithi082.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:25.479 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:25.540 INFO:teuthology.orchestra.run.smithi017.stdout:Updating Subscription Management repositories. 2023-06-23T18:10:26.180 INFO:teuthology.orchestra.run.smithi082.stdout:ceph packages for x86_64 234 kB/s | 81 kB 00:00 2023-06-23T18:10:26.261 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 234 kB/s | 81 kB 00:00 2023-06-23T18:10:26.419 INFO:teuthology.orchestra.run.smithi017.stdout:ceph packages for x86_64 187 kB/s | 81 kB 00:00 2023-06-23T18:10:26.539 INFO:teuthology.orchestra.run.smithi082.stdout:ceph noarch packages 29 kB/s | 9.4 kB 00:00 2023-06-23T18:10:26.662 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 26 kB/s | 9.4 kB 00:00 2023-06-23T18:10:26.787 INFO:teuthology.orchestra.run.smithi017.stdout:ceph noarch packages 28 kB/s | 9.4 kB 00:00 2023-06-23T18:10:26.889 INFO:teuthology.orchestra.run.smithi082.stdout:ceph source packages 4.5 kB/s | 1.5 kB 00:00 2023-06-23T18:10:26.921 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 6.4 kB/s | 1.5 kB 00:00 2023-06-23T18:10:27.129 INFO:teuthology.orchestra.run.smithi017.stdout:ceph source packages 4.7 kB/s | 1.5 kB 00:00 2023-06-23T18:10:29.322 INFO:teuthology.orchestra.run.smithi002.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-23T18:10:29.471 INFO:teuthology.orchestra.run.smithi017.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2023-06-23T18:10:29.540 INFO:teuthology.orchestra.run.smithi082.stdout:Copr repo for python3-asyncssh owned by ceph 1.3 kB/s | 3.5 kB 00:02 2023-06-23T18:10:30.955 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 16 MB 00:01 2023-06-23T18:10:33.157 INFO:teuthology.orchestra.run.smithi082.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 16 MB 00:03 2023-06-23T18:10:34.630 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 441 kB/s | 24 kB 00:00 2023-06-23T18:10:35.557 INFO:teuthology.orchestra.run.smithi002.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-06-23T18:10:36.615 INFO:teuthology.orchestra.run.smithi082.stdout:lab-extras 368 kB/s | 24 kB 00:00 2023-06-23T18:10:37.525 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 63 MB/s | 53 MB 00:00 2023-06-23T18:10:43.349 INFO:teuthology.orchestra.run.smithi017.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 16 MB 00:13 2023-06-23T18:10:44.008 INFO:teuthology.orchestra.run.smithi002.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 59 MB/s | 53 MB 00:00 2023-06-23T18:10:45.666 INFO:teuthology.orchestra.run.smithi082.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-06-23T18:10:47.096 INFO:teuthology.orchestra.run.smithi017.stdout:lab-extras 591 kB/s | 24 kB 00:00 2023-06-23T18:10:47.989 INFO:teuthology.orchestra.run.smithi017.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 58 MB/s | 47 MB 00:00 2023-06-23T18:10:55.940 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:10:55.941 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:10:55.956 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-23T18:10:55.957 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-23T18:10:55.959 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-23T18:10:55.960 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T18:10:55.962 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T18:10:55.965 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-23T18:10:55.966 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-23T18:10:56.108 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-06-23T18:10:56.122 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-06-23T18:10:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2023-06-23T18:10:56.122 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-06-23T18:10:56.122 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2023-06-23T18:10:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-23T18:10:56.122 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T18:10:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.4 k 2023-06-23T18:10:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 5.4 M 2023-06-23T18:10:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 848 k 2023-06-23T18:10:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 165 k 2023-06-23T18:10:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 1.5 M 2023-06-23T18:10:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 128 k 2023-06-23T18:10:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 1.7 M 2023-06-23T18:10:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 7.4 M 2023-06-23T18:10:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 47 k 2023-06-23T18:10:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 13 M 2023-06-23T18:10:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 48 M 2023-06-23T18:10:56.124 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 258 k 2023-06-23T18:10:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 161 k 2023-06-23T18:10:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 258 k 2023-06-23T18:10:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 18 k 2023-06-23T18:10:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-23T18:10:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T18:10:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T18:10:56.125 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 31 k 2023-06-23T18:10:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 771 k 2023-06-23T18:10:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 139 k 2023-06-23T18:10:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T18:10:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-23T18:10:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T18:10:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-23T18:10:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-23T18:10:56.126 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-23T18:10:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 197 k 2023-06-23T18:10:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 370 k 2023-06-23T18:10:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 365 k 2023-06-23T18:10:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 107 k 2023-06-23T18:10:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 94 k 2023-06-23T18:10:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.4 M 2023-06-23T18:10:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 179 k 2023-06-23T18:10:56.127 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-23T18:10:56.128 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2023-06-23T18:10:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-23T18:10:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.7 M 2023-06-23T18:10:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.6 M 2023-06-23T18:10:56.128 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2023-06-23T18:10:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T18:10:56.128 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 22 M 2023-06-23T18:10:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 23 k 2023-06-23T18:10:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 2.3 M 2023-06-23T18:10:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 240 k 2023-06-23T18:10:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 4.6 M 2023-06-23T18:10:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 18 M 2023-06-23T18:10:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 15 k 2023-06-23T18:10:56.129 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 24 k 2023-06-23T18:10:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T18:10:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T18:10:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-23T18:10:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T18:10:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-23T18:10:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 137 k 2023-06-23T18:10:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-23T18:10:56.130 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-23T18:10:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T18:10:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-23T18:10:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-23T18:10:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T18:10:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 474 k 2023-06-23T18:10:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-23T18:10:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.7 M 2023-06-23T18:10:56.131 INFO:teuthology.orchestra.run.smithi002.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T18:10:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-23T18:10:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T18:10:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T18:10:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T18:10:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T18:10:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T18:10:56.132 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-23T18:10:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T18:10:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T18:10:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T18:10:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T18:10:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T18:10:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T18:10:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T18:10:56.133 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T18:10:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 45 k 2023-06-23T18:10:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 119 k 2023-06-23T18:10:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T18:10:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-23T18:10:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T18:10:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T18:10:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T18:10:56.134 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T18:10:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T18:10:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T18:10:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T18:10:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T18:10:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T18:10:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:10:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T18:10:56.135 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T18:10:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T18:10:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T18:10:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T18:10:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T18:10:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T18:10:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T18:10:56.136 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T18:10:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T18:10:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T18:10:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T18:10:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-23T18:10:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T18:10:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T18:10:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-23T18:10:56.137 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T18:10:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T18:10:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:10:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T18:10:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T18:10:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-23T18:10:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T18:10:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T18:10:56.138 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T18:10:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T18:10:56.139 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T18:10:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T18:10:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T18:10:56.144 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T18:10:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T18:10:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T18:10:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-23T18:10:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T18:10:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-23T18:10:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T18:10:56.145 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T18:10:56.146 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling module streams: 2023-06-23T18:10:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql 10 2023-06-23T18:10:56.146 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:56.146 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2023-06-23T18:10:56.146 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-06-23T18:10:56.146 INFO:teuthology.orchestra.run.smithi002.stdout:Install 128 Packages 2023-06-23T18:10:56.146 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 3 Packages 2023-06-23T18:10:56.147 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:10:56.147 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 205 M 2023-06-23T18:10:56.147 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2023-06-23T18:10:56.374 INFO:teuthology.orchestra.run.smithi017.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-06-23T18:10:56.633 INFO:teuthology.orchestra.run.smithi082.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:10:56.634 INFO:teuthology.orchestra.run.smithi082.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:10:56.648 INFO:teuthology.orchestra.run.smithi082.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-23T18:10:56.649 INFO:teuthology.orchestra.run.smithi082.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-23T18:10:56.650 INFO:teuthology.orchestra.run.smithi082.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-23T18:10:56.651 INFO:teuthology.orchestra.run.smithi082.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T18:10:56.653 INFO:teuthology.orchestra.run.smithi082.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T18:10:56.657 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-23T18:10:56.658 INFO:teuthology.orchestra.run.smithi082.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-23T18:10:56.658 INFO:teuthology.orchestra.run.smithi002.stdout:(1/131): ceph-18.0.0-4479.g8627af3e.el8.x86_64. 35 kB/s | 6.4 kB 00:00 2023-06-23T18:10:56.790 INFO:teuthology.orchestra.run.smithi082.stdout:Dependencies resolved. 2023-06-23T18:10:56.802 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-06-23T18:10:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: Package Arch Version Repository Size 2023-06-23T18:10:56.803 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-06-23T18:10:56.803 INFO:teuthology.orchestra.run.smithi082.stdout:Installing: 2023-06-23T18:10:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-23T18:10:56.803 INFO:teuthology.orchestra.run.smithi082.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T18:10:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: ceph x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.4 k 2023-06-23T18:10:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 5.4 M 2023-06-23T18:10:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 848 k 2023-06-23T18:10:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 165 k 2023-06-23T18:10:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 1.5 M 2023-06-23T18:10:56.804 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 128 k 2023-06-23T18:10:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 1.7 M 2023-06-23T18:10:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 7.4 M 2023-06-23T18:10:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 47 k 2023-06-23T18:10:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 13 M 2023-06-23T18:10:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 48 M 2023-06-23T18:10:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 258 k 2023-06-23T18:10:56.805 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 161 k 2023-06-23T18:10:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 258 k 2023-06-23T18:10:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 18 k 2023-06-23T18:10:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-23T18:10:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T18:10:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T18:10:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 31 k 2023-06-23T18:10:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 771 k 2023-06-23T18:10:56.806 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 139 k 2023-06-23T18:10:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T18:10:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-23T18:10:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T18:10:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-23T18:10:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-23T18:10:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-23T18:10:56.807 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 197 k 2023-06-23T18:10:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 370 k 2023-06-23T18:10:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 365 k 2023-06-23T18:10:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 107 k 2023-06-23T18:10:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 94 k 2023-06-23T18:10:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.4 M 2023-06-23T18:10:56.808 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 179 k 2023-06-23T18:10:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-23T18:10:56.809 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrading: 2023-06-23T18:10:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-23T18:10:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: librados2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.7 M 2023-06-23T18:10:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.6 M 2023-06-23T18:10:56.809 INFO:teuthology.orchestra.run.smithi082.stdout:Installing dependencies: 2023-06-23T18:10:56.809 INFO:teuthology.orchestra.run.smithi082.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T18:10:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 22 M 2023-06-23T18:10:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 23 k 2023-06-23T18:10:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 2.3 M 2023-06-23T18:10:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 240 k 2023-06-23T18:10:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 4.6 M 2023-06-23T18:10:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 18 M 2023-06-23T18:10:56.810 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 15 k 2023-06-23T18:10:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 24 k 2023-06-23T18:10:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T18:10:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T18:10:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-23T18:10:56.811 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T18:10:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-23T18:10:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 137 k 2023-06-23T18:10:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-23T18:10:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-23T18:10:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T18:10:56.812 INFO:teuthology.orchestra.run.smithi082.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-23T18:10:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-23T18:10:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T18:10:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 474 k 2023-06-23T18:10:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-23T18:10:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.7 M 2023-06-23T18:10:56.813 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T18:10:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-23T18:10:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T18:10:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T18:10:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T18:10:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T18:10:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T18:10:56.814 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-23T18:10:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T18:10:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T18:10:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T18:10:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T18:10:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T18:10:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T18:10:56.815 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T18:10:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T18:10:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 45 k 2023-06-23T18:10:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 119 k 2023-06-23T18:10:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T18:10:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-23T18:10:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T18:10:56.816 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T18:10:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T18:10:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T18:10:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T18:10:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T18:10:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T18:10:56.817 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T18:10:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T18:10:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:10:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T18:10:56.818 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T18:10:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T18:10:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T18:10:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T18:10:56.819 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T18:10:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T18:10:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T18:10:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T18:10:56.820 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T18:10:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T18:10:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T18:10:56.821 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-23T18:10:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T18:10:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T18:10:56.822 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-23T18:10:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T18:10:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T18:10:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:10:56.823 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T18:10:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T18:10:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-23T18:10:56.824 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T18:10:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T18:10:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T18:10:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T18:10:56.825 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T18:10:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T18:10:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T18:10:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T18:10:56.826 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T18:10:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T18:10:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-23T18:10:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T18:10:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-23T18:10:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T18:10:56.827 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T18:10:56.828 INFO:teuthology.orchestra.run.smithi082.stdout:Enabling module streams: 2023-06-23T18:10:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql 10 2023-06-23T18:10:56.828 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:56.828 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction Summary 2023-06-23T18:10:56.828 INFO:teuthology.orchestra.run.smithi082.stdout:=========================================================================================================================== 2023-06-23T18:10:56.828 INFO:teuthology.orchestra.run.smithi082.stdout:Install 128 Packages 2023-06-23T18:10:56.828 INFO:teuthology.orchestra.run.smithi082.stdout:Upgrade 3 Packages 2023-06-23T18:10:56.829 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:10:56.829 INFO:teuthology.orchestra.run.smithi082.stdout:Total download size: 205 M 2023-06-23T18:10:56.830 INFO:teuthology.orchestra.run.smithi082.stdout:Downloading Packages: 2023-06-23T18:10:57.066 INFO:teuthology.orchestra.run.smithi082.stdout:(1/131): ceph-18.0.0-4479.g8627af3e.el8.x86_64. 29 kB/s | 6.4 kB 00:00 2023-06-23T18:10:57.117 INFO:teuthology.orchestra.run.smithi002.stdout:(2/131): ceph-base-18.0.0-4479.g8627af3e.el8.x8 8.4 MB/s | 5.4 MB 00:00 2023-06-23T18:10:57.142 INFO:teuthology.orchestra.run.smithi002.stdout:(3/131): ceph-fuse-18.0.0-4479.g8627af3e.el8.x8 1.7 MB/s | 848 kB 00:00 2023-06-23T18:10:57.176 INFO:teuthology.orchestra.run.smithi002.stdout:(4/131): ceph-immutable-object-cache-18.0.0-447 2.8 MB/s | 165 kB 00:00 2023-06-23T18:10:57.443 INFO:teuthology.orchestra.run.smithi002.stdout:(5/131): ceph-mgr-18.0.0-4479.g8627af3e.el8.x86 5.8 MB/s | 1.5 MB 00:00 2023-06-23T18:10:57.767 INFO:teuthology.orchestra.run.smithi082.stdout:(2/131): ceph-fuse-18.0.0-4479.g8627af3e.el8.x8 1.2 MB/s | 848 kB 00:00 2023-06-23T18:10:57.984 INFO:teuthology.orchestra.run.smithi082.stdout:(3/131): ceph-immutable-object-cache-18.0.0-447 764 kB/s | 165 kB 00:00 2023-06-23T18:10:58.151 INFO:teuthology.orchestra.run.smithi002.stdout:(6/131): ceph-mon-18.0.0-4479.g8627af3e.el8.x86 6.5 MB/s | 4.6 MB 00:00 2023-06-23T18:10:58.343 INFO:teuthology.orchestra.run.smithi002.stdout:(7/131): ceph-mds-18.0.0-4479.g8627af3e.el8.x86 1.9 MB/s | 2.3 MB 00:01 2023-06-23T18:10:58.744 INFO:teuthology.orchestra.run.smithi002.stdout:(8/131): ceph-common-18.0.0-4479.g8627af3e.el8. 9.7 MB/s | 22 MB 00:02 2023-06-23T18:10:58.776 INFO:teuthology.orchestra.run.smithi082.stdout:(4/131): ceph-base-18.0.0-4479.g8627af3e.el8.x8 2.8 MB/s | 5.4 MB 00:01 2023-06-23T18:10:58.794 INFO:teuthology.orchestra.run.smithi002.stdout:(9/131): ceph-selinux-18.0.0-4479.g8627af3e.el8 488 kB/s | 24 kB 00:00 2023-06-23T18:10:58.942 INFO:teuthology.orchestra.run.smithi082.stdout:(5/131): ceph-mds-18.0.0-4479.g8627af3e.el8.x86 2.4 MB/s | 2.3 MB 00:00 2023-06-23T18:10:59.309 INFO:teuthology.orchestra.run.smithi082.stdout:(6/131): ceph-mgr-18.0.0-4479.g8627af3e.el8.x86 2.9 MB/s | 1.5 MB 00:00 2023-06-23T18:11:00.259 INFO:teuthology.orchestra.run.smithi082.stdout:(7/131): ceph-mon-18.0.0-4479.g8627af3e.el8.x86 3.5 MB/s | 4.6 MB 00:01 2023-06-23T18:11:00.985 INFO:teuthology.orchestra.run.smithi002.stdout:(10/131): ceph-osd-18.0.0-4479.g8627af3e.el8.x8 6.3 MB/s | 18 MB 00:02 2023-06-23T18:11:01.086 INFO:teuthology.orchestra.run.smithi002.stdout:(11/131): cephfs-mirror-18.0.0-4479.g8627af3e.e 2.5 MB/s | 258 kB 00:00 2023-06-23T18:11:01.178 INFO:teuthology.orchestra.run.smithi002.stdout:(12/131): libcephfs-devel-18.0.0-4479.g8627af3e 340 kB/s | 31 kB 00:00 2023-06-23T18:11:01.449 INFO:teuthology.orchestra.run.smithi002.stdout:(13/131): libcephfs2-18.0.0-4479.g8627af3e.el8. 2.8 MB/s | 771 kB 00:00 2023-06-23T18:11:01.536 INFO:teuthology.orchestra.run.smithi002.stdout:(14/131): libcephsqlite-18.0.0-4479.g8627af3e.e 1.5 MB/s | 137 kB 00:00 2023-06-23T18:11:01.620 INFO:teuthology.orchestra.run.smithi002.stdout:(15/131): librados-devel-18.0.0-4479.g8627af3e. 1.6 MB/s | 139 kB 00:00 2023-06-23T18:11:01.779 INFO:teuthology.orchestra.run.smithi002.stdout:(16/131): libradosstriper1-18.0.0-4479.g8627af3 2.9 MB/s | 474 kB 00:00 2023-06-23T18:11:02.151 INFO:teuthology.orchestra.run.smithi082.stdout:(8/131): ceph-common-18.0.0-4479.g8627af3e.el8. 4.2 MB/s | 22 MB 00:05 2023-06-23T18:11:02.285 INFO:teuthology.orchestra.run.smithi082.stdout:(9/131): ceph-radosgw-18.0.0-4479.g8627af3e.el8 6.7 MB/s | 13 MB 00:02 2023-06-23T18:11:02.310 INFO:teuthology.orchestra.run.smithi082.stdout:(10/131): ceph-selinux-18.0.0-4479.g8627af3e.el 153 kB/s | 24 kB 00:00 2023-06-23T18:11:02.452 INFO:teuthology.orchestra.run.smithi082.stdout:(11/131): cephfs-mirror-18.0.0-4479.g8627af3e.e 1.8 MB/s | 258 kB 00:00 2023-06-23T18:11:02.536 INFO:teuthology.orchestra.run.smithi082.stdout:(12/131): libcephfs-devel-18.0.0-4479.g8627af3e 373 kB/s | 31 kB 00:00 2023-06-23T18:11:02.769 INFO:teuthology.orchestra.run.smithi082.stdout:(13/131): libcephfs2-18.0.0-4479.g8627af3e.el8. 3.2 MB/s | 771 kB 00:00 2023-06-23T18:11:02.853 INFO:teuthology.orchestra.run.smithi082.stdout:(14/131): libcephsqlite-18.0.0-4479.g8627af3e.e 1.6 MB/s | 137 kB 00:00 2023-06-23T18:11:02.945 INFO:teuthology.orchestra.run.smithi082.stdout:(15/131): librados-devel-18.0.0-4479.g8627af3e. 1.5 MB/s | 139 kB 00:00 2023-06-23T18:11:03.037 INFO:teuthology.orchestra.run.smithi002.stdout:(17/131): librgw2-18.0.0-4479.g8627af3e.el8.x86 5.4 MB/s | 6.7 MB 00:01 2023-06-23T18:11:03.070 INFO:teuthology.orchestra.run.smithi082.stdout:(16/131): libradosstriper1-18.0.0-4479.g8627af3 3.7 MB/s | 474 kB 00:00 2023-06-23T18:11:03.113 INFO:teuthology.orchestra.run.smithi002.stdout:(18/131): python3-ceph-argparse-18.0.0-4479.g86 597 kB/s | 45 kB 00:00 2023-06-23T18:11:03.188 INFO:teuthology.orchestra.run.smithi002.stdout:(19/131): python3-ceph-common-18.0.0-4479.g8627 1.5 MB/s | 119 kB 00:00 2023-06-23T18:11:03.272 INFO:teuthology.orchestra.run.smithi002.stdout:(20/131): python3-cephfs-18.0.0-4479.g8627af3e. 2.3 MB/s | 197 kB 00:00 2023-06-23T18:11:03.405 INFO:teuthology.orchestra.run.smithi002.stdout:(21/131): python3-rados-18.0.0-4479.g8627af3e.e 2.7 MB/s | 370 kB 00:00 2023-06-23T18:11:03.530 INFO:teuthology.orchestra.run.smithi002.stdout:(22/131): python3-rbd-18.0.0-4479.g8627af3e.el8 2.9 MB/s | 365 kB 00:00 2023-06-23T18:11:03.606 INFO:teuthology.orchestra.run.smithi002.stdout:(23/131): python3-rgw-18.0.0-4479.g8627af3e.el8 1.4 MB/s | 107 kB 00:00 2023-06-23T18:11:03.673 INFO:teuthology.orchestra.run.smithi002.stdout:(24/131): rbd-fuse-18.0.0-4479.g8627af3e.el8.x8 1.4 MB/s | 94 kB 00:00 2023-06-23T18:11:04.106 INFO:teuthology.orchestra.run.smithi002.stdout:(25/131): ceph-test-18.0.0-4479.g8627af3e.el8.x 9.0 MB/s | 48 MB 00:05 2023-06-23T18:11:04.162 INFO:teuthology.orchestra.run.smithi082.stdout:(17/131): librgw2-18.0.0-4479.g8627af3e.el8.x86 6.2 MB/s | 6.7 MB 00:01 2023-06-23T18:11:04.165 INFO:teuthology.orchestra.run.smithi002.stdout:(26/131): rbd-nbd-18.0.0-4479.g8627af3e.el8.x86 3.0 MB/s | 179 kB 00:00 2023-06-23T18:11:04.212 INFO:teuthology.orchestra.run.smithi082.stdout:(18/131): python3-ceph-argparse-18.0.0-4479.g86 900 kB/s | 45 kB 00:00 2023-06-23T18:11:04.223 INFO:teuthology.orchestra.run.smithi002.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4479.g 396 kB/s | 23 kB 00:00 2023-06-23T18:11:04.262 INFO:teuthology.orchestra.run.smithi082.stdout:(19/131): python3-ceph-common-18.0.0-4479.g8627 2.3 MB/s | 119 kB 00:00 2023-06-23T18:11:04.282 INFO:teuthology.orchestra.run.smithi002.stdout:(28/131): rbd-mirror-18.0.0-4479.g8627af3e.el8. 5.7 MB/s | 3.4 MB 00:00 2023-06-23T18:11:04.307 INFO:teuthology.orchestra.run.smithi002.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4479.g8627af3 1.5 MB/s | 128 kB 00:00 2023-06-23T18:11:04.321 INFO:teuthology.orchestra.run.smithi082.stdout:(20/131): python3-cephfs-18.0.0-4479.g8627af3e. 3.3 MB/s | 197 kB 00:00 2023-06-23T18:11:04.446 INFO:teuthology.orchestra.run.smithi082.stdout:(21/131): python3-rados-18.0.0-4479.g8627af3e.e 2.9 MB/s | 370 kB 00:00 2023-06-23T18:11:04.613 INFO:teuthology.orchestra.run.smithi082.stdout:(22/131): ceph-osd-18.0.0-4479.g8627af3e.el8.x8 3.3 MB/s | 18 MB 00:05 2023-06-23T18:11:04.633 INFO:teuthology.orchestra.run.smithi002.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4479.g8627a 4.9 MB/s | 1.7 MB 00:00 2023-06-23T18:11:04.638 INFO:teuthology.orchestra.run.smithi082.stdout:(23/131): python3-rbd-18.0.0-4479.g8627af3e.el8 1.9 MB/s | 365 kB 00:00 2023-06-23T18:11:04.689 INFO:teuthology.orchestra.run.smithi082.stdout:(24/131): python3-rgw-18.0.0-4479.g8627af3e.el8 1.4 MB/s | 107 kB 00:00 2023-06-23T18:11:04.706 INFO:teuthology.orchestra.run.smithi082.stdout:(25/131): rbd-fuse-18.0.0-4479.g8627af3e.el8.x8 1.4 MB/s | 94 kB 00:00 2023-06-23T18:11:04.772 INFO:teuthology.orchestra.run.smithi082.stdout:(26/131): rbd-nbd-18.0.0-4479.g8627af3e.el8.x86 2.6 MB/s | 179 kB 00:00 2023-06-23T18:11:04.775 INFO:teuthology.orchestra.run.smithi002.stdout:(31/131): ceph-radosgw-18.0.0-4479.g8627af3e.el 2.1 MB/s | 13 MB 00:06 2023-06-23T18:11:04.800 INFO:teuthology.orchestra.run.smithi002.stdout:(32/131): ceph-mgr-modules-core-18.0.0-4479.g86 1.4 MB/s | 240 kB 00:00 2023-06-23T18:11:04.831 INFO:teuthology.orchestra.run.smithi082.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4479.g 395 kB/s | 23 kB 00:00 2023-06-23T18:11:04.850 INFO:teuthology.orchestra.run.smithi002.stdout:(33/131): ceph-mgr-rook-18.0.0-4479.g8627af3e.e 634 kB/s | 47 kB 00:00 2023-06-23T18:11:04.875 INFO:teuthology.orchestra.run.smithi002.stdout:(34/131): ceph-prometheus-alerts-18.0.0-4479.g8 193 kB/s | 15 kB 00:00 2023-06-23T18:11:04.890 INFO:teuthology.orchestra.run.smithi082.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4479.g8627af3 2.1 MB/s | 128 kB 00:00 2023-06-23T18:11:04.934 INFO:teuthology.orchestra.run.smithi002.stdout:(35/131): cephadm-18.0.0-4479.g8627af3e.el8.noa 2.7 MB/s | 161 kB 00:00 2023-06-23T18:11:05.051 INFO:teuthology.orchestra.run.smithi002.stdout:(36/131): ceph-mgr-diskprediction-local-18.0.0- 9.9 MB/s | 7.4 MB 00:00 2023-06-23T18:11:05.076 INFO:teuthology.orchestra.run.smithi002.stdout:(37/131): cephfs-top-18.0.0-4479.g8627af3e.el8. 127 kB/s | 18 kB 00:00 2023-06-23T18:11:05.102 INFO:teuthology.orchestra.run.smithi002.stdout:(38/131): ceph-volume-18.0.0-4479.g8627af3e.el8 1.0 MB/s | 258 kB 00:00 2023-06-23T18:11:05.132 INFO:teuthology.orchestra.run.smithi082.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4479.g8627a 7.1 MB/s | 1.7 MB 00:00 2023-06-23T18:11:05.390 INFO:teuthology.orchestra.run.smithi082.stdout:(30/131): rbd-mirror-18.0.0-4479.g8627af3e.el8. 4.9 MB/s | 3.4 MB 00:00 2023-06-23T18:11:05.402 INFO:teuthology.orchestra.run.smithi002.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 181 kB/s | 59 kB 00:00 2023-06-23T18:11:05.427 INFO:teuthology.orchestra.run.smithi002.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 231 kB/s | 75 kB 00:00 2023-06-23T18:11:05.466 INFO:teuthology.orchestra.run.smithi082.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4479.g86 3.1 MB/s | 240 kB 00:00 2023-06-23T18:11:05.511 INFO:teuthology.orchestra.run.smithi002.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 523 kB/s | 44 kB 00:00 2023-06-23T18:11:05.541 INFO:teuthology.orchestra.run.smithi082.stdout:(32/131): ceph-mgr-rook-18.0.0-4479.g8627af3e.e 634 kB/s | 47 kB 00:00 2023-06-23T18:11:05.561 INFO:teuthology.orchestra.run.smithi002.stdout:(42/131): gperftools-libs-2.7-9.el8.x86_64.rpm 601 kB/s | 306 kB 00:00 2023-06-23T18:11:05.599 INFO:teuthology.orchestra.run.smithi082.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4479.g8 250 kB/s | 15 kB 00:00 2023-06-23T18:11:05.645 INFO:teuthology.orchestra.run.smithi002.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 397 kB/s | 33 kB 00:00 2023-06-23T18:11:05.675 INFO:teuthology.orchestra.run.smithi082.stdout:(34/131): ceph-volume-18.0.0-4479.g8627af3e.el8 3.4 MB/s | 258 kB 00:00 2023-06-23T18:11:05.720 INFO:teuthology.orchestra.run.smithi002.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 887 kB/s | 185 kB 00:00 2023-06-23T18:11:05.742 INFO:teuthology.orchestra.run.smithi082.stdout:(35/131): cephadm-18.0.0-4479.g8627af3e.el8.noa 2.4 MB/s | 161 kB 00:00 2023-06-23T18:11:05.745 INFO:teuthology.orchestra.run.smithi002.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2023-06-23T18:11:05.808 INFO:teuthology.orchestra.run.smithi082.stdout:(36/131): cephfs-top-18.0.0-4479.g8627af3e.el8. 271 kB/s | 18 kB 00:00 2023-06-23T18:11:05.820 INFO:teuthology.orchestra.run.smithi002.stdout:(46/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.1 MB/s | 476 kB 00:00 2023-06-23T18:11:05.896 INFO:teuthology.orchestra.run.smithi002.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 982 kB/s | 173 kB 00:00 2023-06-23T18:11:05.917 INFO:teuthology.orchestra.run.smithi082.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 9.4 MB/s | 7.4 MB 00:00 2023-06-23T18:11:05.963 INFO:teuthology.orchestra.run.smithi002.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 572 kB/s | 81 kB 00:00 2023-06-23T18:11:05.976 INFO:teuthology.orchestra.run.smithi082.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.8 MB/s | 306 kB 00:00 2023-06-23T18:11:05.988 INFO:teuthology.orchestra.run.smithi002.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-06-23T18:11:06.001 INFO:teuthology.orchestra.run.smithi082.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 705 kB/s | 59 kB 00:00 2023-06-23T18:11:06.018 INFO:teuthology.orchestra.run.smithi082.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-06-23T18:11:06.038 INFO:teuthology.orchestra.run.smithi002.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2023-06-23T18:11:06.060 INFO:teuthology.orchestra.run.smithi082.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-06-23T18:11:06.072 INFO:teuthology.orchestra.run.smithi002.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.2 MB/s | 384 kB 00:00 2023-06-23T18:11:06.102 INFO:teuthology.orchestra.run.smithi082.stdout:(42/131): python3-asyncssh-2.7.0-2.el8.noarch.r 4.6 MB/s | 476 kB 00:00 2023-06-23T18:11:06.127 INFO:teuthology.orchestra.run.smithi082.stdout:(43/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.7 MB/s | 185 kB 00:00 2023-06-23T18:11:06.139 INFO:teuthology.orchestra.run.smithi002.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2023-06-23T18:11:06.153 INFO:teuthology.orchestra.run.smithi082.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 656 kB/s | 33 kB 00:00 2023-06-23T18:11:06.164 INFO:teuthology.orchestra.run.smithi002.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 638 kB/s | 59 kB 00:00 2023-06-23T18:11:06.178 INFO:teuthology.orchestra.run.smithi082.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 324 kB/s | 16 kB 00:00 2023-06-23T18:11:06.203 INFO:teuthology.orchestra.run.smithi082.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-06-23T18:11:06.214 INFO:teuthology.orchestra.run.smithi002.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 803 kB/s | 60 kB 00:00 2023-06-23T18:11:06.237 INFO:teuthology.orchestra.run.smithi082.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 6.4 MB/s | 384 kB 00:00 2023-06-23T18:11:06.262 INFO:teuthology.orchestra.run.smithi082.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-06-23T18:11:06.287 INFO:teuthology.orchestra.run.smithi082.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 221 kB/s | 11 kB 00:00 2023-06-23T18:11:06.290 INFO:teuthology.orchestra.run.smithi002.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 208 kB/s | 16 kB 00:00 2023-06-23T18:11:06.313 INFO:teuthology.orchestra.run.smithi082.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-06-23T18:11:06.357 INFO:teuthology.orchestra.run.smithi082.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2023-06-23T18:11:06.365 INFO:teuthology.orchestra.run.smithi002.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 449 kB/s | 34 kB 00:00 2023-06-23T18:11:06.390 INFO:teuthology.orchestra.run.smithi002.stdout:(57/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.2 MB/s | 283 kB 00:00 2023-06-23T18:11:06.391 INFO:teuthology.orchestra.run.smithi082.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 11 MB/s | 1.2 MB 00:00 2023-06-23T18:11:06.416 INFO:teuthology.orchestra.run.smithi082.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.0 MB/s | 59 kB 00:00 2023-06-23T18:11:06.441 INFO:teuthology.orchestra.run.smithi082.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-06-23T18:11:06.466 INFO:teuthology.orchestra.run.smithi002.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 821 kB/s | 61 kB 00:00 2023-06-23T18:11:06.475 INFO:teuthology.orchestra.run.smithi082.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-06-23T18:11:06.500 INFO:teuthology.orchestra.run.smithi082.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-06-23T18:11:06.525 INFO:teuthology.orchestra.run.smithi082.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 673 kB/s | 34 kB 00:00 2023-06-23T18:11:06.541 INFO:teuthology.orchestra.run.smithi002.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 250 kB/s | 19 kB 00:00 2023-06-23T18:11:06.550 INFO:teuthology.orchestra.run.smithi082.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-06-23T18:11:06.566 INFO:teuthology.orchestra.run.smithi002.stdout:(60/131): python3-routes-2.4.1-12.el8.noarch.rp 979 kB/s | 196 kB 00:00 2023-06-23T18:11:06.576 INFO:teuthology.orchestra.run.smithi082.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-06-23T18:11:06.601 INFO:teuthology.orchestra.run.smithi082.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2023-06-23T18:11:06.616 INFO:teuthology.orchestra.run.smithi002.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 318 kB/s | 24 kB 00:00 2023-06-23T18:11:06.626 INFO:teuthology.orchestra.run.smithi082.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 476 kB/s | 24 kB 00:00 2023-06-23T18:11:06.642 INFO:teuthology.orchestra.run.smithi002.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2023-06-23T18:11:06.651 INFO:teuthology.orchestra.run.smithi082.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 583 kB/s | 29 kB 00:00 2023-06-23T18:11:06.677 INFO:teuthology.orchestra.run.smithi082.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 549 kB/s | 27 kB 00:00 2023-06-23T18:11:06.683 INFO:teuthology.orchestra.run.smithi002.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 411 kB/s | 27 kB 00:00 2023-06-23T18:11:06.702 INFO:teuthology.orchestra.run.smithi082.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-06-23T18:11:06.725 INFO:teuthology.orchestra.run.smithi002.stdout:(64/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-06-23T18:11:06.727 INFO:teuthology.orchestra.run.smithi082.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-06-23T18:11:06.753 INFO:teuthology.orchestra.run.smithi082.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-06-23T18:11:06.801 INFO:teuthology.orchestra.run.smithi082.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-06-23T18:11:06.809 INFO:teuthology.orchestra.run.smithi002.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 735 kB/s | 61 kB 00:00 2023-06-23T18:11:06.828 INFO:teuthology.orchestra.run.smithi082.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 306 kB/s | 23 kB 00:00 2023-06-23T18:11:06.843 INFO:teuthology.orchestra.run.smithi002.stdout:(66/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-06-23T18:11:06.853 INFO:teuthology.orchestra.run.smithi082.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-06-23T18:11:06.901 INFO:teuthology.orchestra.run.smithi002.stdout:(67/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.1 MB/s | 251 kB 00:00 2023-06-23T18:11:06.904 INFO:teuthology.orchestra.run.smithi082.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 17 MB/s | 1.7 MB 00:00 2023-06-23T18:11:06.927 INFO:teuthology.orchestra.run.smithi002.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 274 kB/s | 23 kB 00:00 2023-06-23T18:11:06.929 INFO:teuthology.orchestra.run.smithi082.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 622 kB/s | 47 kB 00:00 2023-06-23T18:11:06.952 INFO:teuthology.orchestra.run.smithi002.stdout:(69/131): python3-webtest-2.0.33-1.el8.noarch.r 599 kB/s | 85 kB 00:00 2023-06-23T18:11:06.971 INFO:teuthology.orchestra.run.smithi082.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2023-06-23T18:11:06.977 INFO:teuthology.orchestra.run.smithi002.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-06-23T18:11:06.997 INFO:teuthology.orchestra.run.smithi082.stdout:(73/131): libselinux-devel-2.9-5.el8.x86_64.rpm 2.1 MB/s | 200 kB 00:00 2023-06-23T18:11:07.022 INFO:teuthology.orchestra.run.smithi082.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-06-23T18:11:07.047 INFO:teuthology.orchestra.run.smithi082.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 961 kB/s | 48 kB 00:00 2023-06-23T18:11:07.069 INFO:teuthology.orchestra.run.smithi002.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-06-23T18:11:07.072 INFO:teuthology.orchestra.run.smithi082.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-06-23T18:11:07.089 INFO:teuthology.orchestra.run.smithi082.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-06-23T18:11:07.115 INFO:teuthology.orchestra.run.smithi082.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.5 MB/s | 109 kB 00:00 2023-06-23T18:11:07.128 INFO:teuthology.orchestra.run.smithi002.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 933 kB/s | 54 kB 00:00 2023-06-23T18:11:07.140 INFO:teuthology.orchestra.run.smithi082.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-06-23T18:11:07.174 INFO:teuthology.orchestra.run.smithi082.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-06-23T18:11:07.178 INFO:teuthology.orchestra.run.smithi002.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-06-23T18:11:07.191 INFO:teuthology.orchestra.run.smithi082.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 372 kB/s | 19 kB 00:00 2023-06-23T18:11:07.216 INFO:teuthology.orchestra.run.smithi082.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-06-23T18:11:07.237 INFO:teuthology.orchestra.run.smithi002.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-06-23T18:11:07.249 INFO:teuthology.orchestra.run.smithi082.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.4 MB/s | 559 kB 00:00 2023-06-23T18:11:07.275 INFO:teuthology.orchestra.run.smithi082.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 305 kB/s | 18 kB 00:00 2023-06-23T18:11:07.287 INFO:teuthology.orchestra.run.smithi002.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 779 kB/s | 39 kB 00:00 2023-06-23T18:11:07.309 INFO:teuthology.orchestra.run.smithi082.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-06-23T18:11:07.334 INFO:teuthology.orchestra.run.smithi082.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-06-23T18:11:07.367 INFO:teuthology.orchestra.run.smithi082.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.1 MB/s | 72 kB 00:00 2023-06-23T18:11:07.418 INFO:teuthology.orchestra.run.smithi082.stdout:(88/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 21 MB/s | 2.3 MB 00:00 2023-06-23T18:11:07.438 INFO:teuthology.orchestra.run.smithi002.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 28 MB/s | 14 MB 00:00 2023-06-23T18:11:07.451 INFO:teuthology.orchestra.run.smithi082.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 991 kB/s | 83 kB 00:00 2023-06-23T18:11:07.471 INFO:teuthology.orchestra.run.smithi002.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.7 MB/s | 688 kB 00:00 2023-06-23T18:11:07.477 INFO:teuthology.orchestra.run.smithi082.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.0 MB/s | 60 kB 00:00 2023-06-23T18:11:07.502 INFO:teuthology.orchestra.run.smithi082.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-06-23T18:11:07.504 INFO:teuthology.orchestra.run.smithi002.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-06-23T18:11:07.527 INFO:teuthology.orchestra.run.smithi082.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-06-23T18:11:07.530 INFO:teuthology.orchestra.run.smithi002.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-06-23T18:11:07.547 INFO:teuthology.orchestra.run.smithi002.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-06-23T18:11:07.552 INFO:teuthology.orchestra.run.smithi082.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 769 kB/s | 39 kB 00:00 2023-06-23T18:11:07.586 INFO:teuthology.orchestra.run.smithi082.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-06-23T18:11:07.597 INFO:teuthology.orchestra.run.smithi002.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2023-06-23T18:11:07.611 INFO:teuthology.orchestra.run.smithi082.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-06-23T18:11:07.645 INFO:teuthology.orchestra.run.smithi082.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-06-23T18:11:07.664 INFO:teuthology.orchestra.run.smithi002.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 397 kB/s | 27 kB 00:00 2023-06-23T18:11:07.687 INFO:teuthology.orchestra.run.smithi082.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-06-23T18:11:07.715 INFO:teuthology.orchestra.run.smithi002.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-06-23T18:11:07.720 INFO:teuthology.orchestra.run.smithi082.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-06-23T18:11:07.763 INFO:teuthology.orchestra.run.smithi082.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-06-23T18:11:07.773 INFO:teuthology.orchestra.run.smithi002.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-06-23T18:11:07.798 INFO:teuthology.orchestra.run.smithi002.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 526 kB/s | 44 kB 00:00 2023-06-23T18:11:07.804 INFO:teuthology.orchestra.run.smithi082.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-06-23T18:11:07.846 INFO:teuthology.orchestra.run.smithi082.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 931 kB/s | 39 kB 00:00 2023-06-23T18:11:07.857 INFO:teuthology.orchestra.run.smithi002.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.7 MB/s | 457 kB 00:00 2023-06-23T18:11:07.897 INFO:teuthology.orchestra.run.smithi082.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-06-23T18:11:07.924 INFO:teuthology.orchestra.run.smithi002.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 25 MB/s | 3.7 MB 00:00 2023-06-23T18:11:07.949 INFO:teuthology.orchestra.run.smithi002.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2023-06-23T18:11:07.966 INFO:teuthology.orchestra.run.smithi002.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-06-23T18:11:08.000 INFO:teuthology.orchestra.run.smithi002.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-06-23T18:11:08.013 INFO:teuthology.orchestra.run.smithi082.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 4.5 MB/s | 538 kB 00:00 2023-06-23T18:11:08.017 INFO:teuthology.orchestra.run.smithi002.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 398 kB/s | 20 kB 00:00 2023-06-23T18:11:08.042 INFO:teuthology.orchestra.run.smithi002.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-06-23T18:11:08.075 INFO:teuthology.orchestra.run.smithi002.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-06-23T18:11:08.126 INFO:teuthology.orchestra.run.smithi002.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2023-06-23T18:11:08.156 INFO:teuthology.orchestra.run.smithi082.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2023-06-23T18:11:08.218 INFO:teuthology.orchestra.run.smithi002.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 28 MB/s | 4.9 MB 00:00 2023-06-23T18:11:08.268 INFO:teuthology.orchestra.run.smithi002.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.9 MB/s | 202 kB 00:00 2023-06-23T18:11:08.278 INFO:teuthology.orchestra.run.smithi017.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:11:08.280 INFO:teuthology.orchestra.run.smithi017.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-06-23T18:11:08.295 INFO:teuthology.orchestra.run.smithi017.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-06-23T18:11:08.296 INFO:teuthology.orchestra.run.smithi017.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-06-23T18:11:08.297 INFO:teuthology.orchestra.run.smithi017.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-06-23T18:11:08.298 INFO:teuthology.orchestra.run.smithi082.stdout:(105/131): python3-scipy-1.0.0-21.module+el8.5. 20 MB/s | 14 MB 00:00 2023-06-23T18:11:08.298 INFO:teuthology.orchestra.run.smithi017.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-06-23T18:11:08.300 INFO:teuthology.orchestra.run.smithi017.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-06-23T18:11:08.304 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-06-23T18:11:08.305 INFO:teuthology.orchestra.run.smithi017.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-06-23T18:11:08.318 INFO:teuthology.orchestra.run.smithi002.stdout:(97/131): thrift-0.13.0-2.el8.x86_64.rpm 1.2 MB/s | 1.7 MB 00:01 2023-06-23T18:11:08.323 INFO:teuthology.orchestra.run.smithi082.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.1 MB/s | 187 kB 00:00 2023-06-23T18:11:08.352 INFO:teuthology.orchestra.run.smithi002.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 4.1 MB/s | 354 kB 00:00 2023-06-23T18:11:08.357 INFO:teuthology.orchestra.run.smithi082.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-06-23T18:11:08.391 INFO:teuthology.orchestra.run.smithi082.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 800 kB/s | 27 kB 00:00 2023-06-23T18:11:08.432 INFO:teuthology.orchestra.run.smithi082.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-23T18:11:08.448 INFO:teuthology.orchestra.run.smithi017.stdout:Dependencies resolved. 2023-06-23T18:11:08.452 INFO:teuthology.orchestra.run.smithi002.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 15 MB/s | 4.7 MB 00:00 2023-06-23T18:11:08.457 INFO:teuthology.orchestra.run.smithi082.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.7 MB/s | 44 kB 00:00 2023-06-23T18:11:08.461 INFO:teuthology.orchestra.run.smithi017.stdout:=========================================================================================================================== 2023-06-23T18:11:08.461 INFO:teuthology.orchestra.run.smithi017.stdout: Package Arch Version Repository Size 2023-06-23T18:11:08.462 INFO:teuthology.orchestra.run.smithi017.stdout:=========================================================================================================================== 2023-06-23T18:11:08.462 INFO:teuthology.orchestra.run.smithi017.stdout:Installing: 2023-06-23T18:11:08.462 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-06-23T18:11:08.462 INFO:teuthology.orchestra.run.smithi017.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-06-23T18:11:08.462 INFO:teuthology.orchestra.run.smithi017.stdout: ceph x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.4 k 2023-06-23T18:11:08.462 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 5.4 M 2023-06-23T18:11:08.462 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 848 k 2023-06-23T18:11:08.462 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 165 k 2023-06-23T18:11:08.463 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 1.5 M 2023-06-23T18:11:08.463 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 128 k 2023-06-23T18:11:08.463 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 1.7 M 2023-06-23T18:11:08.463 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 7.4 M 2023-06-23T18:11:08.463 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 47 k 2023-06-23T18:11:08.463 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 13 M 2023-06-23T18:11:08.463 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 48 M 2023-06-23T18:11:08.464 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 258 k 2023-06-23T18:11:08.464 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 161 k 2023-06-23T18:11:08.464 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 258 k 2023-06-23T18:11:08.464 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 18 k 2023-06-23T18:11:08.464 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-06-23T18:11:08.464 INFO:teuthology.orchestra.run.smithi017.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-06-23T18:11:08.464 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-06-23T18:11:08.464 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 31 k 2023-06-23T18:11:08.465 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 771 k 2023-06-23T18:11:08.465 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 139 k 2023-06-23T18:11:08.465 INFO:teuthology.orchestra.run.smithi017.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-06-23T18:11:08.465 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-06-23T18:11:08.465 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-06-23T18:11:08.465 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-06-23T18:11:08.465 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-06-23T18:11:08.466 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-06-23T18:11:08.466 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 197 k 2023-06-23T18:11:08.466 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 370 k 2023-06-23T18:11:08.466 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 365 k 2023-06-23T18:11:08.466 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 107 k 2023-06-23T18:11:08.466 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 94 k 2023-06-23T18:11:08.466 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.4 M 2023-06-23T18:11:08.466 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 179 k 2023-06-23T18:11:08.467 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-06-23T18:11:08.467 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrading: 2023-06-23T18:11:08.467 INFO:teuthology.orchestra.run.smithi017.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-06-23T18:11:08.467 INFO:teuthology.orchestra.run.smithi017.stdout: librados2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.7 M 2023-06-23T18:11:08.467 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 3.6 M 2023-06-23T18:11:08.467 INFO:teuthology.orchestra.run.smithi017.stdout:Installing dependencies: 2023-06-23T18:11:08.467 INFO:teuthology.orchestra.run.smithi017.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-06-23T18:11:08.467 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 22 M 2023-06-23T18:11:08.468 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 23 k 2023-06-23T18:11:08.468 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 2.3 M 2023-06-23T18:11:08.468 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 240 k 2023-06-23T18:11:08.468 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 4.6 M 2023-06-23T18:11:08.468 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 18 M 2023-06-23T18:11:08.468 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4479.g8627af3e.el8 ceph-noarch 15 k 2023-06-23T18:11:08.468 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 24 k 2023-06-23T18:11:08.469 INFO:teuthology.orchestra.run.smithi017.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-06-23T18:11:08.469 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-06-23T18:11:08.469 INFO:teuthology.orchestra.run.smithi017.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-06-23T18:11:08.469 INFO:teuthology.orchestra.run.smithi017.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-06-23T18:11:08.469 INFO:teuthology.orchestra.run.smithi017.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-06-23T18:11:08.469 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 137 k 2023-06-23T18:11:08.469 INFO:teuthology.orchestra.run.smithi017.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-06-23T18:11:08.469 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-06-23T18:11:08.470 INFO:teuthology.orchestra.run.smithi017.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-06-23T18:11:08.470 INFO:teuthology.orchestra.run.smithi017.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-06-23T18:11:08.470 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-06-23T18:11:08.470 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-06-23T18:11:08.470 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 474 k 2023-06-23T18:11:08.470 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-06-23T18:11:08.470 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2 x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 6.7 M 2023-06-23T18:11:08.470 INFO:teuthology.orchestra.run.smithi017.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-06-23T18:11:08.471 INFO:teuthology.orchestra.run.smithi017.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-06-23T18:11:08.471 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-06-23T18:11:08.471 INFO:teuthology.orchestra.run.smithi017.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-06-23T18:11:08.471 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-06-23T18:11:08.471 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-06-23T18:11:08.471 INFO:teuthology.orchestra.run.smithi017.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-06-23T18:11:08.472 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-06-23T18:11:08.472 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-06-23T18:11:08.472 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-06-23T18:11:08.473 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-06-23T18:11:08.473 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-06-23T18:11:08.473 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-06-23T18:11:08.473 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-06-23T18:11:08.473 INFO:teuthology.orchestra.run.smithi017.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-06-23T18:11:08.473 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-06-23T18:11:08.473 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 45 k 2023-06-23T18:11:08.474 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common x86_64 2:18.0.0-4479.g8627af3e.el8 ceph 119 k 2023-06-23T18:11:08.474 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-06-23T18:11:08.474 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-06-23T18:11:08.474 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-06-23T18:11:08.474 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-06-23T18:11:08.474 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-06-23T18:11:08.474 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-06-23T18:11:08.474 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-06-23T18:11:08.475 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-06-23T18:11:08.475 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-06-23T18:11:08.475 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-06-23T18:11:08.475 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-06-23T18:11:08.475 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:11:08.475 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-06-23T18:11:08.475 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-06-23T18:11:08.476 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-06-23T18:11:08.476 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-06-23T18:11:08.476 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-06-23T18:11:08.476 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-06-23T18:11:08.476 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-06-23T18:11:08.476 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-06-23T18:11:08.476 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-06-23T18:11:08.476 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-06-23T18:11:08.477 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-06-23T18:11:08.477 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-06-23T18:11:08.477 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-06-23T18:11:08.477 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-06-23T18:11:08.477 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-06-23T18:11:08.477 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-06-23T18:11:08.477 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-06-23T18:11:08.477 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-06-23T18:11:08.478 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-06-23T18:11:08.478 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-06-23T18:11:08.478 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-06-23T18:11:08.478 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-06-23T18:11:08.478 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-06-23T18:11:08.479 INFO:teuthology.orchestra.run.smithi017.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-06-23T18:11:08.479 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-06-23T18:11:08.479 INFO:teuthology.orchestra.run.smithi017.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-06-23T18:11:08.479 INFO:teuthology.orchestra.run.smithi017.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-06-23T18:11:08.479 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-06-23T18:11:08.479 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-06-23T18:11:08.479 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-06-23T18:11:08.479 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-06-23T18:11:08.480 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-06-23T18:11:08.480 INFO:teuthology.orchestra.run.smithi017.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-06-23T18:11:08.480 INFO:teuthology.orchestra.run.smithi017.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-06-23T18:11:08.480 INFO:teuthology.orchestra.run.smithi017.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-06-23T18:11:08.480 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-06-23T18:11:08.480 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-06-23T18:11:08.480 INFO:teuthology.orchestra.run.smithi017.stdout:Enabling module streams: 2023-06-23T18:11:08.481 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql 10 2023-06-23T18:11:08.481 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:11:08.481 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction Summary 2023-06-23T18:11:08.481 INFO:teuthology.orchestra.run.smithi017.stdout:=========================================================================================================================== 2023-06-23T18:11:08.481 INFO:teuthology.orchestra.run.smithi017.stdout:Install 128 Packages 2023-06-23T18:11:08.481 INFO:teuthology.orchestra.run.smithi017.stdout:Upgrade 3 Packages 2023-06-23T18:11:08.481 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:11:08.482 INFO:teuthology.orchestra.run.smithi017.stdout:Total download size: 205 M 2023-06-23T18:11:08.482 INFO:teuthology.orchestra.run.smithi017.stdout:Downloading Packages: 2023-06-23T18:11:08.494 INFO:teuthology.orchestra.run.smithi002.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.9 MB/s | 808 kB 00:00 2023-06-23T18:11:08.528 INFO:teuthology.orchestra.run.smithi002.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-06-23T18:11:08.541 INFO:teuthology.orchestra.run.smithi082.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 22 MB/s | 4.8 MB 00:00 2023-06-23T18:11:08.570 INFO:teuthology.orchestra.run.smithi002.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-06-23T18:11:08.583 INFO:teuthology.orchestra.run.smithi082.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-06-23T18:11:08.595 INFO:teuthology.orchestra.run.smithi002.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-06-23T18:11:08.620 INFO:teuthology.orchestra.run.smithi002.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 845 kB/s | 43 kB 00:00 2023-06-23T18:11:08.625 INFO:teuthology.orchestra.run.smithi082.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-06-23T18:11:08.646 INFO:teuthology.orchestra.run.smithi002.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-06-23T18:11:08.671 INFO:teuthology.orchestra.run.smithi002.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 953 kB/s | 48 kB 00:00 2023-06-23T18:11:08.684 INFO:teuthology.orchestra.run.smithi082.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-06-23T18:11:08.696 INFO:teuthology.orchestra.run.smithi002.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-06-23T18:11:08.701 INFO:teuthology.orchestra.run.smithi082.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 849 kB/s | 64 kB 00:00 2023-06-23T18:11:08.721 INFO:teuthology.orchestra.run.smithi002.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 850 kB/s | 43 kB 00:00 2023-06-23T18:11:08.726 INFO:teuthology.orchestra.run.smithi082.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.9 MB/s | 126 kB 00:00 2023-06-23T18:11:08.743 INFO:teuthology.orchestra.run.smithi082.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 477 kB/s | 20 kB 00:00 2023-06-23T18:11:08.747 INFO:teuthology.orchestra.run.smithi002.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-06-23T18:11:08.768 INFO:teuthology.orchestra.run.smithi082.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-06-23T18:11:08.772 INFO:teuthology.orchestra.run.smithi002.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-06-23T18:11:08.802 INFO:teuthology.orchestra.run.smithi082.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-06-23T18:11:08.805 INFO:teuthology.orchestra.run.smithi002.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-06-23T18:11:08.831 INFO:teuthology.orchestra.run.smithi002.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 319 kB/s | 19 kB 00:00 2023-06-23T18:11:08.843 INFO:teuthology.orchestra.run.smithi082.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-06-23T18:11:08.856 INFO:teuthology.orchestra.run.smithi002.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-06-23T18:11:08.898 INFO:teuthology.orchestra.run.smithi002.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.2 MB/s | 559 kB 00:00 2023-06-23T18:11:08.923 INFO:teuthology.orchestra.run.smithi002.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 267 kB/s | 18 kB 00:00 2023-06-23T18:11:08.924 INFO:teuthology.orchestra.run.smithi017.stdout:(1/131): ceph-18.0.0-4479.g8627af3e.el8.x86_64. 31 kB/s | 6.4 kB 00:00 2023-06-23T18:11:08.936 INFO:teuthology.orchestra.run.smithi082.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 29 MB/s | 4.9 MB 00:00 2023-06-23T18:11:08.978 INFO:teuthology.orchestra.run.smithi082.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-06-23T18:11:08.999 INFO:teuthology.orchestra.run.smithi002.stdout:(116/131): postgresql-server-10.21-2.module+el8 7.4 MB/s | 5.1 MB 00:00 2023-06-23T18:11:09.024 INFO:teuthology.orchestra.run.smithi002.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-06-23T18:11:09.045 INFO:teuthology.orchestra.run.smithi082.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2023-06-23T18:11:09.049 INFO:teuthology.orchestra.run.smithi002.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-06-23T18:11:09.070 INFO:teuthology.orchestra.run.smithi082.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2023-06-23T18:11:09.075 INFO:teuthology.orchestra.run.smithi002.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-06-23T18:11:09.100 INFO:teuthology.orchestra.run.smithi002.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-06-23T18:11:09.120 INFO:teuthology.orchestra.run.smithi082.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-06-23T18:11:09.125 INFO:teuthology.orchestra.run.smithi002.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-06-23T18:11:09.150 INFO:teuthology.orchestra.run.smithi002.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-06-23T18:11:09.176 INFO:teuthology.orchestra.run.smithi002.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-06-23T18:11:09.201 INFO:teuthology.orchestra.run.smithi002.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 767 kB/s | 39 kB 00:00 2023-06-23T18:11:09.204 INFO:teuthology.orchestra.run.smithi082.stdout:(126/131): postgresql-server-10.21-2.module+el8 32 MB/s | 5.1 MB 00:00 2023-06-23T18:11:09.268 INFO:teuthology.orchestra.run.smithi002.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.8 MB/s | 644 kB 00:00 2023-06-23T18:11:09.302 INFO:teuthology.orchestra.run.smithi002.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.7 MB/s | 171 kB 00:00 2023-06-23T18:11:09.327 INFO:teuthology.orchestra.run.smithi002.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.3 MB/s | 138 kB 00:00 2023-06-23T18:11:09.377 INFO:teuthology.orchestra.run.smithi002.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.1 MB/s | 2.3 MB 00:00 2023-06-23T18:11:09.383 INFO:teuthology.orchestra.run.smithi017.stdout:(2/131): ceph-fuse-18.0.0-4479.g8627af3e.el8.x8 1.8 MB/s | 848 kB 00:00 2023-06-23T18:11:09.427 INFO:teuthology.orchestra.run.smithi002.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-06-23T18:11:09.525 INFO:teuthology.orchestra.run.smithi017.stdout:(3/131): ceph-immutable-object-cache-18.0.0-447 1.1 MB/s | 165 kB 00:00 2023-06-23T18:11:09.654 INFO:teuthology.orchestra.run.smithi082.stdout:(127/131): ceph-test-18.0.0-4479.g8627af3e.el8. 6.5 MB/s | 48 MB 00:07 2023-06-23T18:11:09.688 INFO:teuthology.orchestra.run.smithi082.stdout:(128/131): postgresql-10.21-2.module+el8.6.0+15 2.6 MB/s | 1.5 MB 00:00 2023-06-23T18:11:09.738 INFO:teuthology.orchestra.run.smithi082.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-06-23T18:11:10.242 INFO:teuthology.orchestra.run.smithi017.stdout:(4/131): ceph-base-18.0.0-4479.g8627af3e.el8.x8 3.5 MB/s | 5.4 MB 00:01 2023-06-23T18:11:10.318 INFO:teuthology.orchestra.run.smithi002.stdout:(130/131): librbd1-18.0.0-4479.g8627af3e.el8.x8 3.6 MB/s | 3.6 MB 00:00 2023-06-23T18:11:10.626 INFO:teuthology.orchestra.run.smithi017.stdout:(5/131): ceph-common-18.0.0-4479.g8627af3e.el8. 12 MB/s | 22 MB 00:01 2023-06-23T18:11:10.676 INFO:teuthology.orchestra.run.smithi017.stdout:(6/131): ceph-mds-18.0.0-4479.g8627af3e.el8.x86 2.0 MB/s | 2.3 MB 00:01 2023-06-23T18:11:10.763 INFO:teuthology.orchestra.run.smithi082.stdout:(130/131): librbd1-18.0.0-4479.g8627af3e.el8.x8 3.2 MB/s | 3.6 MB 00:01 2023-06-23T18:11:10.888 INFO:teuthology.orchestra.run.smithi082.stdout:(131/131): librados2-18.0.0-4479.g8627af3e.el8. 2.2 MB/s | 3.7 MB 00:01 2023-06-23T18:11:10.891 INFO:teuthology.orchestra.run.smithi082.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:11:10.891 INFO:teuthology.orchestra.run.smithi082.stdout:Total 15 MB/s | 205 MB 00:14 2023-06-23T18:11:10.943 INFO:teuthology.orchestra.run.smithi017.stdout:(7/131): ceph-mon-18.0.0-4479.g8627af3e.el8.x86 15 MB/s | 4.6 MB 00:00 2023-06-23T18:11:11.093 INFO:teuthology.orchestra.run.smithi017.stdout:(8/131): ceph-mgr-18.0.0-4479.g8627af3e.el8.x86 1.8 MB/s | 1.5 MB 00:00 2023-06-23T18:11:11.168 INFO:teuthology.orchestra.run.smithi017.stdout:(9/131): ceph-selinux-18.0.0-4479.g8627af3e.el8 326 kB/s | 24 kB 00:00 2023-06-23T18:11:11.270 INFO:teuthology.orchestra.run.smithi002.stdout:(131/131): librados2-18.0.0-4479.g8627af3e.el8. 1.9 MB/s | 3.7 MB 00:01 2023-06-23T18:11:11.274 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:11:11.274 INFO:teuthology.orchestra.run.smithi002.stdout:Total 14 MB/s | 205 MB 00:15 2023-06-23T18:11:11.735 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction check 2023-06-23T18:11:11.855 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction check succeeded. 2023-06-23T18:11:11.855 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction test 2023-06-23T18:11:11.877 INFO:teuthology.orchestra.run.smithi017.stdout:(10/131): ceph-radosgw-18.0.0-4479.g8627af3e.el 14 MB/s | 13 MB 00:00 2023-06-23T18:11:11.952 INFO:teuthology.orchestra.run.smithi017.stdout:(11/131): cephfs-mirror-18.0.0-4479.g8627af3e.e 3.4 MB/s | 258 kB 00:00 2023-06-23T18:11:11.994 INFO:teuthology.orchestra.run.smithi017.stdout:(12/131): libcephfs-devel-18.0.0-4479.g8627af3e 747 kB/s | 31 kB 00:00 2023-06-23T18:11:12.086 INFO:teuthology.orchestra.run.smithi017.stdout:(13/131): libcephfs2-18.0.0-4479.g8627af3e.el8. 8.2 MB/s | 771 kB 00:00 2023-06-23T18:11:12.144 INFO:teuthology.orchestra.run.smithi017.stdout:(14/131): libcephsqlite-18.0.0-4479.g8627af3e.e 2.3 MB/s | 137 kB 00:00 2023-06-23T18:11:12.159 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2023-06-23T18:11:12.203 INFO:teuthology.orchestra.run.smithi017.stdout:(15/131): librados-devel-18.0.0-4479.g8627af3e. 2.3 MB/s | 139 kB 00:00 2023-06-23T18:11:12.287 INFO:teuthology.orchestra.run.smithi017.stdout:(16/131): libradosstriper1-18.0.0-4479.g8627af3 5.6 MB/s | 474 kB 00:00 2023-06-23T18:11:12.292 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2023-06-23T18:11:12.292 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2023-06-23T18:11:12.695 INFO:teuthology.orchestra.run.smithi017.stdout:(17/131): ceph-osd-18.0.0-4479.g8627af3e.el8.x8 8.8 MB/s | 18 MB 00:02 2023-06-23T18:11:12.827 INFO:teuthology.orchestra.run.smithi017.stdout:(18/131): librgw2-18.0.0-4479.g8627af3e.el8.x86 12 MB/s | 6.7 MB 00:00 2023-06-23T18:11:12.854 INFO:teuthology.orchestra.run.smithi017.stdout:(19/131): python3-ceph-argparse-18.0.0-4479.g86 282 kB/s | 45 kB 00:00 2023-06-23T18:11:12.879 INFO:teuthology.orchestra.run.smithi017.stdout:(20/131): python3-ceph-common-18.0.0-4479.g8627 2.2 MB/s | 119 kB 00:00 2023-06-23T18:11:12.930 INFO:teuthology.orchestra.run.smithi017.stdout:(21/131): python3-rados-18.0.0-4479.g8627af3e.e 7.2 MB/s | 370 kB 00:00 2023-06-23T18:11:12.955 INFO:teuthology.orchestra.run.smithi017.stdout:(22/131): python3-cephfs-18.0.0-4479.g8627af3e. 1.9 MB/s | 197 kB 00:00 2023-06-23T18:11:12.974 INFO:teuthology.orchestra.run.smithi082.stdout:Transaction test succeeded. 2023-06-23T18:11:12.978 INFO:teuthology.orchestra.run.smithi082.stdout:Running transaction 2023-06-23T18:11:12.980 INFO:teuthology.orchestra.run.smithi017.stdout:(23/131): python3-rbd-18.0.0-4479.g8627af3e.el8 7.1 MB/s | 365 kB 00:00 2023-06-23T18:11:13.022 INFO:teuthology.orchestra.run.smithi017.stdout:(24/131): rbd-fuse-18.0.0-4479.g8627af3e.el8.x8 2.2 MB/s | 94 kB 00:00 2023-06-23T18:11:13.048 INFO:teuthology.orchestra.run.smithi017.stdout:(25/131): python3-rgw-18.0.0-4479.g8627af3e.el8 1.1 MB/s | 107 kB 00:00 2023-06-23T18:11:13.123 INFO:teuthology.orchestra.run.smithi017.stdout:(26/131): rbd-nbd-18.0.0-4479.g8627af3e.el8.x86 2.3 MB/s | 179 kB 00:00 2023-06-23T18:11:13.198 INFO:teuthology.orchestra.run.smithi017.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4479.g 308 kB/s | 23 kB 00:00 2023-06-23T18:11:13.257 INFO:teuthology.orchestra.run.smithi017.stdout:(28/131): rbd-mirror-18.0.0-4479.g8627af3e.el8. 15 MB/s | 3.4 MB 00:00 2023-06-23T18:11:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4479.g8627af3 1.7 MB/s | 128 kB 00:00 2023-06-23T18:11:13.440 INFO:teuthology.orchestra.run.smithi017.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4479.g8627a 9.3 MB/s | 1.7 MB 00:00 2023-06-23T18:11:13.471 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2023-06-23T18:11:13.476 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2023-06-23T18:11:13.524 INFO:teuthology.orchestra.run.smithi017.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4479.g86 2.8 MB/s | 240 kB 00:00 2023-06-23T18:11:13.591 INFO:teuthology.orchestra.run.smithi017.stdout:(32/131): ceph-mgr-rook-18.0.0-4479.g8627af3e.e 712 kB/s | 47 kB 00:00 2023-06-23T18:11:13.658 INFO:teuthology.orchestra.run.smithi017.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4479.g8 218 kB/s | 15 kB 00:00 2023-06-23T18:11:13.767 INFO:teuthology.orchestra.run.smithi017.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 15 MB/s | 7.4 MB 00:00 2023-06-23T18:11:13.792 INFO:teuthology.orchestra.run.smithi017.stdout:(35/131): ceph-volume-18.0.0-4479.g8627af3e.el8 1.9 MB/s | 258 kB 00:00 2023-06-23T18:11:13.862 INFO:teuthology.orchestra.run.smithi017.stdout:(36/131): cephadm-18.0.0-4479.g8627af3e.el8.noa 1.7 MB/s | 161 kB 00:00 2023-06-23T18:11:13.884 INFO:teuthology.orchestra.run.smithi017.stdout:(37/131): cephfs-top-18.0.0-4479.g8627af3e.el8. 195 kB/s | 18 kB 00:00 2023-06-23T18:11:14.301 INFO:teuthology.orchestra.run.smithi017.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 141 kB/s | 59 kB 00:00 2023-06-23T18:11:14.460 INFO:teuthology.orchestra.run.smithi017.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 475 kB/s | 75 kB 00:00 2023-06-23T18:11:14.485 INFO:teuthology.orchestra.run.smithi017.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 492 kB/s | 306 kB 00:00 2023-06-23T18:11:14.594 INFO:teuthology.orchestra.run.smithi017.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-06-23T18:11:14.727 INFO:teuthology.orchestra.run.smithi017.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-06-23T18:11:14.761 INFO:teuthology.orchestra.run.smithi017.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.5 MB/s | 476 kB 00:00 2023-06-23T18:11:14.828 INFO:teuthology.orchestra.run.smithi017.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2023-06-23T18:11:14.886 INFO:teuthology.orchestra.run.smithi017.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-06-23T18:11:14.953 INFO:teuthology.orchestra.run.smithi017.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2023-06-23T18:11:15.045 INFO:teuthology.orchestra.run.smithi017.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.4 MB/s | 384 kB 00:00 2023-06-23T18:11:15.062 INFO:teuthology.orchestra.run.smithi017.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 747 kB/s | 81 kB 00:00 2023-06-23T18:11:15.146 INFO:teuthology.orchestra.run.smithi017.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-06-23T18:11:15.171 INFO:teuthology.orchestra.run.smithi017.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 162 kB/s | 18 kB 00:00 2023-06-23T18:11:15.288 INFO:teuthology.orchestra.run.smithi017.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 418 kB/s | 49 kB 00:00 2023-06-23T18:11:15.380 INFO:teuthology.orchestra.run.smithi017.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-06-23T18:11:15.405 INFO:teuthology.orchestra.run.smithi017.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 503 kB/s | 59 kB 00:00 2023-06-23T18:11:15.481 INFO:teuthology.orchestra.run.smithi017.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 603 kB/s | 60 kB 00:00 2023-06-23T18:11:15.556 INFO:teuthology.orchestra.run.smithi017.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-06-23T18:11:15.581 INFO:teuthology.orchestra.run.smithi017.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-06-23T18:11:15.665 INFO:teuthology.orchestra.run.smithi017.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2023-06-23T18:11:15.690 INFO:teuthology.orchestra.run.smithi017.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-06-23T18:11:15.774 INFO:teuthology.orchestra.run.smithi017.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 566 kB/s | 61 kB 00:00 2023-06-23T18:11:15.799 INFO:teuthology.orchestra.run.smithi017.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-06-23T18:11:15.900 INFO:teuthology.orchestra.run.smithi017.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 191 kB/s | 24 kB 00:00 2023-06-23T18:11:15.925 INFO:teuthology.orchestra.run.smithi017.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 233 kB/s | 29 kB 00:00 2023-06-23T18:11:16.016 INFO:teuthology.orchestra.run.smithi017.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 236 kB/s | 27 kB 00:00 2023-06-23T18:11:16.050 INFO:teuthology.orchestra.run.smithi017.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-06-23T18:11:16.176 INFO:teuthology.orchestra.run.smithi017.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 489 kB/s | 61 kB 00:00 2023-06-23T18:11:16.201 INFO:teuthology.orchestra.run.smithi017.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2023-06-23T18:11:16.301 INFO:teuthology.orchestra.run.smithi017.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 682 kB/s | 85 kB 00:00 2023-06-23T18:11:16.318 INFO:teuthology.orchestra.run.smithi017.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-06-23T18:11:16.343 INFO:teuthology.orchestra.run.smithi017.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-06-23T18:11:16.485 INFO:teuthology.orchestra.run.smithi017.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 9.4 MB/s | 1.7 MB 00:00 2023-06-23T18:11:16.544 INFO:teuthology.orchestra.run.smithi017.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-06-23T18:11:16.586 INFO:teuthology.orchestra.run.smithi017.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-06-23T18:11:16.628 INFO:teuthology.orchestra.run.smithi017.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-06-23T18:11:16.670 INFO:teuthology.orchestra.run.smithi017.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-06-23T18:11:16.711 INFO:teuthology.orchestra.run.smithi017.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 933 kB/s | 39 kB 00:00 2023-06-23T18:11:16.871 INFO:teuthology.orchestra.run.smithi017.stdout:(76/131): python3-scipy-1.0.0-21.module+el8.5.0 26 MB/s | 14 MB 00:00 2023-06-23T18:11:16.935 INFO:teuthology.orchestra.run.smithi017.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 3.0 MB/s | 688 kB 00:00 2023-06-23T18:11:16.971 INFO:teuthology.orchestra.run.smithi017.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 5.2 MB/s | 538 kB 00:00 2023-06-23T18:11:16.996 INFO:teuthology.orchestra.run.smithi017.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.5 MB/s | 157 kB 00:00 2023-06-23T18:11:17.022 INFO:teuthology.orchestra.run.smithi017.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2023-06-23T18:11:17.055 INFO:teuthology.orchestra.run.smithi017.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-06-23T18:11:17.097 INFO:teuthology.orchestra.run.smithi017.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 639 kB/s | 27 kB 00:00 2023-06-23T18:11:17.139 INFO:teuthology.orchestra.run.smithi017.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-06-23T18:11:17.181 INFO:teuthology.orchestra.run.smithi017.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2023-06-23T18:11:17.265 INFO:teuthology.orchestra.run.smithi017.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-06-23T18:11:17.307 INFO:teuthology.orchestra.run.smithi017.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-06-23T18:11:17.349 INFO:teuthology.orchestra.run.smithi017.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-06-23T18:11:17.415 INFO:teuthology.orchestra.run.smithi017.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 16 MB/s | 3.7 MB 00:00 2023-06-23T18:11:17.441 INFO:teuthology.orchestra.run.smithi017.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 695 kB/s | 64 kB 00:00 2023-06-23T18:11:17.466 INFO:teuthology.orchestra.run.smithi017.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-06-23T18:11:17.491 INFO:teuthology.orchestra.run.smithi017.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-06-23T18:11:17.517 INFO:teuthology.orchestra.run.smithi017.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-06-23T18:11:17.550 INFO:teuthology.orchestra.run.smithi017.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-06-23T18:11:17.592 INFO:teuthology.orchestra.run.smithi017.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-06-23T18:11:17.904 INFO:teuthology.orchestra.run.smithi082.stdout: Preparing : 1/1 2023-06-23T18:11:18.043 INFO:teuthology.orchestra.run.smithi017.stdout:(95/131): ceph-test-18.0.0-4479.g8627af3e.el8.x 6.9 MB/s | 48 MB 00:06 2023-06-23T18:11:18.110 INFO:teuthology.orchestra.run.smithi017.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2023-06-23T18:11:18.112 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-23T18:11:18.151 INFO:teuthology.orchestra.run.smithi017.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.4 MB/s | 354 kB 00:00 2023-06-23T18:11:18.227 INFO:teuthology.orchestra.run.smithi017.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 6.8 MB/s | 4.9 MB 00:00 2023-06-23T18:11:18.310 INFO:teuthology.orchestra.run.smithi017.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 6.6 MB/s | 4.7 MB 00:00 2023-06-23T18:11:18.312 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-23T18:11:18.344 INFO:teuthology.orchestra.run.smithi017.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-06-23T18:11:18.377 INFO:teuthology.orchestra.run.smithi017.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-06-23T18:11:18.427 INFO:teuthology.orchestra.run.smithi017.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 13 MB/s | 1.5 MB 00:00 2023-06-23T18:11:18.453 INFO:teuthology.orchestra.run.smithi017.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.6 MB/s | 200 kB 00:00 2023-06-23T18:11:18.470 INFO:teuthology.orchestra.run.smithi017.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-06-23T18:11:18.481 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-23T18:11:18.495 INFO:teuthology.orchestra.run.smithi017.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2023-06-23T18:11:18.512 INFO:teuthology.orchestra.run.smithi017.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-06-23T18:11:18.538 INFO:teuthology.orchestra.run.smithi017.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-06-23T18:11:18.563 INFO:teuthology.orchestra.run.smithi017.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 849 kB/s | 43 kB 00:00 2023-06-23T18:11:18.588 INFO:teuthology.orchestra.run.smithi017.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-06-23T18:11:18.592 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-06-23T18:11:18.613 INFO:teuthology.orchestra.run.smithi017.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-06-23T18:11:18.647 INFO:teuthology.orchestra.run.smithi017.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-06-23T18:11:18.663 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2023-06-23T18:11:18.672 INFO:teuthology.orchestra.run.smithi017.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2023-06-23T18:11:18.689 INFO:teuthology.orchestra.run.smithi017.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-06-23T18:11:18.756 INFO:teuthology.orchestra.run.smithi017.stdout:(114/131): postgresql-server-10.21-2.module+el8 8.4 MB/s | 5.1 MB 00:00 2023-06-23T18:11:18.781 INFO:teuthology.orchestra.run.smithi017.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 195 kB/s | 18 kB 00:00 2023-06-23T18:11:18.815 INFO:teuthology.orchestra.run.smithi017.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 3.8 MB/s | 559 kB 00:00 2023-06-23T18:11:18.829 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-23T18:11:18.840 INFO:teuthology.orchestra.run.smithi017.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-06-23T18:11:18.865 INFO:teuthology.orchestra.run.smithi017.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.4 MB/s | 605 kB 00:00 2023-06-23T18:11:18.891 INFO:teuthology.orchestra.run.smithi017.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-06-23T18:11:18.916 INFO:teuthology.orchestra.run.smithi017.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-06-23T18:11:18.941 INFO:teuthology.orchestra.run.smithi017.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-06-23T18:11:18.967 INFO:teuthology.orchestra.run.smithi017.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-06-23T18:11:18.983 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-23T18:11:18.992 INFO:teuthology.orchestra.run.smithi017.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-06-23T18:11:19.039 INFO:teuthology.orchestra.run.smithi017.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 538 kB/s | 39 kB 00:00 2023-06-23T18:11:19.039 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-23T18:11:19.093 INFO:teuthology.orchestra.run.smithi017.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 6.3 MB/s | 644 kB 00:00 2023-06-23T18:11:19.118 INFO:teuthology.orchestra.run.smithi017.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.1 MB/s | 171 kB 00:00 2023-06-23T18:11:19.143 INFO:teuthology.orchestra.run.smithi017.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-06-23T18:11:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-06-23T18:11:19.193 INFO:teuthology.orchestra.run.smithi017.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.1 MB/s | 2.3 MB 00:00 2023-06-23T18:11:19.243 INFO:teuthology.orchestra.run.smithi017.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-06-23T18:11:19.347 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-06-23T18:11:19.443 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-06-23T18:11:19.466 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-06-23T18:11:19.602 INFO:teuthology.orchestra.run.smithi017.stdout:(130/131): librados2-18.0.0-4479.g8627af3e.el8. 7.6 MB/s | 3.7 MB 00:00 2023-06-23T18:11:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-06-23T18:11:19.697 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 5/134 2023-06-23T18:11:19.744 INFO:teuthology.orchestra.run.smithi017.stdout:(131/131): librbd1-18.0.0-4479.g8627af3e.el8.x8 5.9 MB/s | 3.6 MB 00:00 2023-06-23T18:11:19.748 INFO:teuthology.orchestra.run.smithi017.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:11:19.748 INFO:teuthology.orchestra.run.smithi017.stdout:Total 18 MB/s | 205 MB 00:11 2023-06-23T18:11:20.102 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 5/134 2023-06-23T18:11:20.113 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-06-23T18:11:20.289 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:11:20.297 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:11:20.412 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:11:20.606 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction check 2023-06-23T18:11:20.656 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:11:20.718 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:11:20.738 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction check succeeded. 2023-06-23T18:11:20.738 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction test 2023-06-23T18:11:20.750 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:11:20.905 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:11:20.935 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:11:21.033 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:11:21.131 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-06-23T18:11:21.189 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:11:21.285 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:11:21.360 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-23T18:11:21.550 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-23T18:11:21.709 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 9/134 2023-06-23T18:11:21.918 INFO:teuthology.orchestra.run.smithi017.stdout:Transaction test succeeded. 2023-06-23T18:11:21.922 INFO:teuthology.orchestra.run.smithi017.stdout:Running transaction 2023-06-23T18:11:21.934 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-23T18:11:21.941 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-23T18:11:22.127 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-23T18:11:22.159 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-23T18:11:22.338 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:11:22.448 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:11:22.516 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-23T18:11:22.597 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:11:22.710 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-23T18:11:22.748 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:11:22.830 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:11:22.941 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:11:22.967 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-06-23T18:11:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:11:23.150 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-06-23T18:11:23.185 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:11:23.309 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-06-23T18:11:23.329 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-06-23T18:11:23.454 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-23T18:11:23.478 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-06-23T18:11:23.594 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-06-23T18:11:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-06-23T18:11:23.780 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-23T18:11:24.284 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-06-23T18:11:24.379 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-06-23T18:11:24.438 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-06-23T18:11:24.705 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-23T18:11:25.073 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-06-23T18:11:25.238 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-06-23T18:11:25.290 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-06-23T18:11:25.463 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-06-23T18:11:25.981 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-06-23T18:11:26.167 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-06-23T18:11:26.203 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-06-23T18:11:26.410 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-06-23T18:11:26.454 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-06-23T18:11:26.618 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-06-23T18:11:26.668 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-06-23T18:11:26.769 INFO:teuthology.orchestra.run.smithi017.stdout: Preparing : 1/1 2023-06-23T18:11:26.845 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-06-23T18:11:26.863 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-23T18:11:26.921 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-06-23T18:11:27.044 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 26/134 2023-06-23T18:11:27.093 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-06-23T18:11:27.116 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-06-23T18:11:27.181 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 27/134 2023-06-23T18:11:27.332 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-06-23T18:11:27.335 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-23T18:11:27.391 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 28/134 2023-06-23T18:11:27.503 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 26/134 2023-06-23T18:11:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-06-23T18:11:27.587 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-23T18:11:27.609 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-06-23T18:11:27.633 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 27/134 2023-06-23T18:11:27.814 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-06-23T18:11:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-06-23T18:11:28.473 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:11:28.607 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 28/134 2023-06-23T18:11:28.901 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:11:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-06-23T18:11:29.072 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:11:29.187 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-06-23T18:11:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:11:29.345 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-06-23T18:11:29.397 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:11:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-06-23T18:11:29.493 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:11:29.621 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-06-23T18:11:29.657 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 9/134 2023-06-23T18:11:29.880 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-06-23T18:11:30.068 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-06-23T18:11:30.333 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-06-23T18:11:30.457 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-06-23T18:11:30.634 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-06-23T18:11:30.682 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-06-23T18:11:30.831 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:11:30.915 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-06-23T18:11:30.940 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 14/134 2023-06-23T18:11:31.081 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:11:31.110 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-06-23T18:11:31.165 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 15/134 2023-06-23T18:11:31.289 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-06-23T18:11:31.323 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-23T18:11:31.420 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-06-23T18:11:31.502 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-06-23T18:11:31.522 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-23T18:11:31.580 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-06-23T18:11:31.706 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 36/134 2023-06-23T18:11:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-06-23T18:11:31.876 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 37/134 2023-06-23T18:11:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-23T18:11:32.043 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-06-23T18:11:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-23T18:11:32.216 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-06-23T18:11:32.228 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-06-23T18:11:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 36/134 2023-06-23T18:11:32.403 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-06-23T18:11:32.414 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-06-23T18:11:32.456 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-06-23T18:11:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 37/134 2023-06-23T18:11:32.666 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-06-23T18:11:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-06-23T18:11:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-06-23T18:11:32.906 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-06-23T18:11:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-06-23T18:11:33.040 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-06-23T18:11:33.078 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-23T18:11:33.158 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-06-23T18:11:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-06-23T18:11:33.213 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-06-23T18:11:33.365 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-06-23T18:11:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-06-23T18:11:33.514 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-06-23T18:11:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-06-23T18:11:33.709 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-06-23T18:11:33.887 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-06-23T18:11:33.975 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-06-23T18:11:33.986 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-06-23T18:11:34.080 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-06-23T18:11:34.220 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-06-23T18:11:34.222 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-06-23T18:11:34.300 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-06-23T18:11:34.407 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-06-23T18:11:34.447 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-06-23T18:11:34.481 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-06-23T18:11:34.589 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-06-23T18:11:34.652 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-06-23T18:11:34.678 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-06-23T18:11:34.830 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-06-23T18:11:34.934 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-06-23T18:11:34.951 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-06-23T18:11:35.027 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-06-23T18:11:35.228 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-06-23T18:11:35.254 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-06-23T18:11:35.287 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-06-23T18:11:35.403 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-06-23T18:11:35.412 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 26/134 2023-06-23T18:11:35.496 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-06-23T18:11:35.543 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 27/134 2023-06-23T18:11:35.580 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-06-23T18:11:35.723 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-06-23T18:11:35.810 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-06-23T18:11:35.923 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-06-23T18:11:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-06-23T18:11:36.133 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-06-23T18:11:36.243 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-06-23T18:11:36.409 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-06-23T18:11:36.476 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-06-23T18:11:36.500 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 28/134 2023-06-23T18:11:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-06-23T18:11:36.668 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-06-23T18:11:36.804 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-06-23T18:11:36.879 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-06-23T18:11:36.914 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-06-23T18:11:37.072 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-06-23T18:11:37.085 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-06-23T18:11:37.238 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-06-23T18:11:37.261 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-06-23T18:11:37.277 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-06-23T18:11:37.400 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-06-23T18:11:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-06-23T18:11:37.525 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-06-23T18:11:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-06-23T18:11:37.619 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-06-23T18:11:37.696 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-06-23T18:11:37.774 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-06-23T18:11:37.869 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-06-23T18:11:38.007 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-06-23T18:11:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-06-23T18:11:38.244 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-06-23T18:11:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-06-23T18:11:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-06-23T18:11:38.393 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-23T18:11:38.519 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-06-23T18:11:38.562 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-23T18:11:38.721 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-06-23T18:11:38.826 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-23T18:11:38.870 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-23T18:11:39.041 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-23T18:11:39.043 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-23T18:11:39.205 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-23T18:11:39.292 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-23T18:11:39.345 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-23T18:11:39.534 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-23T18:11:39.536 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-23T18:11:39.588 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-06-23T18:11:39.674 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-23T18:11:39.736 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-23T18:11:39.832 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-23T18:11:39.901 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-06-23T18:11:39.910 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-23T18:11:40.033 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-23T18:11:40.086 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-06-23T18:11:40.095 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-23T18:11:40.232 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-23T18:11:40.260 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-23T18:11:40.268 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-06-23T18:11:40.442 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-23T18:11:40.444 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 36/134 2023-06-23T18:11:40.465 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-23T18:11:40.606 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-23T18:11:40.625 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 37/134 2023-06-23T18:11:40.720 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-23T18:11:40.734 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-23T18:11:40.787 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-06-23T18:11:40.906 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-23T18:11:40.940 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-06-23T18:11:40.963 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-23T18:11:41.116 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-06-23T18:11:41.171 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-23T18:11:41.199 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-23T18:11:41.251 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-06-23T18:11:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-23T18:11:41.383 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-23T18:11:41.457 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-06-23T18:11:41.549 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-23T18:11:41.631 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-23T18:11:41.681 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-06-23T18:11:41.775 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-23T18:11:41.852 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-23T18:11:42.003 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-23T18:11:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-06-23T18:11:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-23T18:11:42.342 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-06-23T18:11:42.549 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-06-23T18:11:42.781 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-06-23T18:11:42.893 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-23T18:11:43.003 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-06-23T18:11:43.179 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-23T18:11:43.300 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-06-23T18:11:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-23T18:11:43.479 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-06-23T18:11:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-06-23T18:11:43.766 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-23T18:11:43.870 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-06-23T18:11:43.884 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-23T18:11:44.009 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-06-23T18:11:44.055 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-23T18:11:44.150 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-23T18:11:44.175 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-06-23T18:11:44.248 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-23T18:11:44.403 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-23T18:11:44.491 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-06-23T18:11:44.704 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-23T18:11:44.735 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-06-23T18:11:44.946 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-06-23T18:11:44.986 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-23T18:11:45.039 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-23T18:11:45.167 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-23T18:11:45.169 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-06-23T18:11:45.225 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:11:45.289 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:11:45.320 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-06-23T18:11:45.439 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-06-23T18:11:45.509 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 93/134 2023-06-23T18:11:45.595 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-06-23T18:11:45.667 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-06-23T18:11:45.935 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-06-23T18:11:45.976 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-23T18:11:46.125 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-06-23T18:11:46.159 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:11:46.241 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:11:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-06-23T18:11:46.346 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-06-23T18:11:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 93/134 2023-06-23T18:11:46.570 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-06-23T18:11:46.764 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-06-23T18:11:46.913 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-06-23T18:11:47.070 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-06-23T18:11:47.310 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-06-23T18:11:47.520 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-06-23T18:11:47.642 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-06-23T18:11:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-06-23T18:11:47.985 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-06-23T18:11:48.177 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-06-23T18:11:48.343 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-06-23T18:11:48.490 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-06-23T18:11:48.619 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-06-23T18:11:48.873 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-06-23T18:11:49.067 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-06-23T18:11:49.188 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:49.252 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-06-23T18:11:49.441 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:49.475 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-06-23T18:11:49.662 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-06-23T18:11:49.836 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-06-23T18:11:50.036 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:50.042 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-06-23T18:11:50.263 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:50.288 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:11:50.365 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:11:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:50.901 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:11:51.140 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:51.218 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-06-23T18:11:51.399 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:11:51.467 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:11:51.468 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-06-23T18:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-06-23T18:11:51.751 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:11:52.008 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-06-23T18:11:52.264 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-06-23T18:11:52.445 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-06-23T18:11:53.251 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-06-23T18:11:53.444 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:11:53.505 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 92/134 2023-06-23T18:11:53.670 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 93/134 2023-06-23T18:11:57.516 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:57.753 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:58.405 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 94/134 2023-06-23T18:11:58.642 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:11:58.740 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 95/134 2023-06-23T18:11:58.860 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:12:09.544 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:12:09.544 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /sys 2023-06-23T18:12:09.545 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /proc 2023-06-23T18:12:09.545 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2023-06-23T18:12:09.545 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /run 2023-06-23T18:12:09.546 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /mnt 2023-06-23T18:12:09.546 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /var/tmp 2023-06-23T18:12:09.546 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /home 2023-06-23T18:12:09.547 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /tmp 2023-06-23T18:12:09.547 INFO:teuthology.orchestra.run.smithi082.stdout:skipping the directory /dev 2023-06-23T18:12:09.548 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:12:09.895 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:12:11.457 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:12:11.556 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:12:11.556 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2023-06-23T18:12:11.556 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2023-06-23T18:12:11.556 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /dev 2023-06-23T18:12:11.557 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /run 2023-06-23T18:12:11.557 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2023-06-23T18:12:11.557 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2023-06-23T18:12:11.557 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2023-06-23T18:12:11.557 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2023-06-23T18:12:11.557 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /dev 2023-06-23T18:12:11.557 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:12:11.667 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:12:11.885 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:12:12.006 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:12:12.064 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:12:12.528 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:12:12.677 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:12:12.695 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:12:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:12:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:12:14.069 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:12:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:12:14.346 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:12:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:12:14.682 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 102/134 2023-06-23T18:12:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:12:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:12:14.873 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:12:14.883 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 103/134 2023-06-23T18:12:15.068 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:12:15.284 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:12:15.549 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:12:16.508 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:12:16.659 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:12:16.705 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:12:16.751 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:12:16.889 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:12:17.028 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 102/134 2023-06-23T18:12:17.126 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 107/134 2023-06-23T18:12:17.241 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 103/134 2023-06-23T18:12:17.365 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:12:17.400 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:12:17.405 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:12:17.559 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:12:17.570 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:12:17.627 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:12:18.061 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:12:18.520 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 96/134 2023-06-23T18:12:18.520 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /sys 2023-06-23T18:12:18.521 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /proc 2023-06-23T18:12:18.521 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /dev 2023-06-23T18:12:18.521 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /run 2023-06-23T18:12:18.521 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /mnt 2023-06-23T18:12:18.521 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /var/tmp 2023-06-23T18:12:18.521 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /home 2023-06-23T18:12:18.521 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /tmp 2023-06-23T18:12:18.522 INFO:teuthology.orchestra.run.smithi017.stdout:skipping the directory /dev 2023-06-23T18:12:18.522 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:12:18.864 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 110/134 2023-06-23T18:12:18.867 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:12:19.065 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:12:19.092 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:12:19.103 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:12:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:12:19.259 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:12:19.301 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:12:19.351 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:12:19.442 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:12:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 107/134 2023-06-23T18:12:19.787 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:12:19.869 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:12:19.906 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:12:19.942 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:12:20.063 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:12:20.074 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:12:20.534 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 97/134 2023-06-23T18:12:20.747 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:12:20.979 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 98/134 2023-06-23T18:12:21.150 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:12:21.454 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 110/134 2023-06-23T18:12:21.643 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 99/134 2023-06-23T18:12:21.647 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:12:21.687 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:12:21.783 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:12:21.801 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 100/134 2023-06-23T18:12:21.821 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:12:21.865 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:12:21.996 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:12:22.364 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:12:22.513 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:12:23.523 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:12:23.724 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 101/134 2023-06-23T18:12:23.872 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 102/134 2023-06-23T18:12:24.041 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 103/134 2023-06-23T18:12:24.207 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:12:24.431 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 104/134 2023-06-23T18:12:24.445 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:12:24.721 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:12:25.318 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 115/134 2023-06-23T18:12:25.655 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-06-23T18:12:25.749 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 105/134 2023-06-23T18:12:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 117/134 2023-06-23T18:12:25.899 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:12:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 106/134 2023-06-23T18:12:26.093 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-06-23T18:12:26.249 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-06-23T18:12:26.359 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 107/134 2023-06-23T18:12:26.442 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-06-23T18:12:26.542 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:12:26.578 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 108/134 2023-06-23T18:12:26.678 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-06-23T18:12:26.765 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:12:26.777 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 109/134 2023-06-23T18:12:26.845 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 121/134 2023-06-23T18:12:26.991 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-23T18:12:27.346 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:12:27.600 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 115/134 2023-06-23T18:12:27.895 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-06-23T18:12:28.104 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-06-23T18:12:28.139 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 110/134 2023-06-23T18:12:28.315 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:12:28.357 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 111/134 2023-06-23T18:12:28.490 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:12:28.533 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4479.g8627a 112/134 2023-06-23T18:12:28.682 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:12:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-06-23T18:12:29.049 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 113/134 2023-06-23T18:12:29.117 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:12:29.306 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-06-23T18:12:29.357 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:29.374 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 119/134 2023-06-23T18:12:29.516 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 124/134 2023-06-23T18:12:29.616 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-06-23T18:12:29.792 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 121/134 2023-06-23T18:12:29.815 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 125/134 2023-06-23T18:12:29.938 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-23T18:12:30.141 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 126/134 2023-06-23T18:12:30.481 INFO:teuthology.orchestra.run.smithi082.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-23T18:12:30.664 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-06-23T18:12:30.744 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-06-23T18:12:30.965 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-06-23T18:12:30.977 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-06-23T18:12:31.114 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-23T18:12:31.123 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-06-23T18:12:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-06-23T18:12:31.250 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:31.484 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:31.485 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:12:31.719 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:12:31.913 INFO:teuthology.orchestra.run.smithi082.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:32.211 INFO:teuthology.orchestra.run.smithi082.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:32.211 INFO:teuthology.orchestra.run.smithi082.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T18:12:32.211 INFO:teuthology.orchestra.run.smithi082.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T18:12:32.211 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:12:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:32.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 1/134 2023-06-23T18:12:32.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2/134 2023-06-23T18:12:32.460 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 3/134 2023-06-23T18:12:32.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 4/134 2023-06-23T18:12:32.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 5/134 2023-06-23T18:12:32.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:12:32.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:12:32.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:12:32.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 9/134 2023-06-23T18:12:32.461 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 10/134 2023-06-23T18:12:32.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 11/134 2023-06-23T18:12:32.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 12/134 2023-06-23T18:12:32.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 13/134 2023-06-23T18:12:32.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 14/134 2023-06-23T18:12:32.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 15/134 2023-06-23T18:12:32.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 16/134 2023-06-23T18:12:32.462 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 17/134 2023-06-23T18:12:32.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 18/134 2023-06-23T18:12:32.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 19/134 2023-06-23T18:12:32.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 20/134 2023-06-23T18:12:32.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 21/134 2023-06-23T18:12:32.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 22/134 2023-06-23T18:12:32.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 23/134 2023-06-23T18:12:32.463 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 24/134 2023-06-23T18:12:32.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 25/134 2023-06-23T18:12:32.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 26/134 2023-06-23T18:12:32.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 27/134 2023-06-23T18:12:32.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 28/134 2023-06-23T18:12:32.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 29/134 2023-06-23T18:12:32.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 30/134 2023-06-23T18:12:32.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 31/134 2023-06-23T18:12:32.464 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 32/134 2023-06-23T18:12:32.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 33/134 2023-06-23T18:12:32.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 34/134 2023-06-23T18:12:32.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 35/134 2023-06-23T18:12:32.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 36/134 2023-06-23T18:12:32.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 37/134 2023-06-23T18:12:32.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 38/134 2023-06-23T18:12:32.465 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-23T18:12:32.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-23T18:12:32.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-23T18:12:32.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-23T18:12:32.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-23T18:12:32.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-23T18:12:32.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-23T18:12:32.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-23T18:12:32.466 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-23T18:12:32.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-23T18:12:32.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-23T18:12:32.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-23T18:12:32.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-23T18:12:32.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-23T18:12:32.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-23T18:12:32.467 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-23T18:12:32.468 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-23T18:12:32.468 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-23T18:12:32.468 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-23T18:12:32.468 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-23T18:12:32.468 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-23T18:12:32.468 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-23T18:12:32.468 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-23T18:12:32.468 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-23T18:12:32.469 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-23T18:12:32.469 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-23T18:12:32.469 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-23T18:12:32.469 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-23T18:12:32.469 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-23T18:12:32.469 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-23T18:12:32.469 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-23T18:12:32.470 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-23T18:12:32.470 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-23T18:12:32.470 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-06-23T18:12:32.470 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-06-23T18:12:32.470 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-06-23T18:12:32.470 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-06-23T18:12:32.470 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-06-23T18:12:32.470 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-06-23T18:12:32.471 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-06-23T18:12:32.471 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-06-23T18:12:32.471 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-06-23T18:12:32.471 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-06-23T18:12:32.471 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-06-23T18:12:32.471 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-06-23T18:12:32.471 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-06-23T18:12:32.472 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-06-23T18:12:32.472 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-06-23T18:12:32.472 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-06-23T18:12:32.472 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-06-23T18:12:32.472 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-06-23T18:12:32.472 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-06-23T18:12:32.472 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-06-23T18:12:32.472 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-06-23T18:12:32.473 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-06-23T18:12:32.473 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-06-23T18:12:32.473 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-06-23T18:12:32.473 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-06-23T18:12:32.473 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-06-23T18:12:32.473 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-06-23T18:12:32.473 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-06-23T18:12:32.473 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-06-23T18:12:32.474 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-06-23T18:12:32.474 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-06-23T18:12:32.474 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-06-23T18:12:32.474 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-06-23T18:12:32.474 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-06-23T18:12:32.474 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-06-23T18:12:32.474 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-06-23T18:12:32.475 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-06-23T18:12:32.475 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-06-23T18:12:32.475 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-06-23T18:12:32.475 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-06-23T18:12:32.475 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-06-23T18:12:32.475 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-06-23T18:12:32.475 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-06-23T18:12:32.475 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-06-23T18:12:32.476 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-06-23T18:12:32.476 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-06-23T18:12:32.476 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-06-23T18:12:32.476 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-06-23T18:12:32.476 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-06-23T18:12:32.476 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-06-23T18:12:32.476 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-06-23T18:12:32.476 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-06-23T18:12:32.477 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-06-23T18:12:32.477 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-06-23T18:12:32.477 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-06-23T18:12:32.477 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-06-23T18:12:32.477 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-06-23T18:12:32.477 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 129/134 2023-06-23T18:12:32.477 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-23T18:12:32.478 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 131/134 2023-06-23T18:12:32.478 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:32.478 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-23T18:12:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 124/134 2023-06-23T18:12:32.962 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 125/134 2023-06-23T18:12:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 126/134 2023-06-23T18:12:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-23T18:12:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-23T18:12:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-23T18:12:33.924 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-06-23T18:12:34.026 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 114/134 2023-06-23T18:12:34.111 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-06-23T18:12:34.195 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-06-23T18:12:34.369 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-06-23T18:12:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-06-23T18:12:34.383 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:34.416 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 115/134 2023-06-23T18:12:34.547 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-06-23T18:12:34.572 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:34.573 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:12:34.759 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-06-23T18:12:34.793 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:12:34.885 INFO:teuthology.orchestra.run.smithi082.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:34.886 INFO:teuthology.orchestra.run.smithi082.stdout:Last metadata expiration check: 0:01:49 ago on Fri 23 Jun 2023 06:10:45 PM UTC. 2023-06-23T18:12:35.002 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:35.304 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:35.305 INFO:teuthology.orchestra.run.smithi002.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T18:12:35.305 INFO:teuthology.orchestra.run.smithi002.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T18:12:35.305 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:12:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 1/134 2023-06-23T18:12:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2/134 2023-06-23T18:12:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 3/134 2023-06-23T18:12:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 4/134 2023-06-23T18:12:35.555 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 5/134 2023-06-23T18:12:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:12:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:12:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:12:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 9/134 2023-06-23T18:12:35.556 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 10/134 2023-06-23T18:12:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 11/134 2023-06-23T18:12:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 12/134 2023-06-23T18:12:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 13/134 2023-06-23T18:12:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 14/134 2023-06-23T18:12:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 15/134 2023-06-23T18:12:35.557 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 16/134 2023-06-23T18:12:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 17/134 2023-06-23T18:12:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 18/134 2023-06-23T18:12:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 19/134 2023-06-23T18:12:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 20/134 2023-06-23T18:12:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 21/134 2023-06-23T18:12:35.558 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 22/134 2023-06-23T18:12:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 23/134 2023-06-23T18:12:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 24/134 2023-06-23T18:12:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 25/134 2023-06-23T18:12:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 26/134 2023-06-23T18:12:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 27/134 2023-06-23T18:12:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 28/134 2023-06-23T18:12:35.559 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 29/134 2023-06-23T18:12:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 30/134 2023-06-23T18:12:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 31/134 2023-06-23T18:12:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 32/134 2023-06-23T18:12:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 33/134 2023-06-23T18:12:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 34/134 2023-06-23T18:12:35.560 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 35/134 2023-06-23T18:12:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 36/134 2023-06-23T18:12:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 37/134 2023-06-23T18:12:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 38/134 2023-06-23T18:12:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-23T18:12:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-23T18:12:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-23T18:12:35.561 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-23T18:12:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-23T18:12:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-23T18:12:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-23T18:12:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-23T18:12:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-23T18:12:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-23T18:12:35.562 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-23T18:12:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-23T18:12:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-23T18:12:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-23T18:12:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-23T18:12:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-23T18:12:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-23T18:12:35.563 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-23T18:12:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-23T18:12:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-23T18:12:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-23T18:12:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-23T18:12:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-23T18:12:35.564 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-23T18:12:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-23T18:12:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-23T18:12:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-23T18:12:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-23T18:12:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-23T18:12:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-23T18:12:35.565 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-23T18:12:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-23T18:12:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-23T18:12:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-06-23T18:12:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-06-23T18:12:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-06-23T18:12:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-06-23T18:12:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-06-23T18:12:35.566 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-06-23T18:12:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-06-23T18:12:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-06-23T18:12:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-06-23T18:12:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-06-23T18:12:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-06-23T18:12:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-06-23T18:12:35.567 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-06-23T18:12:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-06-23T18:12:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-06-23T18:12:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-06-23T18:12:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-06-23T18:12:35.568 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-06-23T18:12:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-06-23T18:12:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-06-23T18:12:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-06-23T18:12:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-06-23T18:12:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-06-23T18:12:35.569 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-06-23T18:12:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-06-23T18:12:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-06-23T18:12:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-06-23T18:12:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-06-23T18:12:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-06-23T18:12:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-06-23T18:12:35.570 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-06-23T18:12:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-06-23T18:12:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-06-23T18:12:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-06-23T18:12:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-06-23T18:12:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-06-23T18:12:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-06-23T18:12:35.571 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-06-23T18:12:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-06-23T18:12:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-06-23T18:12:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-06-23T18:12:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-06-23T18:12:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-06-23T18:12:35.572 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-06-23T18:12:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-06-23T18:12:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-06-23T18:12:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-06-23T18:12:35.573 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-06-23T18:12:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-06-23T18:12:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-06-23T18:12:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-23T18:12:35.574 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-06-23T18:12:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-06-23T18:12:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-06-23T18:12:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-06-23T18:12:35.575 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-06-23T18:12:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-23T18:12:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 129/134 2023-06-23T18:12:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-23T18:12:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 131/134 2023-06-23T18:12:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:35.576 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-23T18:12:35.702 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-06-23T18:12:36.042 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-06-23T18:12:36.193 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 119/134 2023-06-23T18:12:36.368 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-06-23T18:12:36.527 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 121/134 2023-06-23T18:12:36.666 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-23T18:12:37.393 INFO:teuthology.orchestra.run.smithi082.stdout:Installed products updated. 2023-06-23T18:12:37.606 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:12:37.607 INFO:teuthology.orchestra.run.smithi082.stdout:Upgraded: 2023-06-23T18:12:37.607 INFO:teuthology.orchestra.run.smithi082.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-23T18:12:37.607 INFO:teuthology.orchestra.run.smithi082.stdout: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.607 INFO:teuthology.orchestra.run.smithi082.stdout: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.607 INFO:teuthology.orchestra.run.smithi082.stdout:Installed: 2023-06-23T18:12:37.607 INFO:teuthology.orchestra.run.smithi082.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-23T18:12:37.608 INFO:teuthology.orchestra.run.smithi082.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T18:12:37.608 INFO:teuthology.orchestra.run.smithi082.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T18:12:37.608 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.608 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.609 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.609 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.609 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.609 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-immutable-object-cache-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.609 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.609 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.609 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.609 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.610 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.610 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.610 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.610 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.610 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.610 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.610 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.611 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.611 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.611 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.611 INFO:teuthology.orchestra.run.smithi082.stdout: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.612 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.612 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:37.612 INFO:teuthology.orchestra.run.smithi082.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-23T18:12:37.612 INFO:teuthology.orchestra.run.smithi082.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T18:12:37.612 INFO:teuthology.orchestra.run.smithi082.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-23T18:12:37.612 INFO:teuthology.orchestra.run.smithi082.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T18:12:37.612 INFO:teuthology.orchestra.run.smithi082.stdout: jq-1.6-3.el8.x86_64 2023-06-23T18:12:37.612 INFO:teuthology.orchestra.run.smithi082.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-23T18:12:37.613 INFO:teuthology.orchestra.run.smithi082.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-23T18:12:37.613 INFO:teuthology.orchestra.run.smithi082.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T18:12:37.613 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.613 INFO:teuthology.orchestra.run.smithi082.stdout: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.613 INFO:teuthology.orchestra.run.smithi082.stdout: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.613 INFO:teuthology.orchestra.run.smithi082.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-23T18:12:37.614 INFO:teuthology.orchestra.run.smithi082.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:12:37.614 INFO:teuthology.orchestra.run.smithi082.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T18:12:37.614 INFO:teuthology.orchestra.run.smithi082.stdout: libpq-13.5-1.el8.x86_64 2023-06-23T18:12:37.614 INFO:teuthology.orchestra.run.smithi082.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:12:37.614 INFO:teuthology.orchestra.run.smithi082.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T18:12:37.614 INFO:teuthology.orchestra.run.smithi082.stdout: librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.615 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.615 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-23T18:12:37.615 INFO:teuthology.orchestra.run.smithi082.stdout: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.615 INFO:teuthology.orchestra.run.smithi082.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T18:12:37.615 INFO:teuthology.orchestra.run.smithi082.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-23T18:12:37.615 INFO:teuthology.orchestra.run.smithi082.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T18:12:37.615 INFO:teuthology.orchestra.run.smithi082.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T18:12:37.615 INFO:teuthology.orchestra.run.smithi082.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-23T18:12:37.616 INFO:teuthology.orchestra.run.smithi082.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T18:12:37.616 INFO:teuthology.orchestra.run.smithi082.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T18:12:37.616 INFO:teuthology.orchestra.run.smithi082.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T18:12:37.616 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-23T18:12:37.616 INFO:teuthology.orchestra.run.smithi082.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-23T18:12:37.616 INFO:teuthology.orchestra.run.smithi082.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-23T18:12:37.616 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T18:12:37.617 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T18:12:37.617 INFO:teuthology.orchestra.run.smithi082.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T18:12:37.617 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:37.617 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:37.617 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:37.617 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T18:12:37.617 INFO:teuthology.orchestra.run.smithi082.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-23T18:12:37.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T18:12:37.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T18:12:37.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T18:12:37.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T18:12:37.618 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-23T18:12:37.619 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T18:12:37.619 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T18:12:37.619 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-23T18:12:37.619 INFO:teuthology.orchestra.run.smithi082.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T18:12:37.619 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T18:12:37.619 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T18:12:37.619 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-23T18:12:37.619 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T18:12:37.620 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T18:12:37.620 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T18:12:37.620 INFO:teuthology.orchestra.run.smithi082.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T18:12:37.620 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T18:12:37.620 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T18:12:37.620 INFO:teuthology.orchestra.run.smithi082.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T18:12:37.620 INFO:teuthology.orchestra.run.smithi082.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T18:12:37.621 INFO:teuthology.orchestra.run.smithi082.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T18:12:37.621 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-23T18:12:37.621 INFO:teuthology.orchestra.run.smithi082.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-23T18:12:37.621 INFO:teuthology.orchestra.run.smithi082.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T18:12:37.621 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T18:12:37.621 INFO:teuthology.orchestra.run.smithi082.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T18:12:37.621 INFO:teuthology.orchestra.run.smithi082.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T18:12:37.621 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-23T18:12:37.622 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T18:12:37.622 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T18:12:37.622 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-23T18:12:37.622 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T18:12:37.622 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.622 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.622 INFO:teuthology.orchestra.run.smithi082.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T18:12:37.623 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T18:12:37.623 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.623 INFO:teuthology.orchestra.run.smithi082.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T18:12:37.623 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T18:12:37.623 INFO:teuthology.orchestra.run.smithi082.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-23T18:12:37.623 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T18:12:37.623 INFO:teuthology.orchestra.run.smithi082.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T18:12:37.623 INFO:teuthology.orchestra.run.smithi082.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T18:12:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T18:12:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T18:12:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T18:12:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T18:12:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T18:12:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T18:12:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T18:12:37.624 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.625 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.625 INFO:teuthology.orchestra.run.smithi082.stdout: rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:37.625 INFO:teuthology.orchestra.run.smithi082.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-23T18:12:37.625 INFO:teuthology.orchestra.run.smithi082.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T18:12:37.625 INFO:teuthology.orchestra.run.smithi082.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-23T18:12:37.625 INFO:teuthology.orchestra.run.smithi082.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-23T18:12:37.626 INFO:teuthology.orchestra.run.smithi082.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T18:12:37.627 INFO:teuthology.orchestra.run.smithi082.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-23T18:12:37.627 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:12:37.627 INFO:teuthology.orchestra.run.smithi082.stdout:Complete! 2023-06-23T18:12:37.821 DEBUG:teuthology.parallel:result is None 2023-06-23T18:12:38.198 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:38.199 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:55 ago on Fri 23 Jun 2023 06:10:43 PM UTC. 2023-06-23T18:12:38.939 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:39.164 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:39.183 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-06-23T18:12:39.323 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 124/134 2023-06-23T18:12:39.623 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 125/134 2023-06-23T18:12:39.955 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 126/134 2023-06-23T18:12:40.232 INFO:teuthology.orchestra.run.smithi017.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-06-23T18:12:40.355 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-23T18:12:40.364 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-23T18:12:40.558 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-06-23T18:12:40.730 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-06-23T18:12:40.818 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-06-23T18:12:40.991 INFO:teuthology.orchestra.run.smithi002.stdout:Installed products updated. 2023-06-23T18:12:41.005 INFO:teuthology.orchestra.run.smithi017.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-06-23T18:12:41.020 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-06-23T18:12:41.021 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:12:41.225 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2023-06-23T18:12:41.225 INFO:teuthology.orchestra.run.smithi002.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-23T18:12:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.226 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2023-06-23T18:12:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-23T18:12:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T18:12:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T18:12:41.226 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.227 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.228 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.229 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-23T18:12:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T18:12:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-23T18:12:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T18:12:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: jq-1.6-3.el8.x86_64 2023-06-23T18:12:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-23T18:12:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-23T18:12:41.230 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T18:12:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-23T18:12:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:12:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T18:12:41.231 INFO:teuthology.orchestra.run.smithi002.stdout: libpq-13.5-1.el8.x86_64 2023-06-23T18:12:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:12:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T18:12:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-23T18:12:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.232 INFO:teuthology.orchestra.run.smithi002.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T18:12:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-23T18:12:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T18:12:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T18:12:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-23T18:12:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T18:12:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T18:12:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T18:12:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-23T18:12:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-23T18:12:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-23T18:12:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T18:12:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T18:12:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T18:12:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:41.234 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T18:12:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-23T18:12:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T18:12:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T18:12:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T18:12:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.235 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T18:12:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-23T18:12:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T18:12:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T18:12:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-23T18:12:41.236 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T18:12:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T18:12:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T18:12:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-23T18:12:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T18:12:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T18:12:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T18:12:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T18:12:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T18:12:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T18:12:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T18:12:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T18:12:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T18:12:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-23T18:12:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-23T18:12:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T18:12:41.238 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T18:12:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T18:12:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T18:12:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-23T18:12:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T18:12:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T18:12:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-23T18:12:41.239 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T18:12:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T18:12:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T18:12:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T18:12:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T18:12:41.240 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-23T18:12:41.241 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T18:12:41.241 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T18:12:41.241 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T18:12:41.241 INFO:teuthology.orchestra.run.smithi002.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T18:12:41.241 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T18:12:41.241 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T18:12:41.241 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T18:12:41.241 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T18:12:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T18:12:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T18:12:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-23T18:12:41.242 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T18:12:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-23T18:12:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-23T18:12:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T18:12:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-23T18:12:41.243 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:12:41.244 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-06-23T18:12:41.251 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:41.251 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:12:41.437 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-06-23T18:12:41.504 DEBUG:teuthology.parallel:result is None 2023-06-23T18:12:41.630 INFO:teuthology.orchestra.run.smithi017.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:41.933 INFO:teuthology.orchestra.run.smithi017.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:41.934 INFO:teuthology.orchestra.run.smithi017.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-06-23T18:12:41.934 INFO:teuthology.orchestra.run.smithi017.stdout:All rules containing unresolvable specifiers will be skipped. 2023-06-23T18:12:41.934 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:12:42.169 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 1/134 2023-06-23T18:12:42.169 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2/134 2023-06-23T18:12:42.169 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 3/134 2023-06-23T18:12:42.169 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 4/134 2023-06-23T18:12:42.169 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4479.g8627a 5/134 2023-06-23T18:12:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 6/134 2023-06-23T18:12:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 7/134 2023-06-23T18:12:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 8/134 2023-06-23T18:12:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 9/134 2023-06-23T18:12:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 10/134 2023-06-23T18:12:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 11/134 2023-06-23T18:12:42.170 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 12/134 2023-06-23T18:12:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 13/134 2023-06-23T18:12:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_ 14/134 2023-06-23T18:12:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 15/134 2023-06-23T18:12:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 16/134 2023-06-23T18:12:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_6 17/134 2023-06-23T18:12:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86 18/134 2023-06-23T18:12:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 19/134 2023-06-23T18:12:42.171 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el 20/134 2023-06-23T18:12:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-ceph-common-2:18.0.0-4479.g8627af3e.el8. 21/134 2023-06-23T18:12:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_6 22/134 2023-06-23T18:12:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 23/134 2023-06-23T18:12:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 24/134 2023-06-23T18:12:42.172 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 25/134 2023-06-23T18:12:42.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 26/134 2023-06-23T18:12:42.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 27/134 2023-06-23T18:12:42.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 28/134 2023-06-23T18:12:42.180 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e. 29/134 2023-06-23T18:12:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noa 30/134 2023-06-23T18:12:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.n 31/134 2023-06-23T18:12:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4479.g862 32/134 2023-06-23T18:12:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el 33/134 2023-06-23T18:12:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 34/134 2023-06-23T18:12:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.e 35/134 2023-06-23T18:12:42.181 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 36/134 2023-06-23T18:12:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 37/134 2023-06-23T18:12:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 38/134 2023-06-23T18:12:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-06-23T18:12:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-06-23T18:12:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-06-23T18:12:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-06-23T18:12:42.182 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-06-23T18:12:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-06-23T18:12:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-06-23T18:12:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-06-23T18:12:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-06-23T18:12:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-06-23T18:12:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-06-23T18:12:42.183 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-06-23T18:12:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-06-23T18:12:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-06-23T18:12:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-06-23T18:12:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-06-23T18:12:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-06-23T18:12:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-06-23T18:12:42.184 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-06-23T18:12:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-06-23T18:12:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-06-23T18:12:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-06-23T18:12:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-06-23T18:12:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-06-23T18:12:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-06-23T18:12:42.185 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-06-23T18:12:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-06-23T18:12:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-06-23T18:12:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-06-23T18:12:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-06-23T18:12:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-06-23T18:12:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-06-23T18:12:42.186 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-06-23T18:12:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-06-23T18:12:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-06-23T18:12:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-06-23T18:12:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-06-23T18:12:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-06-23T18:12:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-06-23T18:12:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-06-23T18:12:42.187 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-06-23T18:12:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-06-23T18:12:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-06-23T18:12:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-06-23T18:12:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-06-23T18:12:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-06-23T18:12:42.188 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-06-23T18:12:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-06-23T18:12:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-06-23T18:12:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-06-23T18:12:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-06-23T18:12:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-06-23T18:12:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-06-23T18:12:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-06-23T18:12:42.189 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-06-23T18:12:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-06-23T18:12:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-06-23T18:12:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-06-23T18:12:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-06-23T18:12:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-06-23T18:12:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-06-23T18:12:42.190 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-06-23T18:12:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-06-23T18:12:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-06-23T18:12:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-06-23T18:12:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-06-23T18:12:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-06-23T18:12:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-06-23T18:12:42.191 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-06-23T18:12:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-06-23T18:12:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-06-23T18:12:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-06-23T18:12:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-06-23T18:12:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-06-23T18:12:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-06-23T18:12:42.192 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-06-23T18:12:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-06-23T18:12:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-06-23T18:12:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-06-23T18:12:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-06-23T18:12:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-06-23T18:12:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-06-23T18:12:42.193 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-06-23T18:12:42.194 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-06-23T18:12:42.194 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-06-23T18:12:42.194 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-06-23T18:12:42.194 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-06-23T18:12:42.194 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-06-23T18:12:42.194 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-06-23T18:12:42.194 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-06-23T18:12:42.195 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 129/134 2023-06-23T18:12:42.195 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-06-23T18:12:42.195 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 131/134 2023-06-23T18:12:42.195 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-06-23T18:12:42.195 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-06-23T18:12:44.605 INFO:teuthology.orchestra.run.smithi017.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-06-23T18:12:44.605 INFO:teuthology.orchestra.run.smithi017.stdout:Last metadata expiration check: 0:01:48 ago on Fri 23 Jun 2023 06:10:56 PM UTC. 2023-06-23T18:12:47.554 INFO:teuthology.orchestra.run.smithi017.stdout:Installed products updated. 2023-06-23T18:12:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:12:47.789 INFO:teuthology.orchestra.run.smithi017.stdout:Upgraded: 2023-06-23T18:12:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-06-23T18:12:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: librados2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.789 INFO:teuthology.orchestra.run.smithi017.stdout: librbd1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.789 INFO:teuthology.orchestra.run.smithi017.stdout:Installed: 2023-06-23T18:12:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-06-23T18:12:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: automake-1.16.1-7.el8.noarch 2023-06-23T18:12:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: bison-3.0.4-10.el8.x86_64 2023-06-23T18:12:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.790 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-grafana-dashboards-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-immutable-object-cache-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mds-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.791 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-rook-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mon-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-osd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-prometheus-alerts-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-radosgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-selinux-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.792 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-test-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-volume-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: cephadm-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-top-2:18.0.0-4479.g8627af3e.el8.noarch 2023-06-23T18:12:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-06-23T18:12:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: flex-2.6.1-9.el8.x86_64 2023-06-23T18:12:47.793 INFO:teuthology.orchestra.run.smithi017.stdout: fuse-2.9.7-15.el8.x86_64 2023-06-23T18:12:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-06-23T18:12:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: jq-1.6-3.el8.x86_64 2023-06-23T18:12:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-06-23T18:12:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-06-23T18:12:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-06-23T18:12:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: libcephfs2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.794 INFO:teuthology.orchestra.run.smithi017.stdout: libcephsqlite-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-06-23T18:12:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:12:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: liboath-2.6.2-3.el8.x86_64 2023-06-23T18:12:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: libpq-13.5-1.el8.x86_64 2023-06-23T18:12:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-06-23T18:12:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-06-23T18:12:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: librados-devel-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.795 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-06-23T18:12:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: librgw2-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-06-23T18:12:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-06-23T18:12:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: libtool-2.4.6-25.el8.x86_64 2023-06-23T18:12:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: libunwind-1.3.1-3.el8.x86_64 2023-06-23T18:12:47.796 INFO:teuthology.orchestra.run.smithi017.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-06-23T18:12:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-06-23T18:12:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-06-23T18:12:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-06-23T18:12:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-0.3.15-3.el8.x86_64 2023-06-23T18:12:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-06-23T18:12:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-06-23T18:12:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-06-23T18:12:47.797 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-06-23T18:12:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-06-23T18:12:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-06-23T18:12:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-06-23T18:12:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: python3-babel-2.5.1-7.el8.noarch 2023-06-23T18:12:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-06-23T18:12:47.798 INFO:teuthology.orchestra.run.smithi017.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-06-23T18:12:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-06-23T18:12:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-common-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cephfs-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-06-23T18:12:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-06-23T18:12:47.799 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-06-23T18:12:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-06-23T18:12:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-06-23T18:12:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-06-23T18:12:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-6.2-6.el8.noarch 2023-06-23T18:12:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-06-23T18:12:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-06-23T18:12:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-06-23T18:12:47.800 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-06-23T18:12:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-06-23T18:12:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-06-23T18:12:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-06-23T18:12:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako-1.0.6-13.el8.noarch 2023-06-23T18:12:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-06-23T18:12:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-06-23T18:12:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-06-23T18:12:47.801 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-06-23T18:12:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-06-23T18:12:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-06-23T18:12:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-06-23T18:12:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: python3-portend-2.6-1.el8.noarch 2023-06-23T18:12:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-06-23T18:12:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-06-23T18:12:47.802 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-06-23T18:12:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-06-23T18:12:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pycparser-2.14-14.el8.noarch 2023-06-23T18:12:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pytz-2017.2-9.el8.noarch 2023-06-23T18:12:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-06-23T18:12:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-06-23T18:12:47.803 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rgw-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: python3-routes-2.4.1-12.el8.noarch 2023-06-23T18:12:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rsa-4.8-1.el8.noarch 2023-06-23T18:12:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-06-23T18:12:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-06-23T18:12:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-06-23T18:12:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-06-23T18:12:47.804 INFO:teuthology.orchestra.run.smithi017.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-06-23T18:12:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-06-23T18:12:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-06-23T18:12:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-06-23T18:12:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-06-23T18:12:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-06-23T18:12:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-06-23T18:12:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-fuse-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.805 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-mirror-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: rbd-nbd-2:18.0.0-4479.g8627af3e.el8.x86_64 2023-06-23T18:12:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: socat-1.7.4.1-1.el8.x86_64 2023-06-23T18:12:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: thrift-0.13.0-2.el8.x86_64 2023-06-23T18:12:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: uuid-1.6.2-43.el8.x86_64 2023-06-23T18:12:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-06-23T18:12:47.806 INFO:teuthology.orchestra.run.smithi017.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-06-23T18:12:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-06-23T18:12:47.807 INFO:teuthology.orchestra.run.smithi017.stdout: 2023-06-23T18:12:47.807 INFO:teuthology.orchestra.run.smithi017.stdout:Complete! 2023-06-23T18:12:47.972 DEBUG:teuthology.parallel:result is None 2023-06-23T18:12:47.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:12:48.201 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T18:12:48.231 INFO:teuthology.orchestra.run.smithi002.stdout:18.0.0-4479.g8627af3e.el8 2023-06-23T18:12:48.231 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4479.g8627af3e.el8 2023-06-23T18:12:48.232 INFO:teuthology.task.install:The correct ceph version 18.0.0-4479.g8627af3e is installed. 2023-06-23T18:12:48.232 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:12:48.367 DEBUG:teuthology.orchestra.run.smithi017:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T18:12:48.392 INFO:teuthology.orchestra.run.smithi017.stdout:18.0.0-4479.g8627af3e.el8 2023-06-23T18:12:48.392 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4479.g8627af3e.el8 2023-06-23T18:12:48.393 INFO:teuthology.task.install:The correct ceph version 18.0.0-4479.g8627af3e is installed. 2023-06-23T18:12:48.393 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:12:48.603 DEBUG:teuthology.orchestra.run.smithi082:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-06-23T18:12:48.631 INFO:teuthology.orchestra.run.smithi082.stdout:18.0.0-4479.g8627af3e.el8 2023-06-23T18:12:48.632 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4479.g8627af3e.el8 2023-06-23T18:12:48.632 INFO:teuthology.task.install:The correct ceph version 18.0.0-4479.g8627af3e is installed. 2023-06-23T18:12:48.632 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-06-23T18:12:48.633 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:12:48.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T18:12:48.682 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:12:48.683 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T18:12:48.733 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:12:48.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-06-23T18:12:48.779 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-06-23T18:12:48.780 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:12:48.780 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T18:12:48.814 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T18:12:48.883 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:12:48.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T18:12:48.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T18:12:48.989 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:12:48.989 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2023-06-23T18:12:49.022 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-06-23T18:12:49.092 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-06-23T18:12:49.093 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:12:49.093 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T18:12:49.126 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T18:12:49.197 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:12:49.198 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T18:12:49.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T18:12:49.306 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:12:49.306 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2023-06-23T18:12:49.338 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-06-23T18:12:49.409 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-06-23T18:12:49.410 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:12:49.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T18:12:49.443 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T18:12:49.516 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:12:49.517 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T18:12:49.551 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T18:12:49.623 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:12:49.623 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2023-06-23T18:12:49.656 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-06-23T18:12:49.727 INFO:teuthology.run_tasks:Running task cephadm... 2023-06-23T18:12:49.823 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-06-23T18:12:49.824 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:12:49.834 INFO:tasks.cephadm:Cluster fsid is 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:12:49.835 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-06-23T18:12:49.836 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.2', 'mon.b': '172.21.15.17', 'mon.c': '172.21.15.82'} 2023-06-23T18:12:49.836 INFO:tasks.cephadm:First mon is mon.a on smithi002 2023-06-23T18:12:49.836 INFO:tasks.cephadm:First mgr is x 2023-06-23T18:12:49.837 INFO:tasks.cephadm:Normalizing hostnames... 2023-06-23T18:12:49.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo hostname $(hostname -s) 2023-06-23T18:12:49.872 DEBUG:teuthology.orchestra.run.smithi017:> sudo hostname $(hostname -s) 2023-06-23T18:12:49.906 DEBUG:teuthology.orchestra.run.smithi082:> sudo hostname $(hostname -s) 2023-06-23T18:12:49.939 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-06-23T18:12:49.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:12:50.172 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8627af3e0adcb765a3c249fcc209cba9f4873e1b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/70742/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4479-g8627af3e', 'node_name': '172.21.4.63+confusa01', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4479.g8627af3e'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-06-21 20:58:11.415463', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-improvements-authmon', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}] 2023-06-23T18:12:50.424 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-rishabh-improvements-authmon, sha1 8627af3e0adcb765a3c249fcc209cba9f4873e1b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=8627af3e0adcb765a3c249fcc209cba9f4873e1b 2023-06-23T18:12:50.425 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/x86_64/flavors/default/cephadm 2023-06-23T18:12:50.425 DEBUG:teuthology.orchestra.run.smithi002:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:50.927 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-rw-r--. 1 ubuntu ubuntu 653374 Jun 23 18:12 /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:50.927 DEBUG:teuthology.orchestra.run.smithi017:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:51.439 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-rw-r--. 1 ubuntu ubuntu 653374 Jun 23 18:12 /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:51.440 DEBUG:teuthology.orchestra.run.smithi082:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-rishabh-improvements-authmon/8627af3e0adcb765a3c249fcc209cba9f4873e1b/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:52.152 INFO:teuthology.orchestra.run.smithi082.stdout:-rw-rw-r--. 1 ubuntu ubuntu 653374 Jun 23 18:12 /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:52.153 DEBUG:teuthology.orchestra.run.smithi002:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:52.175 DEBUG:teuthology.orchestra.run.smithi017:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:52.198 DEBUG:teuthology.orchestra.run.smithi082:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-06-23T18:12:52.235 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b on all hosts... 2023-06-23T18:12:52.235 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b pull 2023-06-23T18:12:52.238 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b pull 2023-06-23T18:12:52.240 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b pull 2023-06-23T18:12:55.295 INFO:teuthology.orchestra.run.smithi082.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b... 2023-06-23T18:12:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b... 2023-06-23T18:12:57.780 INFO:teuthology.orchestra.run.smithi017.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b... 2023-06-23T18:13:24.074 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:13:24.074 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:13:24.074 INFO:teuthology.orchestra.run.smithi002.stdout: "image_id": "2a05a4fe3a20fa291c55052f289a40d5bb240db002532fd9b1c74e32c8a35da2", 2023-06-23T18:13:24.074 INFO:teuthology.orchestra.run.smithi002.stdout: "repo_digests": [ 2023-06-23T18:13:24.074 INFO:teuthology.orchestra.run.smithi002.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f034b9f85e749b7046627792066e5ee84aa168b75729c2e36bfaaf2e3881c763" 2023-06-23T18:13:24.074 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:13:24.075 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:13:25.794 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-06-23T18:13:25.795 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:13:25.795 INFO:teuthology.orchestra.run.smithi082.stdout: "image_id": "2a05a4fe3a20fa291c55052f289a40d5bb240db002532fd9b1c74e32c8a35da2", 2023-06-23T18:13:25.795 INFO:teuthology.orchestra.run.smithi082.stdout: "repo_digests": [ 2023-06-23T18:13:25.795 INFO:teuthology.orchestra.run.smithi082.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f034b9f85e749b7046627792066e5ee84aa168b75729c2e36bfaaf2e3881c763" 2023-06-23T18:13:25.795 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2023-06-23T18:13:25.796 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-06-23T18:13:28.772 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-06-23T18:13:28.772 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:13:28.772 INFO:teuthology.orchestra.run.smithi017.stdout: "image_id": "2a05a4fe3a20fa291c55052f289a40d5bb240db002532fd9b1c74e32c8a35da2", 2023-06-23T18:13:28.772 INFO:teuthology.orchestra.run.smithi017.stdout: "repo_digests": [ 2023-06-23T18:13:28.772 INFO:teuthology.orchestra.run.smithi017.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f034b9f85e749b7046627792066e5ee84aa168b75729c2e36bfaaf2e3881c763" 2023-06-23T18:13:28.773 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2023-06-23T18:13:28.773 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-06-23T18:13:28.793 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph 2023-06-23T18:13:28.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph 2023-06-23T18:13:28.861 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph 2023-06-23T18:13:28.899 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /etc/ceph 2023-06-23T18:13:28.933 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /etc/ceph 2023-06-23T18:13:28.962 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /etc/ceph 2023-06-23T18:13:28.993 INFO:tasks.cephadm:Writing seed config... 2023-06-23T18:13:28.994 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-06-23T18:13:28.995 INFO:tasks.cephadm: override: [client] debug client = 20 2023-06-23T18:13:28.995 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-06-23T18:13:28.995 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-06-23T18:13:28.995 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-06-23T18:13:28.996 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-06-23T18:13:28.996 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-06-23T18:13:28.996 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-06-23T18:13:28.996 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-06-23T18:13:28.997 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-06-23T18:13:28.997 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-06-23T18:13:28.997 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-06-23T18:13:28.997 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-06-23T18:13:28.997 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-06-23T18:13:28.998 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-06-23T18:13:28.998 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-06-23T18:13:28.998 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-06-23T18:13:28.998 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-06-23T18:13:28.998 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-06-23T18:13:28.999 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-06-23T18:13:28.999 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-06-23T18:13:28.999 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-06-23T18:13:28.999 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-06-23T18:13:29.000 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-06-23T18:13:29.000 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-06-23T18:13:29.000 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-06-23T18:13:29.000 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-06-23T18:13:29.000 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-06-23T18:13:29.001 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-06-23T18:13:29.001 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-06-23T18:13:29.001 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-06-23T18:13:29.001 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-06-23T18:13:29.002 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-06-23T18:13:29.002 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-06-23T18:13:29.002 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-06-23T18:13:29.002 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-06-23T18:13:29.002 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-06-23T18:13:29.003 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-06-23T18:13:29.003 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-06-23T18:13:29.003 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-06-23T18:13:29.003 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-06-23T18:13:29.003 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-06-23T18:13:29.004 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-06-23T18:13:29.005 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:13:29.005 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-06-23T18:13:29.026 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 902733cc-11f1-11ee-9b2b-001a4aab830c [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-06-23T18:13:29.026 DEBUG:teuthology.orchestra.run.smithi002:mon.a> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.a.service 2023-06-23T18:13:29.069 DEBUG:teuthology.orchestra.run.smithi002:mgr.x> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.x.service 2023-06-23T18:13:29.100 INFO:journalctl@ceph.mon.a.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:13:29.101 INFO:tasks.cephadm:Bootstrapping... 2023-06-23T18:13:29.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b -v bootstrap --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.2 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:13:29.131 INFO:journalctl@ceph.mgr.x.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:13:29.274 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2023-06-23T18:13:29.274 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b', '-v', 'bootstrap', '--fsid', '902733cc-11f1-11ee-9b2b-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.2', '--skip-admin-label'] 2023-06-23T18:13:29.427 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.1.1 2023-06-23T18:13:29.428 INFO:teuthology.orchestra.run.smithi002.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-06-23T18:13:29.428 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying podman|docker is present... 2023-06-23T18:13:29.574 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.1.1 2023-06-23T18:13:29.575 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying lvm2 is present... 2023-06-23T18:13:29.575 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying time synchronization is in place... 2023-06-23T18:13:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-06-23T18:13:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-06-23T18:13:29.588 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-06-23T18:13:29.588 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-06-23T18:13:29.594 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2023-06-23T18:13:29.600 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2023-06-23T18:13:29.600 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2023-06-23T18:13:29.601 INFO:teuthology.orchestra.run.smithi002.stdout:Repeating the final host check... 2023-06-23T18:13:29.742 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 4.1.1 2023-06-23T18:13:29.742 INFO:teuthology.orchestra.run.smithi002.stdout:podman (/bin/podman) version 4.1.1 is present 2023-06-23T18:13:29.742 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl is present 2023-06-23T18:13:29.742 INFO:teuthology.orchestra.run.smithi002.stdout:lvcreate is present 2023-06-23T18:13:29.748 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-06-23T18:13:29.749 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-06-23T18:13:29.754 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-06-23T18:13:29.754 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-06-23T18:13:29.760 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout enabled 2023-06-23T18:13:29.769 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout active 2023-06-23T18:13:29.769 INFO:teuthology.orchestra.run.smithi002.stdout:Unit chronyd.service is enabled and running 2023-06-23T18:13:29.769 INFO:teuthology.orchestra.run.smithi002.stdout:Host looks OK 2023-06-23T18:13:29.769 INFO:teuthology.orchestra.run.smithi002.stdout:Cluster fsid: 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:13:29.770 INFO:teuthology.orchestra.run.smithi002.stdout:Acquiring lock 139769948662248 on /run/cephadm/902733cc-11f1-11ee-9b2b-001a4aab830c.lock 2023-06-23T18:13:29.770 INFO:teuthology.orchestra.run.smithi002.stdout:Lock 139769948662248 acquired on /run/cephadm/902733cc-11f1-11ee-9b2b-001a4aab830c.lock 2023-06-23T18:13:29.770 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 3300 ... 2023-06-23T18:13:29.770 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying IP 172.21.15.2 port 6789 ... 2023-06-23T18:13:29.770 INFO:teuthology.orchestra.run.smithi002.stdout:Base mon IP(s) is [172.21.15.2:3300, 172.21.15.2:6789], mon addrv is [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2023-06-23T18:13:29.771 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-06-23T18:13:29.771 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.2 metric 100 2023-06-23T18:13:29.774 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-06-23T18:13:29.774 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-06-23T18:13:29.774 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2023-06-23T18:13:29.776 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-06-23T18:13:29.776 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-06-23T18:13:29.777 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-06-23T18:13:29.777 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-06-23T18:13:29.777 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1920/64 scope link 2023-06-23T18:13:29.777 INFO:teuthology.orchestra.run.smithi002.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-06-23T18:13:29.778 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2023-06-23T18:13:29.778 INFO:teuthology.orchestra.run.smithi002.stdout:Mon IP `172.21.15.2` is in CIDR network `172.21.0.0/20` 2023-06-23T18:13:29.778 INFO:teuthology.orchestra.run.smithi002.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-06-23T18:13:29.778 INFO:teuthology.orchestra.run.smithi002.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-06-23T18:13:29.779 INFO:teuthology.orchestra.run.smithi002.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b... 2023-06-23T18:13:30.230 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stdout 2a05a4fe3a20fa291c55052f289a40d5bb240db002532fd9b1c74e32c8a35da2 2023-06-23T18:13:30.231 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b... 2023-06-23T18:13:30.231 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Getting image source signatures 2023-06-23T18:13:30.231 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:adef7e0539e16e4068f7edacedd5197cf9927314f501ac2b79a11598e7d382eb 2023-06-23T18:13:30.232 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying blob sha256:3dbfccf0662dd88e0c2efa9d59916ea5c98f7c5a05bbe18e86c2dc32f0494158 2023-06-23T18:13:30.232 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Copying config sha256:2a05a4fe3a20fa291c55052f289a40d5bb240db002532fd9b1c74e32c8a35da2 2023-06-23T18:13:30.232 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Writing manifest to image destination 2023-06-23T18:13:30.232 INFO:teuthology.orchestra.run.smithi002.stdout:/bin/podman: stderr Storing signatures 2023-06-23T18:13:31.175 INFO:teuthology.orchestra.run.smithi002.stdout:ceph: stdout ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:13:31.175 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph version: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev) 2023-06-23T18:13:31.176 INFO:teuthology.orchestra.run.smithi002.stdout:Extracting ceph user uid/gid from container image... 2023-06-23T18:13:31.824 INFO:teuthology.orchestra.run.smithi002.stdout:stat: stdout 167 167 2023-06-23T18:13:31.825 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial keys... 2023-06-23T18:13:32.571 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDM4JVkidSCDBAA/iSocpGWy3XwzSxA4izErQ== 2023-06-23T18:13:33.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDM4JVkT0i2ORAA2iYmWvM/h6pCNaDW/hlGtQ== 2023-06-23T18:13:35.970 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph-authtool: stdout AQDO4JVk217wKhAAGBjz1IJalc4yj4kxWt9uQw== 2023-06-23T18:13:35.971 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial monmap... 2023-06-23T18:13:36.729 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-06-23T18:13:36.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-06-23T18:13:36.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:13:36.730 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-06-23T18:13:36.730 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool for a [v2:172.21.15.2:3300,v1:172.21.15.2:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-06-23T18:13:36.730 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = pacific 2023-06-23T18:13:36.731 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: set fsid to 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:13:36.731 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-06-23T18:13:36.731 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:13:36.731 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mon... 2023-06-23T18:13:37.586 INFO:teuthology.orchestra.run.smithi002.stdout:create mon.a on 2023-06-23T18:13:37.719 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-06-23T18:13:37.812 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-06-23T18:13:37.938 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c.target → /etc/systemd/system/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c.target. 2023-06-23T18:13:37.939 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c.target → /etc/systemd/system/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c.target. 2023-06-23T18:13:38.098 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.a 2023-06-23T18:13:38.098 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.a.service: Unit ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.a.service not loaded. 2023-06-23T18:13:38.189 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c.target.wants/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@.service. 2023-06-23T18:13:38.465 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:38 smithi002 systemd[1]: Starting Ceph mon.a for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:13:38.725 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:38 smithi002 podman[95408]: 2023-06-23T18:13:38.798 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-23T18:13:38.798 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2023-06-23T18:13:38.805 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-23T18:13:38.805 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-06-23T18:13:38.805 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2023-06-23T18:13:38.806 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2023-06-23T18:13:38.806 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon to start... 2023-06-23T18:13:38.807 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mon... 2023-06-23T18:13:39.042 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:38 smithi002 bash[95408]: 5b6e0d7ed91f3d770213ddf088abc5658dfbcf43cdf16d6d1710c854100d90d8 2023-06-23T18:13:39.042 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:38 smithi002 ceph-mon[95448]: mkfs 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:13:39.042 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:38 smithi002 ceph-mon[95448]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-23T18:13:39.042 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:38 smithi002 systemd[1]: Started Ceph mon.a for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:13:39.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout cluster: 2023-06-23T18:13:39.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout id: 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:13:39.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-06-23T18:13:39.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:39.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout services: 2023-06-23T18:13:39.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.697388s) 2023-06-23T18:13:39.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-06-23T18:13:39.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-06-23T18:13:39.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:39.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout data: 2023-06-23T18:13:39.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-06-23T18:13:39.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-06-23T18:13:39.866 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-06-23T18:13:39.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout pgs: 2023-06-23T18:13:39.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:39.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.298+0000 7eff6ae55700 1 Processor -- start 2023-06-23T18:13:39.867 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.298+0000 7eff6ae55700 1 -- start start 2023-06-23T18:13:39.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.299+0000 7eff6ae55700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff64105160 0x7eff64105570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:39.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.299+0000 7eff6ae55700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64105ab0 con 0x7eff64105160 2023-06-23T18:13:39.868 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.299+0000 7eff68bf1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff64105160 0x7eff64105570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:39.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.299+0000 7eff68bf1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff64105160 0x7eff64105570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51080/0 (socket says 172.21.15.2:51080) 2023-06-23T18:13:39.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.299+0000 7eff68bf1700 1 -- 172.21.15.2:0/1556692614 learned_addr learned my addr 172.21.15.2:0/1556692614 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:39.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.299+0000 7eff68bf1700 1 -- 172.21.15.2:0/1556692614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff64105bf0 con 0x7eff64105160 2023-06-23T18:13:39.869 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff68bf1700 1 --2- 172.21.15.2:0/1556692614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff64105160 0x7eff64105570 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7eff580098a0 tx=0x7eff58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcb822ecc7323c1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:39.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff637fe700 1 -- 172.21.15.2:0/1556692614 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff58004160 con 0x7eff64105160 2023-06-23T18:13:39.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff637fe700 1 -- 172.21.15.2:0/1556692614 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff580042c0 con 0x7eff64105160 2023-06-23T18:13:39.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff637fe700 1 -- 172.21.15.2:0/1556692614 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff580045a0 con 0x7eff64105160 2023-06-23T18:13:39.870 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff6ae55700 1 -- 172.21.15.2:0/1556692614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff64105160 msgr2=0x7eff64105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:39.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff6ae55700 1 --2- 172.21.15.2:0/1556692614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff64105160 0x7eff64105570 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7eff580098a0 tx=0x7eff58009bb0 comp rx=0 tx=0).stop 2023-06-23T18:13:39.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff6ae55700 1 -- 172.21.15.2:0/1556692614 shutdown_connections 2023-06-23T18:13:39.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff6ae55700 1 --2- 172.21.15.2:0/1556692614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff64105160 0x7eff64105570 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:39.871 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.300+0000 7eff6ae55700 1 -- 172.21.15.2:0/1556692614 >> 172.21.15.2:0/1556692614 conn(0x7eff64100810 msgr2=0x7eff64102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:39.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff6ae55700 1 -- 172.21.15.2:0/1556692614 shutdown_connections 2023-06-23T18:13:39.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff6ae55700 1 -- 172.21.15.2:0/1556692614 wait complete. 2023-06-23T18:13:39.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff6ae55700 1 Processor -- start 2023-06-23T18:13:39.872 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff6ae55700 1 -- start start 2023-06-23T18:13:39.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff6ae55700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff641988d0 0x7eff64198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:39.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff6ae55700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64199220 con 0x7eff641988d0 2023-06-23T18:13:39.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff68bf1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff641988d0 0x7eff64198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:39.873 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff68bf1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff641988d0 0x7eff64198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51082/0 (socket says 172.21.15.2:51082) 2023-06-23T18:13:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.301+0000 7eff68bf1700 1 -- 172.21.15.2:0/2996112975 learned_addr learned my addr 172.21.15.2:0/2996112975 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.302+0000 7eff68bf1700 1 -- 172.21.15.2:0/2996112975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58009550 con 0x7eff641988d0 2023-06-23T18:13:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.302+0000 7eff68bf1700 1 --2- 172.21.15.2:0/2996112975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff641988d0 0x7eff64198ce0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7eff58008000 tx=0x7eff58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.302+0000 7eff61ffb700 1 -- 172.21.15.2:0/2996112975 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff58003dc0 con 0x7eff641988d0 2023-06-23T18:13:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.302+0000 7eff61ffb700 1 -- 172.21.15.2:0/2996112975 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff58011630 con 0x7eff641988d0 2023-06-23T18:13:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.302+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff64199420 con 0x7eff641988d0 2023-06-23T18:13:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.302+0000 7eff61ffb700 1 -- 172.21.15.2:0/2996112975 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff58011910 con 0x7eff641988d0 2023-06-23T18:13:39.874 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.302+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6419c080 con 0x7eff641988d0 2023-06-23T18:13:39.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.303+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff64192050 con 0x7eff641988d0 2023-06-23T18:13:39.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.303+0000 7eff61ffb700 1 -- 172.21.15.2:0/2996112975 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7eff58022070 con 0x7eff641988d0 2023-06-23T18:13:39.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.303+0000 7eff61ffb700 1 -- 172.21.15.2:0/2996112975 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7eff5801bdd0 con 0x7eff641988d0 2023-06-23T18:13:39.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.306+0000 7eff61ffb700 1 -- 172.21.15.2:0/2996112975 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7eff58006d80 con 0x7eff641988d0 2023-06-23T18:13:39.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.437+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7eff6402ca40 con 0x7eff641988d0 2023-06-23T18:13:39.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.438+0000 7eff61ffb700 1 -- 172.21.15.2:0/2996112975 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7eff58011d70 con 0x7eff641988d0 2023-06-23T18:13:39.875 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.439+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff641988d0 msgr2=0x7eff64198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:39.876 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.439+0000 7eff6ae55700 1 --2- 172.21.15.2:0/2996112975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff641988d0 0x7eff64198ce0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7eff58008000 tx=0x7eff58004750 comp rx=0 tx=0).stop 2023-06-23T18:13:39.876 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.439+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 shutdown_connections 2023-06-23T18:13:39.876 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.439+0000 7eff6ae55700 1 --2- 172.21.15.2:0/2996112975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff641988d0 0x7eff64198ce0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:39.876 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.439+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 >> 172.21.15.2:0/2996112975 conn(0x7eff64100810 msgr2=0x7eff6418cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:39.876 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.439+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 shutdown_connections 2023-06-23T18:13:39.877 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:39.439+0000 7eff6ae55700 1 -- 172.21.15.2:0/2996112975 wait complete. 2023-06-23T18:13:39.877 INFO:teuthology.orchestra.run.smithi002.stdout:mon is available 2023-06-23T18:13:39.877 INFO:teuthology.orchestra.run.smithi002.stdout:Assimilating anything we can from ceph.conf... 2023-06-23T18:13:40.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:39 smithi002 ceph-mon[95448]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-23T18:13:40.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:39 smithi002 ceph-mon[95448]: monmap e1: 1 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0]} removed_ranks: {} 2023-06-23T18:13:40.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:39 smithi002 ceph-mon[95448]: fsmap 2023-06-23T18:13:40.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:39 smithi002 ceph-mon[95448]: osdmap e1: 0 total, 0 up, 0 in 2023-06-23T18:13:40.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:39 smithi002 ceph-mon[95448]: mgrmap e1: no daemons active 2023-06-23T18:13:40.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:39 smithi002 ceph-mon[95448]: from='client.? 172.21.15.2:0/2996112975' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-06-23T18:13:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:40 smithi002 ceph-mon[95448]: from='client.? 172.21.15.2:0/1349859827' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-06-23T18:13:40.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:40 smithi002 ceph-mon[95448]: from='client.? 172.21.15.2:0/1349859827' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-06-23T18:13:41.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:41.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2023-06-23T18:13:41.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:13:41.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.2:3300,v1:172.21.15.2:6789] 2023-06-23T18:13:41.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-06-23T18:13:41.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-06-23T18:13:41.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-06-23T18:13:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-06-23T18:13:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2023-06-23T18:13:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-06-23T18:13:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2023-06-23T18:13:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-06-23T18:13:41.010 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-06-23T18:13:41.010 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-06-23T18:13:41.010 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-06-23T18:13:41.010 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-06-23T18:13:41.010 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.383+0000 7f4ca520d700 1 Processor -- start 2023-06-23T18:13:41.010 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.383+0000 7f4ca520d700 1 -- start start 2023-06-23T18:13:41.010 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.383+0000 7f4ca520d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca0104470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:41.011 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.383+0000 7f4ca520d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca01049b0 con 0x7f4ca0104060 2023-06-23T18:13:41.011 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.384+0000 7f4c9ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca0104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:41.011 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.384+0000 7f4c9ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca0104470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51084/0 (socket says 172.21.15.2:51084) 2023-06-23T18:13:41.011 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.384+0000 7f4c9ed9d700 1 -- 172.21.15.2:0/2849762807 learned_addr learned my addr 172.21.15.2:0/2849762807 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:41.011 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.384+0000 7f4c9ed9d700 1 -- 172.21.15.2:0/2849762807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ca0104af0 con 0x7f4ca0104060 2023-06-23T18:13:41.011 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.384+0000 7f4c9ed9d700 1 --2- 172.21.15.2:0/2849762807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca0104470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4c940098a0 tx=0x7f4c94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa744a76e7bc3e70 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:41.011 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.384+0000 7f4c9dd9b700 1 -- 172.21.15.2:0/2849762807 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c94004160 con 0x7f4ca0104060 2023-06-23T18:13:41.012 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.384+0000 7f4c9dd9b700 1 -- 172.21.15.2:0/2849762807 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c940042c0 con 0x7f4ca0104060 2023-06-23T18:13:41.012 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.384+0000 7f4c9dd9b700 1 -- 172.21.15.2:0/2849762807 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c940045a0 con 0x7f4ca0104060 2023-06-23T18:13:41.012 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.385+0000 7f4ca520d700 1 -- 172.21.15.2:0/2849762807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 msgr2=0x7f4ca0104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:41.012 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.385+0000 7f4ca520d700 1 --2- 172.21.15.2:0/2849762807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca0104470 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4c940098a0 tx=0x7f4c94009bb0 comp rx=0 tx=0).stop 2023-06-23T18:13:41.012 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.385+0000 7f4ca520d700 1 -- 172.21.15.2:0/2849762807 shutdown_connections 2023-06-23T18:13:41.012 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.385+0000 7f4ca520d700 1 --2- 172.21.15.2:0/2849762807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca0104470 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:41.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.385+0000 7f4ca520d700 1 -- 172.21.15.2:0/2849762807 >> 172.21.15.2:0/2849762807 conn(0x7f4ca00ff6f0 msgr2=0x7f4ca0101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:41.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.385+0000 7f4ca520d700 1 -- 172.21.15.2:0/2849762807 shutdown_connections 2023-06-23T18:13:41.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.385+0000 7f4ca520d700 1 -- 172.21.15.2:0/2849762807 wait complete. 2023-06-23T18:13:41.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.386+0000 7f4ca520d700 1 Processor -- start 2023-06-23T18:13:41.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.386+0000 7f4ca520d700 1 -- start start 2023-06-23T18:13:41.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.386+0000 7f4ca520d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:41.013 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.386+0000 7f4ca520d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca019ae20 con 0x7f4ca0104060 2023-06-23T18:13:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.386+0000 7f4c9ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.386+0000 7f4c9ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51086/0 (socket says 172.21.15.2:51086) 2023-06-23T18:13:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.386+0000 7f4c9ed9d700 1 -- 172.21.15.2:0/1349859827 learned_addr learned my addr 172.21.15.2:0/1349859827 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4c9ed9d700 1 -- 172.21.15.2:0/1349859827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009550 con 0x7f4ca0104060 2023-06-23T18:13:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4c9ed9d700 1 --2- 172.21.15.2:0/1349859827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca019a8e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0104d90 tx=0x7f4c94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4c87fff700 1 -- 172.21.15.2:0/1349859827 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c94011630 con 0x7f4ca0104060 2023-06-23T18:13:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4c87fff700 1 -- 172.21.15.2:0/1349859827 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c94011790 con 0x7f4ca0104060 2023-06-23T18:13:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca019b020 con 0x7f4ca0104060 2023-06-23T18:13:41.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4c87fff700 1 -- 172.21.15.2:0/1349859827 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c94011a30 con 0x7f4ca0104060 2023-06-23T18:13:41.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca019b440 con 0x7f4ca0104060 2023-06-23T18:13:41.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4c87fff700 1 -- 172.21.15.2:0/1349859827 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4c94022070 con 0x7f4ca0104060 2023-06-23T18:13:41.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.387+0000 7f4c87fff700 1 -- 172.21.15.2:0/1349859827 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4c9401c450 con 0x7f4ca0104060 2023-06-23T18:13:41.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.388+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca0194e00 con 0x7f4ca0104060 2023-06-23T18:13:41.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.390+0000 7f4c87fff700 1 -- 172.21.15.2:0/1349859827 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4c9401b450 con 0x7f4ca0104060 2023-06-23T18:13:41.015 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.517+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4ca002ca90 con 0x7f4ca0104060 2023-06-23T18:13:41.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.520+0000 7f4c87fff700 1 -- 172.21.15.2:0/1349859827 <== mon.0 v2:172.21.15.2:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4c9401b5f0 con 0x7f4ca0104060 2023-06-23T18:13:41.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.520+0000 7f4c87fff700 1 -- 172.21.15.2:0/1349859827 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f4c94037540 con 0x7f4ca0104060 2023-06-23T18:13:41.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.521+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 msgr2=0x7f4ca019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:41.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.521+0000 7f4ca520d700 1 --2- 172.21.15.2:0/1349859827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca019a8e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4ca0104d90 tx=0x7f4c94004750 comp rx=0 tx=0).stop 2023-06-23T18:13:41.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.521+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 shutdown_connections 2023-06-23T18:13:41.016 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.521+0000 7f4ca520d700 1 --2- 172.21.15.2:0/1349859827 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ca0104060 0x7f4ca019a8e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:41.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.521+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 >> 172.21.15.2:0/1349859827 conn(0x7f4ca00ff6f0 msgr2=0x7f4ca0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:41.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.521+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 shutdown_connections 2023-06-23T18:13:41.017 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:40.521+0000 7f4ca520d700 1 -- 172.21.15.2:0/1349859827 wait complete. 2023-06-23T18:13:41.017 INFO:teuthology.orchestra.run.smithi002.stdout:Generating new minimal ceph.conf... 2023-06-23T18:13:42.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:41 smithi002 ceph-mon[95448]: from='client.? 172.21.15.2:0/1522804373' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:13:42.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.507+0000 7f1c29e29700 1 Processor -- start 2023-06-23T18:13:42.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.507+0000 7f1c29e29700 1 -- start start 2023-06-23T18:13:42.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.507+0000 7f1c29e29700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:42.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c29e29700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2407a860 con 0x7f1c2407be20 2023-06-23T18:13:42.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c237fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:42.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c237fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51088/0 (socket says 172.21.15.2:51088) 2023-06-23T18:13:42.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c237fe700 1 -- 172.21.15.2:0/1938436905 learned_addr learned my addr 172.21.15.2:0/1938436905 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:42.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c237fe700 1 -- 172.21.15.2:0/1938436905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2407a9a0 con 0x7f1c2407be20 2023-06-23T18:13:42.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c237fe700 1 --2- 172.21.15.2:0/1938436905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2407a320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1c140098a0 tx=0x7f1c14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e488539fd23e38a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:42.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c227fc700 1 -- 172.21.15.2:0/1938436905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c14004160 con 0x7f1c2407be20 2023-06-23T18:13:42.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c227fc700 1 -- 172.21.15.2:0/1938436905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f1c140042c0 con 0x7f1c2407be20 2023-06-23T18:13:42.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.508+0000 7f1c227fc700 1 -- 172.21.15.2:0/1938436905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c14011940 con 0x7f1c2407be20 2023-06-23T18:13:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.509+0000 7f1c29e29700 1 -- 172.21.15.2:0/1938436905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 msgr2=0x7f1c2407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.509+0000 7f1c29e29700 1 --2- 172.21.15.2:0/1938436905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2407a320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1c140098a0 tx=0x7f1c14009bb0 comp rx=0 tx=0).stop 2023-06-23T18:13:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.509+0000 7f1c29e29700 1 -- 172.21.15.2:0/1938436905 shutdown_connections 2023-06-23T18:13:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.509+0000 7f1c29e29700 1 --2- 172.21.15.2:0/1938436905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2407a320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.509+0000 7f1c29e29700 1 -- 172.21.15.2:0/1938436905 >> 172.21.15.2:0/1938436905 conn(0x7f1c24100810 msgr2=0x7f1c24102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.509+0000 7f1c29e29700 1 -- 172.21.15.2:0/1938436905 shutdown_connections 2023-06-23T18:13:42.153 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.509+0000 7f1c29e29700 1 -- 172.21.15.2:0/1938436905 wait complete. 2023-06-23T18:13:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.510+0000 7f1c29e29700 1 Processor -- start 2023-06-23T18:13:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.510+0000 7f1c29e29700 1 -- start start 2023-06-23T18:13:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.510+0000 7f1c29e29700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2419b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.510+0000 7f1c29e29700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2419b5a0 con 0x7f1c2407be20 2023-06-23T18:13:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.510+0000 7f1c237fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2419b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.510+0000 7f1c237fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2419b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51090/0 (socket says 172.21.15.2:51090) 2023-06-23T18:13:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.510+0000 7f1c237fe700 1 -- 172.21.15.2:0/1522804373 learned_addr learned my addr 172.21.15.2:0/1522804373 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:42.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.510+0000 7f1c237fe700 1 -- 172.21.15.2:0/1522804373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c14009550 con 0x7f1c2407be20 2023-06-23T18:13:42.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.511+0000 7f1c237fe700 1 --2- 172.21.15.2:0/1522804373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2419b060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c14000c00 tx=0x7f1c14011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:42.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.511+0000 7f1c20ff9700 1 -- 172.21.15.2:0/1522804373 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c140036a0 con 0x7f1c2407be20 2023-06-23T18:13:42.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.511+0000 7f1c20ff9700 1 -- 172.21.15.2:0/1522804373 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f1c14003c70 con 0x7f1c2407be20 2023-06-23T18:13:42.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.511+0000 7f1c20ff9700 1 -- 172.21.15.2:0/1522804373 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c140193f0 con 0x7f1c2407be20 2023-06-23T18:13:42.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.511+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2419b7a0 con 0x7f1c2407be20 2023-06-23T18:13:42.155 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.511+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2419bbc0 con 0x7f1c2407be20 2023-06-23T18:13:42.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.511+0000 7f1c20ff9700 1 -- 172.21.15.2:0/1522804373 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1c14019920 con 0x7f1c2407be20 2023-06-23T18:13:42.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.511+0000 7f1c20ff9700 1 -- 172.21.15.2:0/1522804373 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1c1401a3d0 con 0x7f1c2407be20 2023-06-23T18:13:42.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.512+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c24061b00 con 0x7f1c2407be20 2023-06-23T18:13:42.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.514+0000 7f1c20ff9700 1 -- 172.21.15.2:0/1522804373 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f1c14029060 con 0x7f1c2407be20 2023-06-23T18:13:42.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.641+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f1c2419e600 con 0x7f1c2407be20 2023-06-23T18:13:42.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.641+0000 7f1c20ff9700 1 -- 172.21.15.2:0/1522804373 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f1c14033090 con 0x7f1c2407be20 2023-06-23T18:13:42.156 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.642+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 msgr2=0x7f1c2419b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:42.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.642+0000 7f1c29e29700 1 --2- 172.21.15.2:0/1522804373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2419b060 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c14000c00 tx=0x7f1c14011e70 comp rx=0 tx=0).stop 2023-06-23T18:13:42.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.642+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 shutdown_connections 2023-06-23T18:13:42.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.642+0000 7f1c29e29700 1 --2- 172.21.15.2:0/1522804373 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c2407be20 0x7f1c2419b060 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:42.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.642+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 >> 172.21.15.2:0/1522804373 conn(0x7f1c24100810 msgr2=0x7f1c24101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:42.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.642+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 shutdown_connections 2023-06-23T18:13:42.157 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:41.642+0000 7f1c29e29700 1 -- 172.21.15.2:0/1522804373 wait complete. 2023-06-23T18:13:42.157 INFO:teuthology.orchestra.run.smithi002.stdout:Restarting the monitor... 2023-06-23T18:13:42.503 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:42 smithi002 systemd[1]: Stopping Ceph mon.a for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:13:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:42 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-a[95424]: 2023-06-23T18:13:42.497+0000 7fc3121dc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:13:42.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:42 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-a[95424]: 2023-06-23T18:13:42.497+0000 7fc3121dc700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-06-23T18:13:43.136 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:42 smithi002 bash[95780]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-a 2023-06-23T18:13:43.490 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 systemd[1]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.a.service: Succeeded. 2023-06-23T18:13:43.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 systemd[1]: Stopped Ceph mon.a for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:13:43.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 systemd[1]: Starting Ceph mon.a for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:13:43.790 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 podman[95906]: 2023-06-23T18:13:43.790 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: set uid:gid to 167:167 (ceph:ceph) 2023-06-23T18:13:43.791 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev), process ceph-mon, pid 7 2023-06-23T18:13:43.791 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: pidfile_write: ignore empty --pid-file 2023-06-23T18:13:43.791 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: load: jerasure load: lrc 2023-06-23T18:13:43.791 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: RocksDB version: 7.9.2 2023-06-23T18:13:43.791 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Git sha 0 2023-06-23T18:13:43.791 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Compile date 2023-06-21 18:00:35 2023-06-23T18:13:43.791 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: DB SUMMARY 2023-06-23T18:13:43.792 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: DB Session ID: NE6AXKTI0N2QQX5OBHXD 2023-06-23T18:13:43.792 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: CURRENT file: CURRENT 2023-06-23T18:13:43.792 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: IDENTITY file: IDENTITY 2023-06-23T18:13:43.792 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-06-23T18:13:43.792 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-06-23T18:13:43.792 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98228 ; 2023-06-23T18:13:43.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.error_if_exists: 0 2023-06-23T18:13:43.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.create_if_missing: 0 2023-06-23T18:13:43.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.paranoid_checks: 1 2023-06-23T18:13:43.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.flush_verify_memtable_count: 1 2023-06-23T18:13:43.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-23T18:13:43.793 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-06-23T18:13:43.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.env: 0x56006fa03720 2023-06-23T18:13:43.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.fs: PosixFileSystem 2023-06-23T18:13:43.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.info_log: 0x560071639260 2023-06-23T18:13:43.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_file_opening_threads: 16 2023-06-23T18:13:43.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.statistics: (nil) 2023-06-23T18:13:43.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.use_fsync: 0 2023-06-23T18:13:43.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_log_file_size: 0 2023-06-23T18:13:43.794 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-23T18:13:43.795 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-23T18:13:43.795 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.keep_log_file_num: 1000 2023-06-23T18:13:43.795 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.recycle_log_file_num: 0 2023-06-23T18:13:43.795 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.allow_fallocate: 1 2023-06-23T18:13:43.795 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.allow_mmap_reads: 0 2023-06-23T18:13:43.795 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.allow_mmap_writes: 0 2023-06-23T18:13:43.795 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.use_direct_reads: 0 2023-06-23T18:13:43.795 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-23T18:13:43.796 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.create_missing_column_families: 0 2023-06-23T18:13:43.796 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.db_log_dir: 2023-06-23T18:13:43.796 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.wal_dir: 2023-06-23T18:13:43.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-23T18:13:43.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-23T18:13:43.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-23T18:13:43.797 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-23T18:13:43.798 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-23T18:13:43.798 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-23T18:13:43.798 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.advise_random_on_open: 1 2023-06-23T18:13:43.798 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.db_write_buffer_size: 0 2023-06-23T18:13:43.799 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.write_buffer_manager: 0x56007155a3c0 2023-06-23T18:13:43.799 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-23T18:13:43.799 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-23T18:13:43.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-23T18:13:43.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.rate_limiter: (nil) 2023-06-23T18:13:43.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-23T18:13:43.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.wal_recovery_mode: 2 2023-06-23T18:13:43.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.enable_thread_tracking: 0 2023-06-23T18:13:43.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.enable_pipelined_write: 0 2023-06-23T18:13:43.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.unordered_write: 0 2023-06-23T18:13:43.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-23T18:13:43.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-23T18:13:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-23T18:13:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-23T18:13:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.row_cache: None 2023-06-23T18:13:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.wal_filter: None 2023-06-23T18:13:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 bash[95906]: b4bd2bd2d7071a1593776276775b4ab6cbd70035cbd343f56c55ae93fd12863e 2023-06-23T18:13:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-23T18:13:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.allow_ingest_behind: 0 2023-06-23T18:13:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.two_write_queues: 0 2023-06-23T18:13:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.manual_wal_flush: 0 2023-06-23T18:13:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.wal_compression: 0 2023-06-23T18:13:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.atomic_flush: 0 2023-06-23T18:13:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-23T18:13:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-23T18:13:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-23T18:13:43.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.log_readahead_size: 0 2023-06-23T18:13:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-23T18:13:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.best_efforts_recovery: 0 2023-06-23T18:13:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-23T18:13:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-23T18:13:43.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.allow_data_in_errors: 0 2023-06-23T18:13:43.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.db_host_id: __hostname__ 2023-06-23T18:13:43.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.enforce_single_del_contracts: true 2023-06-23T18:13:43.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_background_jobs: 2 2023-06-23T18:13:43.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_background_compactions: -1 2023-06-23T18:13:43.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_subcompactions: 1 2023-06-23T18:13:43.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-23T18:13:43.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-23T18:13:43.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-23T18:13:43.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_total_wal_size: 0 2023-06-23T18:13:43.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-23T18:13:43.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-23T18:13:43.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-23T18:13:43.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-23T18:13:43.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_open_files: -1 2023-06-23T18:13:43.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bytes_per_sync: 0 2023-06-23T18:13:43.807 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-23T18:13:43.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-23T18:13:43.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_readahead_size: 0 2023-06-23T18:13:43.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_background_flushes: -1 2023-06-23T18:13:43.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Compression algorithms supported: 2023-06-23T18:13:43.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-23T18:13:43.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: kZSTD supported: 0 2023-06-23T18:13:43.808 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: kXpressCompression supported: 0 2023-06-23T18:13:43.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: kLZ4HCCompression supported: 1 2023-06-23T18:13:43.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: kZlibCompression supported: 1 2023-06-23T18:13:43.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: kSnappyCompression supported: 1 2023-06-23T18:13:43.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: kLZ4Compression supported: 1 2023-06-23T18:13:43.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: kBZip2Compression supported: 0 2023-06-23T18:13:43.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-23T18:13:43.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: DMutex implementation: pthread_mutex_t 2023-06-23T18:13:43.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-06-23T18:13:43.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-06-23T18:13:43.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-23T18:13:43.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.merge_operator: 2023-06-23T18:13:43.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_filter: None 2023-06-23T18:13:43.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_filter_factory: None 2023-06-23T18:13:43.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.sst_partitioner_factory: None 2023-06-23T18:13:43.810 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-23T18:13:43.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-23T18:13:43.811 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560071639360) 2023-06-23T18:13:43.811 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks: 1 2023-06-23T18:13:43.811 INFO:journalctl@ceph.mon.a.smithi002.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-23T18:13:43.811 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-23T18:13:43.811 INFO:journalctl@ceph.mon.a.smithi002.stdout: pin_top_level_index_and_filter: 1 2023-06-23T18:13:43.811 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_type: 0 2023-06-23T18:13:43.811 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_index_type: 0 2023-06-23T18:13:43.812 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_shortening: 1 2023-06-23T18:13:43.812 INFO:journalctl@ceph.mon.a.smithi002.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-23T18:13:43.812 INFO:journalctl@ceph.mon.a.smithi002.stdout: checksum: 4 2023-06-23T18:13:43.812 INFO:journalctl@ceph.mon.a.smithi002.stdout: no_block_cache: 0 2023-06-23T18:13:43.812 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache: 0x5600715f91f0 2023-06-23T18:13:43.813 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_name: BinnedLRUCache 2023-06-23T18:13:43.813 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_options: 2023-06-23T18:13:43.813 INFO:journalctl@ceph.mon.a.smithi002.stdout: capacity : 536870912 2023-06-23T18:13:43.813 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_shard_bits : 4 2023-06-23T18:13:43.813 INFO:journalctl@ceph.mon.a.smithi002.stdout: strict_capacity_limit : 0 2023-06-23T18:13:43.813 INFO:journalctl@ceph.mon.a.smithi002.stdout: high_pri_pool_ratio: 0.000 2023-06-23T18:13:43.813 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_cache_compressed: (nil) 2023-06-23T18:13:43.814 INFO:journalctl@ceph.mon.a.smithi002.stdout: persistent_cache: (nil) 2023-06-23T18:13:43.814 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size: 4096 2023-06-23T18:13:43.814 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_size_deviation: 10 2023-06-23T18:13:43.814 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_restart_interval: 16 2023-06-23T18:13:43.814 INFO:journalctl@ceph.mon.a.smithi002.stdout: index_block_restart_interval: 1 2023-06-23T18:13:43.814 INFO:journalctl@ceph.mon.a.smithi002.stdout: metadata_block_size: 4096 2023-06-23T18:13:43.814 INFO:journalctl@ceph.mon.a.smithi002.stdout: partition_filters: 0 2023-06-23T18:13:43.815 INFO:journalctl@ceph.mon.a.smithi002.stdout: use_delta_encoding: 1 2023-06-23T18:13:43.815 INFO:journalctl@ceph.mon.a.smithi002.stdout: filter_policy: bloomfilter 2023-06-23T18:13:43.815 INFO:journalctl@ceph.mon.a.smithi002.stdout: whole_key_filtering: 1 2023-06-23T18:13:43.815 INFO:journalctl@ceph.mon.a.smithi002.stdout: verify_compression: 0 2023-06-23T18:13:43.815 INFO:journalctl@ceph.mon.a.smithi002.stdout: read_amp_bytes_per_bit: 0 2023-06-23T18:13:43.815 INFO:journalctl@ceph.mon.a.smithi002.stdout: format_version: 5 2023-06-23T18:13:43.815 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_index_compression: 1 2023-06-23T18:13:43.815 INFO:journalctl@ceph.mon.a.smithi002.stdout: block_align: 0 2023-06-23T18:13:43.816 INFO:journalctl@ceph.mon.a.smithi002.stdout: max_auto_readahead_size: 262144 2023-06-23T18:13:43.816 INFO:journalctl@ceph.mon.a.smithi002.stdout: prepopulate_block_cache: 0 2023-06-23T18:13:43.816 INFO:journalctl@ceph.mon.a.smithi002.stdout: initial_auto_readahead_size: 8192 2023-06-23T18:13:43.816 INFO:journalctl@ceph.mon.a.smithi002.stdout: num_file_reads_for_auto_readahead: 2 2023-06-23T18:13:43.816 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.write_buffer_size: 33554432 2023-06-23T18:13:43.816 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_write_buffer_number: 2 2023-06-23T18:13:43.816 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression: NoCompression 2023-06-23T18:13:43.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression: Disabled 2023-06-23T18:13:43.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.prefix_extractor: nullptr 2023-06-23T18:13:43.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-23T18:13:43.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.num_levels: 7 2023-06-23T18:13:43.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-23T18:13:43.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-23T18:13:43.817 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-23T18:13:43.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-23T18:13:43.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-23T18:13:43.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-23T18:13:43.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-23T18:13:43.818 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-23T18:13:43.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-23T18:13:43.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-23T18:13:43.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-06-23T18:13:43.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-06-23T18:13:43.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-23T18:13:43.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.level: 32767 2023-06-23T18:13:43.819 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.strategy: 0 2023-06-23T18:13:43.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-23T18:13:43.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-23T18:13:43.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-06-23T18:13:43.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-23T18:13:43.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.enabled: false 2023-06-23T18:13:43.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-06-23T18:13:43.820 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-23T18:13:43.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-23T18:13:43.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-23T18:13:43.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.target_file_size_base: 67108864 2023-06-23T18:13:43.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-23T18:13:43.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-23T18:13:43.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-23T18:13:43.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-23T18:13:43.821 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-23T18:13:43.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-23T18:13:43.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-23T18:13:43.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-23T18:13:43.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-23T18:13:43.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-23T18:13:43.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-23T18:13:43.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-23T18:13:43.822 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-23T18:13:43.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-06-23T18:13:43.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.arena_block_size: 1048576 2023-06-23T18:13:43.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-23T18:13:43.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-23T18:13:43.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.disable_auto_compactions: 0 2023-06-23T18:13:43.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-23T18:13:43.823 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-23T18:13:43.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-23T18:13:43.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-23T18:13:43.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-23T18:13:43.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-23T18:13:43.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-23T18:13:43.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-23T18:13:43.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-23T18:13:43.824 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-23T18:13:43.825 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.table_properties_collectors: 2023-06-23T18:13:43.825 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.inplace_update_support: 0 2023-06-23T18:13:43.825 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-23T18:13:43.825 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-23T18:13:43.825 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-23T18:13:43.825 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-23T18:13:43.826 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.bloom_locality: 0 2023-06-23T18:13:43.826 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.max_successive_merges: 0 2023-06-23T18:13:43.826 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-23T18:13:43.826 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.paranoid_file_checks: 0 2023-06-23T18:13:43.826 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.force_consistency_checks: 1 2023-06-23T18:13:43.826 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.report_bg_io_stats: 0 2023-06-23T18:13:43.827 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.ttl: 2592000 2023-06-23T18:13:43.827 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-23T18:13:43.827 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-06-23T18:13:43.827 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-06-23T18:13:43.827 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.enable_blob_files: false 2023-06-23T18:13:43.827 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.min_blob_size: 0 2023-06-23T18:13:43.828 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.blob_file_size: 268435456 2023-06-23T18:13:43.828 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-23T18:13:43.828 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-23T18:13:43.828 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-23T18:13:43.828 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-06-23T18:13:43.828 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-06-23T18:13:43.828 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.blob_file_starting_level: 0 2023-06-23T18:13:43.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-06-23T18:13:43.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-06-23T18:13:43.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-06-23T18:13:43.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 48953f0c-a63f-462c-9f1a-c928d9b6d0de 2023-06-23T18:13:43.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687544023710083, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-06-23T18:13:43.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-06-23T18:13:43.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687544023711760, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93530, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 312, "table_properties": {"data_size": 91530, "index_size": 282, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13527, "raw_average_key_size": 49, "raw_value_size": 84149, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 271, "num_filter_entries": 271, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1687544023, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "48953f0c-a63f-462c-9f1a-c928d9b6d0de", "db_session_id": "NE6AXKTI0N2QQX5OBHXD", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-06-23T18:13:43.829 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687544023711863, "job": 1, "event": "recovery_finished"} 2023-06-23T18:13:43.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-06-23T18:13:43.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-23T18:13:43.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560071692000 2023-06-23T18:13:43.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: DB pointer 0x56007167e000 2023-06-23T18:13:43.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-06-23T18:13:43.830 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-06-23T18:13:43.830 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** DB Stats ** 2023-06-23T18:13:43.831 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T18:13:43.831 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-23T18:13:43.831 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-23T18:13:43.831 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-23T18:13:43.831 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-23T18:13:43.831 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-23T18:13:43.831 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-23T18:13:43.831 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-06-23T18:13:43.832 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2023-06-23T18:13:43.832 INFO:journalctl@ceph.mon.a.smithi002.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-06-23T18:13:43.832 INFO:journalctl@ceph.mon.a.smithi002.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-06-23T18:13:43.832 INFO:journalctl@ceph.mon.a.smithi002.stdout: L0 2/0 93.17 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-23T18:13:43.832 INFO:journalctl@ceph.mon.a.smithi002.stdout: Sum 2/0 93.17 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-23T18:13:43.832 INFO:journalctl@ceph.mon.a.smithi002.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-23T18:13:43.832 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-06-23T18:13:43.832 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** Compaction Stats [default] ** 2023-06-23T18:13:43.833 INFO:journalctl@ceph.mon.a.smithi002.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-06-23T18:13:43.833 INFO:journalctl@ceph.mon.a.smithi002.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T18:13:43.833 INFO:journalctl@ceph.mon.a.smithi002.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 86.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-23T18:13:43.833 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-06-23T18:13:43.833 INFO:journalctl@ceph.mon.a.smithi002.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-06-23T18:13:43.833 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-06-23T18:13:43.833 INFO:journalctl@ceph.mon.a.smithi002.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T18:13:43.833 INFO:journalctl@ceph.mon.a.smithi002.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-23T18:13:43.834 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-23T18:13:43.834 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-23T18:13:43.834 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-23T18:13:43.834 INFO:journalctl@ceph.mon.a.smithi002.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-23T18:13:43.834 INFO:journalctl@ceph.mon.a.smithi002.stdout: Cumulative compaction: 0.00 GB write, 18.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T18:13:43.834 INFO:journalctl@ceph.mon.a.smithi002.stdout: Interval compaction: 0.00 GB write, 18.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T18:13:43.834 INFO:journalctl@ceph.mon.a.smithi002.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-23T18:13:43.834 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache BinnedLRUCache@0x5600715f91f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-06-23T18:13:43.835 INFO:journalctl@ceph.mon.a.smithi002.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-06-23T18:13:43.835 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-06-23T18:13:43.835 INFO:journalctl@ceph.mon.a.smithi002.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-23T18:13:43.835 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: starting mon.a rank 0 at public addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] at bind addrs [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:13:43.835 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a@-1(???) e1 preinit fsid 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:13:43.835 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a@-1(???).mds e1 new map 2023-06-23T18:13:43.835 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a@-1(???).mds e1 print_map 2023-06-23T18:13:43.836 INFO:journalctl@ceph.mon.a.smithi002.stdout: e1 2023-06-23T18:13:43.836 INFO:journalctl@ceph.mon.a.smithi002.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-23T18:13:43.836 INFO:journalctl@ceph.mon.a.smithi002.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-23T18:13:43.836 INFO:journalctl@ceph.mon.a.smithi002.stdout: legacy client fscid: -1 2023-06-23T18:13:43.836 INFO:journalctl@ceph.mon.a.smithi002.stdout: 2023-06-23T18:13:43.836 INFO:journalctl@ceph.mon.a.smithi002.stdout: No filesystems configured 2023-06-23T18:13:43.836 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-23T18:13:43.836 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T18:13:43.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T18:13:43.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T18:13:43.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-06-23T18:13:43.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta expand map: {default=false} 2023-06-23T18:13:43.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta from 'false' to 'false' 2023-06-23T18:13:43.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta expanded map: {default=false} 2023-06-23T18:13:43.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta expand map: {default=info} 2023-06-23T18:13:43.837 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta from 'info' to 'info' 2023-06-23T18:13:43.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta expanded map: {default=info} 2023-06-23T18:13:43.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta expand map: {default=daemon} 2023-06-23T18:13:43.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta from 'daemon' to 'daemon' 2023-06-23T18:13:43.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta expanded map: {default=daemon} 2023-06-23T18:13:43.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta expand map: {default=debug} 2023-06-23T18:13:43.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta from 'debug' to 'debug' 2023-06-23T18:13:43.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: expand_channel_meta expanded map: {default=debug} 2023-06-23T18:13:43.838 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mon.a is new leader, mons a in quorum (ranks 0) 2023-06-23T18:13:43.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: monmap e1: 1 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0]} removed_ranks: {} 2023-06-23T18:13:43.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: fsmap 2023-06-23T18:13:43.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: osdmap e1: 0 total, 0 up, 0 in 2023-06-23T18:13:43.839 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 ceph-mon[95947]: mgrmap e1: no daemons active 2023-06-23T18:13:43.840 INFO:teuthology.orchestra.run.smithi002.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-06-23T18:13:44.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:43 smithi002 systemd[1]: Started Ceph mon.a for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:13:44.969 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.378+0000 7facf2455700 1 Processor -- start 2023-06-23T18:13:44.969 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.378+0000 7facf2455700 1 -- start start 2023-06-23T18:13:44.969 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.379+0000 7facf2455700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec105550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:44.970 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.379+0000 7facf2455700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec105a90 con 0x7facec105140 2023-06-23T18:13:44.970 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.379+0000 7facebfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec105550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:44.970 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.379+0000 7facebfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec105550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51092/0 (socket says 172.21.15.2:51092) 2023-06-23T18:13:44.970 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.379+0000 7facebfff700 1 -- 172.21.15.2:0/2438665741 learned_addr learned my addr 172.21.15.2:0/2438665741 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:44.970 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.380+0000 7facebfff700 1 -- 172.21.15.2:0/2438665741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec105bd0 con 0x7facec105140 2023-06-23T18:13:44.970 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.380+0000 7facebfff700 1 --2- 172.21.15.2:0/2438665741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec105550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7facdc0098a0 tx=0x7facdc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f4edbbfc74a1dc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:44.971 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.380+0000 7faceaffd700 1 -- 172.21.15.2:0/2438665741 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facdc004160 con 0x7facec105140 2023-06-23T18:13:44.971 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.380+0000 7faceaffd700 1 -- 172.21.15.2:0/2438665741 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7facdc0042c0 con 0x7facec105140 2023-06-23T18:13:44.971 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.380+0000 7faceaffd700 1 -- 172.21.15.2:0/2438665741 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facdc011940 con 0x7facec105140 2023-06-23T18:13:44.971 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 -- 172.21.15.2:0/2438665741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 msgr2=0x7facec105550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:44.971 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 --2- 172.21.15.2:0/2438665741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec105550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7facdc0098a0 tx=0x7facdc009bb0 comp rx=0 tx=0).stop 2023-06-23T18:13:44.971 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 -- 172.21.15.2:0/2438665741 shutdown_connections 2023-06-23T18:13:44.971 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 --2- 172.21.15.2:0/2438665741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec105550 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:44.971 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 -- 172.21.15.2:0/2438665741 >> 172.21.15.2:0/2438665741 conn(0x7facec1007b0 msgr2=0x7facec102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:44.972 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 -- 172.21.15.2:0/2438665741 shutdown_connections 2023-06-23T18:13:44.972 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 -- 172.21.15.2:0/2438665741 wait complete. 2023-06-23T18:13:44.972 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 Processor -- start 2023-06-23T18:13:44.972 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.381+0000 7facf2455700 1 -- start start 2023-06-23T18:13:44.972 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7facf2455700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec07bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:44.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7facf2455700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec07c190 con 0x7facec105140 2023-06-23T18:13:44.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7facebfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec07bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:44.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7facebfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec07bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51094/0 (socket says 172.21.15.2:51094) 2023-06-23T18:13:44.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7facebfff700 1 -- 172.21.15.2:0/3002700178 learned_addr learned my addr 172.21.15.2:0/3002700178 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:44.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7facebfff700 1 -- 172.21.15.2:0/3002700178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc009550 con 0x7facec105140 2023-06-23T18:13:44.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7facebfff700 1 --2- 172.21.15.2:0/3002700178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec07bc50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7facdc006af0 tx=0x7facdc011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:44.973 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7face97fa700 1 -- 172.21.15.2:0/3002700178 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facdc003bf0 con 0x7facec105140 2023-06-23T18:13:44.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7face97fa700 1 -- 172.21.15.2:0/3002700178 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7facdc01b8b0 con 0x7facec105140 2023-06-23T18:13:44.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec07a2a0 con 0x7facec105140 2023-06-23T18:13:44.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.382+0000 7face97fa700 1 -- 172.21.15.2:0/3002700178 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7facdc0197b0 con 0x7facec105140 2023-06-23T18:13:44.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.383+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec07a6c0 con 0x7facec105140 2023-06-23T18:13:44.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.383+0000 7face97fa700 1 -- 172.21.15.2:0/3002700178 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7facdc0233f0 con 0x7facec105140 2023-06-23T18:13:44.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.383+0000 7face97fa700 1 -- 172.21.15.2:0/3002700178 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7facdc01a750 con 0x7facec105140 2023-06-23T18:13:44.974 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.384+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec061b00 con 0x7facec105140 2023-06-23T18:13:44.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.386+0000 7face97fa700 1 -- 172.21.15.2:0/3002700178 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7facdc028060 con 0x7facec105140 2023-06-23T18:13:44.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.512+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7facec19edb0 con 0x7facec105140 2023-06-23T18:13:44.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.514+0000 7face97fa700 1 -- 172.21.15.2:0/3002700178 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7facdc01bd70 con 0x7facec105140 2023-06-23T18:13:44.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.515+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 msgr2=0x7facec07bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:44.975 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.515+0000 7facf2455700 1 --2- 172.21.15.2:0/3002700178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec07bc50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7facdc006af0 tx=0x7facdc011a30 comp rx=0 tx=0).stop 2023-06-23T18:13:44.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.515+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 shutdown_connections 2023-06-23T18:13:44.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.515+0000 7facf2455700 1 --2- 172.21.15.2:0/3002700178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec105140 0x7facec07bc50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:44.976 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.515+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 >> 172.21.15.2:0/3002700178 conn(0x7facec1007b0 msgr2=0x7facec1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:44.977 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.515+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 shutdown_connections 2023-06-23T18:13:44.977 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:44.515+0000 7facf2455700 1 -- 172.21.15.2:0/3002700178 wait complete. 2023-06-23T18:13:44.977 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote config to /etc/ceph/ceph.conf 2023-06-23T18:13:44.995 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:13:44.995 INFO:teuthology.orchestra.run.smithi002.stdout:Creating mgr... 2023-06-23T18:13:44.995 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 9283 ... 2023-06-23T18:13:44.996 INFO:teuthology.orchestra.run.smithi002.stdout:Verifying port 8765 ... 2023-06-23T18:13:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.x 2023-06-23T18:13:45.198 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Failed to reset failed state of unit ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.x.service: Unit ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.x.service not loaded. 2023-06-23T18:13:45.297 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c.target.wants/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@.service. 2023-06-23T18:13:45.518 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:45 smithi002 systemd[1]: Starting Ceph mgr.x for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:13:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:45 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3002700178' entity='client.admin' 2023-06-23T18:13:45.802 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:45 smithi002 podman[96174]: 2023-06-23T18:13:45.904 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-23T18:13:45.904 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2023-06-23T18:13:45.912 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-23T18:13:45.912 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-06-23T18:13:45.912 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2023-06-23T18:13:45.912 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to enable service . firewalld.service is not available 2023-06-23T18:13:45.920 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-23T18:13:45.920 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2023-06-23T18:13:45.928 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-23T18:13:45.928 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-06-23T18:13:45.928 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2023-06-23T18:13:45.928 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-06-23T18:13:45.929 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr to start... 2023-06-23T18:13:45.929 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr... 2023-06-23T18:13:46.149 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:45 smithi002 bash[96174]: 816801aad85c926fc57b2cadcb314eef1bccc131f34a68adc833323e34017010 2023-06-23T18:13:46.150 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:45 smithi002 systemd[1]: Started Ceph mgr.x for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:13:46.150 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:45 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:45.989+0000 7f185b502200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:13:46.150 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:46 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:46.087+0000 7f185b502200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:13:46.523 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:46 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:46.200+0000 7f185b502200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:13:46.523 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:46 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:46.312+0000 7f185b502200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:13:47.198 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:46 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1794640070' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-23T18:13:47.199 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:46 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:46.518+0000 7f185b502200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:13:47.199 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:46 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:46.933+0000 7f185b502200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:13:47.199 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:47 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:47.042+0000 7f185b502200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:13:47.199 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:47 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:47.194+0000 7f185b502200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:13:47.551 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:47 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:47.312+0000 7f185b502200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:13:48.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:47 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:47.824+0000 7f185b502200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:13:48.580 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:48.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-06-23T18:13:48.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:13:48.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2023-06-23T18:13:48.594 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-23T18:13:48.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-23T18:13:48.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-23T18:13:48.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:48.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-23T18:13:48.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-23T18:13:48.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2023-06-23T18:13:48.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:48.595 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-23T18:13:48.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2023-06-23T18:13:48.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:48.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-06-23T18:13:48.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-23T18:13:48.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:48.596 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-06-23T18:13:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-23T18:13:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-23T18:13:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-23T18:13:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-23T18:13:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-23T18:13:48.597 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-23T18:13:48.598 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-23T18:13:48.598 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-23T18:13:48.598 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:48.598 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-23T18:13:48.598 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-23T18:13:48.598 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-23T18:13:48.598 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-23T18:13:48.599 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-23T18:13:48.599 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-23T18:13:48.599 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-23T18:13:48.599 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-23T18:13:48.599 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-23T18:13:48.599 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:48.599 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-23T18:13:48.599 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:48.600 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-23T18:13:48.600 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-23T18:13:48.600 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:48.600 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-23T18:13:48.600 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-23T18:13:48.600 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-23T18:13:48.600 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-23T18:13:48.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-23T18:13:48.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-23T18:13:48.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2023-06-23T18:13:48.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:48.601 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T18:13:48.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:48.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-23T18:13:48.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:48.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2023-06-23T18:13:38.742210+0000", 2023-06-23T18:13:48.602 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T18:13:48.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:48.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-23T18:13:48.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-06-23T18:13:48.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.747+0000 7f881900d700 1 Processor -- start 2023-06-23T18:13:48.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.747+0000 7f881900d700 1 -- start start 2023-06-23T18:13:48.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.747+0000 7f881900d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881407a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:48.603 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.747+0000 7f881900d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881407a880 con 0x7f881407be40 2023-06-23T18:13:48.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.747+0000 7f8812d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881407a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:48.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.747+0000 7f8812d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881407a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51100/0 (socket says 172.21.15.2:51100) 2023-06-23T18:13:48.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.747+0000 7f8812d9d700 1 -- 172.21.15.2:0/3696826142 learned_addr learned my addr 172.21.15.2:0/3696826142 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:48.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.747+0000 7f8812d9d700 1 -- 172.21.15.2:0/3696826142 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881407a9c0 con 0x7f881407be40 2023-06-23T18:13:48.604 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.748+0000 7f8812d9d700 1 --2- 172.21.15.2:0/3696826142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881407a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88040098a0 tx=0x7f8804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed5075c6c59de1c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.748+0000 7f8811d9b700 1 -- 172.21.15.2:0/3696826142 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8804004160 con 0x7f881407be40 2023-06-23T18:13:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.748+0000 7f8811d9b700 1 -- 172.21.15.2:0/3696826142 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f88040042c0 con 0x7f881407be40 2023-06-23T18:13:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.748+0000 7f8811d9b700 1 -- 172.21.15.2:0/3696826142 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8804011940 con 0x7f881407be40 2023-06-23T18:13:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.749+0000 7f881900d700 1 -- 172.21.15.2:0/3696826142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 msgr2=0x7f881407a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.749+0000 7f881900d700 1 --2- 172.21.15.2:0/3696826142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881407a340 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f88040098a0 tx=0x7f8804009bb0 comp rx=0 tx=0).stop 2023-06-23T18:13:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.749+0000 7f881900d700 1 -- 172.21.15.2:0/3696826142 shutdown_connections 2023-06-23T18:13:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.749+0000 7f881900d700 1 --2- 172.21.15.2:0/3696826142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881407a340 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:48.605 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.749+0000 7f881900d700 1 -- 172.21.15.2:0/3696826142 >> 172.21.15.2:0/3696826142 conn(0x7f8814100820 msgr2=0x7f8814102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:48.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.749+0000 7f881900d700 1 -- 172.21.15.2:0/3696826142 shutdown_connections 2023-06-23T18:13:48.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.749+0000 7f881900d700 1 -- 172.21.15.2:0/3696826142 wait complete. 2023-06-23T18:13:48.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.749+0000 7f881900d700 1 Processor -- start 2023-06-23T18:13:48.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.750+0000 7f881900d700 1 -- start start 2023-06-23T18:13:48.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.750+0000 7f881900d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:48.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.750+0000 7f881900d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881419b5c0 con 0x7f881407be40 2023-06-23T18:13:48.606 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.750+0000 7f8812d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:48.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.750+0000 7f8812d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881419b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51102/0 (socket says 172.21.15.2:51102) 2023-06-23T18:13:48.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.750+0000 7f8812d9d700 1 -- 172.21.15.2:0/1794640070 learned_addr learned my addr 172.21.15.2:0/1794640070 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:48.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.750+0000 7f8812d9d700 1 -- 172.21.15.2:0/1794640070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804009550 con 0x7f881407be40 2023-06-23T18:13:48.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.750+0000 7f8812d9d700 1 --2- 172.21.15.2:0/1794640070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881419b080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8804006af0 tx=0x7f8804011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:48.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.751+0000 7f880bfff700 1 -- 172.21.15.2:0/1794640070 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8804003bf0 con 0x7f881407be40 2023-06-23T18:13:48.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.751+0000 7f880bfff700 1 -- 172.21.15.2:0/1794640070 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f880401b8b0 con 0x7f881407be40 2023-06-23T18:13:48.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.751+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881419b7c0 con 0x7f881407be40 2023-06-23T18:13:48.607 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.751+0000 7f880bfff700 1 -- 172.21.15.2:0/1794640070 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88040197e0 con 0x7f881407be40 2023-06-23T18:13:48.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.751+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881419bbe0 con 0x7f881407be40 2023-06-23T18:13:48.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.751+0000 7f880bfff700 1 -- 172.21.15.2:0/1794640070 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f88040233f0 con 0x7f881407be40 2023-06-23T18:13:48.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.751+0000 7f880bfff700 1 -- 172.21.15.2:0/1794640070 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f880401a710 con 0x7f881407be40 2023-06-23T18:13:48.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.752+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8814061b00 con 0x7f881407be40 2023-06-23T18:13:48.608 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.756+0000 7f880bfff700 1 -- 172.21.15.2:0/1794640070 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f8804028060 con 0x7f881407be40 2023-06-23T18:13:48.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.900+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f881419e620 con 0x7f881407be40 2023-06-23T18:13:48.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.901+0000 7f880bfff700 1 -- 172.21.15.2:0/1794640070 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f880401bd70 con 0x7f881407be40 2023-06-23T18:13:48.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.902+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 msgr2=0x7f881419b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:48.609 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.902+0000 7f881900d700 1 --2- 172.21.15.2:0/1794640070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881419b080 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8804006af0 tx=0x7f8804011a30 comp rx=0 tx=0).stop 2023-06-23T18:13:48.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.902+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 shutdown_connections 2023-06-23T18:13:48.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.902+0000 7f881900d700 1 --2- 172.21.15.2:0/1794640070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f881407be40 0x7f881419b080 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:48.610 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.902+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 >> 172.21.15.2:0/1794640070 conn(0x7f8814100820 msgr2=0x7f8814101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:48.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.902+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 shutdown_connections 2023-06-23T18:13:48.611 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:46.902+0000 7f881900d700 1 -- 172.21.15.2:0/1794640070 wait complete. 2023-06-23T18:13:48.611 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (1/15)... 2023-06-23T18:13:48.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:48 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:48.521+0000 7f185b502200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T18:13:48.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:48 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:48.716+0000 7f185b502200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:13:50.010 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:49.679+0000 7f185b502200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:13:50.010 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:49.789+0000 7f185b502200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:13:50.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:50 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:50.005+0000 7f185b502200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:13:50.609 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:50 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:50.350+0000 7f185b502200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T18:13:51.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:50 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:50.604+0000 7f185b502200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:13:51.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:50 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:50.787+0000 7f185b502200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:13:51.551 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:51 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:51.112+0000 7f185b502200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:13:52.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:51 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:51.931+0000 7f185b502200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:13:52.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:52 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:52.087+0000 7f185b502200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:13:52.592 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:52 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:52.329+0000 7f185b502200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:13:52.592 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:52 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:52.468+0000 7f185b502200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:13:53.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:52 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:52.742+0000 7f185b502200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:13:53.798 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:53 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/796625179' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-23T18:13:54.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:13:53 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:13:53.793+0000 7f185b502200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:13:55.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: Activating manager daemon x 2023-06-23T18:13:55.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: mgrmap e2: x(active, starting, since 0.00280349s) 2023-06-23T18:13:55.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-23T18:13:55.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-23T18:13:55.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-23T18:13:55.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:13:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-23T18:13:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: Manager daemon x is now available 2023-06-23T18:13:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-23T18:13:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-23T18:13:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' 2023-06-23T18:13:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' 2023-06-23T18:13:55.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:54 smithi002 ceph-mon[95947]: from='mgr.14100 172.21.15.2:0/5443387' entity='mgr.x' 2023-06-23T18:13:55.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:55.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-06-23T18:13:55.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:13:55.843 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2023-06-23T18:13:55.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-23T18:13:55.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-23T18:13:55.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-23T18:13:55.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:55.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-23T18:13:55.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-23T18:13:55.844 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2023-06-23T18:13:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-23T18:13:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2023-06-23T18:13:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-06-23T18:13:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-23T18:13:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:55.845 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-06-23T18:13:55.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-23T18:13:55.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:55.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-23T18:13:55.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:55.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-23T18:13:55.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-23T18:13:55.846 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-23T18:13:55.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-23T18:13:55.852 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-23T18:13:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-23T18:13:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-23T18:13:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-23T18:13:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-23T18:13:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-23T18:13:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-23T18:13:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-23T18:13:55.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-23T18:13:55.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-23T18:13:55.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-23T18:13:55.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:55.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-23T18:13:55.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:55.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-23T18:13:55.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-23T18:13:55.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:55.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-23T18:13:55.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": false, 2023-06-23T18:13:55.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-23T18:13:55.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-23T18:13:55.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-23T18:13:55.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-23T18:13:55.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2023-06-23T18:13:55.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:55.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T18:13:55.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:55.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-23T18:13:55.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:55.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2023-06-23T18:13:38.742210+0000", 2023-06-23T18:13:55.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T18:13:55.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:55.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-23T18:13:55.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-06-23T18:13:55.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.227+0000 7fbaedfce700 1 Processor -- start 2023-06-23T18:13:55.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.227+0000 7fbaedfce700 1 -- start start 2023-06-23T18:13:55.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.227+0000 7fbaedfce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae807a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:55.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.227+0000 7fbaedfce700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae807a840 con 0x7fbae807be00 2023-06-23T18:13:55.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.228+0000 7fbae77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae807a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:55.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.228+0000 7fbae77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae807a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51104/0 (socket says 172.21.15.2:51104) 2023-06-23T18:13:55.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.228+0000 7fbae77fe700 1 -- 172.21.15.2:0/3383958856 learned_addr learned my addr 172.21.15.2:0/3383958856 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:55.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.228+0000 7fbae77fe700 1 -- 172.21.15.2:0/3383958856 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae807a980 con 0x7fbae807be00 2023-06-23T18:13:55.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.228+0000 7fbae77fe700 1 --2- 172.21.15.2:0/3383958856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae807a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098a0 tx=0x7fbad8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e1f059514e5301 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:55.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.228+0000 7fbae67fc700 1 -- 172.21.15.2:0/3383958856 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad8004160 con 0x7fbae807be00 2023-06-23T18:13:55.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.228+0000 7fbae67fc700 1 -- 172.21.15.2:0/3383958856 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fbad80042c0 con 0x7fbae807be00 2023-06-23T18:13:55.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.228+0000 7fbae67fc700 1 -- 172.21.15.2:0/3383958856 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad8011940 con 0x7fbae807be00 2023-06-23T18:13:55.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.229+0000 7fbaedfce700 1 -- 172.21.15.2:0/3383958856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 msgr2=0x7fbae807a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:55.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.229+0000 7fbaedfce700 1 --2- 172.21.15.2:0/3383958856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae807a300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098a0 tx=0x7fbad8009bb0 comp rx=0 tx=0).stop 2023-06-23T18:13:55.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.229+0000 7fbaedfce700 1 -- 172.21.15.2:0/3383958856 shutdown_connections 2023-06-23T18:13:55.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.229+0000 7fbaedfce700 1 --2- 172.21.15.2:0/3383958856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae807a300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:55.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.229+0000 7fbaedfce700 1 -- 172.21.15.2:0/3383958856 >> 172.21.15.2:0/3383958856 conn(0x7fbae8100810 msgr2=0x7fbae8102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:55.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.229+0000 7fbaedfce700 1 -- 172.21.15.2:0/3383958856 shutdown_connections 2023-06-23T18:13:55.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.229+0000 7fbaedfce700 1 -- 172.21.15.2:0/3383958856 wait complete. 2023-06-23T18:13:55.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.230+0000 7fbaedfce700 1 Processor -- start 2023-06-23T18:13:55.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.230+0000 7fbaedfce700 1 -- start start 2023-06-23T18:13:55.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.230+0000 7fbaedfce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae819b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:55.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.230+0000 7fbaedfce700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae819b5e0 con 0x7fbae807be00 2023-06-23T18:13:55.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.230+0000 7fbae77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae819b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:55.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.230+0000 7fbae77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae819b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51106/0 (socket says 172.21.15.2:51106) 2023-06-23T18:13:55.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.230+0000 7fbae77fe700 1 -- 172.21.15.2:0/796625179 learned_addr learned my addr 172.21.15.2:0/796625179 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:55.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbae77fe700 1 -- 172.21.15.2:0/796625179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad8009550 con 0x7fbae807be00 2023-06-23T18:13:55.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbae77fe700 1 --2- 172.21.15.2:0/796625179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae819b0a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbad8006af0 tx=0x7fbad80119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:55.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbae4ff9700 1 -- 172.21.15.2:0/796625179 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad8003a70 con 0x7fbae807be00 2023-06-23T18:13:55.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbae4ff9700 1 -- 172.21.15.2:0/796625179 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fbad8003bd0 con 0x7fbae807be00 2023-06-23T18:13:55.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae819b7e0 con 0x7fbae807be00 2023-06-23T18:13:55.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbae4ff9700 1 -- 172.21.15.2:0/796625179 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbad80195e0 con 0x7fbae807be00 2023-06-23T18:13:55.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae819bc00 con 0x7fbae807be00 2023-06-23T18:13:55.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbae4ff9700 1 -- 172.21.15.2:0/796625179 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbad8019b70 con 0x7fbae807be00 2023-06-23T18:13:55.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.231+0000 7fbae4ff9700 1 -- 172.21.15.2:0/796625179 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbad8024730 con 0x7fbae807be00 2023-06-23T18:13:55.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.232+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae8061b00 con 0x7fbae807be00 2023-06-23T18:13:55.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.235+0000 7fbae4ff9700 1 -- 172.21.15.2:0/796625179 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fbad801a9a0 con 0x7fbae807be00 2023-06-23T18:13:55.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.368+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbae819e640 con 0x7fbae807be00 2023-06-23T18:13:55.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.369+0000 7fbae4ff9700 1 -- 172.21.15.2:0/796625179 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fbad801a430 con 0x7fbae807be00 2023-06-23T18:13:55.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.370+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 msgr2=0x7fbae819b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:55.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.370+0000 7fbaedfce700 1 --2- 172.21.15.2:0/796625179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae819b0a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbad8006af0 tx=0x7fbad80119c0 comp rx=0 tx=0).stop 2023-06-23T18:13:55.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.370+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 shutdown_connections 2023-06-23T18:13:55.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.370+0000 7fbaedfce700 1 --2- 172.21.15.2:0/796625179 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbae807be00 0x7fbae819b0a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:55.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.370+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 >> 172.21.15.2:0/796625179 conn(0x7fbae8100810 msgr2=0x7fbae8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:55.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.370+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 shutdown_connections 2023-06-23T18:13:55.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:53.370+0000 7fbaedfce700 1 -- 172.21.15.2:0/796625179 wait complete. 2023-06-23T18:13:55.863 INFO:teuthology.orchestra.run.smithi002.stdout:mgr not available, waiting (2/15)... 2023-06-23T18:13:56.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:55 smithi002 ceph-mon[95947]: mgrmap e3: x(active, since 1.00725s) 2023-06-23T18:13:58.089 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:57 smithi002 ceph-mon[95947]: mgrmap e4: x(active, since 3s) 2023-06-23T18:13:59.301 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:13:59.302 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-06-23T18:13:59.302 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:13:59.302 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "health": { 2023-06-23T18:13:59.302 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-06-23T18:13:59.302 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-06-23T18:13:59.302 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-06-23T18:13:59.302 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:59.303 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-06-23T18:13:59.303 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-06-23T18:13:59.303 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 0 2023-06-23T18:13:59.303 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:59.303 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-06-23T18:13:59.303 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "a" 2023-06-23T18:13:59.303 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2023-06-23T18:13:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "monmap": { 2023-06-23T18:13:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-06-23T18:13:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-06-23T18:13:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:59.304 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-06-23T18:13:59.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:59.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-06-23T18:13:59.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-06-23T18:13:59.305 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-06-23T18:13:59.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-06-23T18:13:59.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-06-23T18:13:59.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-06-23T18:13:59.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:59.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-06-23T18:13:59.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-06-23T18:13:59.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-06-23T18:13:59.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-06-23T18:13:59.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-06-23T18:13:59.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-06-23T18:13:59.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-06-23T18:13:59.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-06-23T18:13:59.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-06-23T18:13:59.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-06-23T18:13:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-06-23T18:13:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-06-23T18:13:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-06-23T18:13:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-23T18:13:59.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-06-23T18:13:59.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modules": [ 2023-06-23T18:13:59.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "iostat", 2023-06-23T18:13:59.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "nfs", 2023-06-23T18:13:59.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "restful" 2023-06-23T18:13:59.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ], 2023-06-23T18:13:59.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T18:13:59.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:59.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-06-23T18:13:59.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-06-23T18:13:59.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "modified": "2023-06-23T18:13:38.742210+0000", 2023-06-23T18:13:59.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "services": {} 2023-06-23T18:13:59.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout }, 2023-06-23T18:13:59.316 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-06-23T18:13:59.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-06-23T18:13:59.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.373+0000 7f2785033700 1 Processor -- start 2023-06-23T18:13:59.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.373+0000 7f2785033700 1 -- start start 2023-06-23T18:13:59.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.374+0000 7f2785033700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:59.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.374+0000 7f2785033700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2780079d80 con 0x7f2780105c70 2023-06-23T18:13:59.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.374+0000 7f277ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:59.317 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.374+0000 7f277ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51132/0 (socket says 172.21.15.2:51132) 2023-06-23T18:13:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.374+0000 7f277ed9d700 1 -- 172.21.15.2:0/2864182819 learned_addr learned my addr 172.21.15.2:0/2864182819 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.374+0000 7f277ed9d700 1 -- 172.21.15.2:0/2864182819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2780079ec0 con 0x7f2780105c70 2023-06-23T18:13:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f277ed9d700 1 --2- 172.21.15.2:0/2864182819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780108050 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f27740098a0 tx=0x7f2774009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e86313c9e09b67 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f277dd9b700 1 -- 172.21.15.2:0/2864182819 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2774004160 con 0x7f2780105c70 2023-06-23T18:13:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f277dd9b700 1 -- 172.21.15.2:0/2864182819 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f27740042c0 con 0x7f2780105c70 2023-06-23T18:13:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f277dd9b700 1 -- 172.21.15.2:0/2864182819 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2774011940 con 0x7f2780105c70 2023-06-23T18:13:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f2785033700 1 -- 172.21.15.2:0/2864182819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 msgr2=0x7f2780108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:59.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f2785033700 1 --2- 172.21.15.2:0/2864182819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780108050 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f27740098a0 tx=0x7f2774009bb0 comp rx=0 tx=0).stop 2023-06-23T18:13:59.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f2785033700 1 -- 172.21.15.2:0/2864182819 shutdown_connections 2023-06-23T18:13:59.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f2785033700 1 --2- 172.21.15.2:0/2864182819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780108050 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:59.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f2785033700 1 -- 172.21.15.2:0/2864182819 >> 172.21.15.2:0/2864182819 conn(0x7f27800ff710 msgr2=0x7f2780101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:59.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.375+0000 7f2785033700 1 -- 172.21.15.2:0/2864182819 shutdown_connections 2023-06-23T18:13:59.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.376+0000 7f2785033700 1 -- 172.21.15.2:0/2864182819 wait complete. 2023-06-23T18:13:59.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.376+0000 7f2785033700 1 Processor -- start 2023-06-23T18:13:59.319 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.377+0000 7f2785033700 1 -- start start 2023-06-23T18:13:59.320 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.377+0000 7f2785033700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:59.320 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.377+0000 7f2785033700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27801993a0 con 0x7f2780105c70 2023-06-23T18:13:59.320 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.377+0000 7f277ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:59.320 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.377+0000 7f277ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51134/0 (socket says 172.21.15.2:51134) 2023-06-23T18:13:59.320 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.377+0000 7f277ed9d700 1 -- 172.21.15.2:0/413730857 learned_addr learned my addr 172.21.15.2:0/413730857 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:13:59.320 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.377+0000 7f277ed9d700 1 -- 172.21.15.2:0/413730857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2774009550 con 0x7f2780105c70 2023-06-23T18:13:59.320 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.377+0000 7f277ed9d700 1 --2- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780198e60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2774000c00 tx=0x7f2774011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:59.321 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.378+0000 7f2767fff700 1 -- 172.21.15.2:0/413730857 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27740036a0 con 0x7f2780105c70 2023-06-23T18:13:59.321 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.378+0000 7f2767fff700 1 -- 172.21.15.2:0/413730857 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f2774003c70 con 0x7f2780105c70 2023-06-23T18:13:59.321 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.378+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27801995a0 con 0x7f2780105c70 2023-06-23T18:13:59.321 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.378+0000 7f2767fff700 1 -- 172.21.15.2:0/413730857 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27740193f0 con 0x7f2780105c70 2023-06-23T18:13:59.321 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.378+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27801999c0 con 0x7f2780105c70 2023-06-23T18:13:59.321 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.379+0000 7f2767fff700 1 -- 172.21.15.2:0/413730857 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f2774019620 con 0x7f2780105c70 2023-06-23T18:13:59.321 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.379+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2780061b00 con 0x7f2780105c70 2023-06-23T18:13:59.322 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.379+0000 7f2767fff700 1 --2- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f2768038710 0x7f276803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:13:59.322 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.379+0000 7f2767fff700 1 -- 172.21.15.2:0/413730857 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f277404c490 con 0x7f2780105c70 2023-06-23T18:13:59.322 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.379+0000 7f277e59c700 1 --2- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f2768038710 0x7f276803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:13:59.322 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.380+0000 7f277e59c700 1 --2- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f2768038710 0x7f276803abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2770006fd0 tx=0x7f2770006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:13:59.322 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.384+0000 7f2767fff700 1 -- 172.21.15.2:0/413730857 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2774019e40 con 0x7f2780105c70 2023-06-23T18:13:59.322 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.900+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f278019cb70 con 0x7f2780105c70 2023-06-23T18:13:59.322 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.901+0000 7f2767fff700 1 -- 172.21.15.2:0/413730857 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f277401f070 con 0x7f2780105c70 2023-06-23T18:13:59.323 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f2768038710 msgr2=0x7f276803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:59.323 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 --2- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f2768038710 0x7f276803abc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2770006fd0 tx=0x7f2770006e40 comp rx=0 tx=0).stop 2023-06-23T18:13:59.323 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 msgr2=0x7f2780198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:13:59.323 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 --2- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780198e60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2774000c00 tx=0x7f2774011e70 comp rx=0 tx=0).stop 2023-06-23T18:13:59.323 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 shutdown_connections 2023-06-23T18:13:59.323 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 --2- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f2768038710 0x7f276803abc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:59.323 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 --2- 172.21.15.2:0/413730857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2780105c70 0x7f2780198e60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:13:59.324 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 >> 172.21.15.2:0/413730857 conn(0x7f27800ff710 msgr2=0x7f2780100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:13:59.324 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 shutdown_connections 2023-06-23T18:13:59.324 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:58.903+0000 7f2785033700 1 -- 172.21.15.2:0/413730857 wait complete. 2023-06-23T18:13:59.324 INFO:teuthology.orchestra.run.smithi002.stdout:mgr is available 2023-06-23T18:14:00.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:13:59 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/413730857' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-06-23T18:14:00.755 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:14:00.755 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [global] 2023-06-23T18:14:00.755 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout fsid = 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:14:00.755 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-06-23T18:14:00.755 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-06-23T18:14:00.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-06-23T18:14:00.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-06-23T18:14:00.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:14:00.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [mgr] 2023-06-23T18:14:00.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-06-23T18:14:00.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 2023-06-23T18:14:00.756 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout [osd] 2023-06-23T18:14:00.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-06-23T18:14:00.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-06-23T18:14:00.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-06-23T18:14:00.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-06-23T18:14:00.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-06-23T18:14:00.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.921+0000 7fd36c207700 1 Processor -- start 2023-06-23T18:14:00.757 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.921+0000 7fd36c207700 1 -- start start 2023-06-23T18:14:00.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.921+0000 7fd36c207700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd364105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:00.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.921+0000 7fd36c207700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd364079d80 con 0x7fd364103a60 2023-06-23T18:14:00.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.922+0000 7fd369fa3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd364105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:00.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.922+0000 7fd369fa3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd364105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51138/0 (socket says 172.21.15.2:51138) 2023-06-23T18:14:00.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.922+0000 7fd369fa3700 1 -- 172.21.15.2:0/870059022 learned_addr learned my addr 172.21.15.2:0/870059022 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:00.758 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.922+0000 7fd369fa3700 1 -- 172.21.15.2:0/870059022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364079ec0 con 0x7fd364103a60 2023-06-23T18:14:00.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.922+0000 7fd369fa3700 1 --2- 172.21.15.2:0/870059022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd364105e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098a0 tx=0x7fd360009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee07b967c21f54b6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:00.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.922+0000 7fd368fa1700 1 -- 172.21.15.2:0/870059022 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd360004160 con 0x7fd364103a60 2023-06-23T18:14:00.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.922+0000 7fd368fa1700 1 -- 172.21.15.2:0/870059022 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd3600042c0 con 0x7fd364103a60 2023-06-23T18:14:00.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.922+0000 7fd368fa1700 1 -- 172.21.15.2:0/870059022 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd360011940 con 0x7fd364103a60 2023-06-23T18:14:00.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.923+0000 7fd36c207700 1 -- 172.21.15.2:0/870059022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 msgr2=0x7fd364105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:00.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.923+0000 7fd36c207700 1 --2- 172.21.15.2:0/870059022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd364105e80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd3600098a0 tx=0x7fd360009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:00.759 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.923+0000 7fd36c207700 1 -- 172.21.15.2:0/870059022 shutdown_connections 2023-06-23T18:14:00.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.923+0000 7fd36c207700 1 --2- 172.21.15.2:0/870059022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd364105e80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:00.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.923+0000 7fd36c207700 1 -- 172.21.15.2:0/870059022 >> 172.21.15.2:0/870059022 conn(0x7fd3640ff6f0 msgr2=0x7fd364101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:00.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.923+0000 7fd36c207700 1 -- 172.21.15.2:0/870059022 shutdown_connections 2023-06-23T18:14:00.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.923+0000 7fd36c207700 1 -- 172.21.15.2:0/870059022 wait complete. 2023-06-23T18:14:00.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd36c207700 1 Processor -- start 2023-06-23T18:14:00.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd36c207700 1 -- start start 2023-06-23T18:14:00.760 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd36c207700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd36419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:00.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd36c207700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36419f9a0 con 0x7fd364103a60 2023-06-23T18:14:00.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd369fa3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd36419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:00.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd369fa3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd36419f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51140/0 (socket says 172.21.15.2:51140) 2023-06-23T18:14:00.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd369fa3700 1 -- 172.21.15.2:0/2342802195 learned_addr learned my addr 172.21.15.2:0/2342802195 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:00.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd369fa3700 1 -- 172.21.15.2:0/2342802195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd360009550 con 0x7fd364103a60 2023-06-23T18:14:00.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd369fa3700 1 --2- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd36419f460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd360000c00 tx=0x7fd360011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:00.761 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd356ffd700 1 -- 172.21.15.2:0/2342802195 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3600036a0 con 0x7fd364103a60 2023-06-23T18:14:00.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd356ffd700 1 -- 172.21.15.2:0/2342802195 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd360003c70 con 0x7fd364103a60 2023-06-23T18:14:00.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36419fba0 con 0x7fd364103a60 2023-06-23T18:14:00.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.924+0000 7fd356ffd700 1 -- 172.21.15.2:0/2342802195 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd3600193f0 con 0x7fd364103a60 2023-06-23T18:14:00.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.925+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36419ffc0 con 0x7fd364103a60 2023-06-23T18:14:00.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.925+0000 7fd356ffd700 1 -- 172.21.15.2:0/2342802195 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fd36001b440 con 0x7fd364103a60 2023-06-23T18:14:00.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.926+0000 7fd356ffd700 1 --2- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fd350038710 0x7fd35003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:00.762 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.926+0000 7fd356ffd700 1 -- 172.21.15.2:0/2342802195 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd36004c470 con 0x7fd364103a60 2023-06-23T18:14:00.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.926+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd364061b00 con 0x7fd364103a60 2023-06-23T18:14:00.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.926+0000 7fd3697a2700 1 --2- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fd350038710 0x7fd35003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:00.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.926+0000 7fd3697a2700 1 --2- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fd350038710 0x7fd35003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd358006fd0 tx=0x7fd358006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:00.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:13:59.931+0000 7fd356ffd700 1 -- 172.21.15.2:0/2342802195 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd36001f020 con 0x7fd364103a60 2023-06-23T18:14:00.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.288+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd3641a2a00 con 0x7fd364103a60 2023-06-23T18:14:00.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.289+0000 7fd356ffd700 1 -- 172.21.15.2:0/2342802195 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd36004a020 con 0x7fd364103a60 2023-06-23T18:14:00.763 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fd350038710 msgr2=0x7fd35003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:00.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 --2- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fd350038710 0x7fd35003abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd358006fd0 tx=0x7fd358006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:00.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 msgr2=0x7fd36419f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:00.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 --2- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd36419f460 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd360000c00 tx=0x7fd360011e70 comp rx=0 tx=0).stop 2023-06-23T18:14:00.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 shutdown_connections 2023-06-23T18:14:00.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 --2- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fd350038710 0x7fd35003abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:00.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 --2- 172.21.15.2:0/2342802195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd364103a60 0x7fd36419f460 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:00.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 >> 172.21.15.2:0/2342802195 conn(0x7fd3640ff6f0 msgr2=0x7fd364100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:00.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 shutdown_connections 2023-06-23T18:14:00.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:00.291+0000 7fd36c207700 1 -- 172.21.15.2:0/2342802195 wait complete. 2023-06-23T18:14:00.765 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling cephadm module... 2023-06-23T18:14:01.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:00 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2342802195' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-06-23T18:14:02.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:01 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3791247792' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-06-23T18:14:02.210 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:01 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: ignoring --setuser ceph since I am not root 2023-06-23T18:14:02.210 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:01 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: ignoring --setgroup ceph since I am not root 2023-06-23T18:14:02.210 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:02.108+0000 7f1843652200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:14:02.401 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.382+0000 7fb5c57af700 1 Processor -- start 2023-06-23T18:14:02.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.382+0000 7fb5c57af700 1 -- start start 2023-06-23T18:14:02.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.383+0000 7fb5c57af700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c0108020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:02.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.383+0000 7fb5c57af700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c0079d80 con 0x7fb5c0105c40 2023-06-23T18:14:02.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.383+0000 7fb5beffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c0108020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:02.403 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.383+0000 7fb5beffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c0108020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51144/0 (socket says 172.21.15.2:51144) 2023-06-23T18:14:02.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.383+0000 7fb5beffd700 1 -- 172.21.15.2:0/3364912514 learned_addr learned my addr 172.21.15.2:0/3364912514 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:02.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.383+0000 7fb5beffd700 1 -- 172.21.15.2:0/3364912514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0079ec0 con 0x7fb5c0105c40 2023-06-23T18:14:02.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.383+0000 7fb5beffd700 1 --2- 172.21.15.2:0/3364912514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c0108020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40098a0 tx=0x7fb5b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693fd45d9620efbb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:02.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5bdffb700 1 -- 172.21.15.2:0/3364912514 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b4004160 con 0x7fb5c0105c40 2023-06-23T18:14:02.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5bdffb700 1 -- 172.21.15.2:0/3364912514 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb5b40042c0 con 0x7fb5c0105c40 2023-06-23T18:14:02.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5bdffb700 1 -- 172.21.15.2:0/3364912514 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b4011940 con 0x7fb5c0105c40 2023-06-23T18:14:02.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5c57af700 1 -- 172.21.15.2:0/3364912514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 msgr2=0x7fb5c0108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:02.404 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5c57af700 1 --2- 172.21.15.2:0/3364912514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c0108020 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40098a0 tx=0x7fb5b4009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:02.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5c57af700 1 -- 172.21.15.2:0/3364912514 shutdown_connections 2023-06-23T18:14:02.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5c57af700 1 --2- 172.21.15.2:0/3364912514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c0108020 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:02.405 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5c57af700 1 -- 172.21.15.2:0/3364912514 >> 172.21.15.2:0/3364912514 conn(0x7fb5c00ff6a0 msgr2=0x7fb5c0101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:02.407 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5c57af700 1 -- 172.21.15.2:0/3364912514 shutdown_connections 2023-06-23T18:14:02.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.384+0000 7fb5c57af700 1 -- 172.21.15.2:0/3364912514 wait complete. 2023-06-23T18:14:02.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.385+0000 7fb5c57af700 1 Processor -- start 2023-06-23T18:14:02.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.385+0000 7fb5c57af700 1 -- start start 2023-06-23T18:14:02.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.385+0000 7fb5c57af700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c01055a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:02.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.385+0000 7fb5c57af700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c0105ae0 con 0x7fb5c0105c40 2023-06-23T18:14:02.408 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.385+0000 7fb5beffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c01055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:02.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.385+0000 7fb5beffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c01055a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51146/0 (socket says 172.21.15.2:51146) 2023-06-23T18:14:02.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.385+0000 7fb5beffd700 1 -- 172.21.15.2:0/3791247792 learned_addr learned my addr 172.21.15.2:0/3791247792 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:02.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.386+0000 7fb5beffd700 1 -- 172.21.15.2:0/3791247792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b4009550 con 0x7fb5c0105c40 2023-06-23T18:14:02.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.386+0000 7fb5beffd700 1 --2- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c01055a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:02.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.386+0000 7fb5a7fff700 1 -- 172.21.15.2:0/3791247792 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b40036a0 con 0x7fb5c0105c40 2023-06-23T18:14:02.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.386+0000 7fb5a7fff700 1 -- 172.21.15.2:0/3791247792 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fb5b4003c70 con 0x7fb5c0105c40 2023-06-23T18:14:02.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.386+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c0103bf0 con 0x7fb5c0105c40 2023-06-23T18:14:02.409 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.386+0000 7fb5a7fff700 1 -- 172.21.15.2:0/3791247792 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b40193f0 con 0x7fb5c0105c40 2023-06-23T18:14:02.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.386+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c0104010 con 0x7fb5c0105c40 2023-06-23T18:14:02.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.387+0000 7fb5a7fff700 1 -- 172.21.15.2:0/3791247792 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fb5b4019620 con 0x7fb5c0105c40 2023-06-23T18:14:02.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.387+0000 7fb5a7fff700 1 --2- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fb5a8038760 0x7fb5a803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:02.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.387+0000 7fb5a7fff700 1 -- 172.21.15.2:0/3791247792 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb5b404c490 con 0x7fb5c0105c40 2023-06-23T18:14:02.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.387+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c01091d0 con 0x7fb5c0105c40 2023-06-23T18:14:02.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.387+0000 7fb5be7fc700 1 --2- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fb5a8038760 0x7fb5a803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:02.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.388+0000 7fb5be7fc700 1 --2- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fb5a8038760 0x7fb5a803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006fd0 tx=0x7fb5b0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:02.410 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.392+0000 7fb5a7fff700 1 -- 172.21.15.2:0/3791247792 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb5b401a3d0 con 0x7fb5c0105c40 2023-06-23T18:14:02.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.861+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb5c019ee30 con 0x7fb5c0105c40 2023-06-23T18:14:02.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.915+0000 7fb5a7fff700 1 -- 172.21.15.2:0/3791247792 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fb5b401b440 con 0x7fb5c0105c40 2023-06-23T18:14:02.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.915+0000 7fb5a7fff700 1 -- 172.21.15.2:0/3791247792 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb5b4019e40 con 0x7fb5c0105c40 2023-06-23T18:14:02.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fb5a8038760 msgr2=0x7fb5a803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:02.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 --2- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fb5a8038760 0x7fb5a803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0006fd0 tx=0x7fb5b0006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:02.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 msgr2=0x7fb5c01055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:02.411 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 --2- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c01055a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4011e70 comp rx=0 tx=0).stop 2023-06-23T18:14:02.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 shutdown_connections 2023-06-23T18:14:02.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 --2- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fb5a8038760 0x7fb5a803ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:02.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 --2- 172.21.15.2:0/3791247792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5c0105c40 0x7fb5c01055a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:02.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 >> 172.21.15.2:0/3791247792 conn(0x7fb5c00ff6a0 msgr2=0x7fb5c0100430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:02.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 shutdown_connections 2023-06-23T18:14:02.412 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:01.918+0000 7fb5c57af700 1 -- 172.21.15.2:0/3791247792 wait complete. 2023-06-23T18:14:02.551 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:02.203+0000 7f1843652200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:14:02.552 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:02.312+0000 7f1843652200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:14:02.552 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:02.429+0000 7f1843652200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:14:02.922 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:02.642+0000 7f1843652200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:14:03.179 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:03.065+0000 7f1843652200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:14:03.179 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:02 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3791247792' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-06-23T18:14:03.179 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:02 smithi002 ceph-mon[95947]: mgrmap e5: x(active, since 8s) 2023-06-23T18:14:03.441 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:03.173+0000 7f1843652200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:14:03.441 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:03.324+0000 7f1843652200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:14:03.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:03.442+0000 7f1843652200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:14:03.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-06-23T18:14:03.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-06-23T18:14:03.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-23T18:14:03.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-06-23T18:14:03.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-06-23T18:14:03.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-06-23T18:14:03.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.917+0000 7fc72f5e8700 1 Processor -- start 2023-06-23T18:14:03.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.917+0000 7fc72f5e8700 1 -- start start 2023-06-23T18:14:03.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.917+0000 7fc72f5e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc728079d00 0x7fc72810a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:03.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.917+0000 7fc72f5e8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc728105af0 con 0x7fc728079d00 2023-06-23T18:14:03.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.918+0000 7fc72d384700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc728079d00 0x7fc72810a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.918+0000 7fc72d384700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc728079d00 0x7fc72810a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51160/0 (socket says 172.21.15.2:51160) 2023-06-23T18:14:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.918+0000 7fc72d384700 1 -- 172.21.15.2:0/2944368175 learned_addr learned my addr 172.21.15.2:0/2944368175 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.918+0000 7fc72d384700 1 -- 172.21.15.2:0/2944368175 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72810a640 con 0x7fc728079d00 2023-06-23T18:14:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.918+0000 7fc72d384700 1 --2- 172.21.15.2:0/2944368175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc728079d00 0x7fc72810a100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc7240098a0 tx=0x7fc724009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9cb2383e81e1a4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.919+0000 7fc71ffff700 1 -- 172.21.15.2:0/2944368175 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc724004160 con 0x7fc728079d00 2023-06-23T18:14:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.919+0000 7fc71ffff700 1 -- 172.21.15.2:0/2944368175 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc7240042c0 con 0x7fc728079d00 2023-06-23T18:14:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.919+0000 7fc71ffff700 1 -- 172.21.15.2:0/2944368175 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc724011940 con 0x7fc728079d00 2023-06-23T18:14:03.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.920+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2944368175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc728079d00 msgr2=0x7fc72810a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:03.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.920+0000 7fc72f5e8700 1 --2- 172.21.15.2:0/2944368175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc728079d00 0x7fc72810a100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc7240098a0 tx=0x7fc724009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:03.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.920+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2944368175 shutdown_connections 2023-06-23T18:14:03.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.920+0000 7fc72f5e8700 1 --2- 172.21.15.2:0/2944368175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc728079d00 0x7fc72810a100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:03.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.920+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2944368175 >> 172.21.15.2:0/2944368175 conn(0x7fc7280ff6f0 msgr2=0x7fc728101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:03.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.920+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2944368175 shutdown_connections 2023-06-23T18:14:03.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.920+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2944368175 wait complete. 2023-06-23T18:14:03.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.921+0000 7fc72f5e8700 1 Processor -- start 2023-06-23T18:14:03.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.921+0000 7fc72f5e8700 1 -- start start 2023-06-23T18:14:03.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.921+0000 7fc72f5e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc72819b010 0x7fc72819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:03.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.921+0000 7fc72f5e8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72819b960 con 0x7fc72819b010 2023-06-23T18:14:03.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.921+0000 7fc72d384700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc72819b010 0x7fc72819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:03.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.921+0000 7fc72d384700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc72819b010 0x7fc72819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51162/0 (socket says 172.21.15.2:51162) 2023-06-23T18:14:03.861 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.921+0000 7fc72d384700 1 -- 172.21.15.2:0/2716268343 learned_addr learned my addr 172.21.15.2:0/2716268343 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:03.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.921+0000 7fc72d384700 1 -- 172.21.15.2:0/2716268343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724009550 con 0x7fc72819b010 2023-06-23T18:14:03.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.922+0000 7fc72d384700 1 --2- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc72819b010 0x7fc72819b420 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc724006af0 tx=0x7fc724011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:03.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.922+0000 7fc71e7fc700 1 -- 172.21.15.2:0/2716268343 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc724003bf0 con 0x7fc72819b010 2023-06-23T18:14:03.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.922+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72819bb60 con 0x7fc72819b010 2023-06-23T18:14:03.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.922+0000 7fc71e7fc700 1 -- 172.21.15.2:0/2716268343 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fc72401b8b0 con 0x7fc72819b010 2023-06-23T18:14:03.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.922+0000 7fc71e7fc700 1 -- 172.21.15.2:0/2716268343 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7240197e0 con 0x7fc72819b010 2023-06-23T18:14:03.862 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.922+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72819e7c0 con 0x7fc72819b010 2023-06-23T18:14:03.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.923+0000 7fc71e7fc700 1 -- 172.21.15.2:0/2716268343 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fc724019aa0 con 0x7fc72819b010 2023-06-23T18:14:03.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.923+0000 7fc71e7fc700 1 --2- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fc7140386c0 0x7fc71403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:03.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.923+0000 7fc71e7fc700 1 -- 172.21.15.2:0/2716268343 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc72404ccf0 con 0x7fc72819b010 2023-06-23T18:14:03.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.923+0000 7fc72cb83700 1 -- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fc7140386c0 msgr2=0x7fc71403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/647669756 2023-06-23T18:14:03.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.923+0000 7fc72cb83700 1 --2- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fc7140386c0 0x7fc71403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:14:03.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.923+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc728061b00 con 0x7fc72819b010 2023-06-23T18:14:03.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:02.933+0000 7fc71e7fc700 1 -- 172.21.15.2:0/2716268343 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc7240189a0 con 0x7fc72819b010 2023-06-23T18:14:03.863 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.124+0000 7fc72cb83700 1 -- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fc7140386c0 msgr2=0x7fc71403ab70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/647669756 2023-06-23T18:14:03.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.124+0000 7fc72cb83700 1 --2- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fc7140386c0 0x7fc71403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T18:14:03.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.413+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc72819eae0 con 0x7fc72819b010 2023-06-23T18:14:03.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.413+0000 7fc71e7fc700 1 -- 172.21.15.2:0/2716268343 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc724040070 con 0x7fc72819b010 2023-06-23T18:14:03.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.415+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fc7140386c0 msgr2=0x7fc71403ab70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:14:03.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 --2- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fc7140386c0 0x7fc71403ab70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:03.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc72819b010 msgr2=0x7fc72819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:03.864 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 --2- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc72819b010 0x7fc72819b420 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc724006af0 tx=0x7fc724011a30 comp rx=0 tx=0).stop 2023-06-23T18:14:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 shutdown_connections 2023-06-23T18:14:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 --2- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7fc7140386c0 0x7fc71403ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 --2- 172.21.15.2:0/2716268343 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc72819b010 0x7fc72819b420 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 >> 172.21.15.2:0/2716268343 conn(0x7fc7280ff6f0 msgr2=0x7fc728100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 shutdown_connections 2023-06-23T18:14:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:03.416+0000 7fc72f5e8700 1 -- 172.21.15.2:0/2716268343 wait complete. 2023-06-23T18:14:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2023-06-23T18:14:03.865 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 5... 2023-06-23T18:14:04.090 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:03 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2716268343' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-06-23T18:14:04.091 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:03.964+0000 7f1843652200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:14:05.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:04.653+0000 7f1843652200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T18:14:05.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:04.844+0000 7f1843652200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:14:06.135 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:05 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:05.803+0000 7f1843652200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:14:06.135 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:05 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:05.913+0000 7f1843652200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:14:06.732 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:06.130+0000 7f1843652200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:14:06.732 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:06.474+0000 7f1843652200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T18:14:07.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:06.726+0000 7f1843652200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:14:07.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:06.907+0000 7f1843652200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:14:07.551 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:07 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:07.233+0000 7f1843652200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:14:08.456 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:08.052+0000 7f1843652200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:14:08.456 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:08.209+0000 7f1843652200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:14:08.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:08.450+0000 7f1843652200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:14:08.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:08.586+0000 7f1843652200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:14:09.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:08.856+0000 7f1843652200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:14:10.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:09.888+0000 7f1843652200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:14:10.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: Active manager daemon x restarted 2023-06-23T18:14:10.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: Activating manager daemon x 2023-06-23T18:14:10.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: osdmap e2: 0 total, 0 up, 0 in 2023-06-23T18:14:10.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: mgrmap e6: x(active, starting, since 0.00360952s) 2023-06-23T18:14:10.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:14:10.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-23T18:14:10.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-23T18:14:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-23T18:14:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-23T18:14:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: Manager daemon x is now available 2023-06-23T18:14:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:10.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:09 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:11.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:10 smithi002 ceph-mon[95947]: Found migration_current of "None". Setting to last migration. 2023-06-23T18:14:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:10 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:10 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-23T18:14:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:10 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-23T18:14:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:10 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:10 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:10 smithi002 ceph-mon[95947]: mgrmap e7: x(active, since 1.0051s) 2023-06-23T18:14:11.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-06-23T18:14:11.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-06-23T18:14:11.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2023-06-23T18:14:11.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-06-23T18:14:11.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.353+0000 7f4ff9ecb700 1 Processor -- start 2023-06-23T18:14:11.517 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.353+0000 7f4ff9ecb700 1 -- start start 2023-06-23T18:14:11.518 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.353+0000 7f4ff9ecb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff4104470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:11.518 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.353+0000 7f4ff9ecb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff41049b0 con 0x7f4ff4104060 2023-06-23T18:14:11.518 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.353+0000 7f4ff37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff4104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:11.518 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.353+0000 7f4ff37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff4104470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51170/0 (socket says 172.21.15.2:51170) 2023-06-23T18:14:11.518 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.353+0000 7f4ff37fe700 1 -- 172.21.15.2:0/3097927780 learned_addr learned my addr 172.21.15.2:0/3097927780 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:11.518 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.354+0000 7f4ff37fe700 1 -- 172.21.15.2:0/3097927780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff4104af0 con 0x7f4ff4104060 2023-06-23T18:14:11.518 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.354+0000 7f4ff37fe700 1 --2- 172.21.15.2:0/3097927780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff4104470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400bb40 tx=0x7f4fe400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0e92b1c549fb89 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.354+0000 7f4ff27fc700 1 -- 172.21.15.2:0/3097927780 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe400dbe0 con 0x7f4ff4104060 2023-06-23T18:14:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.354+0000 7f4ff27fc700 1 -- 172.21.15.2:0/3097927780 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4fe40044a0 con 0x7f4ff4104060 2023-06-23T18:14:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.354+0000 7f4ff27fc700 1 -- 172.21.15.2:0/3097927780 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe4003da0 con 0x7f4ff4104060 2023-06-23T18:14:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.355+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/3097927780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 msgr2=0x7f4ff4104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.355+0000 7f4ff9ecb700 1 --2- 172.21.15.2:0/3097927780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff4104470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400bb40 tx=0x7f4fe400be50 comp rx=0 tx=0).stop 2023-06-23T18:14:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.355+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/3097927780 shutdown_connections 2023-06-23T18:14:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.355+0000 7f4ff9ecb700 1 --2- 172.21.15.2:0/3097927780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff4104470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:11.519 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.355+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/3097927780 >> 172.21.15.2:0/3097927780 conn(0x7f4ff40ff6f0 msgr2=0x7f4ff4101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:11.520 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.355+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/3097927780 shutdown_connections 2023-06-23T18:14:11.520 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.355+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/3097927780 wait complete. 2023-06-23T18:14:11.520 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.356+0000 7f4ff9ecb700 1 Processor -- start 2023-06-23T18:14:11.520 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.356+0000 7f4ff9ecb700 1 -- start start 2023-06-23T18:14:11.520 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.356+0000 7f4ff9ecb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:11.520 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.356+0000 7f4ff9ecb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff419b5c0 con 0x7f4ff4104060 2023-06-23T18:14:11.520 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.356+0000 7f4ff37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:11.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.356+0000 7f4ff37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff419b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51172/0 (socket says 172.21.15.2:51172) 2023-06-23T18:14:11.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.356+0000 7f4ff37fe700 1 -- 172.21.15.2:0/403785577 learned_addr learned my addr 172.21.15.2:0/403785577 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:11.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.357+0000 7f4ff37fe700 1 -- 172.21.15.2:0/403785577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe400b7f0 con 0x7f4ff4104060 2023-06-23T18:14:11.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.357+0000 7f4ff37fe700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff419b080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40039a0 tx=0x7f4fe4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:11.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.357+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe40105f0 con 0x7f4ff4104060 2023-06-23T18:14:11.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.357+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4fe4010bc0 con 0x7f4ff4104060 2023-06-23T18:14:11.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.357+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff419b7c0 con 0x7f4ff4104060 2023-06-23T18:14:11.521 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.357+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe4021a60 con 0x7f4ff4104060 2023-06-23T18:14:11.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.357+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff419bb80 con 0x7f4ff4104060 2023-06-23T18:14:11.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.358+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4fe4021c90 con 0x7f4ff4104060 2023-06-23T18:14:11.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.358+0000 7f4ff0ff9700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 0x7f4fdc03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:11.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.358+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 --> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fdc03b320 con 0x7f4fdc038760 2023-06-23T18:14:11.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.358+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4fe404c940 con 0x7f4ff4104060 2023-06-23T18:14:11.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.358+0000 7f4ff2ffd700 1 -- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 msgr2=0x7f4fdc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/647669756 2023-06-23T18:14:11.522 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.358+0000 7f4ff2ffd700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 0x7f4fdc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:14:11.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.559+0000 7f4ff2ffd700 1 -- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 msgr2=0x7f4fdc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/647669756 2023-06-23T18:14:11.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.559+0000 7f4ff2ffd700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 0x7f4fdc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T18:14:11.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.960+0000 7f4ff2ffd700 1 -- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 msgr2=0x7f4fdc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/647669756 2023-06-23T18:14:11.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:04.960+0000 7f4ff2ffd700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 0x7f4fdc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-23T18:14:11.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:05.761+0000 7f4ff2ffd700 1 -- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 msgr2=0x7f4fdc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/647669756 2023-06-23T18:14:11.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:05.761+0000 7f4ff2ffd700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 0x7f4fdc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-23T18:14:11.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:07.362+0000 7f4ff2ffd700 1 -- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 msgr2=0x7f4fdc03ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/647669756 2023-06-23T18:14:11.523 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:07.362+0000 7f4ff2ffd700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 0x7f4fdc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-06-23T18:14:11.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:09.904+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f4fe40193f0 con 0x7f4ff4104060 2023-06-23T18:14:11.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:09.904+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 msgr2=0x7f4fdc03ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:14:11.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:09.904+0000 7f4ff0ff9700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 0x7f4fdc03ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:11.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.905+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f4fe4019770 con 0x7f4ff4104060 2023-06-23T18:14:11.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.905+0000 7f4ff0ff9700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f4fdc03bfe0 0x7f4fdc03e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:11.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.905+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fdc03b320 con 0x7f4fdc03bfe0 2023-06-23T18:14:11.524 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.906+0000 7f4ff2ffd700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f4fdc03bfe0 0x7f4fdc03e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:11.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.906+0000 7f4ff2ffd700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f4fdc03bfe0 0x7f4fdc03e3c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8003a10 tx=0x7f4fe80092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:11.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.907+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mgr.14118 v2:172.21.15.2:6800/3628945044 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f4fdc03b320 con 0x7f4fdc03bfe0 2023-06-23T18:14:11.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4ff419c3c0 con 0x7f4fdc03bfe0 2023-06-23T18:14:11.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff0ff9700 1 -- 172.21.15.2:0/403785577 <== mgr.14118 v2:172.21.15.2:6800/3628945044 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4ff419c3c0 con 0x7f4fdc03bfe0 2023-06-23T18:14:11.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f4fdc03bfe0 msgr2=0x7f4fdc03e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:11.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f4fdc03bfe0 0x7f4fdc03e3c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8003a10 tx=0x7f4fe80092b0 comp rx=0 tx=0).stop 2023-06-23T18:14:11.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 msgr2=0x7f4ff419b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:11.525 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff419b080 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40039a0 tx=0x7f4fe4003f70 comp rx=0 tx=0).stop 2023-06-23T18:14:11.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 shutdown_connections 2023-06-23T18:14:11.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f4fdc03bfe0 0x7f4fdc03e3c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:11.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ff4104060 0x7f4ff419b080 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:11.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 --2- 172.21.15.2:0/403785577 >> [v2:172.21.15.2:6800/647669756,v1:172.21.15.2:6801/647669756] conn(0x7f4fdc038760 0x7f4fdc03ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:11.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.918+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 >> 172.21.15.2:0/403785577 conn(0x7f4ff40ff6f0 msgr2=0x7f4ff418dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:11.526 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.919+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 shutdown_connections 2023-06-23T18:14:11.527 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:10.919+0000 7f4ff9ecb700 1 -- 172.21.15.2:0/403785577 wait complete. 2023-06-23T18:14:11.527 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 5 is available 2023-06-23T18:14:11.527 INFO:teuthology.orchestra.run.smithi002.stdout:Setting orchestrator backend to cephadm... 2023-06-23T18:14:11.963 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:11 smithi002 ceph-mon[95947]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-06-23T18:14:11.963 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:11 smithi002 ceph-mon[95947]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-06-23T18:14:12.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:11 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:12.764 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.980+0000 7f32b0e8e700 1 Processor -- start 2023-06-23T18:14:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.980+0000 7f32b0e8e700 1 -- start start 2023-06-23T18:14:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.981+0000 7f32b0e8e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.981+0000 7f32b0e8e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1036a0 con 0x7f32ac102cc0 2023-06-23T18:14:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.981+0000 7f32aa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.981+0000 7f32aa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac1030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51224/0 (socket says 172.21.15.2:51224) 2023-06-23T18:14:12.765 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.981+0000 7f32aa59c700 1 -- 172.21.15.2:0/1666483963 learned_addr learned my addr 172.21.15.2:0/1666483963 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.981+0000 7f32aa59c700 1 -- 172.21.15.2:0/1666483963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac1037e0 con 0x7f32ac102cc0 2023-06-23T18:14:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.981+0000 7f32aa59c700 1 --2- 172.21.15.2:0/1666483963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac1030d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098a0 tx=0x7f329c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88ef68e818201c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32a959a700 1 -- 172.21.15.2:0/1666483963 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f329c004160 con 0x7f32ac102cc0 2023-06-23T18:14:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32a959a700 1 -- 172.21.15.2:0/1666483963 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f329c0042c0 con 0x7f32ac102cc0 2023-06-23T18:14:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32a959a700 1 -- 172.21.15.2:0/1666483963 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f329c011940 con 0x7f32ac102cc0 2023-06-23T18:14:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32b0e8e700 1 -- 172.21.15.2:0/1666483963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 msgr2=0x7f32ac1030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32b0e8e700 1 --2- 172.21.15.2:0/1666483963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac1030d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098a0 tx=0x7f329c009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:12.766 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32b0e8e700 1 -- 172.21.15.2:0/1666483963 shutdown_connections 2023-06-23T18:14:12.767 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32b0e8e700 1 --2- 172.21.15.2:0/1666483963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac1030d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:12.767 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32b0e8e700 1 -- 172.21.15.2:0/1666483963 >> 172.21.15.2:0/1666483963 conn(0x7f32ac0fe9f0 msgr2=0x7f32ac100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:12.767 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32b0e8e700 1 -- 172.21.15.2:0/1666483963 shutdown_connections 2023-06-23T18:14:12.767 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.982+0000 7f32b0e8e700 1 -- 172.21.15.2:0/1666483963 wait complete. 2023-06-23T18:14:12.767 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.983+0000 7f32b0e8e700 1 Processor -- start 2023-06-23T18:14:12.767 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.983+0000 7f32b0e8e700 1 -- start start 2023-06-23T18:14:12.767 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.983+0000 7f32b0e8e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:12.768 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.983+0000 7f32b0e8e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1992a0 con 0x7f32ac102cc0 2023-06-23T18:14:12.768 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32aa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:12.768 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32aa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51226/0 (socket says 172.21.15.2:51226) 2023-06-23T18:14:12.768 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32aa59c700 1 -- 172.21.15.2:0/3234611989 learned_addr learned my addr 172.21.15.2:0/3234611989 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:12.768 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32aa59c700 1 -- 172.21.15.2:0/3234611989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c009550 con 0x7f32ac102cc0 2023-06-23T18:14:12.768 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32aa59c700 1 --2- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac198d60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f329c011dc0 tx=0x7f329c011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:12.768 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32a37fe700 1 -- 172.21.15.2:0/3234611989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f329c0037d0 con 0x7f32ac102cc0 2023-06-23T18:14:12.769 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32a37fe700 1 -- 172.21.15.2:0/3234611989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f329c01b440 con 0x7f32ac102cc0 2023-06-23T18:14:12.769 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac1994a0 con 0x7f32ac102cc0 2023-06-23T18:14:12.769 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32a37fe700 1 -- 172.21.15.2:0/3234611989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f329c01a560 con 0x7f32ac102cc0 2023-06-23T18:14:12.769 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.984+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac1998c0 con 0x7f32ac102cc0 2023-06-23T18:14:12.769 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.985+0000 7f32a37fe700 1 -- 172.21.15.2:0/3234611989 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f329c01b5b0 con 0x7f32ac102cc0 2023-06-23T18:14:12.769 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.985+0000 7f32a37fe700 1 --2- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3298038710 0x7f329803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:12.769 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.986+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac061b00 con 0x7f32ac102cc0 2023-06-23T18:14:12.769 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.986+0000 7f32a37fe700 1 -- 172.21.15.2:0/3234611989 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f329c04c650 con 0x7f32ac102cc0 2023-06-23T18:14:12.770 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.987+0000 7f32a9d9b700 1 --2- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3298038710 0x7f329803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:12.770 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.987+0000 7f32a9d9b700 1 --2- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3298038710 0x7f329803abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3294006fd0 tx=0x7f3294006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:12.770 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:11.991+0000 7f32a37fe700 1 -- 172.21.15.2:0/3234611989 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f329c01f080 con 0x7f32ac102cc0 2023-06-23T18:14:12.770 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.376+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f32ac0ffe30 con 0x7f3298038710 2023-06-23T18:14:12.770 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.383+0000 7f32a37fe700 1 -- 172.21.15.2:0/3234611989 <== mgr.14118 v2:172.21.15.2:6800/3628945044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f32ac0ffe30 con 0x7f3298038710 2023-06-23T18:14:12.770 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.385+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3298038710 msgr2=0x7f329803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:12.770 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.385+0000 7f32b0e8e700 1 --2- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3298038710 0x7f329803abc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3294006fd0 tx=0x7f3294006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.385+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 msgr2=0x7f32ac198d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.385+0000 7f32b0e8e700 1 --2- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac198d60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f329c011dc0 tx=0x7f329c011ea0 comp rx=0 tx=0).stop 2023-06-23T18:14:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.385+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 shutdown_connections 2023-06-23T18:14:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.385+0000 7f32b0e8e700 1 --2- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3298038710 0x7f329803abc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.385+0000 7f32b0e8e700 1 --2- 172.21.15.2:0/3234611989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32ac102cc0 0x7f32ac198d60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.386+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 >> 172.21.15.2:0/3234611989 conn(0x7f32ac0fe9f0 msgr2=0x7f32ac0ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.386+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 shutdown_connections 2023-06-23T18:14:12.771 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:12.386+0000 7f32b0e8e700 1 -- 172.21.15.2:0/3234611989 wait complete. 2023-06-23T18:14:13.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:12 smithi002 ceph-mon[95947]: [23/Jun/2023:18:14:11] ENGINE Bus STARTING 2023-06-23T18:14:13.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:12 smithi002 ceph-mon[95947]: [23/Jun/2023:18:14:11] ENGINE Serving on http://172.21.15.2:8765 2023-06-23T18:14:13.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:12 smithi002 ceph-mon[95947]: [23/Jun/2023:18:14:11] ENGINE Serving on https://172.21.15.2:7150 2023-06-23T18:14:13.034 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:12 smithi002 ceph-mon[95947]: [23/Jun/2023:18:14:11] ENGINE Bus STARTED 2023-06-23T18:14:13.035 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:12 smithi002 ceph-mon[95947]: mgrmap e8: x(active, since 2s) 2023-06-23T18:14:13.035 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:12 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:13.035 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:12 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:14.110 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout value unchanged 2023-06-23T18:14:14.110 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.295+0000 7f85be067700 1 Processor -- start 2023-06-23T18:14:14.111 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.295+0000 7f85be067700 1 -- start start 2023-06-23T18:14:14.111 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.295+0000 7f85be067700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8108050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:14.111 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.295+0000 7f85be067700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8079d80 con 0x7f85b8105c70 2023-06-23T18:14:14.111 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.295+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:14.111 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.296+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51230/0 (socket says 172.21.15.2:51230) 2023-06-23T18:14:14.111 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.296+0000 7f85b77fe700 1 -- 172.21.15.2:0/3986045782 learned_addr learned my addr 172.21.15.2:0/3986045782 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:14.111 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.296+0000 7f85b77fe700 1 -- 172.21.15.2:0/3986045782 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b8079ec0 con 0x7f85b8105c70 2023-06-23T18:14:14.112 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.296+0000 7f85b77fe700 1 --2- 172.21.15.2:0/3986045782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8108050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85a80098a0 tx=0x7f85a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cdbeb56413ca1b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:14.112 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.296+0000 7f85b67fc700 1 -- 172.21.15.2:0/3986045782 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85a8004160 con 0x7f85b8105c70 2023-06-23T18:14:14.112 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.296+0000 7f85b67fc700 1 -- 172.21.15.2:0/3986045782 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f85a80042c0 con 0x7f85b8105c70 2023-06-23T18:14:14.112 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.296+0000 7f85b67fc700 1 -- 172.21.15.2:0/3986045782 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85a8011940 con 0x7f85b8105c70 2023-06-23T18:14:14.112 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.297+0000 7f85be067700 1 -- 172.21.15.2:0/3986045782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 msgr2=0x7f85b8108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:14.112 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.297+0000 7f85be067700 1 --2- 172.21.15.2:0/3986045782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8108050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85a80098a0 tx=0x7f85a8009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:14.112 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.297+0000 7f85be067700 1 -- 172.21.15.2:0/3986045782 shutdown_connections 2023-06-23T18:14:14.113 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.297+0000 7f85be067700 1 --2- 172.21.15.2:0/3986045782 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8108050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:14.113 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.297+0000 7f85be067700 1 -- 172.21.15.2:0/3986045782 >> 172.21.15.2:0/3986045782 conn(0x7f85b80ff6f0 msgr2=0x7f85b8101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:14.113 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.297+0000 7f85be067700 1 -- 172.21.15.2:0/3986045782 shutdown_connections 2023-06-23T18:14:14.113 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.297+0000 7f85be067700 1 -- 172.21.15.2:0/3986045782 wait complete. 2023-06-23T18:14:14.113 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.297+0000 7f85be067700 1 Processor -- start 2023-06-23T18:14:14.113 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.298+0000 7f85be067700 1 -- start start 2023-06-23T18:14:14.113 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.298+0000 7f85be067700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:14.113 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.298+0000 7f85be067700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8199380 con 0x7f85b8105c70 2023-06-23T18:14:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.298+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.298+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51232/0 (socket says 172.21.15.2:51232) 2023-06-23T18:14:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.298+0000 7f85b77fe700 1 -- 172.21.15.2:0/221246491 learned_addr learned my addr 172.21.15.2:0/221246491 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.298+0000 7f85b77fe700 1 -- 172.21.15.2:0/221246491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8009550 con 0x7f85b8105c70 2023-06-23T18:14:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.298+0000 7f85b77fe700 1 --2- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8198e40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.299+0000 7f85b4ff9700 1 -- 172.21.15.2:0/221246491 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85a80036a0 con 0x7f85b8105c70 2023-06-23T18:14:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.299+0000 7f85b4ff9700 1 -- 172.21.15.2:0/221246491 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f85a8003c70 con 0x7f85b8105c70 2023-06-23T18:14:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.299+0000 7f85b4ff9700 1 -- 172.21.15.2:0/221246491 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85a80193f0 con 0x7f85b8105c70 2023-06-23T18:14:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.299+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b8199580 con 0x7f85b8105c70 2023-06-23T18:14:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.299+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b81999a0 con 0x7f85b8105c70 2023-06-23T18:14:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.300+0000 7f85b4ff9700 1 -- 172.21.15.2:0/221246491 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f85a8019670 con 0x7f85b8105c70 2023-06-23T18:14:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.300+0000 7f85b4ff9700 1 --2- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85a0038710 0x7f85a003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.300+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b8061b00 con 0x7f85b8105c70 2023-06-23T18:14:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.300+0000 7f85b4ff9700 1 -- 172.21.15.2:0/221246491 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f85a804c840 con 0x7f85b8105c70 2023-06-23T18:14:14.115 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.302+0000 7f85b6ffd700 1 --2- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85a0038710 0x7f85a003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:14.116 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.302+0000 7f85b6ffd700 1 --2- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85a0038710 0x7f85a003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85ac006fd0 tx=0x7f85ac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:14.116 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.305+0000 7f85b4ff9700 1 -- 172.21.15.2:0/221246491 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f85a8022950 con 0x7f85b8105c70 2023-06-23T18:14:14.116 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.665+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f85b819c4a0 con 0x7f85a0038710 2023-06-23T18:14:14.116 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.666+0000 7f85b4ff9700 1 -- 172.21.15.2:0/221246491 <== mgr.14118 v2:172.21.15.2:6800/3628945044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f85b819c4a0 con 0x7f85a0038710 2023-06-23T18:14:14.116 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85a0038710 msgr2=0x7f85a003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:14.116 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 --2- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85a0038710 0x7f85a003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85ac006fd0 tx=0x7f85ac006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:14.116 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 msgr2=0x7f85b8198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 --2- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8198e40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f85a8000c00 tx=0x7f85a8011e70 comp rx=0 tx=0).stop 2023-06-23T18:14:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 shutdown_connections 2023-06-23T18:14:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 --2- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85a0038710 0x7f85a003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 --2- 172.21.15.2:0/221246491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f85b8105c70 0x7f85b8198e40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 >> 172.21.15.2:0/221246491 conn(0x7f85b80ff6f0 msgr2=0x7f85b8100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 shutdown_connections 2023-06-23T18:14:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:13.669+0000 7f85be067700 1 -- 172.21.15.2:0/221246491 wait complete. 2023-06-23T18:14:14.117 INFO:teuthology.orchestra.run.smithi002.stdout:Generating ssh key... 2023-06-23T18:14:14.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:13 smithi002 ceph-mon[95947]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:15.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:14 smithi002 ceph-mon[95947]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:15.302 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: Generating public/private rsa key pair. 2023-06-23T18:14:15.302 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: Your identification has been saved in /tmp/tmp9_g_muej/key. 2023-06-23T18:14:15.302 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: Your public key has been saved in /tmp/tmp9_g_muej/key.pub. 2023-06-23T18:14:15.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: The key fingerprint is: 2023-06-23T18:14:15.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: SHA256:XzTuVS2AV3RKqy6Z9EaWu62dFWkjF93YuvkM+LQ1dNo ceph-902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:14:15.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: The key's randomart image is: 2023-06-23T18:14:15.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: +---[RSA 3072]----+ 2023-06-23T18:14:15.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | ..o+ .| 2023-06-23T18:14:15.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | . .o *+| 2023-06-23T18:14:15.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | .o =.*| 2023-06-23T18:14:15.303 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | o + +o| 2023-06-23T18:14:15.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | S . B.oB.| 2023-06-23T18:14:15.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | o X +==+| 2023-06-23T18:14:15.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | = B *.E| 2023-06-23T18:14:15.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | o * O.| 2023-06-23T18:14:15.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: | o.* o| 2023-06-23T18:14:15.304 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: +----[SHA256]-----+ 2023-06-23T18:14:15.556 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.618+0000 7f3d6ca49700 1 Processor -- start 2023-06-23T18:14:15.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.618+0000 7f3d6ca49700 1 -- start start 2023-06-23T18:14:15.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.618+0000 7f3d6ca49700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68104470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:15.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.618+0000 7f3d6ca49700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d681049b0 con 0x7f3d68104060 2023-06-23T18:14:15.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.618+0000 7f3d6659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:15.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.618+0000 7f3d6659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68104470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51236/0 (socket says 172.21.15.2:51236) 2023-06-23T18:14:15.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.618+0000 7f3d6659c700 1 -- 172.21.15.2:0/430043426 learned_addr learned my addr 172.21.15.2:0/430043426 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:15.557 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.618+0000 7f3d6659c700 1 -- 172.21.15.2:0/430043426 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68104af0 con 0x7f3d68104060 2023-06-23T18:14:15.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.619+0000 7f3d6659c700 1 --2- 172.21.15.2:0/430043426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68104470 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d580098a0 tx=0x7f3d58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a29fa810b53cf5f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:15.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.619+0000 7f3d6559a700 1 -- 172.21.15.2:0/430043426 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d58004160 con 0x7f3d68104060 2023-06-23T18:14:15.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.619+0000 7f3d6559a700 1 -- 172.21.15.2:0/430043426 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3d580042c0 con 0x7f3d68104060 2023-06-23T18:14:15.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.619+0000 7f3d6559a700 1 -- 172.21.15.2:0/430043426 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d58011940 con 0x7f3d68104060 2023-06-23T18:14:15.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.619+0000 7f3d6ca49700 1 -- 172.21.15.2:0/430043426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 msgr2=0x7f3d68104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:15.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.619+0000 7f3d6ca49700 1 --2- 172.21.15.2:0/430043426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68104470 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d580098a0 tx=0x7f3d58009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:15.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.620+0000 7f3d6ca49700 1 -- 172.21.15.2:0/430043426 shutdown_connections 2023-06-23T18:14:15.558 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.620+0000 7f3d6ca49700 1 --2- 172.21.15.2:0/430043426 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68104470 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.620+0000 7f3d6ca49700 1 -- 172.21.15.2:0/430043426 >> 172.21.15.2:0/430043426 conn(0x7f3d680ff710 msgr2=0x7f3d68101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.620+0000 7f3d6ca49700 1 -- 172.21.15.2:0/430043426 shutdown_connections 2023-06-23T18:14:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.620+0000 7f3d6ca49700 1 -- 172.21.15.2:0/430043426 wait complete. 2023-06-23T18:14:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.620+0000 7f3d6ca49700 1 Processor -- start 2023-06-23T18:14:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.620+0000 7f3d6ca49700 1 -- start start 2023-06-23T18:14:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6ca49700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68079b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6ca49700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6819bb60 con 0x7f3d68104060 2023-06-23T18:14:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68079b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:15.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68079b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51238/0 (socket says 172.21.15.2:51238) 2023-06-23T18:14:15.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6659c700 1 -- 172.21.15.2:0/4186251553 learned_addr learned my addr 172.21.15.2:0/4186251553 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:15.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6659c700 1 -- 172.21.15.2:0/4186251553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d58009550 con 0x7f3d68104060 2023-06-23T18:14:15.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6659c700 1 --2- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68079b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3d58006af0 tx=0x7f3d58011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:15.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d577fe700 1 -- 172.21.15.2:0/4186251553 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d58003bf0 con 0x7f3d68104060 2023-06-23T18:14:15.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d577fe700 1 -- 172.21.15.2:0/4186251553 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f3d5801b8b0 con 0x7f3d68104060 2023-06-23T18:14:15.560 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6819bca0 con 0x7f3d68104060 2023-06-23T18:14:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d577fe700 1 -- 172.21.15.2:0/4186251553 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d580197b0 con 0x7f3d68104060 2023-06-23T18:14:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.621+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6819c0c0 con 0x7f3d68104060 2023-06-23T18:14:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.622+0000 7f3d577fe700 1 -- 172.21.15.2:0/4186251553 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f3d58019a70 con 0x7f3d68104060 2023-06-23T18:14:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.622+0000 7f3d577fe700 1 --2- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3d500386c0 0x7f3d5003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.623+0000 7f3d577fe700 1 -- 172.21.15.2:0/4186251553 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3d5804cc90 con 0x7f3d68104060 2023-06-23T18:14:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.623+0000 7f3d65d9b700 1 --2- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3d500386c0 0x7f3d5003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.623+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d68061b00 con 0x7f3d68104060 2023-06-23T18:14:15.561 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.623+0000 7f3d65d9b700 1 --2- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3d500386c0 0x7f3d5003ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006fd0 tx=0x7f3d5c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:15.562 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.628+0000 7f3d577fe700 1 -- 172.21.15.2:0/4186251553 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3d580189a0 con 0x7f3d68104060 2023-06-23T18:14:15.562 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:14.989+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d681019b0 con 0x7f3d500386c0 2023-06-23T18:14:15.562 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.136+0000 7f3d577fe700 1 -- 172.21.15.2:0/4186251553 <== mgr.14118 v2:172.21.15.2:6800/3628945044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3d681019b0 con 0x7f3d500386c0 2023-06-23T18:14:15.562 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3d500386c0 msgr2=0x7f3d5003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:15.562 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 --2- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3d500386c0 0x7f3d5003ab70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006fd0 tx=0x7f3d5c006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:15.562 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 msgr2=0x7f3d68079b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:15.562 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 --2- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68079b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3d58006af0 tx=0x7f3d58011a30 comp rx=0 tx=0).stop 2023-06-23T18:14:15.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 shutdown_connections 2023-06-23T18:14:15.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 --2- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f3d500386c0 0x7f3d5003ab70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:15.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 --2- 172.21.15.2:0/4186251553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d68104060 0x7f3d68079b50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:15.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 >> 172.21.15.2:0/4186251553 conn(0x7f3d680ff710 msgr2=0x7f3d68100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:15.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 shutdown_connections 2023-06-23T18:14:15.563 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:15.138+0000 7f3d6ca49700 1 -- 172.21.15.2:0/4186251553 wait complete. 2023-06-23T18:14:16.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:16 smithi002 ceph-mon[95947]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:16.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:16 smithi002 ceph-mon[95947]: Generating ssh key... 2023-06-23T18:14:16.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:16 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:16.467 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:16 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:17.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3UxW0vk5sj0hepuJozcIPkmmrGMuP/+LjXr4Z2Xn1b49oxRm9b5g3Hj3AEvNSW8mn5BuOueRSh1aINIKQ0eia4W3kwEUvjDmQKj4qHefoMSua5U9pbFLlJEpGtHTrUguH7ppsEoLzeqAuBAleLTN49gPPC/UKgFwCtgKsnGINPFLSd5ebdAg5hIz0+6eps+oKe/eo5seyr5b6FgukxtFxNa8aiaVKrMNo7PhKbvwloMFeeWU/QpaMODQvQ1jSWr1AHpbQZuTySK9jDPE/1TGlHyt0A2hiRaJSNAdbbxRa0JYGCtT1aay0nzG7xiDuoPqE4pYal2gDzZX54RWbb0TthtgrJ3xxMZi6qYqALYgobFvebJQBbUhgng0B8SdvEpUIQRCTmgddWaqaUo+BFOt2QPmkaFryGKMx9ubX2LdCL/YKPGgKWJe/acmoO6XVqMUBkG1z2wi4PyfJEllH6CwAENvo2hlc27gDC+oiZ73xAgho1+tFB93HZ+RW600vJD8= ceph-902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:14:17.002 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.056+0000 7fe957480700 1 Processor -- start 2023-06-23T18:14:17.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.056+0000 7fe957480700 1 -- start start 2023-06-23T18:14:17.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe957480700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950104470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:17.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe957480700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501049b0 con 0x7fe950104060 2023-06-23T18:14:17.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe95521c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:17.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe95521c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950104470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51242/0 (socket says 172.21.15.2:51242) 2023-06-23T18:14:17.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe95521c700 1 -- 172.21.15.2:0/1555115999 learned_addr learned my addr 172.21.15.2:0/1555115999 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:17.003 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe95521c700 1 -- 172.21.15.2:0/1555115999 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950104af0 con 0x7fe950104060 2023-06-23T18:14:17.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe95521c700 1 --2- 172.21.15.2:0/1555115999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950104470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0098a0 tx=0x7fe94c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5892a4b6a4aea35a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:17.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe947fff700 1 -- 172.21.15.2:0/1555115999 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe94c004160 con 0x7fe950104060 2023-06-23T18:14:17.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.057+0000 7fe947fff700 1 -- 172.21.15.2:0/1555115999 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe94c0042c0 con 0x7fe950104060 2023-06-23T18:14:17.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.058+0000 7fe947fff700 1 -- 172.21.15.2:0/1555115999 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe94c011940 con 0x7fe950104060 2023-06-23T18:14:17.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.058+0000 7fe957480700 1 -- 172.21.15.2:0/1555115999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 msgr2=0x7fe950104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:17.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.058+0000 7fe957480700 1 --2- 172.21.15.2:0/1555115999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950104470 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe94c0098a0 tx=0x7fe94c009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:17.004 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.058+0000 7fe957480700 1 -- 172.21.15.2:0/1555115999 shutdown_connections 2023-06-23T18:14:17.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.058+0000 7fe957480700 1 --2- 172.21.15.2:0/1555115999 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950104470 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:17.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.058+0000 7fe957480700 1 -- 172.21.15.2:0/1555115999 >> 172.21.15.2:0/1555115999 conn(0x7fe9500ff6f0 msgr2=0x7fe950101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:17.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.058+0000 7fe957480700 1 -- 172.21.15.2:0/1555115999 shutdown_connections 2023-06-23T18:14:17.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.058+0000 7fe957480700 1 -- 172.21.15.2:0/1555115999 wait complete. 2023-06-23T18:14:17.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.059+0000 7fe957480700 1 Processor -- start 2023-06-23T18:14:17.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.059+0000 7fe957480700 1 -- start start 2023-06-23T18:14:17.005 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.059+0000 7fe957480700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950079b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.059+0000 7fe957480700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95019bb80 con 0x7fe950104060 2023-06-23T18:14:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.059+0000 7fe95521c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950079b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.059+0000 7fe95521c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950079b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51244/0 (socket says 172.21.15.2:51244) 2023-06-23T18:14:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.059+0000 7fe95521c700 1 -- 172.21.15.2:0/2594654182 learned_addr learned my addr 172.21.15.2:0/2594654182 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.059+0000 7fe95521c700 1 -- 172.21.15.2:0/2594654182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c009550 con 0x7fe950104060 2023-06-23T18:14:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.060+0000 7fe95521c700 1 --2- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950079b30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe94c006af0 tx=0x7fe94c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.060+0000 7fe9467fc700 1 -- 172.21.15.2:0/2594654182 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe94c003bf0 con 0x7fe950104060 2023-06-23T18:14:17.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.060+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95019bcc0 con 0x7fe950104060 2023-06-23T18:14:17.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.060+0000 7fe9467fc700 1 -- 172.21.15.2:0/2594654182 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe94c01b8b0 con 0x7fe950104060 2023-06-23T18:14:17.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.060+0000 7fe9467fc700 1 -- 172.21.15.2:0/2594654182 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe94c0197b0 con 0x7fe950104060 2023-06-23T18:14:17.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.060+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95019c0e0 con 0x7fe950104060 2023-06-23T18:14:17.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.061+0000 7fe9467fc700 1 -- 172.21.15.2:0/2594654182 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe94c01ba20 con 0x7fe950104060 2023-06-23T18:14:17.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.061+0000 7fe9467fc700 1 --2- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe93c0386c0 0x7fe93c03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:17.007 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.061+0000 7fe9467fc700 1 -- 172.21.15.2:0/2594654182 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe94c04ca30 con 0x7fe950104060 2023-06-23T18:14:17.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.061+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe950061b00 con 0x7fe950104060 2023-06-23T18:14:17.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.061+0000 7fe954a1b700 1 --2- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe93c0386c0 0x7fe93c03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:17.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.061+0000 7fe954a1b700 1 --2- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe93c0386c0 0x7fe93c03ab70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe940006fd0 tx=0x7fe940006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:17.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.066+0000 7fe9467fc700 1 -- 172.21.15.2:0/2594654182 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe94c02bb40 con 0x7fe950104060 2023-06-23T18:14:17.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.429+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe950101990 con 0x7fe93c0386c0 2023-06-23T18:14:17.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.429+0000 7fe9467fc700 1 -- 172.21.15.2:0/2594654182 <== mgr.14118 v2:172.21.15.2:6800/3628945044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fe950101990 con 0x7fe93c0386c0 2023-06-23T18:14:17.008 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe93c0386c0 msgr2=0x7fe93c03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 --2- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe93c0386c0 0x7fe93c03ab70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe940006fd0 tx=0x7fe940006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 msgr2=0x7fe950079b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 --2- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950079b30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe94c006af0 tx=0x7fe94c011a30 comp rx=0 tx=0).stop 2023-06-23T18:14:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 shutdown_connections 2023-06-23T18:14:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 --2- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe93c0386c0 0x7fe93c03ab70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 --2- 172.21.15.2:0/2594654182 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe950104060 0x7fe950079b30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 >> 172.21.15.2:0/2594654182 conn(0x7fe9500ff6f0 msgr2=0x7fe950100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 shutdown_connections 2023-06-23T18:14:17.010 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:16.432+0000 7fe957480700 1 -- 172.21.15.2:0/2594654182 wait complete. 2023-06-23T18:14:17.010 INFO:teuthology.orchestra.run.smithi002.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-06-23T18:14:17.010 INFO:teuthology.orchestra.run.smithi002.stdout:Adding key to root@localhost authorized_keys... 2023-06-23T18:14:17.010 INFO:teuthology.orchestra.run.smithi002.stdout:Adding host smithi002... 2023-06-23T18:14:17.227 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:17 smithi002 ceph-mon[95947]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:18.189 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:18 smithi002 ceph-mon[95947]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:20.026 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:19 smithi002 ceph-mon[95947]: Deploying cephadm binary to smithi002 2023-06-23T18:14:21.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Added host 'smithi002' with addr '172.21.15.2' 2023-06-23T18:14:21.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.514+0000 7fe817a4d700 1 Processor -- start 2023-06-23T18:14:21.055 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.514+0000 7fe817a4d700 1 -- start start 2023-06-23T18:14:21.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.514+0000 7fe817a4d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810108080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:21.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.514+0000 7fe817a4d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810079d20 con 0x7fe810105ca0 2023-06-23T18:14:21.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.514+0000 7fe8157e9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:21.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.514+0000 7fe8157e9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51248/0 (socket says 172.21.15.2:51248) 2023-06-23T18:14:21.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.514+0000 7fe8157e9700 1 -- 172.21.15.2:0/3260441072 learned_addr learned my addr 172.21.15.2:0/3260441072 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:21.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.515+0000 7fe8157e9700 1 -- 172.21.15.2:0/3260441072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810079e60 con 0x7fe810105ca0 2023-06-23T18:14:21.056 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.515+0000 7fe8157e9700 1 --2- 172.21.15.2:0/3260441072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810108080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0098a0 tx=0x7fe80c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f26ef834dd3e4e5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:21.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.515+0000 7fe807fff700 1 -- 172.21.15.2:0/3260441072 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe80c004160 con 0x7fe810105ca0 2023-06-23T18:14:21.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.515+0000 7fe807fff700 1 -- 172.21.15.2:0/3260441072 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe80c0042c0 con 0x7fe810105ca0 2023-06-23T18:14:21.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.515+0000 7fe807fff700 1 -- 172.21.15.2:0/3260441072 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe80c011940 con 0x7fe810105ca0 2023-06-23T18:14:21.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.516+0000 7fe817a4d700 1 -- 172.21.15.2:0/3260441072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 msgr2=0x7fe810108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:21.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.516+0000 7fe817a4d700 1 --2- 172.21.15.2:0/3260441072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810108080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0098a0 tx=0x7fe80c009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:21.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.516+0000 7fe817a4d700 1 -- 172.21.15.2:0/3260441072 shutdown_connections 2023-06-23T18:14:21.057 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.516+0000 7fe817a4d700 1 --2- 172.21.15.2:0/3260441072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810108080 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:21.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.516+0000 7fe817a4d700 1 -- 172.21.15.2:0/3260441072 >> 172.21.15.2:0/3260441072 conn(0x7fe8100ff6a0 msgr2=0x7fe810101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:21.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.516+0000 7fe817a4d700 1 -- 172.21.15.2:0/3260441072 shutdown_connections 2023-06-23T18:14:21.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.516+0000 7fe817a4d700 1 -- 172.21.15.2:0/3260441072 wait complete. 2023-06-23T18:14:21.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.516+0000 7fe817a4d700 1 Processor -- start 2023-06-23T18:14:21.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe817a4d700 1 -- start start 2023-06-23T18:14:21.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe817a4d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810105600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:21.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe817a4d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810105b40 con 0x7fe810105ca0 2023-06-23T18:14:21.058 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe8157e9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810105600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe8157e9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810105600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51250/0 (socket says 172.21.15.2:51250) 2023-06-23T18:14:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe8157e9700 1 -- 172.21.15.2:0/3043504612 learned_addr learned my addr 172.21.15.2:0/3043504612 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe8157e9700 1 -- 172.21.15.2:0/3043504612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c009550 con 0x7fe810105ca0 2023-06-23T18:14:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe8157e9700 1 --2- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810105600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe80c006af0 tx=0x7fe80c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe8067fc700 1 -- 172.21.15.2:0/3043504612 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe80c003bf0 con 0x7fe810105ca0 2023-06-23T18:14:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe8067fc700 1 -- 172.21.15.2:0/3043504612 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe80c01b8b0 con 0x7fe810105ca0 2023-06-23T18:14:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe810103c50 con 0x7fe810105ca0 2023-06-23T18:14:21.059 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe8067fc700 1 -- 172.21.15.2:0/3043504612 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe80c0197e0 con 0x7fe810105ca0 2023-06-23T18:14:21.060 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.517+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe810104070 con 0x7fe810105ca0 2023-06-23T18:14:21.060 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.519+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8101091d0 con 0x7fe810105ca0 2023-06-23T18:14:21.060 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.519+0000 7fe8067fc700 1 -- 172.21.15.2:0/3043504612 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe80c01ba20 con 0x7fe810105ca0 2023-06-23T18:14:21.060 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.519+0000 7fe8067fc700 1 --2- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe7fc038710 0x7fe7fc03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:21.060 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.519+0000 7fe8067fc700 1 -- 172.21.15.2:0/3043504612 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe80c04cd00 con 0x7fe810105ca0 2023-06-23T18:14:21.060 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.519+0000 7fe814fe8700 1 --2- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe7fc038710 0x7fe7fc03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:21.060 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.520+0000 7fe814fe8700 1 --2- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe7fc038710 0x7fe7fc03abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe800006fd0 tx=0x7fe800006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:21.061 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.524+0000 7fe8067fc700 1 -- 172.21.15.2:0/3043504612 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe80c00b9b0 con 0x7fe810105ca0 2023-06-23T18:14:21.061 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:17.886+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi002", "addr": "172.21.15.2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe810100b90 con 0x7fe7fc038710 2023-06-23T18:14:21.061 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.634+0000 7fe8067fc700 1 -- 172.21.15.2:0/3043504612 <== mgr.14118 v2:172.21.15.2:6800/3628945044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fe810100b90 con 0x7fe7fc038710 2023-06-23T18:14:21.061 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe7fc038710 msgr2=0x7fe7fc03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:21.061 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 --2- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe7fc038710 0x7fe7fc03abc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe800006fd0 tx=0x7fe800006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:21.061 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 msgr2=0x7fe810105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:21.061 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 --2- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810105600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe80c006af0 tx=0x7fe80c011a30 comp rx=0 tx=0).stop 2023-06-23T18:14:21.061 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 shutdown_connections 2023-06-23T18:14:21.062 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 --2- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fe7fc038710 0x7fe7fc03abc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:21.062 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 --2- 172.21.15.2:0/3043504612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe810105ca0 0x7fe810105600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:21.062 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 >> 172.21.15.2:0/3043504612 conn(0x7fe8100ff6a0 msgr2=0x7fe810100480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:21.062 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 shutdown_connections 2023-06-23T18:14:21.062 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:20.637+0000 7fe817a4d700 1 -- 172.21.15.2:0/3043504612 wait complete. 2023-06-23T18:14:21.062 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mon service... 2023-06-23T18:14:22.009 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:21 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:22.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:21 smithi002 ceph-mon[95947]: Added host smithi002 2023-06-23T18:14:22.010 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:21 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:22.500 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-06-23T18:14:22.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.624+0000 7f277292f700 1 Processor -- start 2023-06-23T18:14:22.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.624+0000 7f277292f700 1 -- start start 2023-06-23T18:14:22.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.624+0000 7f277292f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c105590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:22.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.624+0000 7f277292f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c105ad0 con 0x7f276c105180 2023-06-23T18:14:22.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.624+0000 7f276bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:22.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.624+0000 7f276bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c105590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51284/0 (socket says 172.21.15.2:51284) 2023-06-23T18:14:22.501 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.624+0000 7f276bfff700 1 -- 172.21.15.2:0/3744870369 learned_addr learned my addr 172.21.15.2:0/3744870369 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:22.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.625+0000 7f276bfff700 1 -- 172.21.15.2:0/3744870369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c105c10 con 0x7f276c105180 2023-06-23T18:14:22.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.625+0000 7f276bfff700 1 --2- 172.21.15.2:0/3744870369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c105590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f275c0098a0 tx=0x7f275c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d891dbdc3daed1e0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:22.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.625+0000 7f276affd700 1 -- 172.21.15.2:0/3744870369 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f275c004160 con 0x7f276c105180 2023-06-23T18:14:22.502 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.625+0000 7f276affd700 1 -- 172.21.15.2:0/3744870369 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f275c0042c0 con 0x7f276c105180 2023-06-23T18:14:22.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.625+0000 7f276affd700 1 -- 172.21.15.2:0/3744870369 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f275c011940 con 0x7f276c105180 2023-06-23T18:14:22.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.626+0000 7f277292f700 1 -- 172.21.15.2:0/3744870369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 msgr2=0x7f276c105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:22.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.626+0000 7f277292f700 1 --2- 172.21.15.2:0/3744870369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c105590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f275c0098a0 tx=0x7f275c009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:22.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.626+0000 7f277292f700 1 -- 172.21.15.2:0/3744870369 shutdown_connections 2023-06-23T18:14:22.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.626+0000 7f277292f700 1 --2- 172.21.15.2:0/3744870369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c105590 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:22.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.626+0000 7f277292f700 1 -- 172.21.15.2:0/3744870369 >> 172.21.15.2:0/3744870369 conn(0x7f276c100810 msgr2=0x7f276c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:22.503 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.626+0000 7f277292f700 1 -- 172.21.15.2:0/3744870369 shutdown_connections 2023-06-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.626+0000 7f277292f700 1 -- 172.21.15.2:0/3744870369 wait complete. 2023-06-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.627+0000 7f277292f700 1 Processor -- start 2023-06-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.627+0000 7f277292f700 1 -- start start 2023-06-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.627+0000 7f277292f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c07bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.627+0000 7f277292f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c07c190 con 0x7f276c105180 2023-06-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.627+0000 7f276bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c07bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.627+0000 7f276bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c07bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51286/0 (socket says 172.21.15.2:51286) 2023-06-23T18:14:22.504 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.627+0000 7f276bfff700 1 -- 172.21.15.2:0/506091744 learned_addr learned my addr 172.21.15.2:0/506091744 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:22.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.627+0000 7f276bfff700 1 -- 172.21.15.2:0/506091744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c009550 con 0x7f276c105180 2023-06-23T18:14:22.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.628+0000 7f276bfff700 1 --2- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c07bc50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f275c006af0 tx=0x7f275c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:22.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.628+0000 7f27697fa700 1 -- 172.21.15.2:0/506091744 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f275c003bf0 con 0x7f276c105180 2023-06-23T18:14:22.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.628+0000 7f27697fa700 1 -- 172.21.15.2:0/506091744 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f275c01b8b0 con 0x7f276c105180 2023-06-23T18:14:22.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.628+0000 7f27697fa700 1 -- 172.21.15.2:0/506091744 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f275c0197b0 con 0x7f276c105180 2023-06-23T18:14:22.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.628+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c07a2a0 con 0x7f276c105180 2023-06-23T18:14:22.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.628+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c07a6c0 con 0x7f276c105180 2023-06-23T18:14:22.505 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.629+0000 7f27697fa700 1 -- 172.21.15.2:0/506091744 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f275c019a70 con 0x7f276c105180 2023-06-23T18:14:22.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.629+0000 7f27697fa700 1 --2- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f27540386c0 0x7f275403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:22.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.629+0000 7f27697fa700 1 -- 172.21.15.2:0/506091744 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f275c04cc00 con 0x7f276c105180 2023-06-23T18:14:22.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.629+0000 7f276b7fe700 1 --2- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f27540386c0 0x7f275403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:22.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.629+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f276c061b00 con 0x7f276c105180 2023-06-23T18:14:22.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.630+0000 7f276b7fe700 1 --2- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f27540386c0 0x7f275403ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2760006fd0 tx=0x7f2760006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:22.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:21.634+0000 7f27697fa700 1 -- 172.21.15.2:0/506091744 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f275c0189a0 con 0x7f276c105180 2023-06-23T18:14:22.506 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.024+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f276c07b140 con 0x7f27540386c0 2023-06-23T18:14:22.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.029+0000 7f27697fa700 1 -- 172.21.15.2:0/506091744 <== mgr.14118 v2:172.21.15.2:6800/3628945044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f276c07b140 con 0x7f27540386c0 2023-06-23T18:14:22.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f27540386c0 msgr2=0x7f275403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:22.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 --2- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f27540386c0 0x7f275403ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2760006fd0 tx=0x7f2760006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:22.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 msgr2=0x7f276c07bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:22.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 --2- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c07bc50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f275c006af0 tx=0x7f275c011a30 comp rx=0 tx=0).stop 2023-06-23T18:14:22.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 shutdown_connections 2023-06-23T18:14:22.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 --2- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f27540386c0 0x7f275403ab70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:22.507 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 --2- 172.21.15.2:0/506091744 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f276c105180 0x7f276c07bc50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:22.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 >> 172.21.15.2:0/506091744 conn(0x7f276c100810 msgr2=0x7f276c101560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:22.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 shutdown_connections 2023-06-23T18:14:22.508 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:22.032+0000 7f277292f700 1 -- 172.21.15.2:0/506091744 wait complete. 2023-06-23T18:14:22.508 INFO:teuthology.orchestra.run.smithi002.stdout:Deploying unmanaged mgr service... 2023-06-23T18:14:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:23 smithi002 ceph-mon[95947]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:23 smithi002 ceph-mon[95947]: Saving service mon spec with placement count:5 2023-06-23T18:14:23.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:23 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:24.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-06-23T18:14:24.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.317+0000 7f85734c8700 1 Processor -- start 2023-06-23T18:14:24.145 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.317+0000 7f85734c8700 1 -- start start 2023-06-23T18:14:24.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.317+0000 7f85734c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c104440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:24.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.317+0000 7f85734c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c104980 con 0x7f856c104030 2023-06-23T18:14:24.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.317+0000 7f8571264700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:24.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.317+0000 7f8571264700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c104440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51290/0 (socket says 172.21.15.2:51290) 2023-06-23T18:14:24.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.317+0000 7f8571264700 1 -- 172.21.15.2:0/807115752 learned_addr learned my addr 172.21.15.2:0/807115752 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:24.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.318+0000 7f8571264700 1 -- 172.21.15.2:0/807115752 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c104ac0 con 0x7f856c104030 2023-06-23T18:14:24.146 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.318+0000 7f8571264700 1 --2- 172.21.15.2:0/807115752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c104440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f856800bb40 tx=0x7f856800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbcf164362445b8f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:24.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.318+0000 7f8563fff700 1 -- 172.21.15.2:0/807115752 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f856800dbe0 con 0x7f856c104030 2023-06-23T18:14:24.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.318+0000 7f8563fff700 1 -- 172.21.15.2:0/807115752 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f85680044c0 con 0x7f856c104030 2023-06-23T18:14:24.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.318+0000 7f8563fff700 1 -- 172.21.15.2:0/807115752 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8568003d50 con 0x7f856c104030 2023-06-23T18:14:24.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.319+0000 7f85734c8700 1 -- 172.21.15.2:0/807115752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 msgr2=0x7f856c104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:24.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.319+0000 7f85734c8700 1 --2- 172.21.15.2:0/807115752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c104440 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f856800bb40 tx=0x7f856800be50 comp rx=0 tx=0).stop 2023-06-23T18:14:24.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.319+0000 7f85734c8700 1 -- 172.21.15.2:0/807115752 shutdown_connections 2023-06-23T18:14:24.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.319+0000 7f85734c8700 1 --2- 172.21.15.2:0/807115752 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c104440 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:24.147 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.319+0000 7f85734c8700 1 -- 172.21.15.2:0/807115752 >> 172.21.15.2:0/807115752 conn(0x7f856c0ff6a0 msgr2=0x7f856c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:24.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.319+0000 7f85734c8700 1 -- 172.21.15.2:0/807115752 shutdown_connections 2023-06-23T18:14:24.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.319+0000 7f85734c8700 1 -- 172.21.15.2:0/807115752 wait complete. 2023-06-23T18:14:24.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.319+0000 7f85734c8700 1 Processor -- start 2023-06-23T18:14:24.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.320+0000 7f85734c8700 1 -- start start 2023-06-23T18:14:24.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.320+0000 7f85734c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c19b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:24.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.320+0000 7f85734c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c19b580 con 0x7f856c104030 2023-06-23T18:14:24.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.320+0000 7f8571264700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c19b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:24.148 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.320+0000 7f8571264700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c19b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51292/0 (socket says 172.21.15.2:51292) 2023-06-23T18:14:24.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.320+0000 7f8571264700 1 -- 172.21.15.2:0/2325813297 learned_addr learned my addr 172.21.15.2:0/2325813297 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:24.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.320+0000 7f8571264700 1 -- 172.21.15.2:0/2325813297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856800b7f0 con 0x7f856c104030 2023-06-23T18:14:24.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.320+0000 7f8571264700 1 --2- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c19b040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8568003f60 tx=0x7f8568004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:24.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.321+0000 7f85627fc700 1 -- 172.21.15.2:0/2325813297 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8568010800 con 0x7f856c104030 2023-06-23T18:14:24.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.321+0000 7f85627fc700 1 -- 172.21.15.2:0/2325813297 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8568010df0 con 0x7f856c104030 2023-06-23T18:14:24.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.321+0000 7f85627fc700 1 -- 172.21.15.2:0/2325813297 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8568021cc0 con 0x7f856c104030 2023-06-23T18:14:24.149 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.321+0000 7f85734c8700 1 -- 172.21.15.2:0/2325813297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856c19b780 con 0x7f856c104030 2023-06-23T18:14:24.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.321+0000 7f85734c8700 1 -- 172.21.15.2:0/2325813297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856c19bb40 con 0x7f856c104030 2023-06-23T18:14:24.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.322+0000 7f85627fc700 1 -- 172.21.15.2:0/2325813297 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8568021510 con 0x7f856c104030 2023-06-23T18:14:24.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.322+0000 7f85627fc700 1 --2- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85580386c0 0x7f855803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:24.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.322+0000 7f85627fc700 1 -- 172.21.15.2:0/2325813297 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f856804cc30 con 0x7f856c104030 2023-06-23T18:14:24.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.322+0000 7f8557fff700 1 -- 172.21.15.2:0/2325813297 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f854c0052f0 con 0x7f856c104030 2023-06-23T18:14:24.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.322+0000 7f8570a63700 1 --2- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85580386c0 0x7f855803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:24.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.323+0000 7f8570a63700 1 --2- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85580386c0 0x7f855803ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f855c006fd0 tx=0x7f855c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:24.150 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.331+0000 7f85627fc700 1 -- 172.21.15.2:0/2325813297 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f856800fa20 con 0x7f856c104030 2023-06-23T18:14:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.722+0000 7f8557fff700 1 -- 172.21.15.2:0/2325813297 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f854c000bc0 con 0x7f85580386c0 2023-06-23T18:14:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.727+0000 7f85627fc700 1 -- 172.21.15.2:0/2325813297 <== mgr.14118 v2:172.21.15.2:6800/3628945044 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f854c000bc0 con 0x7f85580386c0 2023-06-23T18:14:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 -- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85580386c0 msgr2=0x7f855803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 --2- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85580386c0 0x7f855803ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f855c006fd0 tx=0x7f855c006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 -- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 msgr2=0x7f856c19b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 --2- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c19b040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8568003f60 tx=0x7f8568004040 comp rx=0 tx=0).stop 2023-06-23T18:14:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 -- 172.21.15.2:0/2325813297 shutdown_connections 2023-06-23T18:14:24.151 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 --2- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f85580386c0 0x7f855803ab70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:24.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 --2- 172.21.15.2:0/2325813297 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f856c104030 0x7f856c19b040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:24.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 -- 172.21.15.2:0/2325813297 >> 172.21.15.2:0/2325813297 conn(0x7f856c0ff6a0 msgr2=0x7f856c18daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:24.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 -- 172.21.15.2:0/2325813297 shutdown_connections 2023-06-23T18:14:24.152 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:23.730+0000 7f8557fff700 1 -- 172.21.15.2:0/2325813297 wait complete. 2023-06-23T18:14:24.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:24 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:24.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:24 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:24.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:24 smithi002 ceph-mon[95947]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:24.383 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:24 smithi002 ceph-mon[95947]: Saving service mgr spec with placement count:2 2023-06-23T18:14:24.384 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:24 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:25.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.657+0000 7fec387c8700 1 Processor -- start 2023-06-23T18:14:25.693 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.657+0000 7fec387c8700 1 -- start start 2023-06-23T18:14:25.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.657+0000 7fec387c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec30079d00 0x7fec3010a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:25.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.657+0000 7fec387c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec30105af0 con 0x7fec30079d00 2023-06-23T18:14:25.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.658+0000 7fec36564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec30079d00 0x7fec3010a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:25.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.658+0000 7fec36564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec30079d00 0x7fec3010a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51296/0 (socket says 172.21.15.2:51296) 2023-06-23T18:14:25.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.658+0000 7fec36564700 1 -- 172.21.15.2:0/3163697890 learned_addr learned my addr 172.21.15.2:0/3163697890 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:25.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.658+0000 7fec36564700 1 -- 172.21.15.2:0/3163697890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3010a640 con 0x7fec30079d00 2023-06-23T18:14:25.694 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.658+0000 7fec36564700 1 --2- 172.21.15.2:0/3163697890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec30079d00 0x7fec3010a100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fec200098a0 tx=0x7fec20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390fc2878f1a4f7f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:25.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.658+0000 7fec35562700 1 -- 172.21.15.2:0/3163697890 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec20004160 con 0x7fec30079d00 2023-06-23T18:14:25.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.658+0000 7fec35562700 1 -- 172.21.15.2:0/3163697890 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec200042c0 con 0x7fec30079d00 2023-06-23T18:14:25.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.658+0000 7fec35562700 1 -- 172.21.15.2:0/3163697890 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec200118e0 con 0x7fec30079d00 2023-06-23T18:14:25.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.659+0000 7fec387c8700 1 -- 172.21.15.2:0/3163697890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec30079d00 msgr2=0x7fec3010a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:25.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.659+0000 7fec387c8700 1 --2- 172.21.15.2:0/3163697890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec30079d00 0x7fec3010a100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fec200098a0 tx=0x7fec20009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:25.695 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.659+0000 7fec387c8700 1 -- 172.21.15.2:0/3163697890 shutdown_connections 2023-06-23T18:14:25.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.659+0000 7fec387c8700 1 --2- 172.21.15.2:0/3163697890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec30079d00 0x7fec3010a100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:25.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.659+0000 7fec387c8700 1 -- 172.21.15.2:0/3163697890 >> 172.21.15.2:0/3163697890 conn(0x7fec300ff6f0 msgr2=0x7fec30101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:25.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.659+0000 7fec387c8700 1 -- 172.21.15.2:0/3163697890 shutdown_connections 2023-06-23T18:14:25.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.659+0000 7fec387c8700 1 -- 172.21.15.2:0/3163697890 wait complete. 2023-06-23T18:14:25.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec387c8700 1 Processor -- start 2023-06-23T18:14:25.696 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec387c8700 1 -- start start 2023-06-23T18:14:25.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec387c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec3019b100 0x7fec3019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:25.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec387c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3019ba50 con 0x7fec3019b100 2023-06-23T18:14:25.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec36564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec3019b100 0x7fec3019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:25.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec36564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec3019b100 0x7fec3019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51298/0 (socket says 172.21.15.2:51298) 2023-06-23T18:14:25.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec36564700 1 -- 172.21.15.2:0/1288213011 learned_addr learned my addr 172.21.15.2:0/1288213011 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:25.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec36564700 1 -- 172.21.15.2:0/1288213011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec20009550 con 0x7fec3019b100 2023-06-23T18:14:25.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.660+0000 7fec36564700 1 --2- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec3019b100 0x7fec3019b510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec20003e30 tx=0x7fec200119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:25.697 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.661+0000 7fec277fe700 1 -- 172.21.15.2:0/1288213011 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec200036a0 con 0x7fec3019b100 2023-06-23T18:14:25.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.661+0000 7fec277fe700 1 -- 172.21.15.2:0/1288213011 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec2001b440 con 0x7fec3019b100 2023-06-23T18:14:25.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.661+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3019bc50 con 0x7fec3019b100 2023-06-23T18:14:25.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.661+0000 7fec277fe700 1 -- 172.21.15.2:0/1288213011 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec20023400 con 0x7fec3019b100 2023-06-23T18:14:25.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.661+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3019e8b0 con 0x7fec3019b100 2023-06-23T18:14:25.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.662+0000 7fec277fe700 1 -- 172.21.15.2:0/1288213011 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fec2001b5b0 con 0x7fec3019b100 2023-06-23T18:14:25.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.662+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec301950f0 con 0x7fec3019b100 2023-06-23T18:14:25.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.662+0000 7fec277fe700 1 --2- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fec1c038760 0x7fec1c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:25.698 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.662+0000 7fec277fe700 1 -- 172.21.15.2:0/1288213011 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fec2004c6b0 con 0x7fec3019b100 2023-06-23T18:14:25.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.662+0000 7fec35d63700 1 --2- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fec1c038760 0x7fec1c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:25.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.662+0000 7fec35d63700 1 --2- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fec1c038760 0x7fec1c03ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec2c006fd0 tx=0x7fec2c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:25.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:24.667+0000 7fec277fe700 1 -- 172.21.15.2:0/1288213011 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fec2001ba90 con 0x7fec3019b100 2023-06-23T18:14:25.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.033+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fec30079d00 con 0x7fec3019b100 2023-06-23T18:14:25.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.037+0000 7fec277fe700 1 -- 172.21.15.2:0/1288213011 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fec2000b3e0 con 0x7fec3019b100 2023-06-23T18:14:25.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fec1c038760 msgr2=0x7fec1c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:25.699 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 --2- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fec1c038760 0x7fec1c03ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fec2c006fd0 tx=0x7fec2c006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:25.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec3019b100 msgr2=0x7fec3019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:25.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 --2- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec3019b100 0x7fec3019b510 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec20003e30 tx=0x7fec200119a0 comp rx=0 tx=0).stop 2023-06-23T18:14:25.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 shutdown_connections 2023-06-23T18:14:25.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 --2- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fec1c038760 0x7fec1c03ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:25.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 --2- 172.21.15.2:0/1288213011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec3019b100 0x7fec3019b510 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:25.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 >> 172.21.15.2:0/1288213011 conn(0x7fec300ff6f0 msgr2=0x7fec30100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:25.700 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.039+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 shutdown_connections 2023-06-23T18:14:25.701 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:25.040+0000 7fec387c8700 1 -- 172.21.15.2:0/1288213011 wait complete. 2023-06-23T18:14:26.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:26 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1288213011' entity='client.admin' 2023-06-23T18:14:26.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:26 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:27.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.246+0000 7fbbfdf38700 1 Processor -- start 2023-06-23T18:14:27.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.246+0000 7fbbfdf38700 1 -- start start 2023-06-23T18:14:27.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.246+0000 7fbbfdf38700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8105590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:27.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.246+0000 7fbbfdf38700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf8105ad0 con 0x7fbbf8105180 2023-06-23T18:14:27.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.246+0000 7fbbf77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8105590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:27.339 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.246+0000 7fbbf77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8105590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51302/0 (socket says 172.21.15.2:51302) 2023-06-23T18:14:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.247+0000 7fbbf77fe700 1 -- 172.21.15.2:0/177752754 learned_addr learned my addr 172.21.15.2:0/177752754 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.247+0000 7fbbf77fe700 1 -- 172.21.15.2:0/177752754 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8105c10 con 0x7fbbf8105180 2023-06-23T18:14:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.247+0000 7fbbf77fe700 1 --2- 172.21.15.2:0/177752754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8105590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0098a0 tx=0x7fbbec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7241c6f62fd5228d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.247+0000 7fbbf67fc700 1 -- 172.21.15.2:0/177752754 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec004160 con 0x7fbbf8105180 2023-06-23T18:14:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.247+0000 7fbbf67fc700 1 -- 172.21.15.2:0/177752754 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbec0042c0 con 0x7fbbf8105180 2023-06-23T18:14:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.247+0000 7fbbf67fc700 1 -- 172.21.15.2:0/177752754 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec0118e0 con 0x7fbbf8105180 2023-06-23T18:14:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.248+0000 7fbbfdf38700 1 -- 172.21.15.2:0/177752754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 msgr2=0x7fbbf8105590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.248+0000 7fbbfdf38700 1 --2- 172.21.15.2:0/177752754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8105590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0098a0 tx=0x7fbbec009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.248+0000 7fbbfdf38700 1 -- 172.21.15.2:0/177752754 shutdown_connections 2023-06-23T18:14:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.248+0000 7fbbfdf38700 1 --2- 172.21.15.2:0/177752754 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8105590 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.248+0000 7fbbfdf38700 1 -- 172.21.15.2:0/177752754 >> 172.21.15.2:0/177752754 conn(0x7fbbf8100810 msgr2=0x7fbbf8102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.248+0000 7fbbfdf38700 1 -- 172.21.15.2:0/177752754 shutdown_connections 2023-06-23T18:14:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.248+0000 7fbbfdf38700 1 -- 172.21.15.2:0/177752754 wait complete. 2023-06-23T18:14:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.248+0000 7fbbfdf38700 1 Processor -- start 2023-06-23T18:14:27.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbfdf38700 1 -- start start 2023-06-23T18:14:27.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbfdf38700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:27.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbfdf38700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf8199390 con 0x7fbbf8105180 2023-06-23T18:14:27.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbf77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:27.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbf77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51304/0 (socket says 172.21.15.2:51304) 2023-06-23T18:14:27.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbf77fe700 1 -- 172.21.15.2:0/3105291080 learned_addr learned my addr 172.21.15.2:0/3105291080 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:27.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbf77fe700 1 -- 172.21.15.2:0/3105291080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec009550 con 0x7fbbf8105180 2023-06-23T18:14:27.342 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbf77fe700 1 --2- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8198e50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006af0 tx=0x7fbbec011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:27.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbf4ff9700 1 -- 172.21.15.2:0/3105291080 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec0037d0 con 0x7fbbf8105180 2023-06-23T18:14:27.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf8199590 con 0x7fbbf8105180 2023-06-23T18:14:27.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbf4ff9700 1 -- 172.21.15.2:0/3105291080 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbec01b440 con 0x7fbbf8105180 2023-06-23T18:14:27.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.249+0000 7fbbf4ff9700 1 -- 172.21.15.2:0/3105291080 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec023400 con 0x7fbbf8105180 2023-06-23T18:14:27.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.250+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf81999b0 con 0x7fbbf8105180 2023-06-23T18:14:27.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.251+0000 7fbbf4ff9700 1 -- 172.21.15.2:0/3105291080 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fbbec0236c0 con 0x7fbbf8105180 2023-06-23T18:14:27.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.251+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf81095c0 con 0x7fbbf8105180 2023-06-23T18:14:27.343 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.251+0000 7fbbf4ff9700 1 --2- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fbbe00386c0 0x7fbbe003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:27.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.251+0000 7fbbf4ff9700 1 -- 172.21.15.2:0/3105291080 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbbec04c900 con 0x7fbbf8105180 2023-06-23T18:14:27.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.251+0000 7fbbf6ffd700 1 --2- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fbbe00386c0 0x7fbbe003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:27.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.251+0000 7fbbf6ffd700 1 --2- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fbbe00386c0 0x7fbbe003ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8006fd0 tx=0x7fbbe8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:27.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.256+0000 7fbbf4ff9700 1 -- 172.21.15.2:0/3105291080 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbbec02bd20 con 0x7fbbf8105180 2023-06-23T18:14:27.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.618+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fbbf819c810 con 0x7fbbf8105180 2023-06-23T18:14:27.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.621+0000 7fbbf4ff9700 1 -- 172.21.15.2:0/3105291080 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fbbec02b6d0 con 0x7fbbf8105180 2023-06-23T18:14:27.344 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.625+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fbbe00386c0 msgr2=0x7fbbe003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:27.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.625+0000 7fbbfdf38700 1 --2- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fbbe00386c0 0x7fbbe003ab70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbbe8006fd0 tx=0x7fbbe8006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:27.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.625+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 msgr2=0x7fbbf8198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:27.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.625+0000 7fbbfdf38700 1 --2- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8198e50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbbec006af0 tx=0x7fbbec011ed0 comp rx=0 tx=0).stop 2023-06-23T18:14:27.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.626+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 shutdown_connections 2023-06-23T18:14:27.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.626+0000 7fbbfdf38700 1 --2- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fbbe00386c0 0x7fbbe003ab70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:27.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.626+0000 7fbbfdf38700 1 --2- 172.21.15.2:0/3105291080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbbf8105180 0x7fbbf8198e50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:27.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.626+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 >> 172.21.15.2:0/3105291080 conn(0x7fbbf8100810 msgr2=0x7fbbf8101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:27.345 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.626+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 shutdown_connections 2023-06-23T18:14:27.346 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:26.626+0000 7fbbfdf38700 1 -- 172.21.15.2:0/3105291080 wait complete. 2023-06-23T18:14:27.346 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling the dashboard module... 2023-06-23T18:14:27.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:27 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3105291080' entity='client.admin' 2023-06-23T18:14:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:27 smithi002 ceph-mon[95947]: from='mgr.14118 172.21.15.2:0/3654261927' entity='mgr.x' 2023-06-23T18:14:28.927 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:28 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1315734728' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-06-23T18:14:28.927 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: ignoring --setuser ceph since I am not root 2023-06-23T18:14:28.928 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: ignoring --setgroup ceph since I am not root 2023-06-23T18:14:28.928 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:28.826+0000 7ff1c0540200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:14:28.928 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:28.922+0000 7ff1c0540200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:14:29.085 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.837+0000 7f16bf820700 1 Processor -- start 2023-06-23T18:14:29.085 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.837+0000 7f16bf820700 1 -- start start 2023-06-23T18:14:29.085 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.838+0000 7f16bf820700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b8104440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:29.085 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.838+0000 7f16bf820700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8104980 con 0x7f16b8104030 2023-06-23T18:14:29.086 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.838+0000 7f16bd5bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b8104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:29.086 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.838+0000 7f16bd5bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b8104440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51308/0 (socket says 172.21.15.2:51308) 2023-06-23T18:14:29.086 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.838+0000 7f16bd5bc700 1 -- 172.21.15.2:0/656238619 learned_addr learned my addr 172.21.15.2:0/656238619 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:29.086 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.838+0000 7f16bd5bc700 1 -- 172.21.15.2:0/656238619 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8104ac0 con 0x7f16b8104030 2023-06-23T18:14:29.086 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.838+0000 7f16bd5bc700 1 --2- 172.21.15.2:0/656238619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b8104440 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16a80098a0 tx=0x7f16a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9edd003d05f043 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:29.086 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.838+0000 7f16affff700 1 -- 172.21.15.2:0/656238619 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16a8004160 con 0x7f16b8104030 2023-06-23T18:14:29.086 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16affff700 1 -- 172.21.15.2:0/656238619 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16a80042c0 con 0x7f16b8104030 2023-06-23T18:14:29.086 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16affff700 1 -- 172.21.15.2:0/656238619 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16a80118e0 con 0x7f16b8104030 2023-06-23T18:14:29.087 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16bf820700 1 -- 172.21.15.2:0/656238619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 msgr2=0x7f16b8104440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:29.087 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16bf820700 1 --2- 172.21.15.2:0/656238619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b8104440 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f16a80098a0 tx=0x7f16a8009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:29.087 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16bf820700 1 -- 172.21.15.2:0/656238619 shutdown_connections 2023-06-23T18:14:29.087 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16bf820700 1 --2- 172.21.15.2:0/656238619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b8104440 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:29.087 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16bf820700 1 -- 172.21.15.2:0/656238619 >> 172.21.15.2:0/656238619 conn(0x7f16b80ff6a0 msgr2=0x7f16b8101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:29.087 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16bf820700 1 -- 172.21.15.2:0/656238619 shutdown_connections 2023-06-23T18:14:29.087 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.839+0000 7f16bf820700 1 -- 172.21.15.2:0/656238619 wait complete. 2023-06-23T18:14:29.088 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.840+0000 7f16bf820700 1 Processor -- start 2023-06-23T18:14:29.088 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.840+0000 7f16bf820700 1 -- start start 2023-06-23T18:14:29.088 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.840+0000 7f16bf820700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b818f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:29.088 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.840+0000 7f16bf820700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b818f6e0 con 0x7f16b8104030 2023-06-23T18:14:29.088 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.840+0000 7f16bd5bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b818f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:29.088 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.840+0000 7f16bd5bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b818f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51310/0 (socket says 172.21.15.2:51310) 2023-06-23T18:14:29.088 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.840+0000 7f16bd5bc700 1 -- 172.21.15.2:0/1315734728 learned_addr learned my addr 172.21.15.2:0/1315734728 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:29.089 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.841+0000 7f16bd5bc700 1 -- 172.21.15.2:0/1315734728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8009550 con 0x7f16b8104030 2023-06-23T18:14:29.089 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.841+0000 7f16bd5bc700 1 --2- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b818f1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16a8006af0 tx=0x7f16a8011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:29.089 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.841+0000 7f16ae7fc700 1 -- 172.21.15.2:0/1315734728 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16a80037d0 con 0x7f16b8104030 2023-06-23T18:14:29.089 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.841+0000 7f16ae7fc700 1 -- 172.21.15.2:0/1315734728 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16a801b440 con 0x7f16b8104030 2023-06-23T18:14:29.089 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.841+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b818d7f0 con 0x7f16b8104030 2023-06-23T18:14:29.089 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.841+0000 7f16ae7fc700 1 -- 172.21.15.2:0/1315734728 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16a8023400 con 0x7f16b8104030 2023-06-23T18:14:29.089 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.841+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b818dc10 con 0x7f16b8104030 2023-06-23T18:14:29.089 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.842+0000 7f16ae7fc700 1 -- 172.21.15.2:0/1315734728 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f16a8003930 con 0x7f16b8104030 2023-06-23T18:14:29.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.842+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b8061b00 con 0x7f16b8104030 2023-06-23T18:14:29.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.842+0000 7f16ae7fc700 1 --2- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f16a4038710 0x7f16a403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:29.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.842+0000 7f16ae7fc700 1 -- 172.21.15.2:0/1315734728 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f16a804c6f0 con 0x7f16b8104030 2023-06-23T18:14:29.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.842+0000 7f16bcdbb700 1 --2- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f16a4038710 0x7f16a403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:29.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.843+0000 7f16bcdbb700 1 --2- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f16a4038710 0x7f16a403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006fd0 tx=0x7f16b4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:29.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:27.847+0000 7f16ae7fc700 1 -- 172.21.15.2:0/1315734728 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f16a801a3d0 con 0x7f16b8104030 2023-06-23T18:14:29.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.322+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f16b8105ed0 con 0x7f16b8104030 2023-06-23T18:14:29.090 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.624+0000 7f16ae7fc700 1 -- 172.21.15.2:0/1315734728 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f16a801b8d0 con 0x7f16b8104030 2023-06-23T18:14:29.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.625+0000 7f16ae7fc700 1 -- 172.21.15.2:0/1315734728 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f16a8003be0 con 0x7f16b8104030 2023-06-23T18:14:29.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f16a4038710 msgr2=0x7f16a403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:29.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 --2- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f16a4038710 0x7f16a403abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006fd0 tx=0x7f16b4006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:29.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 msgr2=0x7f16b818f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:29.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 --2- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b818f1a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f16a8006af0 tx=0x7f16a8011ed0 comp rx=0 tx=0).stop 2023-06-23T18:14:29.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 shutdown_connections 2023-06-23T18:14:29.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 --2- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f16a4038710 0x7f16a403abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:29.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 --2- 172.21.15.2:0/1315734728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16b8104030 0x7f16b818f1a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:29.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 >> 172.21.15.2:0/1315734728 conn(0x7f16b80ff6a0 msgr2=0x7f16b8079960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:29.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 shutdown_connections 2023-06-23T18:14:29.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:28.628+0000 7f16bf820700 1 -- 172.21.15.2:0/1315734728 wait complete. 2023-06-23T18:14:29.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:29.031+0000 7ff1c0540200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:14:29.302 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:29.146+0000 7ff1c0540200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:14:29.628 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:29.356+0000 7ff1c0540200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:14:30.022 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:29.757+0000 7ff1c0540200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:14:30.023 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:29.866+0000 7ff1c0540200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:14:30.023 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:29 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1315734728' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-06-23T18:14:30.023 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:29 smithi002 ceph-mon[95947]: mgrmap e9: x(active, since 18s) 2023-06-23T18:14:30.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:30.018+0000 7ff1c0540200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:14:30.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:30.134+0000 7ff1c0540200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:14:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-06-23T18:14:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-06-23T18:14:30.536 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "available": true, 2023-06-23T18:14:30.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-06-23T18:14:30.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-06-23T18:14:30.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-06-23T18:14:30.537 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.653+0000 7fc80f0ac700 1 Processor -- start 2023-06-23T18:14:30.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.653+0000 7fc80f0ac700 1 -- start start 2023-06-23T18:14:30.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.653+0000 7fc80f0ac700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc8081080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:30.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.653+0000 7fc80f0ac700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808079d20 con 0x7fc808105cd0 2023-06-23T18:14:30.538 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.653+0000 7fc80ce48700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc8081080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:30.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.653+0000 7fc80ce48700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc8081080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51324/0 (socket says 172.21.15.2:51324) 2023-06-23T18:14:30.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.653+0000 7fc80ce48700 1 -- 172.21.15.2:0/1128603329 learned_addr learned my addr 172.21.15.2:0/1128603329 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:30.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.653+0000 7fc80ce48700 1 -- 172.21.15.2:0/1128603329 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808079e60 con 0x7fc808105cd0 2023-06-23T18:14:30.539 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.654+0000 7fc80ce48700 1 --2- 172.21.15.2:0/1128603329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc8081080b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098a0 tx=0x7fc7fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd72271b2d052ff7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:30.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.654+0000 7fc8077fe700 1 -- 172.21.15.2:0/1128603329 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7fc004160 con 0x7fc808105cd0 2023-06-23T18:14:30.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.654+0000 7fc8077fe700 1 -- 172.21.15.2:0/1128603329 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7fc0042c0 con 0x7fc808105cd0 2023-06-23T18:14:30.540 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.654+0000 7fc8077fe700 1 -- 172.21.15.2:0/1128603329 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7fc0118e0 con 0x7fc808105cd0 2023-06-23T18:14:30.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 -- 172.21.15.2:0/1128603329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 msgr2=0x7fc8081080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:30.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 --2- 172.21.15.2:0/1128603329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc8081080b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098a0 tx=0x7fc7fc009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:30.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 -- 172.21.15.2:0/1128603329 shutdown_connections 2023-06-23T18:14:30.541 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 --2- 172.21.15.2:0/1128603329 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc8081080b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:30.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 -- 172.21.15.2:0/1128603329 >> 172.21.15.2:0/1128603329 conn(0x7fc8080ff6f0 msgr2=0x7fc808101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:30.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 -- 172.21.15.2:0/1128603329 shutdown_connections 2023-06-23T18:14:30.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 -- 172.21.15.2:0/1128603329 wait complete. 2023-06-23T18:14:30.542 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 Processor -- start 2023-06-23T18:14:30.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 -- start start 2023-06-23T18:14:30.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.655+0000 7fc80f0ac700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc808198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:30.543 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc80f0ac700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808199470 con 0x7fc808105cd0 2023-06-23T18:14:30.544 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc80ce48700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc808198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:30.544 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc80ce48700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc808198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51326/0 (socket says 172.21.15.2:51326) 2023-06-23T18:14:30.544 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc80ce48700 1 -- 172.21.15.2:0/2612548946 learned_addr learned my addr 172.21.15.2:0/2612548946 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:30.544 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc80ce48700 1 -- 172.21.15.2:0/2612548946 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc009550 con 0x7fc808105cd0 2023-06-23T18:14:30.545 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc80ce48700 1 --2- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc808198f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc003e30 tx=0x7fc7fc0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:30.545 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc805ffb700 1 -- 172.21.15.2:0/2612548946 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7fc0036a0 con 0x7fc808105cd0 2023-06-23T18:14:30.545 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc805ffb700 1 -- 172.21.15.2:0/2612548946 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7fc01b440 con 0x7fc808105cd0 2023-06-23T18:14:30.545 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc808199670 con 0x7fc808105cd0 2023-06-23T18:14:30.546 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.656+0000 7fc805ffb700 1 -- 172.21.15.2:0/2612548946 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7fc023400 con 0x7fc808105cd0 2023-06-23T18:14:30.546 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.657+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc808199a90 con 0x7fc808105cd0 2023-06-23T18:14:30.546 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.657+0000 7fc805ffb700 1 -- 172.21.15.2:0/2612548946 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fc7fc023680 con 0x7fc808105cd0 2023-06-23T18:14:30.546 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.658+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808192ee0 con 0x7fc808105cd0 2023-06-23T18:14:30.547 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.658+0000 7fc805ffb700 1 --2- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fc7f8038760 0x7fc7f803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:30.547 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.658+0000 7fc805ffb700 1 -- 172.21.15.2:0/2612548946 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc7fc04c7a0 con 0x7fc808105cd0 2023-06-23T18:14:30.547 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.658+0000 7fc807fff700 1 -- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fc7f8038760 msgr2=0x7fc7f803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3628945044 2023-06-23T18:14:30.548 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.658+0000 7fc807fff700 1 --2- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fc7f8038760 0x7fc7f803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:14:30.548 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.663+0000 7fc805ffb700 1 -- 172.21.15.2:0/2612548946 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc7fc02c910 con 0x7fc808105cd0 2023-06-23T18:14:30.548 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.859+0000 7fc807fff700 1 -- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fc7f8038760 msgr2=0x7fc7f803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3628945044 2023-06-23T18:14:30.548 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:29.859+0000 7fc807fff700 1 --2- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fc7f8038760 0x7fc7f803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T18:14:30.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.139+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc808061b00 con 0x7fc808105cd0 2023-06-23T18:14:30.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.139+0000 7fc805ffb700 1 -- 172.21.15.2:0/2612548946 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc7fc02bb40 con 0x7fc808105cd0 2023-06-23T18:14:30.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fc7f8038760 msgr2=0x7fc7f803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:14:30.549 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 --2- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fc7f8038760 0x7fc7f803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:30.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 msgr2=0x7fc808198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:30.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 --2- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc808198f30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc003e30 tx=0x7fc7fc0119a0 comp rx=0 tx=0).stop 2023-06-23T18:14:30.550 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 shutdown_connections 2023-06-23T18:14:30.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 --2- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7fc7f8038760 0x7fc7f803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:30.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 --2- 172.21.15.2:0/2612548946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc808105cd0 0x7fc808198f30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:30.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 >> 172.21.15.2:0/2612548946 conn(0x7fc8080ff6f0 msgr2=0x7fc808100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:30.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 shutdown_connections 2023-06-23T18:14:30.551 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:30.142+0000 7fc80f0ac700 1 -- 172.21.15.2:0/2612548946 wait complete. 2023-06-23T18:14:30.551 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for the mgr to restart... 2023-06-23T18:14:30.552 INFO:teuthology.orchestra.run.smithi002.stdout:Waiting for mgr epoch 9... 2023-06-23T18:14:30.812 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:30 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2612548946' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-06-23T18:14:30.813 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:30.660+0000 7ff1c0540200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:14:31.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:31 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:31.357+0000 7ff1c0540200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T18:14:31.809 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:31 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:31.550+0000 7ff1c0540200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:14:32.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:32.514+0000 7ff1c0540200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:14:32.802 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:32.624+0000 7ff1c0540200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:14:33.197 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:32.845+0000 7ff1c0540200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:14:33.451 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:33 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:33.192+0000 7ff1c0540200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T18:14:33.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:33 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:33.447+0000 7ff1c0540200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:14:33.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:33 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:33.630+0000 7ff1c0540200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:14:34.301 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:33 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:33.959+0000 7ff1c0540200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:14:35.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:34 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:34.784+0000 7ff1c0540200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:14:35.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:34 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:34.943+0000 7ff1c0540200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:14:35.551 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:35 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:35.186+0000 7ff1c0540200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:14:35.551 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:35 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:35.322+0000 7ff1c0540200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:14:36.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:35 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:35.593+0000 7ff1c0540200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:14:37.051 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:14:36.640+0000 7ff1c0540200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:14:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: Active manager daemon x restarted 2023-06-23T18:14:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: Activating manager daemon x 2023-06-23T18:14:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: osdmap e3: 0 total, 0 up, 0 in 2023-06-23T18:14:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: mgrmap e10: x(active, starting, since 0.00342318s) 2023-06-23T18:14:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:14:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-06-23T18:14:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-06-23T18:14:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-06-23T18:14:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-06-23T18:14:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: Manager daemon x is now available 2023-06-23T18:14:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:38.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-06-23T18:14:38.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-06-23T18:14:38.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:37 smithi002 ceph-mon[95947]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-06-23T18:14:38.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:37 smithi002 ceph-mon[95947]: mgrmap e11: x(active, since 1.006s) 2023-06-23T18:14:38.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:37 smithi002 ceph-mon[95947]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-06-23T18:14:38.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout { 2023-06-23T18:14:38.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-06-23T18:14:38.091 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout "initialized": true 2023-06-23T18:14:38.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout } 2023-06-23T18:14:38.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.042+0000 7f2ba935a700 1 Processor -- start 2023-06-23T18:14:38.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.043+0000 7f2ba935a700 1 -- start start 2023-06-23T18:14:38.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.043+0000 7f2ba935a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba4079d00 0x7f2ba410a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:38.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.043+0000 7f2ba935a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba4105af0 con 0x7f2ba4079d00 2023-06-23T18:14:38.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.043+0000 7f2ba2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba4079d00 0x7f2ba410a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:38.092 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.043+0000 7f2ba2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba4079d00 0x7f2ba410a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51334/0 (socket says 172.21.15.2:51334) 2023-06-23T18:14:38.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.043+0000 7f2ba2ffd700 1 -- 172.21.15.2:0/1027252877 learned_addr learned my addr 172.21.15.2:0/1027252877 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:38.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.043+0000 7f2ba2ffd700 1 -- 172.21.15.2:0/1027252877 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba410a640 con 0x7f2ba4079d00 2023-06-23T18:14:38.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.043+0000 7f2ba2ffd700 1 --2- 172.21.15.2:0/1027252877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba4079d00 0x7f2ba410a100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2b940098a0 tx=0x7f2b94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72556e06c4482a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:38.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.044+0000 7f2ba1ffb700 1 -- 172.21.15.2:0/1027252877 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b94004160 con 0x7f2ba4079d00 2023-06-23T18:14:38.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.044+0000 7f2ba1ffb700 1 -- 172.21.15.2:0/1027252877 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b940042c0 con 0x7f2ba4079d00 2023-06-23T18:14:38.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.044+0000 7f2ba1ffb700 1 -- 172.21.15.2:0/1027252877 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b940118e0 con 0x7f2ba4079d00 2023-06-23T18:14:38.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.044+0000 7f2ba935a700 1 -- 172.21.15.2:0/1027252877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba4079d00 msgr2=0x7f2ba410a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:38.093 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.044+0000 7f2ba935a700 1 --2- 172.21.15.2:0/1027252877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba4079d00 0x7f2ba410a100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2b940098a0 tx=0x7f2b94009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:38.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.044+0000 7f2ba935a700 1 -- 172.21.15.2:0/1027252877 shutdown_connections 2023-06-23T18:14:38.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.044+0000 7f2ba935a700 1 --2- 172.21.15.2:0/1027252877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba4079d00 0x7f2ba410a100 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:38.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.044+0000 7f2ba935a700 1 -- 172.21.15.2:0/1027252877 >> 172.21.15.2:0/1027252877 conn(0x7f2ba40ff6f0 msgr2=0x7f2ba4101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:38.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.045+0000 7f2ba935a700 1 -- 172.21.15.2:0/1027252877 shutdown_connections 2023-06-23T18:14:38.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.045+0000 7f2ba935a700 1 -- 172.21.15.2:0/1027252877 wait complete. 2023-06-23T18:14:38.094 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.045+0000 7f2ba935a700 1 Processor -- start 2023-06-23T18:14:38.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.045+0000 7f2ba935a700 1 -- start start 2023-06-23T18:14:38.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba935a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba419b0d0 0x7f2ba419b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:38.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba935a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba419ba20 con 0x7f2ba419b0d0 2023-06-23T18:14:38.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba419b0d0 0x7f2ba419b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:38.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba419b0d0 0x7f2ba419b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51336/0 (socket says 172.21.15.2:51336) 2023-06-23T18:14:38.095 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba2ffd700 1 -- 172.21.15.2:0/153381940 learned_addr learned my addr 172.21.15.2:0/153381940 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba2ffd700 1 -- 172.21.15.2:0/153381940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b94009550 con 0x7f2ba419b0d0 2023-06-23T18:14:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba2ffd700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba419b0d0 0x7f2ba419b4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006af0 tx=0x7f2b94011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b940037d0 con 0x7f2ba419b0d0 2023-06-23T18:14:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b9401b440 con 0x7f2ba419b0d0 2023-06-23T18:14:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b94023400 con 0x7f2ba419b0d0 2023-06-23T18:14:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba935a700 1 -- 172.21.15.2:0/153381940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba419bc20 con 0x7f2ba419b0d0 2023-06-23T18:14:38.097 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.046+0000 7f2ba935a700 1 -- 172.21.15.2:0/153381940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba419e880 con 0x7f2ba419b0d0 2023-06-23T18:14:38.097 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.047+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f2b940236c0 con 0x7f2ba419b0d0 2023-06-23T18:14:38.097 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.048+0000 7f2b8bfff700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 0x7f2b8c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:38.099 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.048+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 --> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b8c03b2d0 con 0x7f2b8c038710 2023-06-23T18:14:38.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.048+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2b9404c8e0 con 0x7f2ba419b0d0 2023-06-23T18:14:38.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.048+0000 7f2ba27fc700 1 -- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 msgr2=0x7f2b8c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3628945044 2023-06-23T18:14:38.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.048+0000 7f2ba27fc700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 0x7f2b8c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:14:38.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.249+0000 7f2ba27fc700 1 -- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 msgr2=0x7f2b8c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3628945044 2023-06-23T18:14:38.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.249+0000 7f2ba27fc700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 0x7f2b8c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-06-23T18:14:38.100 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.650+0000 7f2ba27fc700 1 -- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 msgr2=0x7f2b8c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3628945044 2023-06-23T18:14:38.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:31.650+0000 7f2ba27fc700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 0x7f2b8c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-06-23T18:14:38.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:32.451+0000 7f2ba27fc700 1 -- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 msgr2=0x7f2b8c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3628945044 2023-06-23T18:14:38.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:32.451+0000 7f2ba27fc700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 0x7f2b8c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-06-23T18:14:38.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:34.052+0000 7f2ba27fc700 1 -- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 msgr2=0x7f2b8c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.2:6800/3628945044 2023-06-23T18:14:38.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:34.052+0000 7f2ba27fc700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 0x7f2b8c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-06-23T18:14:38.101 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:36.655+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7f2b94023ba0 con 0x7f2ba419b0d0 2023-06-23T18:14:38.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:36.655+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 msgr2=0x7f2b8c03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:14:38.102 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:36.655+0000 7f2b8bfff700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 0x7f2b8c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:38.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.657+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f2b9404bfa0 con 0x7f2ba419b0d0 2023-06-23T18:14:38.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.657+0000 7f2b8bfff700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2b8c03bff0 0x7f2b8c03e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:38.103 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.657+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b8c03b2d0 con 0x7f2b8c03bff0 2023-06-23T18:14:38.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.657+0000 7f2ba27fc700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2b8c03bff0 0x7f2b8c03e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:38.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.658+0000 7f2ba27fc700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2b8c03bff0 0x7f2b8c03e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2b98003a10 tx=0x7f2b980092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:38.104 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.659+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f2b8c03b2d0 con 0x7f2b8c03bff0 2023-06-23T18:14:38.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.669+0000 7f2b89ffb700 1 -- 172.21.15.2:0/153381940 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f2b84002600 con 0x7f2b8c03bff0 2023-06-23T18:14:38.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b8bfff700 1 -- 172.21.15.2:0/153381940 <== mgr.14150 v2:172.21.15.2:6800/3622114004 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f2b84002600 con 0x7f2b8c03bff0 2023-06-23T18:14:38.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 -- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2b8c03bff0 msgr2=0x7f2b8c03e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:38.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2b8c03bff0 0x7f2b8c03e3d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2b98003a10 tx=0x7f2b980092b0 comp rx=0 tx=0).stop 2023-06-23T18:14:38.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 -- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba419b0d0 msgr2=0x7f2ba419b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:38.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba419b0d0 0x7f2ba419b4e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2b94006af0 tx=0x7f2b94011ed0 comp rx=0 tx=0).stop 2023-06-23T18:14:38.105 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 -- 172.21.15.2:0/153381940 shutdown_connections 2023-06-23T18:14:38.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2b8c03bff0 0x7f2b8c03e3d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:38.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2ba419b0d0 0x7f2ba419b4e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:38.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 --2- 172.21.15.2:0/153381940 >> [v2:172.21.15.2:6800/3628945044,v1:172.21.15.2:6801/3628945044] conn(0x7f2b8c038710 0x7f2b8c03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:38.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 -- 172.21.15.2:0/153381940 >> 172.21.15.2:0/153381940 conn(0x7f2ba40ff6f0 msgr2=0x7f2ba4100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:38.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 -- 172.21.15.2:0/153381940 shutdown_connections 2023-06-23T18:14:38.106 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:37.670+0000 7f2b89ffb700 1 -- 172.21.15.2:0/153381940 wait complete. 2023-06-23T18:14:38.106 INFO:teuthology.orchestra.run.smithi002.stdout:mgr epoch 9 is available 2023-06-23T18:14:38.107 INFO:teuthology.orchestra.run.smithi002.stdout:Generating a dashboard self-signed certificate... 2023-06-23T18:14:39.648 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-06-23T18:14:39.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.784+0000 7f51229e8700 1 Processor -- start 2023-06-23T18:14:39.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.784+0000 7f51229e8700 1 -- start start 2023-06-23T18:14:39.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.784+0000 7f51229e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c104560 0x7f511c104970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:39.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.784+0000 7f51229e8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c104eb0 con 0x7f511c104560 2023-06-23T18:14:39.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.784+0000 7f511bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c104560 0x7f511c104970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:39.649 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.784+0000 7f511bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c104560 0x7f511c104970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51390/0 (socket says 172.21.15.2:51390) 2023-06-23T18:14:39.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.784+0000 7f511bfff700 1 -- 172.21.15.2:0/3168636713 learned_addr learned my addr 172.21.15.2:0/3168636713 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:39.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.785+0000 7f511bfff700 1 -- 172.21.15.2:0/3168636713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c104ff0 con 0x7f511c104560 2023-06-23T18:14:39.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.785+0000 7f511bfff700 1 --2- 172.21.15.2:0/3168636713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c104560 0x7f511c104970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f510c0098a0 tx=0x7f510c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7a0c21ef7fb659 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:39.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.785+0000 7f511affd700 1 -- 172.21.15.2:0/3168636713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f510c004160 con 0x7f511c104560 2023-06-23T18:14:39.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.785+0000 7f511affd700 1 -- 172.21.15.2:0/3168636713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f510c0042c0 con 0x7f511c104560 2023-06-23T18:14:39.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.785+0000 7f511affd700 1 -- 172.21.15.2:0/3168636713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f510c0118e0 con 0x7f511c104560 2023-06-23T18:14:39.650 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.786+0000 7f51229e8700 1 -- 172.21.15.2:0/3168636713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c104560 msgr2=0x7f511c104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:39.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.786+0000 7f51229e8700 1 --2- 172.21.15.2:0/3168636713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c104560 0x7f511c104970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f510c0098a0 tx=0x7f510c009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:39.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.786+0000 7f51229e8700 1 -- 172.21.15.2:0/3168636713 shutdown_connections 2023-06-23T18:14:39.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.786+0000 7f51229e8700 1 --2- 172.21.15.2:0/3168636713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c104560 0x7f511c104970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:39.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.786+0000 7f51229e8700 1 -- 172.21.15.2:0/3168636713 >> 172.21.15.2:0/3168636713 conn(0x7f511c0fee70 msgr2=0x7f511c101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:39.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.786+0000 7f51229e8700 1 -- 172.21.15.2:0/3168636713 shutdown_connections 2023-06-23T18:14:39.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.786+0000 7f51229e8700 1 -- 172.21.15.2:0/3168636713 wait complete. 2023-06-23T18:14:39.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.787+0000 7f51229e8700 1 Processor -- start 2023-06-23T18:14:39.651 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.787+0000 7f51229e8700 1 -- start start 2023-06-23T18:14:39.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.787+0000 7f51229e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c103ca0 0x7f511c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:39.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.787+0000 7f51229e8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1040b0 con 0x7f511c103ca0 2023-06-23T18:14:39.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.787+0000 7f511bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c103ca0 0x7f511c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:39.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.787+0000 7f511bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c103ca0 0x7f511c19beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51392/0 (socket says 172.21.15.2:51392) 2023-06-23T18:14:39.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.787+0000 7f511bfff700 1 -- 172.21.15.2:0/4153588548 learned_addr learned my addr 172.21.15.2:0/4153588548 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:39.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.788+0000 7f511bfff700 1 -- 172.21.15.2:0/4153588548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c009550 con 0x7f511c103ca0 2023-06-23T18:14:39.652 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.788+0000 7f511bfff700 1 --2- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c103ca0 0x7f511c19beb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f511c104290 tx=0x7f510c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:39.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.788+0000 7f51197fa700 1 -- 172.21.15.2:0/4153588548 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f510c0117f0 con 0x7f511c103ca0 2023-06-23T18:14:39.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.788+0000 7f51197fa700 1 -- 172.21.15.2:0/4153588548 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f510c011de0 con 0x7f511c103ca0 2023-06-23T18:14:39.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.788+0000 7f51197fa700 1 -- 172.21.15.2:0/4153588548 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f510c025d40 con 0x7f511c103ca0 2023-06-23T18:14:39.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.788+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c19c3f0 con 0x7f511c103ca0 2023-06-23T18:14:39.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.788+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c19c7b0 con 0x7f511c103ca0 2023-06-23T18:14:39.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.789+0000 7f51197fa700 1 -- 172.21.15.2:0/4153588548 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f510c022070 con 0x7f511c103ca0 2023-06-23T18:14:39.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.789+0000 7f51197fa700 1 --2- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51040387b0 0x7f510403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:39.653 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.789+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c191e00 con 0x7f511c103ca0 2023-06-23T18:14:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.789+0000 7f51197fa700 1 -- 172.21.15.2:0/4153588548 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f510c04cd40 con 0x7f511c103ca0 2023-06-23T18:14:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.790+0000 7f511b7fe700 1 --2- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51040387b0 0x7f510403ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.790+0000 7f511b7fe700 1 --2- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51040387b0 0x7f510403ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5110006fd0 tx=0x7f5110006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:38.795+0000 7f51197fa700 1 -- 172.21.15.2:0/4153588548 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f510c01b680 con 0x7f511c103ca0 2023-06-23T18:14:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.177+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f511c060910 con 0x7f51040387b0 2023-06-23T18:14:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.276+0000 7f51197fa700 1 -- 172.21.15.2:0/4153588548 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f511c060910 con 0x7f51040387b0 2023-06-23T18:14:39.654 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51040387b0 msgr2=0x7f510403ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:39.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 --2- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51040387b0 0x7f510403ac60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5110006fd0 tx=0x7f5110006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:39.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c103ca0 msgr2=0x7f511c19beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:39.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 --2- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c103ca0 0x7f511c19beb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f511c104290 tx=0x7f510c0039d0 comp rx=0 tx=0).stop 2023-06-23T18:14:39.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 shutdown_connections 2023-06-23T18:14:39.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 --2- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51040387b0 0x7f510403ac60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:39.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 --2- 172.21.15.2:0/4153588548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511c103ca0 0x7f511c19beb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:39.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 >> 172.21.15.2:0/4153588548 conn(0x7f511c0fee70 msgr2=0x7f511c100650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:39.655 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 shutdown_connections 2023-06-23T18:14:39.656 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:39.279+0000 7f51229e8700 1 -- 172.21.15.2:0/4153588548 wait complete. 2023-06-23T18:14:39.656 INFO:teuthology.orchestra.run.smithi002.stdout:Creating initial admin user... 2023-06-23T18:14:39.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:39 smithi002 ceph-mon[95947]: [23/Jun/2023:18:14:38] ENGINE Bus STARTING 2023-06-23T18:14:39.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:39 smithi002 ceph-mon[95947]: [23/Jun/2023:18:14:38] ENGINE Serving on https://172.21.15.2:7150 2023-06-23T18:14:39.907 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:39 smithi002 ceph-mon[95947]: [23/Jun/2023:18:14:38] ENGINE Serving on http://172.21.15.2:8765 2023-06-23T18:14:39.908 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:39 smithi002 ceph-mon[95947]: [23/Jun/2023:18:14:38] ENGINE Bus STARTED 2023-06-23T18:14:39.908 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:39.908 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:39 smithi002 ceph-mon[95947]: mgrmap e12: x(active, since 2s) 2023-06-23T18:14:39.908 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:39.908 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:40.854 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:40 smithi002 ceph-mon[95947]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:40.855 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:41.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$kVb/dfMFoKX7rdMUSUv0xeGPmk5dPrKwrjWYXxl47j6uasx9Dez9a", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1687544080, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-06-23T18:14:41.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.156+0000 7ff33c9b5700 1 Processor -- start 2023-06-23T18:14:41.492 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.156+0000 7ff33c9b5700 1 -- start start 2023-06-23T18:14:41.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.156+0000 7ff33c9b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff338104470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:41.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.156+0000 7ff33c9b5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3381049b0 con 0x7ff338104060 2023-06-23T18:14:41.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.157+0000 7ff33659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff338104470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:41.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.157+0000 7ff33659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff338104470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51396/0 (socket says 172.21.15.2:51396) 2023-06-23T18:14:41.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.157+0000 7ff33659c700 1 -- 172.21.15.2:0/1317475366 learned_addr learned my addr 172.21.15.2:0/1317475366 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:41.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.157+0000 7ff33659c700 1 -- 172.21.15.2:0/1317475366 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338104af0 con 0x7ff338104060 2023-06-23T18:14:41.493 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.157+0000 7ff33659c700 1 --2- 172.21.15.2:0/1317475366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff338104470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00bb40 tx=0x7ff32c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6f48241e17a40d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:41.494 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.157+0000 7ff33559a700 1 -- 172.21.15.2:0/1317475366 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff32c00dbe0 con 0x7ff338104060 2023-06-23T18:14:41.494 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.157+0000 7ff33559a700 1 -- 172.21.15.2:0/1317475366 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff32c0044c0 con 0x7ff338104060 2023-06-23T18:14:41.494 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.157+0000 7ff33559a700 1 -- 172.21.15.2:0/1317475366 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff32c003d50 con 0x7ff338104060 2023-06-23T18:14:41.494 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.158+0000 7ff33c9b5700 1 -- 172.21.15.2:0/1317475366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 msgr2=0x7ff338104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:41.494 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.158+0000 7ff33c9b5700 1 --2- 172.21.15.2:0/1317475366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff338104470 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00bb40 tx=0x7ff32c00be50 comp rx=0 tx=0).stop 2023-06-23T18:14:41.494 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.158+0000 7ff33c9b5700 1 -- 172.21.15.2:0/1317475366 shutdown_connections 2023-06-23T18:14:41.494 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.158+0000 7ff33c9b5700 1 --2- 172.21.15.2:0/1317475366 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff338104470 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:41.494 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.158+0000 7ff33c9b5700 1 -- 172.21.15.2:0/1317475366 >> 172.21.15.2:0/1317475366 conn(0x7ff3380ff710 msgr2=0x7ff338101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:41.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.158+0000 7ff33c9b5700 1 -- 172.21.15.2:0/1317475366 shutdown_connections 2023-06-23T18:14:41.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.158+0000 7ff33c9b5700 1 -- 172.21.15.2:0/1317475366 wait complete. 2023-06-23T18:14:41.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.159+0000 7ff33c9b5700 1 Processor -- start 2023-06-23T18:14:41.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.159+0000 7ff33c9b5700 1 -- start start 2023-06-23T18:14:41.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.159+0000 7ff33c9b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff33819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:41.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.159+0000 7ff33c9b5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33819b5f0 con 0x7ff338104060 2023-06-23T18:14:41.495 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.159+0000 7ff33659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff33819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:41.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.159+0000 7ff33659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff33819b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51398/0 (socket says 172.21.15.2:51398) 2023-06-23T18:14:41.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.159+0000 7ff33659c700 1 -- 172.21.15.2:0/260300939 learned_addr learned my addr 172.21.15.2:0/260300939 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:41.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.159+0000 7ff33659c700 1 -- 172.21.15.2:0/260300939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c00b7f0 con 0x7ff338104060 2023-06-23T18:14:41.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.160+0000 7ff33659c700 1 --2- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff33819b0b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff32c003f60 tx=0x7ff32c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:41.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.160+0000 7ff3277fe700 1 -- 172.21.15.2:0/260300939 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff32c010800 con 0x7ff338104060 2023-06-23T18:14:41.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.160+0000 7ff3277fe700 1 -- 172.21.15.2:0/260300939 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff32c010df0 con 0x7ff338104060 2023-06-23T18:14:41.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.160+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33819b7f0 con 0x7ff338104060 2023-06-23T18:14:41.496 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.160+0000 7ff3277fe700 1 -- 172.21.15.2:0/260300939 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff32c021cc0 con 0x7ff338104060 2023-06-23T18:14:41.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.160+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33819bbb0 con 0x7ff338104060 2023-06-23T18:14:41.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.161+0000 7ff3277fe700 1 -- 172.21.15.2:0/260300939 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7ff32c0193f0 con 0x7ff338104060 2023-06-23T18:14:41.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.161+0000 7ff3277fe700 1 --2- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff320038710 0x7ff32003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:41.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.161+0000 7ff3277fe700 1 -- 172.21.15.2:0/260300939 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff32c04cce0 con 0x7ff338104060 2023-06-23T18:14:41.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.161+0000 7ff335d9b700 1 --2- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff320038710 0x7ff32003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:41.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.161+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff338190990 con 0x7ff338104060 2023-06-23T18:14:41.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.161+0000 7ff335d9b700 1 --2- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff320038710 0x7ff32003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff328006fd0 tx=0x7ff328006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:41.497 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.166+0000 7ff3277fe700 1 -- 172.21.15.2:0/260300939 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff32c00e950 con 0x7ff338104060 2023-06-23T18:14:41.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.572+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff338061920 con 0x7ff320038710 2023-06-23T18:14:41.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.821+0000 7ff3277fe700 1 -- 172.21.15.2:0/260300939 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7ff338061920 con 0x7ff320038710 2023-06-23T18:14:41.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.824+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff320038710 msgr2=0x7ff32003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:41.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.824+0000 7ff33c9b5700 1 --2- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff320038710 0x7ff32003abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff328006fd0 tx=0x7ff328006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:41.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.824+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 msgr2=0x7ff33819b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:41.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.824+0000 7ff33c9b5700 1 --2- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff33819b0b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff32c003f60 tx=0x7ff32c004040 comp rx=0 tx=0).stop 2023-06-23T18:14:41.498 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.824+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 shutdown_connections 2023-06-23T18:14:41.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.824+0000 7ff33c9b5700 1 --2- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff320038710 0x7ff32003abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:41.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.824+0000 7ff33c9b5700 1 --2- 172.21.15.2:0/260300939 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff338104060 0x7ff33819b0b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:41.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.824+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 >> 172.21.15.2:0/260300939 conn(0x7ff3380ff710 msgr2=0x7ff33818dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:41.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.825+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 shutdown_connections 2023-06-23T18:14:41.499 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:40.825+0000 7ff33c9b5700 1 -- 172.21.15.2:0/260300939 wait complete. 2023-06-23T18:14:41.499 INFO:teuthology.orchestra.run.smithi002.stdout:Fetching dashboard port number... 2023-06-23T18:14:41.757 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:41 smithi002 ceph-mon[95947]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:41.758 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:42 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/16913731' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-06-23T18:14:42.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stdout 8443 2023-06-23T18:14:42.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.065+0000 7fed770d6700 1 Processor -- start 2023-06-23T18:14:42.942 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.065+0000 7fed770d6700 1 -- start start 2023-06-23T18:14:42.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.065+0000 7fed770d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70105a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:42.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.065+0000 7fed770d6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed70106000 con 0x7fed70105620 2023-06-23T18:14:42.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.066+0000 7fed74e72700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70105a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:42.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.066+0000 7fed74e72700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70105a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51406/0 (socket says 172.21.15.2:51406) 2023-06-23T18:14:42.943 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.066+0000 7fed74e72700 1 -- 172.21.15.2:0/247395175 learned_addr learned my addr 172.21.15.2:0/247395175 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:42.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.066+0000 7fed74e72700 1 -- 172.21.15.2:0/247395175 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed70106140 con 0x7fed70105620 2023-06-23T18:14:42.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.066+0000 7fed74e72700 1 --2- 172.21.15.2:0/247395175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70105a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed600098a0 tx=0x7fed60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ebc2c924d57f0a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:42.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.066+0000 7fed6f7fe700 1 -- 172.21.15.2:0/247395175 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed60004160 con 0x7fed70105620 2023-06-23T18:14:42.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.066+0000 7fed6f7fe700 1 -- 172.21.15.2:0/247395175 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed600042c0 con 0x7fed70105620 2023-06-23T18:14:42.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.066+0000 7fed6f7fe700 1 -- 172.21.15.2:0/247395175 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed600118e0 con 0x7fed70105620 2023-06-23T18:14:42.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.067+0000 7fed770d6700 1 -- 172.21.15.2:0/247395175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 msgr2=0x7fed70105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:42.944 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.067+0000 7fed770d6700 1 --2- 172.21.15.2:0/247395175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70105a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fed600098a0 tx=0x7fed60009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:42.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.067+0000 7fed770d6700 1 -- 172.21.15.2:0/247395175 shutdown_connections 2023-06-23T18:14:42.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.067+0000 7fed770d6700 1 --2- 172.21.15.2:0/247395175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70105a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:42.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.067+0000 7fed770d6700 1 -- 172.21.15.2:0/247395175 >> 172.21.15.2:0/247395175 conn(0x7fed700fee20 msgr2=0x7fed70101230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:42.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.067+0000 7fed770d6700 1 -- 172.21.15.2:0/247395175 shutdown_connections 2023-06-23T18:14:42.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.067+0000 7fed770d6700 1 -- 172.21.15.2:0/247395175 wait complete. 2023-06-23T18:14:42.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.067+0000 7fed770d6700 1 Processor -- start 2023-06-23T18:14:42.945 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed770d6700 1 -- start start 2023-06-23T18:14:42.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed770d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70103dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:42.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed770d6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7019bb50 con 0x7fed70105620 2023-06-23T18:14:42.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed74e72700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70103dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:42.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed74e72700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70103dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51408/0 (socket says 172.21.15.2:51408) 2023-06-23T18:14:42.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed74e72700 1 -- 172.21.15.2:0/16913731 learned_addr learned my addr 172.21.15.2:0/16913731 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:42.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed74e72700 1 -- 172.21.15.2:0/16913731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed60009550 con 0x7fed70105620 2023-06-23T18:14:42.946 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed74e72700 1 --2- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70103dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed60006af0 tx=0x7fed60011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:42.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed6dffb700 1 -- 172.21.15.2:0/16913731 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed600037d0 con 0x7fed70105620 2023-06-23T18:14:42.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed6dffb700 1 -- 172.21.15.2:0/16913731 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed6001b440 con 0x7fed70105620 2023-06-23T18:14:42.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7019bc90 con 0x7fed70105620 2023-06-23T18:14:42.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.068+0000 7fed6dffb700 1 -- 172.21.15.2:0/16913731 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed60023400 con 0x7fed70105620 2023-06-23T18:14:42.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.069+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7019c0b0 con 0x7fed70105620 2023-06-23T18:14:42.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.069+0000 7fed6dffb700 1 -- 172.21.15.2:0/16913731 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fed60003930 con 0x7fed70105620 2023-06-23T18:14:42.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.070+0000 7fed6dffb700 1 --2- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fed5c038760 0x7fed5c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:42.947 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.070+0000 7fed6dffb700 1 -- 172.21.15.2:0/16913731 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fed6004c860 con 0x7fed70105620 2023-06-23T18:14:42.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.070+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed70061b00 con 0x7fed70105620 2023-06-23T18:14:42.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.070+0000 7fed6ffff700 1 --2- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fed5c038760 0x7fed5c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:42.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.070+0000 7fed6ffff700 1 --2- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fed5c038760 0x7fed5c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed58006fd0 tx=0x7fed58006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:42.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.075+0000 7fed6dffb700 1 -- 172.21.15.2:0/16913731 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fed60003c30 con 0x7fed70105620 2023-06-23T18:14:42.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.431+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fed7019c800 con 0x7fed70105620 2023-06-23T18:14:42.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.431+0000 7fed6dffb700 1 -- 172.21.15.2:0/16913731 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fed6000b3e0 con 0x7fed70105620 2023-06-23T18:14:42.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fed5c038760 msgr2=0x7fed5c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:42.948 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 --2- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fed5c038760 0x7fed5c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed58006fd0 tx=0x7fed58006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:42.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 msgr2=0x7fed70103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:42.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 --2- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70103dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fed60006af0 tx=0x7fed60011ed0 comp rx=0 tx=0).stop 2023-06-23T18:14:42.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 shutdown_connections 2023-06-23T18:14:42.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 --2- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fed5c038760 0x7fed5c03ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:42.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 --2- 172.21.15.2:0/16913731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed70105620 0x7fed70103dd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:42.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 >> 172.21.15.2:0/16913731 conn(0x7fed700fee20 msgr2=0x7fed701004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:42.949 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 shutdown_connections 2023-06-23T18:14:42.950 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:42.434+0000 7fed770d6700 1 -- 172.21.15.2:0/16913731 wait complete. 2023-06-23T18:14:42.950 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-06-23T18:14:42.951 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout disabled 2023-06-23T18:14:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-06-23T18:14:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:systemctl: stdout inactive 2023-06-23T18:14:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:firewalld.service is not enabled 2023-06-23T18:14:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-06-23T18:14:42.959 INFO:teuthology.orchestra.run.smithi002.stdout:Ceph Dashboard is now available at: 2023-06-23T18:14:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: URL: https://smithi002:8443/ 2023-06-23T18:14:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: User: admin 2023-06-23T18:14:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: Password: q1wswunq9s 2023-06-23T18:14:42.960 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:Saving cluster configuration to /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config directory 2023-06-23T18:14:42.961 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling autotune for osd_memory_target 2023-06-23T18:14:44.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.444+0000 7f00c1e34700 1 Processor -- start 2023-06-23T18:14:44.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.444+0000 7f00c1e34700 1 -- start start 2023-06-23T18:14:44.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.445+0000 7f00c1e34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc103830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:44.307 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.445+0000 7f00c1e34700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc103d70 con 0x7f00bc103420 2023-06-23T18:14:44.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.445+0000 7f00bb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:44.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.445+0000 7f00bb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc103830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51412/0 (socket says 172.21.15.2:51412) 2023-06-23T18:14:44.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.445+0000 7f00bb7fe700 1 -- 172.21.15.2:0/4241535804 learned_addr learned my addr 172.21.15.2:0/4241535804 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:44.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.445+0000 7f00bb7fe700 1 -- 172.21.15.2:0/4241535804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00bc103eb0 con 0x7f00bc103420 2023-06-23T18:14:44.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.445+0000 7f00bb7fe700 1 --2- 172.21.15.2:0/4241535804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc103830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0098a0 tx=0x7f00ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dbba0515b5110a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:44.308 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.445+0000 7f00ba7fc700 1 -- 172.21.15.2:0/4241535804 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00ac004160 con 0x7f00bc103420 2023-06-23T18:14:44.309 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00ba7fc700 1 -- 172.21.15.2:0/4241535804 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00ac0042c0 con 0x7f00bc103420 2023-06-23T18:14:44.309 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00ba7fc700 1 -- 172.21.15.2:0/4241535804 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00ac0118e0 con 0x7f00bc103420 2023-06-23T18:14:44.309 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00c1e34700 1 -- 172.21.15.2:0/4241535804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 msgr2=0x7f00bc103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:44.309 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00c1e34700 1 --2- 172.21.15.2:0/4241535804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc103830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0098a0 tx=0x7f00ac009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:44.309 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00c1e34700 1 -- 172.21.15.2:0/4241535804 shutdown_connections 2023-06-23T18:14:44.310 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00c1e34700 1 --2- 172.21.15.2:0/4241535804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc103830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:44.310 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00c1e34700 1 -- 172.21.15.2:0/4241535804 >> 172.21.15.2:0/4241535804 conn(0x7f00bc0fee20 msgr2=0x7f00bc101230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:44.310 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00c1e34700 1 -- 172.21.15.2:0/4241535804 shutdown_connections 2023-06-23T18:14:44.310 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.446+0000 7f00c1e34700 1 -- 172.21.15.2:0/4241535804 wait complete. 2023-06-23T18:14:44.310 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.447+0000 7f00c1e34700 1 Processor -- start 2023-06-23T18:14:44.310 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.447+0000 7f00c1e34700 1 -- start start 2023-06-23T18:14:44.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.447+0000 7f00c1e34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc105db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:44.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.447+0000 7f00c1e34700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00bc1062f0 con 0x7f00bc103420 2023-06-23T18:14:44.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.447+0000 7f00bb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc105db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:44.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.447+0000 7f00bb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc105db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51414/0 (socket says 172.21.15.2:51414) 2023-06-23T18:14:44.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.447+0000 7f00bb7fe700 1 -- 172.21.15.2:0/1537438185 learned_addr learned my addr 172.21.15.2:0/1537438185 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:44.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.448+0000 7f00bb7fe700 1 -- 172.21.15.2:0/1537438185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac009550 con 0x7f00bc103420 2023-06-23T18:14:44.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.448+0000 7f00bb7fe700 1 --2- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc105db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00ac003e30 tx=0x7f00ac0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:44.311 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.448+0000 7f00b8ff9700 1 -- 172.21.15.2:0/1537438185 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00ac0036a0 con 0x7f00bc103420 2023-06-23T18:14:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.448+0000 7f00b8ff9700 1 -- 172.21.15.2:0/1537438185 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00ac01b440 con 0x7f00bc103420 2023-06-23T18:14:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.448+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00bc104400 con 0x7f00bc103420 2023-06-23T18:14:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.448+0000 7f00b8ff9700 1 -- 172.21.15.2:0/1537438185 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00ac023400 con 0x7f00bc103420 2023-06-23T18:14:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.448+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00bc104820 con 0x7f00bc103420 2023-06-23T18:14:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.449+0000 7f00b8ff9700 1 -- 172.21.15.2:0/1537438185 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f00ac023680 con 0x7f00bc103420 2023-06-23T18:14:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.449+0000 7f00b8ff9700 1 --2- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00a4038760 0x7f00a403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:44.312 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.449+0000 7f00b8ff9700 1 -- 172.21.15.2:0/1537438185 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f00ac04c790 con 0x7f00bc103420 2023-06-23T18:14:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.449+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00bc061b00 con 0x7f00bc103420 2023-06-23T18:14:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.449+0000 7f00baffd700 1 --2- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00a4038760 0x7f00a403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.450+0000 7f00baffd700 1 --2- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00a4038760 0x7f00a403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f00b0006fd0 tx=0x7f00b0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.454+0000 7f00b8ff9700 1 -- 172.21.15.2:0/1537438185 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f00ac018350 con 0x7f00bc103420 2023-06-23T18:14:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.674+0000 7f00b8ff9700 1 -- 172.21.15.2:0/1537438185 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f00ac01b5b0 con 0x7f00bc103420 2023-06-23T18:14:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.811+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f00bc104e80 con 0x7f00bc103420 2023-06-23T18:14:44.313 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.811+0000 7f00b8ff9700 1 -- 172.21.15.2:0/1537438185 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f00ac018350 con 0x7f00bc103420 2023-06-23T18:14:44.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00a4038760 msgr2=0x7f00a403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:44.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 --2- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00a4038760 0x7f00a403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f00b0006fd0 tx=0x7f00b0006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:44.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 msgr2=0x7f00bc105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:44.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 --2- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc105db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f00ac003e30 tx=0x7f00ac0119a0 comp rx=0 tx=0).stop 2023-06-23T18:14:44.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 shutdown_connections 2023-06-23T18:14:44.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 --2- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00a4038760 0x7f00a403ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:44.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 --2- 172.21.15.2:0/1537438185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00bc103420 0x7f00bc105db0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:44.314 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 >> 172.21.15.2:0/1537438185 conn(0x7f00bc0fee20 msgr2=0x7f00bc1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:44.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 shutdown_connections 2023-06-23T18:14:44.315 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:43.813+0000 7f00c1e34700 1 -- 172.21.15.2:0/1537438185 wait complete. 2023-06-23T18:14:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:44 smithi002 ceph-mon[95947]: mgrmap e13: x(active, since 7s) 2023-06-23T18:14:45.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.822+0000 7f428672c700 1 Processor -- start 2023-06-23T18:14:45.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.822+0000 7f428672c700 1 -- start start 2023-06-23T18:14:45.853 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f428672c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280103830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:45.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f428672c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4280103d70 con 0x7f4280103420 2023-06-23T18:14:45.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f427ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280103830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:45.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f427ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280103830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51418/0 (socket says 172.21.15.2:51418) 2023-06-23T18:14:45.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f427ffff700 1 -- 172.21.15.2:0/3263416529 learned_addr learned my addr 172.21.15.2:0/3263416529 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:45.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f427ffff700 1 -- 172.21.15.2:0/3263416529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4280103eb0 con 0x7f4280103420 2023-06-23T18:14:45.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f427ffff700 1 --2- 172.21.15.2:0/3263416529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280103830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42700098a0 tx=0x7f4270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e0a1ba33b236e9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:45.854 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f427effd700 1 -- 172.21.15.2:0/3263416529 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4270004160 con 0x7f4280103420 2023-06-23T18:14:45.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f427effd700 1 -- 172.21.15.2:0/3263416529 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42700042c0 con 0x7f4280103420 2023-06-23T18:14:45.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.823+0000 7f427effd700 1 -- 172.21.15.2:0/3263416529 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42700118e0 con 0x7f4280103420 2023-06-23T18:14:45.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.824+0000 7f428672c700 1 -- 172.21.15.2:0/3263416529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 msgr2=0x7f4280103830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:45.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.824+0000 7f428672c700 1 --2- 172.21.15.2:0/3263416529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280103830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42700098a0 tx=0x7f4270009bb0 comp rx=0 tx=0).stop 2023-06-23T18:14:45.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.824+0000 7f428672c700 1 -- 172.21.15.2:0/3263416529 shutdown_connections 2023-06-23T18:14:45.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.824+0000 7f428672c700 1 --2- 172.21.15.2:0/3263416529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280103830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:45.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.824+0000 7f428672c700 1 -- 172.21.15.2:0/3263416529 >> 172.21.15.2:0/3263416529 conn(0x7f42800fee20 msgr2=0x7f4280101230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:45.855 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.824+0000 7f428672c700 1 -- 172.21.15.2:0/3263416529 shutdown_connections 2023-06-23T18:14:45.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.824+0000 7f428672c700 1 -- 172.21.15.2:0/3263416529 wait complete. 2023-06-23T18:14:45.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.825+0000 7f428672c700 1 Processor -- start 2023-06-23T18:14:45.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.825+0000 7f428672c700 1 -- start start 2023-06-23T18:14:45.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.825+0000 7f428672c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:45.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.825+0000 7f428672c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4280199380 con 0x7f4280103420 2023-06-23T18:14:45.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.825+0000 7f427ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:45.856 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.825+0000 7f427ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51420/0 (socket says 172.21.15.2:51420) 2023-06-23T18:14:45.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.825+0000 7f427ffff700 1 -- 172.21.15.2:0/2623793153 learned_addr learned my addr 172.21.15.2:0/2623793153 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:45.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.826+0000 7f427ffff700 1 -- 172.21.15.2:0/2623793153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4270009550 con 0x7f4280103420 2023-06-23T18:14:45.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.826+0000 7f427ffff700 1 --2- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280198e40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4270003e30 tx=0x7f42700119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:45.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.826+0000 7f427d7fa700 1 -- 172.21.15.2:0/2623793153 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42700036a0 con 0x7f4280103420 2023-06-23T18:14:45.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.826+0000 7f427d7fa700 1 -- 172.21.15.2:0/2623793153 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f427001b440 con 0x7f4280103420 2023-06-23T18:14:45.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.826+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4280199580 con 0x7f4280103420 2023-06-23T18:14:45.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.826+0000 7f427d7fa700 1 -- 172.21.15.2:0/2623793153 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4270023400 con 0x7f4280103420 2023-06-23T18:14:45.857 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.826+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42801999a0 con 0x7f4280103420 2023-06-23T18:14:45.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.827+0000 7f427d7fa700 1 -- 172.21.15.2:0/2623793153 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f4270023630 con 0x7f4280103420 2023-06-23T18:14:45.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.827+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4280061b00 con 0x7f4280103420 2023-06-23T18:14:45.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.827+0000 7f427d7fa700 1 --2- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42680388a0 0x7f426803ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:45.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.827+0000 7f427d7fa700 1 -- 172.21.15.2:0/2623793153 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f427004c9f0 con 0x7f4280103420 2023-06-23T18:14:45.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.827+0000 7f427f7fe700 1 --2- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42680388a0 0x7f426803ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:45.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.828+0000 7f427f7fe700 1 --2- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42680388a0 0x7f426803ad50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4274006fd0 tx=0x7f4274006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:45.858 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:44.832+0000 7f427d7fa700 1 -- 172.21.15.2:0/2623793153 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f427002b710 con 0x7f4280103420 2023-06-23T18:14:45.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.401+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f428019c800 con 0x7f4280103420 2023-06-23T18:14:45.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.403+0000 7f427d7fa700 1 -- 172.21.15.2:0/2623793153 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f427002b350 con 0x7f4280103420 2023-06-23T18:14:45.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42680388a0 msgr2=0x7f426803ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:45.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 --2- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42680388a0 0x7f426803ad50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4274006fd0 tx=0x7f4274006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:45.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 msgr2=0x7f4280198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:45.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 --2- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280198e40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4270003e30 tx=0x7f42700119a0 comp rx=0 tx=0).stop 2023-06-23T18:14:45.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 shutdown_connections 2023-06-23T18:14:45.859 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 --2- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42680388a0 0x7f426803ad50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:45.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 --2- 172.21.15.2:0/2623793153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4280103420 0x7f4280198e40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:45.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 >> 172.21.15.2:0/2623793153 conn(0x7f42800fee20 msgr2=0x7f42801004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:45.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 shutdown_connections 2023-06-23T18:14:45.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr 2023-06-23T18:14:45.406+0000 7f428672c700 1 -- 172.21.15.2:0/2623793153 wait complete. 2023-06-23T18:14:45.860 INFO:teuthology.orchestra.run.smithi002.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-06-23T18:14:45.860 INFO:teuthology.orchestra.run.smithi002.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-06-23T18:14:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:14:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:45.861 INFO:teuthology.orchestra.run.smithi002.stdout:Or, if you are only running a single cluster on this host: 2023-06-23T18:14:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-06-23T18:14:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:45.861 INFO:teuthology.orchestra.run.smithi002.stdout:Please consider enabling telemetry to help improve Ceph: 2023-06-23T18:14:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: ceph telemetry on 2023-06-23T18:14:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:45.862 INFO:teuthology.orchestra.run.smithi002.stdout:For more information see: 2023-06-23T18:14:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-06-23T18:14:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:45.862 INFO:teuthology.orchestra.run.smithi002.stdout:Bootstrap complete. 2023-06-23T18:14:45.890 INFO:tasks.cephadm:Fetching config... 2023-06-23T18:14:45.891 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:14:45.891 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-06-23T18:14:45.908 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-06-23T18:14:45.909 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:14:45.909 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-06-23T18:14:45.967 INFO:tasks.cephadm:Fetching mon keyring... 2023-06-23T18:14:45.968 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:14:45.968 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-06-23T18:14:46.035 INFO:tasks.cephadm:Fetching pub ssh key... 2023-06-23T18:14:46.036 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:14:46.036 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-06-23T18:14:46.093 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-06-23T18:14:46.093 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3UxW0vk5sj0hepuJozcIPkmmrGMuP/+LjXr4Z2Xn1b49oxRm9b5g3Hj3AEvNSW8mn5BuOueRSh1aINIKQ0eia4W3kwEUvjDmQKj4qHefoMSua5U9pbFLlJEpGtHTrUguH7ppsEoLzeqAuBAleLTN49gPPC/UKgFwCtgKsnGINPFLSd5ebdAg5hIz0+6eps+oKe/eo5seyr5b6FgukxtFxNa8aiaVKrMNo7PhKbvwloMFeeWU/QpaMODQvQ1jSWr1AHpbQZuTySK9jDPE/1TGlHyt0A2hiRaJSNAdbbxRa0JYGCtT1aay0nzG7xiDuoPqE4pYal2gDzZX54RWbb0TthtgrJ3xxMZi6qYqALYgobFvebJQBbUhgng0B8SdvEpUIQRCTmgddWaqaUo+BFOt2QPmkaFryGKMx9ubX2LdCL/YKPGgKWJe/acmoO6XVqMUBkG1z2wi4PyfJEllH6CwAENvo2hlc27gDC+oiZ73xAgho1+tFB93HZ+RW600vJD8= ceph-902733cc-11f1-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-23T18:14:46.170 INFO:teuthology.orchestra.run.smithi002.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3UxW0vk5sj0hepuJozcIPkmmrGMuP/+LjXr4Z2Xn1b49oxRm9b5g3Hj3AEvNSW8mn5BuOueRSh1aINIKQ0eia4W3kwEUvjDmQKj4qHefoMSua5U9pbFLlJEpGtHTrUguH7ppsEoLzeqAuBAleLTN49gPPC/UKgFwCtgKsnGINPFLSd5ebdAg5hIz0+6eps+oKe/eo5seyr5b6FgukxtFxNa8aiaVKrMNo7PhKbvwloMFeeWU/QpaMODQvQ1jSWr1AHpbQZuTySK9jDPE/1TGlHyt0A2hiRaJSNAdbbxRa0JYGCtT1aay0nzG7xiDuoPqE4pYal2gDzZX54RWbb0TthtgrJ3xxMZi6qYqALYgobFvebJQBbUhgng0B8SdvEpUIQRCTmgddWaqaUo+BFOt2QPmkaFryGKMx9ubX2LdCL/YKPGgKWJe/acmoO6XVqMUBkG1z2wi4PyfJEllH6CwAENvo2hlc27gDC+oiZ73xAgho1+tFB93HZ+RW600vJD8= ceph-902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:14:46.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3UxW0vk5sj0hepuJozcIPkmmrGMuP/+LjXr4Z2Xn1b49oxRm9b5g3Hj3AEvNSW8mn5BuOueRSh1aINIKQ0eia4W3kwEUvjDmQKj4qHefoMSua5U9pbFLlJEpGtHTrUguH7ppsEoLzeqAuBAleLTN49gPPC/UKgFwCtgKsnGINPFLSd5ebdAg5hIz0+6eps+oKe/eo5seyr5b6FgukxtFxNa8aiaVKrMNo7PhKbvwloMFeeWU/QpaMODQvQ1jSWr1AHpbQZuTySK9jDPE/1TGlHyt0A2hiRaJSNAdbbxRa0JYGCtT1aay0nzG7xiDuoPqE4pYal2gDzZX54RWbb0TthtgrJ3xxMZi6qYqALYgobFvebJQBbUhgng0B8SdvEpUIQRCTmgddWaqaUo+BFOt2QPmkaFryGKMx9ubX2LdCL/YKPGgKWJe/acmoO6XVqMUBkG1z2wi4PyfJEllH6CwAENvo2hlc27gDC+oiZ73xAgho1+tFB93HZ+RW600vJD8= ceph-902733cc-11f1-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-23T18:14:46.229 INFO:teuthology.orchestra.run.smithi017.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3UxW0vk5sj0hepuJozcIPkmmrGMuP/+LjXr4Z2Xn1b49oxRm9b5g3Hj3AEvNSW8mn5BuOueRSh1aINIKQ0eia4W3kwEUvjDmQKj4qHefoMSua5U9pbFLlJEpGtHTrUguH7ppsEoLzeqAuBAleLTN49gPPC/UKgFwCtgKsnGINPFLSd5ebdAg5hIz0+6eps+oKe/eo5seyr5b6FgukxtFxNa8aiaVKrMNo7PhKbvwloMFeeWU/QpaMODQvQ1jSWr1AHpbQZuTySK9jDPE/1TGlHyt0A2hiRaJSNAdbbxRa0JYGCtT1aay0nzG7xiDuoPqE4pYal2gDzZX54RWbb0TthtgrJ3xxMZi6qYqALYgobFvebJQBbUhgng0B8SdvEpUIQRCTmgddWaqaUo+BFOt2QPmkaFryGKMx9ubX2LdCL/YKPGgKWJe/acmoO6XVqMUBkG1z2wi4PyfJEllH6CwAENvo2hlc27gDC+oiZ73xAgho1+tFB93HZ+RW600vJD8= ceph-902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:14:46.242 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3UxW0vk5sj0hepuJozcIPkmmrGMuP/+LjXr4Z2Xn1b49oxRm9b5g3Hj3AEvNSW8mn5BuOueRSh1aINIKQ0eia4W3kwEUvjDmQKj4qHefoMSua5U9pbFLlJEpGtHTrUguH7ppsEoLzeqAuBAleLTN49gPPC/UKgFwCtgKsnGINPFLSd5ebdAg5hIz0+6eps+oKe/eo5seyr5b6FgukxtFxNa8aiaVKrMNo7PhKbvwloMFeeWU/QpaMODQvQ1jSWr1AHpbQZuTySK9jDPE/1TGlHyt0A2hiRaJSNAdbbxRa0JYGCtT1aay0nzG7xiDuoPqE4pYal2gDzZX54RWbb0TthtgrJ3xxMZi6qYqALYgobFvebJQBbUhgng0B8SdvEpUIQRCTmgddWaqaUo+BFOt2QPmkaFryGKMx9ubX2LdCL/YKPGgKWJe/acmoO6XVqMUBkG1z2wi4PyfJEllH6CwAENvo2hlc27gDC+oiZ73xAgho1+tFB93HZ+RW600vJD8= ceph-902733cc-11f1-11ee-9b2b-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-06-23T18:14:46.287 INFO:teuthology.orchestra.run.smithi082.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3UxW0vk5sj0hepuJozcIPkmmrGMuP/+LjXr4Z2Xn1b49oxRm9b5g3Hj3AEvNSW8mn5BuOueRSh1aINIKQ0eia4W3kwEUvjDmQKj4qHefoMSua5U9pbFLlJEpGtHTrUguH7ppsEoLzeqAuBAleLTN49gPPC/UKgFwCtgKsnGINPFLSd5ebdAg5hIz0+6eps+oKe/eo5seyr5b6FgukxtFxNa8aiaVKrMNo7PhKbvwloMFeeWU/QpaMODQvQ1jSWr1AHpbQZuTySK9jDPE/1TGlHyt0A2hiRaJSNAdbbxRa0JYGCtT1aay0nzG7xiDuoPqE4pYal2gDzZX54RWbb0TthtgrJ3xxMZi6qYqALYgobFvebJQBbUhgng0B8SdvEpUIQRCTmgddWaqaUo+BFOt2QPmkaFryGKMx9ubX2LdCL/YKPGgKWJe/acmoO6XVqMUBkG1z2wi4PyfJEllH6CwAENvo2hlc27gDC+oiZ73xAgho1+tFB93HZ+RW600vJD8= ceph-902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:14:46.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-06-23T18:14:46.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:46 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2623793153' entity='client.admin' 2023-06-23T18:14:46.867 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:14:48.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.397+0000 7faf65b9e700 1 -- 172.21.15.2:0/1795751197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 msgr2=0x7faf600fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:48.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.397+0000 7faf65b9e700 1 --2- 172.21.15.2:0/1795751197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 0x7faf600fe8b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7faf50009910 tx=0x7faf50009c20 comp rx=0 tx=0).stop 2023-06-23T18:14:48.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.397+0000 7faf65b9e700 1 -- 172.21.15.2:0/1795751197 shutdown_connections 2023-06-23T18:14:48.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.397+0000 7faf65b9e700 1 --2- 172.21.15.2:0/1795751197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 0x7faf600fe8b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:48.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.397+0000 7faf65b9e700 1 -- 172.21.15.2:0/1795751197 >> 172.21.15.2:0/1795751197 conn(0x7faf600f9a30 msgr2=0x7faf600fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:48.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.397+0000 7faf65b9e700 1 -- 172.21.15.2:0/1795751197 shutdown_connections 2023-06-23T18:14:48.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.397+0000 7faf65b9e700 1 -- 172.21.15.2:0/1795751197 wait complete. 2023-06-23T18:14:48.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.398+0000 7faf65b9e700 1 Processor -- start 2023-06-23T18:14:48.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.398+0000 7faf65b9e700 1 -- start start 2023-06-23T18:14:48.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.398+0000 7faf65b9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 0x7faf60196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:48.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.398+0000 7faf65b9e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf60196dc0 con 0x7faf600fe4a0 2023-06-23T18:14:48.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.398+0000 7faf5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 0x7faf60196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:48.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.398+0000 7faf5f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 0x7faf60196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51426/0 (socket says 172.21.15.2:51426) 2023-06-23T18:14:48.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.398+0000 7faf5f7fe700 1 -- 172.21.15.2:0/4227176386 learned_addr learned my addr 172.21.15.2:0/4227176386 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:48.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.398+0000 7faf5f7fe700 1 -- 172.21.15.2:0/4227176386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500095f0 con 0x7faf600fe4a0 2023-06-23T18:14:48.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.399+0000 7faf5f7fe700 1 --2- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 0x7faf60196880 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf50006010 tx=0x7faf50005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:48.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.399+0000 7faf5cff9700 1 -- 172.21.15.2:0/4227176386 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf5000ba90 con 0x7faf600fe4a0 2023-06-23T18:14:48.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.399+0000 7faf5cff9700 1 -- 172.21.15.2:0/4227176386 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf5000bbf0 con 0x7faf600fe4a0 2023-06-23T18:14:48.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.399+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf60196fc0 con 0x7faf600fe4a0 2023-06-23T18:14:48.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.399+0000 7faf5cff9700 1 -- 172.21.15.2:0/4227176386 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf50017440 con 0x7faf600fe4a0 2023-06-23T18:14:48.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.399+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf60197460 con 0x7faf600fe4a0 2023-06-23T18:14:48.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.400+0000 7faf5cff9700 1 -- 172.21.15.2:0/4227176386 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7faf500175a0 con 0x7faf600fe4a0 2023-06-23T18:14:48.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.400+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf60190330 con 0x7faf600fe4a0 2023-06-23T18:14:48.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.400+0000 7faf5cff9700 1 --2- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf4c038850 0x7faf4c03ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:48.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.400+0000 7faf5cff9700 1 -- 172.21.15.2:0/4227176386 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faf5004cc70 con 0x7faf600fe4a0 2023-06-23T18:14:48.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.400+0000 7faf5effd700 1 --2- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf4c038850 0x7faf4c03ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:48.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.401+0000 7faf5effd700 1 --2- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf4c038850 0x7faf4c03ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faf48006fd0 tx=0x7faf48006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:48.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.405+0000 7faf5cff9700 1 -- 172.21.15.2:0/4227176386 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7faf50015350 con 0x7faf600fe4a0 2023-06-23T18:14:48.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.612+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7faf6002cc80 con 0x7faf600fe4a0 2023-06-23T18:14:48.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.615+0000 7faf5cff9700 1 -- 172.21.15.2:0/4227176386 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7faf50027030 con 0x7faf600fe4a0 2023-06-23T18:14:48.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf4c038850 msgr2=0x7faf4c03ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 --2- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf4c038850 0x7faf4c03ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faf48006fd0 tx=0x7faf48006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 msgr2=0x7faf60196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 --2- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 0x7faf60196880 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf50006010 tx=0x7faf50005810 comp rx=0 tx=0).stop 2023-06-23T18:14:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 shutdown_connections 2023-06-23T18:14:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 --2- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf4c038850 0x7faf4c03ad00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:48.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 --2- 172.21.15.2:0/4227176386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faf600fe4a0 0x7faf60196880 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:48.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 >> 172.21.15.2:0/4227176386 conn(0x7faf600f9a30 msgr2=0x7faf60068060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:48.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 shutdown_connections 2023-06-23T18:14:48.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:48.617+0000 7faf65b9e700 1 -- 172.21.15.2:0/4227176386 wait complete. 2023-06-23T18:14:49.124 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-06-23T18:14:49.124 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-06-23T18:14:49.736 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:14:49.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:49 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/4227176386' entity='client.admin' 2023-06-23T18:14:49.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:49 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:49.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:49 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:49.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:49 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi002", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:14:49.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:49 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:49.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:49 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:49.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:49 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:51.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.181+0000 7f7ed6132700 1 -- 172.21.15.2:0/2065187913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 msgr2=0x7f7ed00ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:51.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.181+0000 7f7ed6132700 1 --2- 172.21.15.2:0/2065187913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 0x7f7ed00ff9c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009910 tx=0x7f7ec0009c20 comp rx=0 tx=0).stop 2023-06-23T18:14:51.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.181+0000 7f7ed6132700 1 -- 172.21.15.2:0/2065187913 shutdown_connections 2023-06-23T18:14:51.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.181+0000 7f7ed6132700 1 --2- 172.21.15.2:0/2065187913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 0x7f7ed00ff9c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:51.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.181+0000 7f7ed6132700 1 -- 172.21.15.2:0/2065187913 >> 172.21.15.2:0/2065187913 conn(0x7f7ed00fab40 msgr2=0x7f7ed00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:51.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.181+0000 7f7ed6132700 1 -- 172.21.15.2:0/2065187913 shutdown_connections 2023-06-23T18:14:51.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.181+0000 7f7ed6132700 1 -- 172.21.15.2:0/2065187913 wait complete. 2023-06-23T18:14:51.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.182+0000 7f7ed6132700 1 Processor -- start 2023-06-23T18:14:51.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ed6132700 1 -- start start 2023-06-23T18:14:51.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ed6132700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 0x7f7ed0194630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:51.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ed6132700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed0194b70 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ecf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 0x7f7ed0194630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:51.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ecf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 0x7f7ed0194630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51432/0 (socket says 172.21.15.2:51432) 2023-06-23T18:14:51.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ecf7fe700 1 -- 172.21.15.2:0/1754877985 learned_addr learned my addr 172.21.15.2:0/1754877985 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:51.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ecf7fe700 1 -- 172.21.15.2:0/1754877985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec00095f0 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ecf7fe700 1 --2- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 0x7f7ed0194630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0006010 tx=0x7f7ec0005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:51.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7eccff9700 1 -- 172.21.15.2:0/1754877985 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec000ba90 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed0194d70 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7eccff9700 1 -- 172.21.15.2:0/1754877985 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ec000bbf0 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7eccff9700 1 -- 172.21.15.2:0/1754877985 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ec0017440 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.183+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed0195210 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.184+0000 7f7eccff9700 1 -- 172.21.15.2:0/1754877985 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f7ec0017670 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.185+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed018e030 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.185+0000 7f7eccff9700 1 --2- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7eb80387b0 0x7f7eb803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:51.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.185+0000 7f7eccff9700 1 -- 172.21.15.2:0/1754877985 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7ec004cd50 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.185+0000 7f7eceffd700 1 --2- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7eb80387b0 0x7f7eb803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:51.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.185+0000 7f7eceffd700 1 --2- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7eb80387b0 0x7f7eb803ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006fd0 tx=0x7f7ec4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:51.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.190+0000 7f7eccff9700 1 -- 172.21.15.2:0/1754877985 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7ec0050030 con 0x7f7ed00ff5b0 2023-06-23T18:14:51.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.391+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ed00609a0 con 0x7f7eb80387b0 2023-06-23T18:14:51.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.394+0000 7f7eccff9700 1 -- 172.21.15.2:0/1754877985 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7ed00609a0 con 0x7f7eb80387b0 2023-06-23T18:14:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.396+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7eb80387b0 msgr2=0x7f7eb803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.396+0000 7f7ed6132700 1 --2- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7eb80387b0 0x7f7eb803ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006fd0 tx=0x7f7ec4006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.396+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 msgr2=0x7f7ed0194630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:51.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.396+0000 7f7ed6132700 1 --2- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 0x7f7ed0194630 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0006010 tx=0x7f7ec0005810 comp rx=0 tx=0).stop 2023-06-23T18:14:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.396+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 shutdown_connections 2023-06-23T18:14:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.396+0000 7f7ed6132700 1 --2- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7eb80387b0 0x7f7eb803ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.396+0000 7f7ed6132700 1 --2- 172.21.15.2:0/1754877985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed00ff5b0 0x7f7ed0194630 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.396+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 >> 172.21.15.2:0/1754877985 conn(0x7f7ed00fab40 msgr2=0x7f7ed00fb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.397+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 shutdown_connections 2023-06-23T18:14:51.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:51.397+0000 7f7ed6132700 1 -- 172.21.15.2:0/1754877985 wait complete. 2023-06-23T18:14:51.895 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi017 2023-06-23T18:14:51.895 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:14:51.896 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/etc/ceph/ceph.conf 2023-06-23T18:14:51.916 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:14:51.917 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:14:51.977 INFO:tasks.cephadm:Adding host smithi017 to orchestrator... 2023-06-23T18:14:51.977 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch host add smithi017 2023-06-23T18:14:52.547 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:14:52.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:52 smithi002 ceph-mon[95947]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:14:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:14:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:52 smithi002 ceph-mon[95947]: Updating smithi002:/etc/ceph/ceph.conf 2023-06-23T18:14:52.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:52 smithi002 ceph-mon[95947]: Updating smithi002:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:14:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:53 smithi002 ceph-mon[95947]: Updating smithi002:/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:14:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:53 smithi002 ceph-mon[95947]: Updating smithi002:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T18:14:53.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:53.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:53.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:53.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.821+0000 7fdc7e263700 1 -- 172.21.15.2:0/2603242762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 msgr2=0x7fdc78104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.821+0000 7fdc7e263700 1 --2- 172.21.15.2:0/2603242762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 0x7fdc78104880 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009910 tx=0x7fdc68009c20 comp rx=0 tx=0).stop 2023-06-23T18:14:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.821+0000 7fdc7e263700 1 -- 172.21.15.2:0/2603242762 shutdown_connections 2023-06-23T18:14:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.821+0000 7fdc7e263700 1 --2- 172.21.15.2:0/2603242762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 0x7fdc78104880 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.821+0000 7fdc7e263700 1 -- 172.21.15.2:0/2603242762 >> 172.21.15.2:0/2603242762 conn(0x7fdc780f9fd0 msgr2=0x7fdc780fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.821+0000 7fdc7e263700 1 -- 172.21.15.2:0/2603242762 shutdown_connections 2023-06-23T18:14:53.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.821+0000 7fdc7e263700 1 -- 172.21.15.2:0/2603242762 wait complete. 2023-06-23T18:14:53.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.822+0000 7fdc7e263700 1 Processor -- start 2023-06-23T18:14:53.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.822+0000 7fdc7e263700 1 -- start start 2023-06-23T18:14:53.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.822+0000 7fdc7e263700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 0x7fdc7819c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:53.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.822+0000 7fdc7e263700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7819cef0 con 0x7fdc781044b0 2023-06-23T18:14:53.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.822+0000 7fdc77fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 0x7fdc7819c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:53.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.822+0000 7fdc77fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 0x7fdc7819c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51438/0 (socket says 172.21.15.2:51438) 2023-06-23T18:14:53.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.822+0000 7fdc77fff700 1 -- 172.21.15.2:0/809268440 learned_addr learned my addr 172.21.15.2:0/809268440 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:53.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.823+0000 7fdc77fff700 1 -- 172.21.15.2:0/809268440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680095f0 con 0x7fdc781044b0 2023-06-23T18:14:53.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.823+0000 7fdc77fff700 1 --2- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 0x7fdc7819c9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc68004f40 tx=0x7fdc68005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:53.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.823+0000 7fdc75ffb700 1 -- 172.21.15.2:0/809268440 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc6801d070 con 0x7fdc781044b0 2023-06-23T18:14:53.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.823+0000 7fdc75ffb700 1 -- 172.21.15.2:0/809268440 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdc680044b0 con 0x7fdc781044b0 2023-06-23T18:14:53.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.823+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7819d0f0 con 0x7fdc781044b0 2023-06-23T18:14:53.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.823+0000 7fdc75ffb700 1 -- 172.21.15.2:0/809268440 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc68003860 con 0x7fdc781044b0 2023-06-23T18:14:53.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.823+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7819d4b0 con 0x7fdc781044b0 2023-06-23T18:14:53.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.824+0000 7fdc75ffb700 1 -- 172.21.15.2:0/809268440 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fdc68003a90 con 0x7fdc781044b0 2023-06-23T18:14:53.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.824+0000 7fdc75ffb700 1 --2- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdc64038800 0x7fdc6403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:53.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.824+0000 7fdc75ffb700 1 -- 172.21.15.2:0/809268440 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdc6804c670 con 0x7fdc781044b0 2023-06-23T18:14:53.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.824+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc78061b00 con 0x7fdc781044b0 2023-06-23T18:14:53.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.824+0000 7fdc6ffff700 1 --2- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdc64038800 0x7fdc6403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:53.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.825+0000 7fdc6ffff700 1 --2- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdc64038800 0x7fdc6403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc60006fd0 tx=0x7fdc60006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:53.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:53.829+0000 7fdc75ffb700 1 -- 172.21.15.2:0/809268440 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdc6802a770 con 0x7fdc781044b0 2023-06-23T18:14:54.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:54.039+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi017", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc78068670 con 0x7fdc64038800 2023-06-23T18:14:55.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:54 smithi002 ceph-mon[95947]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi017", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:14:55.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:54 smithi002 ceph-mon[95947]: Deploying cephadm binary to smithi017 2023-06-23T18:14:56.806 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi017' with addr '172.21.15.17' 2023-06-23T18:14:56.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.800+0000 7fdc75ffb700 1 -- 172.21.15.2:0/809268440 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fdc78068670 con 0x7fdc64038800 2023-06-23T18:14:56.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.802+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdc64038800 msgr2=0x7fdc6403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:56.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.802+0000 7fdc7e263700 1 --2- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdc64038800 0x7fdc6403acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdc60006fd0 tx=0x7fdc60006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:56.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.802+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 msgr2=0x7fdc7819c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:56.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.802+0000 7fdc7e263700 1 --2- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 0x7fdc7819c9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdc68004f40 tx=0x7fdc68005020 comp rx=0 tx=0).stop 2023-06-23T18:14:56.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.803+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 shutdown_connections 2023-06-23T18:14:56.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.803+0000 7fdc7e263700 1 --2- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdc64038800 0x7fdc6403acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:56.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.803+0000 7fdc7e263700 1 --2- 172.21.15.2:0/809268440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdc781044b0 0x7fdc7819c9b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:56.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.803+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 >> 172.21.15.2:0/809268440 conn(0x7fdc780f9fd0 msgr2=0x7fdc78067f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:56.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.803+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 shutdown_connections 2023-06-23T18:14:56.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:56.803+0000 7fdc7e263700 1 -- 172.21.15.2:0/809268440 wait complete. 2023-06-23T18:14:57.200 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch host ls --format=json 2023-06-23T18:14:57.764 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:14:58.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:57 smithi002 ceph-mon[95947]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:14:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:14:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:58.932 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:58 smithi002 ceph-mon[95947]: Added host smithi017 2023-06-23T18:14:58.933 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:14:59.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.374+0000 7fc8a6afe700 1 -- 172.21.15.2:0/3134705823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 msgr2=0x7fc8a0104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.374+0000 7fc8a6afe700 1 --2- 172.21.15.2:0/3134705823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 0x7fc8a0104880 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc894009910 tx=0x7fc894009c20 comp rx=0 tx=0).stop 2023-06-23T18:14:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.374+0000 7fc8a6afe700 1 -- 172.21.15.2:0/3134705823 shutdown_connections 2023-06-23T18:14:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.374+0000 7fc8a6afe700 1 --2- 172.21.15.2:0/3134705823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 0x7fc8a0104880 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.374+0000 7fc8a6afe700 1 -- 172.21.15.2:0/3134705823 >> 172.21.15.2:0/3134705823 conn(0x7fc8a00f9fd0 msgr2=0x7fc8a00fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:59.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.374+0000 7fc8a6afe700 1 -- 172.21.15.2:0/3134705823 shutdown_connections 2023-06-23T18:14:59.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.374+0000 7fc8a6afe700 1 -- 172.21.15.2:0/3134705823 wait complete. 2023-06-23T18:14:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a6afe700 1 Processor -- start 2023-06-23T18:14:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a6afe700 1 -- start start 2023-06-23T18:14:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a6afe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 0x7fc8a0195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:59.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a6afe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0195d80 con 0x7fc8a01044b0 2023-06-23T18:14:59.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a489a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 0x7fc8a0195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:59.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a489a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 0x7fc8a0195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51464/0 (socket says 172.21.15.2:51464) 2023-06-23T18:14:59.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a489a700 1 -- 172.21.15.2:0/659917298 learned_addr learned my addr 172.21.15.2:0/659917298 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:14:59.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a489a700 1 -- 172.21.15.2:0/659917298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8940095f0 con 0x7fc8a01044b0 2023-06-23T18:14:59.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc8a489a700 1 --2- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 0x7fc8a0195840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc894006010 tx=0x7fc894004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:59.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc89dffb700 1 -- 172.21.15.2:0/659917298 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc89401d070 con 0x7fc8a01044b0 2023-06-23T18:14:59.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.375+0000 7fc89dffb700 1 -- 172.21.15.2:0/659917298 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc89400bdf0 con 0x7fc8a01044b0 2023-06-23T18:14:59.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.376+0000 7fc89dffb700 1 -- 172.21.15.2:0/659917298 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc894022cd0 con 0x7fc8a01044b0 2023-06-23T18:14:59.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.376+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a0195f20 con 0x7fc8a01044b0 2023-06-23T18:14:59.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.376+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a0196340 con 0x7fc8a01044b0 2023-06-23T18:14:59.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.377+0000 7fc89dffb700 1 -- 172.21.15.2:0/659917298 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fc894022470 con 0x7fc8a01044b0 2023-06-23T18:14:59.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.377+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a0061b00 con 0x7fc8a01044b0 2023-06-23T18:14:59.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.377+0000 7fc89dffb700 1 --2- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc890038800 0x7fc89003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:14:59.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.377+0000 7fc89dffb700 1 -- 172.21.15.2:0/659917298 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc89404c990 con 0x7fc8a01044b0 2023-06-23T18:14:59.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.377+0000 7fc89ffff700 1 --2- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc890038800 0x7fc89003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:14:59.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.377+0000 7fc89ffff700 1 --2- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc890038800 0x7fc89003acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc88c006fd0 tx=0x7fc88c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:14:59.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.382+0000 7fc89dffb700 1 -- 172.21.15.2:0/659917298 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc89402bba0 con 0x7fc8a01044b0 2023-06-23T18:14:59.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.583+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8a01966e0 con 0x7fc890038800 2023-06-23T18:14:59.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.585+0000 7fc89dffb700 1 -- 172.21.15.2:0/659917298 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7fc8a01966e0 con 0x7fc890038800 2023-06-23T18:14:59.590 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:14:59.590 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.17", "hostname": "smithi017", "labels": [], "status": ""}] 2023-06-23T18:14:59.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.586+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc890038800 msgr2=0x7fc89003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:59.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.586+0000 7fc8a6afe700 1 --2- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc890038800 0x7fc89003acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc88c006fd0 tx=0x7fc88c006e40 comp rx=0 tx=0).stop 2023-06-23T18:14:59.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.586+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 msgr2=0x7fc8a0195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:14:59.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.586+0000 7fc8a6afe700 1 --2- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 0x7fc8a0195840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc894006010 tx=0x7fc894004ff0 comp rx=0 tx=0).stop 2023-06-23T18:14:59.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.587+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 shutdown_connections 2023-06-23T18:14:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.587+0000 7fc8a6afe700 1 --2- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc890038800 0x7fc89003acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.587+0000 7fc8a6afe700 1 --2- 172.21.15.2:0/659917298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8a01044b0 0x7fc8a0195840 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:14:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.587+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 >> 172.21.15.2:0/659917298 conn(0x7fc8a00f9fd0 msgr2=0x7fc8a00fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:14:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.587+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 shutdown_connections 2023-06-23T18:14:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:14:59.587+0000 7fc8a6afe700 1 -- 172.21.15.2:0/659917298 wait complete. 2023-06-23T18:14:59.978 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:59 smithi002 ceph-mon[95947]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:14:59.979 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:14:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:00.013 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi082 2023-06-23T18:15:00.014 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:15:00.014 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.conf 2023-06-23T18:15:00.034 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:15:00.035 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:15:00.094 INFO:tasks.cephadm:Adding host smithi082 to orchestrator... 2023-06-23T18:15:00.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch host add smithi082 2023-06-23T18:15:00.655 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:15:00.925 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:00 smithi002 ceph-mon[95947]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:15:01.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.965+0000 7f52dbcce700 1 -- 172.21.15.2:0/1961860661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 msgr2=0x7f52d41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:01.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.965+0000 7f52dbcce700 1 --2- 172.21.15.2:0/1961860661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 0x7f52d41059e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52c400b0a0 tx=0x7f52c400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:15:01.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.965+0000 7f52dbcce700 1 -- 172.21.15.2:0/1961860661 shutdown_connections 2023-06-23T18:15:01.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.965+0000 7f52dbcce700 1 --2- 172.21.15.2:0/1961860661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 0x7f52d41059e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:01.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.965+0000 7f52dbcce700 1 -- 172.21.15.2:0/1961860661 >> 172.21.15.2:0/1961860661 conn(0x7f52d40fb0f0 msgr2=0x7f52d40fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:01.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.965+0000 7f52dbcce700 1 -- 172.21.15.2:0/1961860661 shutdown_connections 2023-06-23T18:15:01.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.965+0000 7f52dbcce700 1 -- 172.21.15.2:0/1961860661 wait complete. 2023-06-23T18:15:01.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.966+0000 7f52dbcce700 1 Processor -- start 2023-06-23T18:15:01.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.966+0000 7f52dbcce700 1 -- start start 2023-06-23T18:15:01.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52dbcce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 0x7f52d419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:01.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52dbcce700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d419ac00 con 0x7f52d4105610 2023-06-23T18:15:01.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52d9a6a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 0x7f52d419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:01.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52d9a6a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 0x7f52d419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51470/0 (socket says 172.21.15.2:51470) 2023-06-23T18:15:01.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52d9a6a700 1 -- 172.21.15.2:0/1218543652 learned_addr learned my addr 172.21.15.2:0/1218543652 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:15:01.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52d9a6a700 1 -- 172.21.15.2:0/1218543652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009d00 con 0x7f52d4105610 2023-06-23T18:15:01.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52d9a6a700 1 --2- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 0x7f52d419a6c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52c40062a0 tx=0x7f52c400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:01.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52d2ffd700 1 -- 172.21.15.2:0/1218543652 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52c400e040 con 0x7f52d4105610 2023-06-23T18:15:01.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d419ada0 con 0x7f52d4105610 2023-06-23T18:15:01.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52d2ffd700 1 -- 172.21.15.2:0/1218543652 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52c40095a0 con 0x7f52d4105610 2023-06-23T18:15:01.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52d2ffd700 1 -- 172.21.15.2:0/1218543652 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f52c401d610 con 0x7f52d4105610 2023-06-23T18:15:01.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.967+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d419b220 con 0x7f52d4105610 2023-06-23T18:15:01.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.968+0000 7f52d2ffd700 1 -- 172.21.15.2:0/1218543652 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f52c4004040 con 0x7f52d4105610 2023-06-23T18:15:01.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.968+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d4061b00 con 0x7f52d4105610 2023-06-23T18:15:01.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.968+0000 7f52d2ffd700 1 --2- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f52bc038800 0x7f52bc03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:01.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.968+0000 7f52d2ffd700 1 -- 172.21.15.2:0/1218543652 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f52c404cac0 con 0x7f52d4105610 2023-06-23T18:15:01.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.969+0000 7f52d9269700 1 --2- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f52bc038800 0x7f52bc03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:01.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.969+0000 7f52d9269700 1 --2- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f52bc038800 0x7f52bc03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f52c8006fd0 tx=0x7f52c8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:01.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:01.973+0000 7f52d2ffd700 1 -- 172.21.15.2:0/1218543652 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f52c4025360 con 0x7f52d4105610 2023-06-23T18:15:02.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:01 smithi002 ceph-mon[95947]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:02.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:02.177+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}) v1 -- 0x7f52d40fcf40 con 0x7f52bc038800 2023-06-23T18:15:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:03.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:03.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:03.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi017", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:15:03.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:03.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:03.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: Updating smithi017:/etc/ceph/ceph.conf 2023-06-23T18:15:03.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: Updating smithi017:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:03.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:03.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:15:03.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: Updating smithi017:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T18:15:03.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:02 smithi002 ceph-mon[95947]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:04.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:03 smithi002 ceph-mon[95947]: Deploying cephadm binary to smithi082 2023-06-23T18:15:04.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:04.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:04.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.038+0000 7f52d2ffd700 1 -- 172.21.15.2:0/1218543652 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f52d40fcf40 con 0x7f52bc038800 2023-06-23T18:15:05.045 INFO:teuthology.orchestra.run.smithi002.stdout:Added host 'smithi082' with addr '172.21.15.82' 2023-06-23T18:15:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f52bc038800 msgr2=0x7f52bc03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 --2- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f52bc038800 0x7f52bc03acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f52c8006fd0 tx=0x7f52c8006e40 comp rx=0 tx=0).stop 2023-06-23T18:15:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 msgr2=0x7f52d419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 --2- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 0x7f52d419a6c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52c40062a0 tx=0x7f52c400bd40 comp rx=0 tx=0).stop 2023-06-23T18:15:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 shutdown_connections 2023-06-23T18:15:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 --2- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f52bc038800 0x7f52bc03acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 --2- 172.21.15.2:0/1218543652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52d4105610 0x7f52d419a6c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 >> 172.21.15.2:0/1218543652 conn(0x7f52d40fb0f0 msgr2=0x7f52d40fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 shutdown_connections 2023-06-23T18:15:05.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:05.041+0000 7f52dbcce700 1 -- 172.21.15.2:0/1218543652 wait complete. 2023-06-23T18:15:05.188 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:04 smithi002 ceph-mon[95947]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:05.512 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch host ls --format=json 2023-06-23T18:15:06.078 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:15:06.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:06 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:06.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:06 smithi002 ceph-mon[95947]: Added host smithi082 2023-06-23T18:15:06.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:06 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:06.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:06 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:07.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:07.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:07 smithi002 ceph-mon[95947]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:07.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.389+0000 7f5ab272d700 1 -- 172.21.15.2:0/4223611904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 msgr2=0x7f5aac104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:07.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.389+0000 7f5ab272d700 1 --2- 172.21.15.2:0/4223611904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 0x7f5aac104880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009910 tx=0x7f5a9c009c20 comp rx=0 tx=0).stop 2023-06-23T18:15:07.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.389+0000 7f5ab272d700 1 -- 172.21.15.2:0/4223611904 shutdown_connections 2023-06-23T18:15:07.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.389+0000 7f5ab272d700 1 --2- 172.21.15.2:0/4223611904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 0x7f5aac104880 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:07.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.389+0000 7f5ab272d700 1 -- 172.21.15.2:0/4223611904 >> 172.21.15.2:0/4223611904 conn(0x7f5aac074270 msgr2=0x7f5aac074670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:07.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.389+0000 7f5ab272d700 1 -- 172.21.15.2:0/4223611904 shutdown_connections 2023-06-23T18:15:07.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.389+0000 7f5ab272d700 1 -- 172.21.15.2:0/4223611904 wait complete. 2023-06-23T18:15:07.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.390+0000 7f5ab272d700 1 Processor -- start 2023-06-23T18:15:07.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.390+0000 7f5ab272d700 1 -- start start 2023-06-23T18:15:07.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.390+0000 7f5ab272d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 0x7f5aac19a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:07.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.390+0000 7f5ab272d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac19ad60 con 0x7f5aac1044b0 2023-06-23T18:15:07.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.390+0000 7f5aabfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 0x7f5aac19a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:07.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.390+0000 7f5aabfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 0x7f5aac19a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51496/0 (socket says 172.21.15.2:51496) 2023-06-23T18:15:07.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.390+0000 7f5aabfff700 1 -- 172.21.15.2:0/605336518 learned_addr learned my addr 172.21.15.2:0/605336518 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:15:07.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.390+0000 7f5aabfff700 1 -- 172.21.15.2:0/605336518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9c0095f0 con 0x7f5aac1044b0 2023-06-23T18:15:07.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.391+0000 7f5aabfff700 1 --2- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 0x7f5aac19a820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c006010 tx=0x7f5a9c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:07.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.391+0000 7f5aa97fa700 1 -- 172.21.15.2:0/605336518 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a9c01d070 con 0x7f5aac1044b0 2023-06-23T18:15:07.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.391+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac19af60 con 0x7f5aac1044b0 2023-06-23T18:15:07.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.391+0000 7f5aa97fa700 1 -- 172.21.15.2:0/605336518 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a9c00bdf0 con 0x7f5aac1044b0 2023-06-23T18:15:07.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.391+0000 7f5aa97fa700 1 -- 172.21.15.2:0/605336518 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a9c022d80 con 0x7f5aac1044b0 2023-06-23T18:15:07.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.391+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac19b380 con 0x7f5aac1044b0 2023-06-23T18:15:07.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.392+0000 7f5aa97fa700 1 -- 172.21.15.2:0/605336518 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f5a9c00f460 con 0x7f5aac1044b0 2023-06-23T18:15:07.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.392+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac061b00 con 0x7f5aac1044b0 2023-06-23T18:15:07.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.392+0000 7f5aa97fa700 1 --2- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a980387b0 0x7f5a9803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:07.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.392+0000 7f5aa97fa700 1 -- 172.21.15.2:0/605336518 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5a9c04cc70 con 0x7f5aac1044b0 2023-06-23T18:15:07.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.392+0000 7f5aab7fe700 1 --2- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a980387b0 0x7f5a9803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:07.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.392+0000 7f5aab7fe700 1 --2- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a980387b0 0x7f5a9803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5a94006fd0 tx=0x7f5a94006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:07.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.397+0000 7f5aa97fa700 1 -- 172.21.15.2:0/605336518 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5a9c02c440 con 0x7f5aac1044b0 2023-06-23T18:15:07.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.603+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5aac067de0 con 0x7f5a980387b0 2023-06-23T18:15:07.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.605+0000 7f5aa97fa700 1 -- 172.21.15.2:0/605336518 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+237 (secure 0 0 0) 0x7f5aac067de0 con 0x7f5a980387b0 2023-06-23T18:15:07.611 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:15:07.611 INFO:teuthology.orchestra.run.smithi002.stdout:[{"addr": "172.21.15.2", "hostname": "smithi002", "labels": [], "status": ""}, {"addr": "172.21.15.17", "hostname": "smithi017", "labels": [], "status": ""}, {"addr": "172.21.15.82", "hostname": "smithi082", "labels": [], "status": ""}] 2023-06-23T18:15:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a980387b0 msgr2=0x7f5a9803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 --2- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a980387b0 0x7f5a9803ac60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5a94006fd0 tx=0x7f5a94006e40 comp rx=0 tx=0).stop 2023-06-23T18:15:07.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 msgr2=0x7f5aac19a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 --2- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 0x7f5aac19a820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c006010 tx=0x7f5a9c004ff0 comp rx=0 tx=0).stop 2023-06-23T18:15:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 shutdown_connections 2023-06-23T18:15:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 --2- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a980387b0 0x7f5a9803ac60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:07.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 --2- 172.21.15.2:0/605336518 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1044b0 0x7f5aac19a820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 >> 172.21.15.2:0/605336518 conn(0x7f5aac074270 msgr2=0x7f5aac0676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 shutdown_connections 2023-06-23T18:15:07.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:07.607+0000 7f5ab272d700 1 -- 172.21.15.2:0/605336518 wait complete. 2023-06-23T18:15:08.042 INFO:tasks.cephadm:Setting crush tunables to default 2023-06-23T18:15:08.043 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd crush tunables default 2023-06-23T18:15:08.603 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:15:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:08 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:08 smithi002 ceph-mon[95947]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:15:09.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:09 smithi002 ceph-mon[95947]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:09.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.898+0000 7f7edefed700 1 -- 172.21.15.2:0/2290691164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 msgr2=0x7f7ed8104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:09.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.898+0000 7f7edefed700 1 --2- 172.21.15.2:0/2290691164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 0x7f7ed8104880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009910 tx=0x7f7ecc009c20 comp rx=0 tx=0).stop 2023-06-23T18:15:09.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.898+0000 7f7edefed700 1 -- 172.21.15.2:0/2290691164 shutdown_connections 2023-06-23T18:15:09.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.898+0000 7f7edefed700 1 --2- 172.21.15.2:0/2290691164 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 0x7f7ed8104880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:09.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.898+0000 7f7edefed700 1 -- 172.21.15.2:0/2290691164 >> 172.21.15.2:0/2290691164 conn(0x7f7ed8074270 msgr2=0x7f7ed8074670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:09.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.898+0000 7f7edefed700 1 -- 172.21.15.2:0/2290691164 shutdown_connections 2023-06-23T18:15:09.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.898+0000 7f7edefed700 1 -- 172.21.15.2:0/2290691164 wait complete. 2023-06-23T18:15:09.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.899+0000 7f7edefed700 1 Processor -- start 2023-06-23T18:15:09.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.899+0000 7f7edefed700 1 -- start start 2023-06-23T18:15:09.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.899+0000 7f7edefed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 0x7f7ed819a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:09.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.899+0000 7f7edefed700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed819ad60 con 0x7f7ed81044b0 2023-06-23T18:15:09.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.899+0000 7f7edcd89700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 0x7f7ed819a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:09.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7edcd89700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 0x7f7ed819a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51502/0 (socket says 172.21.15.2:51502) 2023-06-23T18:15:09.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7edcd89700 1 -- 172.21.15.2:0/3247693326 learned_addr learned my addr 172.21.15.2:0/3247693326 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:15:09.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7edcd89700 1 -- 172.21.15.2:0/3247693326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0095f0 con 0x7f7ed81044b0 2023-06-23T18:15:09.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7edcd89700 1 --2- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 0x7f7ed819a820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc004f40 tx=0x7f7ecc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:09.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7ed5ffb700 1 -- 172.21.15.2:0/3247693326 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ecc01d070 con 0x7f7ed81044b0 2023-06-23T18:15:09.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7ed5ffb700 1 -- 172.21.15.2:0/3247693326 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ecc0044b0 con 0x7f7ed81044b0 2023-06-23T18:15:09.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed819af60 con 0x7f7ed81044b0 2023-06-23T18:15:09.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7ed5ffb700 1 -- 172.21.15.2:0/3247693326 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ecc003860 con 0x7f7ed81044b0 2023-06-23T18:15:09.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.900+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed819b320 con 0x7f7ed81044b0 2023-06-23T18:15:09.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.901+0000 7f7ed5ffb700 1 -- 172.21.15.2:0/3247693326 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f7ecc003ae0 con 0x7f7ed81044b0 2023-06-23T18:15:09.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.901+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed8061b00 con 0x7f7ed81044b0 2023-06-23T18:15:09.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.901+0000 7f7ed5ffb700 1 --2- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7ec80387b0 0x7f7ec803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:09.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.901+0000 7f7ed5ffb700 1 -- 172.21.15.2:0/3247693326 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7ecc04c650 con 0x7f7ed81044b0 2023-06-23T18:15:09.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.901+0000 7f7ed7fff700 1 --2- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7ec80387b0 0x7f7ec803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:09.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.902+0000 7f7ed7fff700 1 --2- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7ec80387b0 0x7f7ec803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006fd0 tx=0x7f7ec4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:09.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:09.906+0000 7f7ed5ffb700 1 -- 172.21.15.2:0/3247693326 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7ecc02a540 con 0x7f7ed81044b0 2023-06-23T18:15:10.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.115+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7ed8195db0 con 0x7f7ed81044b0 2023-06-23T18:15:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.989+0000 7f7ed5ffb700 1 -- 172.21.15.2:0/3247693326 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7ecc02a360 con 0x7f7ed81044b0 2023-06-23T18:15:10.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7ec80387b0 msgr2=0x7f7ec803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 --2- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7ec80387b0 0x7f7ec803ac60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006fd0 tx=0x7f7ec4006e40 comp rx=0 tx=0).stop 2023-06-23T18:15:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 msgr2=0x7f7ed819a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 --2- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 0x7f7ed819a820 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc004f40 tx=0x7f7ecc005020 comp rx=0 tx=0).stop 2023-06-23T18:15:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 shutdown_connections 2023-06-23T18:15:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 --2- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7ec80387b0 0x7f7ec803ac60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 --2- 172.21.15.2:0/3247693326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ed81044b0 0x7f7ed819a820 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 >> 172.21.15.2:0/3247693326 conn(0x7f7ed8074270 msgr2=0x7f7ed80676d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 shutdown_connections 2023-06-23T18:15:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:10.991+0000 7f7edefed700 1 -- 172.21.15.2:0/3247693326 wait complete. 2023-06-23T18:15:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2023-06-23T18:15:11.237 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:11.237 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:11.237 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:11.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:11.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:15:11.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:11.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:11.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: Updating smithi082:/etc/ceph/ceph.conf 2023-06-23T18:15:11.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3247693326' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-06-23T18:15:11.238 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: Updating smithi082:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:11.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:15:11.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: Updating smithi082:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T18:15:11.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:11.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:11.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:11.239 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:11.556 INFO:tasks.cephadm:Adding mon.a on smithi002 2023-06-23T18:15:11.557 INFO:tasks.cephadm:Adding mon.b on smithi017 2023-06-23T18:15:11.557 INFO:tasks.cephadm:Adding mon.c on smithi082 2023-06-23T18:15:11.557 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch apply mon '3;smithi002:172.21.15.2=a;smithi017:172.21.15.17=b;smithi082:172.21.15.82=c' 2023-06-23T18:15:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:12.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:11 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3247693326' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-06-23T18:15:12.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:11 smithi002 ceph-mon[95947]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T18:15:13.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:12 smithi002 ceph-mon[95947]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.501+0000 7f9cb5115700 1 -- 172.21.15.82:0/1045453493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 msgr2=0x7f9cb01059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.501+0000 7f9cb5115700 1 --2- 172.21.15.82:0/1045453493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 0x7f9cb01059b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009910 tx=0x7f9ca4009c20 comp rx=0 tx=0).stop 2023-06-23T18:15:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.501+0000 7f9cb5115700 1 -- 172.21.15.82:0/1045453493 shutdown_connections 2023-06-23T18:15:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.501+0000 7f9cb5115700 1 --2- 172.21.15.82:0/1045453493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 0x7f9cb01059b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.501+0000 7f9cb5115700 1 -- 172.21.15.82:0/1045453493 >> 172.21.15.82:0/1045453493 conn(0x7f9cb00fb0c0 msgr2=0x7f9cb00fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.501+0000 7f9cb5115700 1 -- 172.21.15.82:0/1045453493 shutdown_connections 2023-06-23T18:15:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.501+0000 7f9cb5115700 1 -- 172.21.15.82:0/1045453493 wait complete. 2023-06-23T18:15:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.502+0000 7f9cb5115700 1 Processor -- start 2023-06-23T18:15:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.502+0000 7f9cb5115700 1 -- start start 2023-06-23T18:15:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.503+0000 7f9cb5115700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 0x7f9cb019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.503+0000 7f9cb5115700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb019abf0 con 0x7f9cb01055e0 2023-06-23T18:15:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.503+0000 7f9caed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 0x7f9cb019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.503+0000 7f9caed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 0x7f9cb019a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.82:38408/0 (socket says 172.21.15.82:38408) 2023-06-23T18:15:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.503+0000 7f9caed9d700 1 -- 172.21.15.82:0/1863478427 learned_addr learned my addr 172.21.15.82:0/1863478427 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:15:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.503+0000 7f9caed9d700 1 -- 172.21.15.82:0/1863478427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca40095f0 con 0x7f9cb01055e0 2023-06-23T18:15:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.503+0000 7f9caed9d700 1 --2- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 0x7f9cb019a6b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4004f40 tx=0x7f9ca4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.504+0000 7f9c97fff700 1 -- 172.21.15.82:0/1863478427 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca401d070 con 0x7f9cb01055e0 2023-06-23T18:15:13.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.504+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb019adf0 con 0x7f9cb01055e0 2023-06-23T18:15:13.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.504+0000 7f9c97fff700 1 -- 172.21.15.82:0/1863478427 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9ca40044b0 con 0x7f9cb01055e0 2023-06-23T18:15:13.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.504+0000 7f9c97fff700 1 -- 172.21.15.82:0/1863478427 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ca4003810 con 0x7f9cb01055e0 2023-06-23T18:15:13.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.504+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb0194730 con 0x7f9cb01055e0 2023-06-23T18:15:13.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.504+0000 7f9c97fff700 1 -- 172.21.15.82:0/1863478427 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f9ca4003a90 con 0x7f9cb01055e0 2023-06-23T18:15:13.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.505+0000 7f9c97fff700 1 --2- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c980387a0 0x7f9c9803ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:13.511 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.505+0000 7f9c97fff700 1 -- 172.21.15.82:0/1863478427 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9ca404c6d0 con 0x7f9cb01055e0 2023-06-23T18:15:13.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.505+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb0061b00 con 0x7f9cb01055e0 2023-06-23T18:15:13.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.505+0000 7f9cae59c700 1 --2- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c980387a0 0x7f9c9803ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:13.512 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.505+0000 7f9cae59c700 1 --2- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c980387a0 0x7f9c9803ac50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0006fd0 tx=0x7f9ca0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:13.515 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.511+0000 7f9c97fff700 1 -- 172.21.15.82:0/1863478427 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9ca402a770 con 0x7f9cb01055e0 2023-06-23T18:15:13.710 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.705+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi017:172.21.15.17=b;smithi082:172.21.15.82=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cb0194970 con 0x7f9c980387a0 2023-06-23T18:15:13.718 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.714+0000 7f9c97fff700 1 -- 172.21.15.82:0/1863478427 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9cb0194970 con 0x7f9c980387a0 2023-06-23T18:15:13.718 INFO:teuthology.orchestra.run.smithi082.stdout:Scheduled mon update... 2023-06-23T18:15:13.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c980387a0 msgr2=0x7f9c9803ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:13.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 --2- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c980387a0 0x7f9c9803ac50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9ca0006fd0 tx=0x7f9ca0006e40 comp rx=0 tx=0).stop 2023-06-23T18:15:13.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 msgr2=0x7f9cb019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:13.720 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 --2- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 0x7f9cb019a6b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4004f40 tx=0x7f9ca4005020 comp rx=0 tx=0).stop 2023-06-23T18:15:13.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 shutdown_connections 2023-06-23T18:15:13.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 --2- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c980387a0 0x7f9c9803ac50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:13.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 --2- 172.21.15.82:0/1863478427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cb01055e0 0x7f9cb019a6b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:13.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 >> 172.21.15.82:0/1863478427 conn(0x7f9cb00fb0c0 msgr2=0x7f9cb00fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:13.721 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 shutdown_connections 2023-06-23T18:15:13.722 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:13.715+0000 7f9cb5115700 1 -- 172.21.15.82:0/1863478427 wait complete. 2023-06-23T18:15:14.098 DEBUG:teuthology.orchestra.run.smithi017:mon.b> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.b.service 2023-06-23T18:15:14.103 DEBUG:teuthology.orchestra.run.smithi082:mon.c> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.c.service 2023-06-23T18:15:14.107 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-06-23T18:15:14.108 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-23T18:15:14.139 INFO:journalctl@ceph.mon.b.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:15:14.140 INFO:journalctl@ceph.mon.c.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:15:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:15.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi017:172.21.15.17=b;smithi082:172.21.15.82=c", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:15.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi017:172.21.15.17=b;smithi082:172.21.15.82=c;count:3 2023-06-23T18:15:15.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:15.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:15.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:15.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:15.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:15.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:15.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:15.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: Deploying daemon mon.c on smithi082 2023-06-23T18:15:15.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:14 smithi002 ceph-mon[95947]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:16.699 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.694+0000 7f86db925700 1 -- 172.21.15.82:0/4068593501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 msgr2=0x7f86d4104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:16.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.694+0000 7f86db925700 1 --2- 172.21.15.82:0/4068593501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 0x7f86d4104880 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009910 tx=0x7f86d0009c20 comp rx=0 tx=0).stop 2023-06-23T18:15:16.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 -- 172.21.15.82:0/4068593501 shutdown_connections 2023-06-23T18:15:16.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 --2- 172.21.15.82:0/4068593501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 0x7f86d4104880 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:16.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 -- 172.21.15.82:0/4068593501 >> 172.21.15.82:0/4068593501 conn(0x7f86d40f9fd0 msgr2=0x7f86d40fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:16.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 -- 172.21.15.82:0/4068593501 shutdown_connections 2023-06-23T18:15:16.700 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 -- 172.21.15.82:0/4068593501 wait complete. 2023-06-23T18:15:16.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 Processor -- start 2023-06-23T18:15:16.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 -- start start 2023-06-23T18:15:16.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 0x7f86d4193650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:16.701 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.695+0000 7f86db925700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d4068e40 con 0x7f86d41044b0 2023-06-23T18:15:16.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.696+0000 7f86d96c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 0x7f86d4193650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:16.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.696+0000 7f86d96c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 0x7f86d4193650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.82:38414/0 (socket says 172.21.15.82:38414) 2023-06-23T18:15:16.702 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.696+0000 7f86d96c1700 1 -- 172.21.15.82:0/1480404770 learned_addr learned my addr 172.21.15.82:0/1480404770 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:15:16.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.696+0000 7f86d96c1700 1 -- 172.21.15.82:0/1480404770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d00095f0 con 0x7f86d41044b0 2023-06-23T18:15:16.703 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.696+0000 7f86d96c1700 1 --2- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 0x7f86d4193650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86d0004f40 tx=0x7f86d0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.696+0000 7f86ca7fc700 1 -- 172.21.15.82:0/1480404770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86d001d070 con 0x7f86d41044b0 2023-06-23T18:15:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.696+0000 7f86ca7fc700 1 -- 172.21.15.82:0/1480404770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86d00044b0 con 0x7f86d41044b0 2023-06-23T18:15:16.704 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.696+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d4193b90 con 0x7f86d41044b0 2023-06-23T18:15:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.697+0000 7f86ca7fc700 1 -- 172.21.15.82:0/1480404770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86d0003860 con 0x7f86d41044b0 2023-06-23T18:15:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.697+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d4193f50 con 0x7f86d41044b0 2023-06-23T18:15:16.705 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.698+0000 7f86ca7fc700 1 -- 172.21.15.82:0/1480404770 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f86d0003ae0 con 0x7f86d41044b0 2023-06-23T18:15:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.698+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d4061b00 con 0x7f86d41044b0 2023-06-23T18:15:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.698+0000 7f86ca7fc700 1 --2- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f86c00387b0 0x7f86c003ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.698+0000 7f86ca7fc700 1 -- 172.21.15.82:0/1480404770 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f86d004c850 con 0x7f86d41044b0 2023-06-23T18:15:16.706 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.698+0000 7f86d8ec0700 1 --2- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f86c00387b0 0x7f86c003ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:16.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.699+0000 7f86d8ec0700 1 --2- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f86c00387b0 0x7f86c003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f86c4006fd0 tx=0x7f86c4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:16.707 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.703+0000 7f86ca7fc700 1 -- 172.21.15.82:0/1480404770 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f86d002a770 con 0x7f86d41044b0 2023-06-23T18:15:16.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.985+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f86d4194270 con 0x7f86d41044b0 2023-06-23T18:15:16.990 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.986+0000 7f86ca7fc700 1 -- 172.21.15.82:0/1480404770 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7f86d000e3e0 con 0x7f86d41044b0 2023-06-23T18:15:16.991 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:15:16.991 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":1,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","modified":"2023-06-23T18:13:36.370680Z","created":"2023-06-23T18:13:36.370680Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-06-23T18:15:16.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f86c00387b0 msgr2=0x7f86c003ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:16.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 --2- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f86c00387b0 0x7f86c003ac60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f86c4006fd0 tx=0x7f86c4006e40 comp rx=0 tx=0).stop 2023-06-23T18:15:16.992 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 msgr2=0x7f86d4193650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:16.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 --2- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 0x7f86d4193650 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86d0004f40 tx=0x7f86d0005020 comp rx=0 tx=0).stop 2023-06-23T18:15:16.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 shutdown_connections 2023-06-23T18:15:16.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 --2- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f86c00387b0 0x7f86c003ac60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:16.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 --2- 172.21.15.82:0/1480404770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d41044b0 0x7f86d4193650 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:16.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 >> 172.21.15.82:0/1480404770 conn(0x7f86d40f9fd0 msgr2=0x7f86d40fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:16.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 shutdown_connections 2023-06-23T18:15:16.993 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:16.988+0000 7f86db925700 1 -- 172.21.15.82:0/1480404770 wait complete. 2023-06-23T18:15:16.994 INFO:teuthology.orchestra.run.smithi082.stderr:dumped monmap epoch 1 2023-06-23T18:15:17.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:16 smithi002 ceph-mon[95947]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:17.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:17 smithi082 ceph-mon[97889]: expand_channel_meta expanded map: {default=debug} 2023-06-23T18:15:17.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:17 smithi082 systemd[1]: Started Ceph mon.c for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:15:18.442 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-06-23T18:15:18.442 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-23T18:15:19.008 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:15:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:15:22.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: mon.a calling monitor election 2023-06-23T18:15:22.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: mon.c calling monitor election 2023-06-23T18:15:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T18:15:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: monmap e2: 2 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-06-23T18:15:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: fsmap 2023-06-23T18:15:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T18:15:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: mgrmap e13: x(active, since 45s) 2023-06-23T18:15:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: overall HEALTH_OK 2023-06-23T18:15:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:22.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:22.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:22.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:22.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:22.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:15:22.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: mon.a calling monitor election 2023-06-23T18:15:22.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:22.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: mon.c calling monitor election 2023-06-23T18:15:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T18:15:22.603 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: monmap e2: 2 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-06-23T18:15:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: fsmap 2023-06-23T18:15:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T18:15:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: mgrmap e13: x(active, since 45s) 2023-06-23T18:15:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: overall HEALTH_OK 2023-06-23T18:15:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:22.604 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:22.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:22.605 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:23.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.174+0000 7f7496899700 1 -- 172.21.15.82:0/1978417575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f747c0035c0 msgr2=0x7f747c005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:23.180 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.174+0000 7f7496899700 1 --2- 172.21.15.82:0/1978417575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f747c0035c0 0x7f747c005a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f74900fe510 tx=0x7f747800feb0 comp rx=0 tx=0).stop 2023-06-23T18:15:23.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.174+0000 7f7496899700 1 -- 172.21.15.82:0/1978417575 shutdown_connections 2023-06-23T18:15:23.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.174+0000 7f7496899700 1 --2- 172.21.15.82:0/1978417575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f747c0035c0 0x7f747c005a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:23.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.174+0000 7f7496899700 1 -- 172.21.15.82:0/1978417575 >> 172.21.15.82:0/1978417575 conn(0x7f74900fa120 msgr2=0x7f74900fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:23.181 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.174+0000 7f7496899700 1 -- 172.21.15.82:0/1978417575 shutdown_connections 2023-06-23T18:15:23.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.174+0000 7f7496899700 1 -- 172.21.15.82:0/1978417575 wait complete. 2023-06-23T18:15:23.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.175+0000 7f7496899700 1 Processor -- start 2023-06-23T18:15:23.182 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.175+0000 7f7496899700 1 -- start start 2023-06-23T18:15:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.175+0000 7f7496899700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 0x7f7490196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:23.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.175+0000 7f7496899700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7490197200 0x7f749019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:23.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.175+0000 7f7496899700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7490197780 con 0x7f7490197200 2023-06-23T18:15:23.186 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.175+0000 7f7496899700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74901978c0 con 0x7f747c0035c0 2023-06-23T18:15:23.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 0x7f7490196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:23.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 0x7f7490196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40462/0 (socket says 172.21.15.82:40462) 2023-06-23T18:15:23.187 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748ffff700 1 -- 172.21.15.82:0/3994559003 learned_addr learned my addr 172.21.15.82:0/3994559003 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:15:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748ffff700 1 -- 172.21.15.82:0/3994559003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 msgr2=0x7f7490196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-06-23T18:15:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748ffff700 1 -- 172.21.15.82:0/3994559003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 msgr2=0x7f7490196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-06-23T18:15:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748ffff700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 0x7f7490196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:15:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748f7fe700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7490197200 0x7f749019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748ffff700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 0x7f7490196cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:15:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748f7fe700 1 -- 172.21.15.82:0/3994559003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 msgr2=0x7f7490196cc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:15:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748f7fe700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 0x7f7490196cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:23.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.176+0000 7f748f7fe700 1 -- 172.21.15.82:0/3994559003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747800f900 con 0x7f7490197200 2023-06-23T18:15:23.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.177+0000 7f748f7fe700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7490197200 0x7f749019a5c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f748000b4d0 tx=0x7f748000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:23.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.177+0000 7f748d7fa700 1 -- 172.21.15.82:0/3994559003 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7480011630 con 0x7f7490197200 2023-06-23T18:15:23.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.177+0000 7f748d7fa700 1 -- 172.21.15.82:0/3994559003 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7480011c20 con 0x7f7490197200 2023-06-23T18:15:23.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.177+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749019ab00 con 0x7f7490197200 2023-06-23T18:15:23.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.177+0000 7f748d7fa700 1 -- 172.21.15.82:0/3994559003 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7480010c60 con 0x7f7490197200 2023-06-23T18:15:23.189 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.177+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749019b050 con 0x7f7490197200 2023-06-23T18:15:23.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.178+0000 7f748d7fa700 1 -- 172.21.15.82:0/3994559003 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f748001a400 con 0x7f7490197200 2023-06-23T18:15:23.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.178+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7490061b00 con 0x7f7490197200 2023-06-23T18:15:23.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.178+0000 7f748d7fa700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74700385f0 0x7f747003aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:23.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.178+0000 7f748d7fa700 1 -- 172.21.15.82:0/3994559003 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f748004ae40 con 0x7f7490197200 2023-06-23T18:15:23.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.178+0000 7f748ffff700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74700385f0 0x7f747003aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:23.190 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.179+0000 7f748ffff700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74700385f0 0x7f747003aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7478005b40 tx=0x7f7478005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:23.191 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.184+0000 7f748d7fa700 1 -- 172.21.15.82:0/3994559003 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7480010dc0 con 0x7f7490197200 2023-06-23T18:15:23.460 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.455+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f74901a1940 con 0x7f7490197200 2023-06-23T18:15:23.461 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.456+0000 7f748d7fa700 1 -- 172.21.15.82:0/3994559003 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+998 (secure 0 0 0) 0x7f74800172c0 con 0x7f7490197200 2023-06-23T18:15:23.461 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:15:23.461 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","modified":"2023-06-23T18:15:17.025455Z","created":"2023-06-23T18:13:36.370680Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-06-23T18:15:23.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74700385f0 msgr2=0x7f747003aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:23.463 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74700385f0 0x7f747003aaa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7478005b40 tx=0x7f7478005a90 comp rx=0 tx=0).stop 2023-06-23T18:15:23.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7490197200 msgr2=0x7f749019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:23.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7490197200 0x7f749019a5c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f748000b4d0 tx=0x7f748000b890 comp rx=0 tx=0).stop 2023-06-23T18:15:23.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 shutdown_connections 2023-06-23T18:15:23.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74700385f0 0x7f747003aaa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:23.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f747c0035c0 0x7f7490196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:23.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 --2- 172.21.15.82:0/3994559003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7490197200 0x7f749019a5c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:23.464 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 >> 172.21.15.82:0/3994559003 conn(0x7f74900fa120 msgr2=0x7f749018fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:23.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 shutdown_connections 2023-06-23T18:15:23.465 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:23.458+0000 7f7496899700 1 -- 172.21.15.82:0/3994559003 wait complete. 2023-06-23T18:15:23.465 INFO:teuthology.orchestra.run.smithi082.stderr:dumped monmap epoch 2 2023-06-23T18:15:23.482 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:23 smithi082 ceph-mon[97889]: Deploying daemon mon.b on smithi017 2023-06-23T18:15:23.483 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:23 smithi082 ceph-mon[97889]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:23.483 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:23 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:23.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:23 smithi002 ceph-mon[95947]: Deploying daemon mon.b on smithi017 2023-06-23T18:15:23.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:23 smithi002 ceph-mon[95947]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:23.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:23 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:24.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:24 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/3994559003' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T18:15:24.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:24 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/3994559003' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T18:15:24.867 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-06-23T18:15:24.867 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph mon dump -f json 2023-06-23T18:15:25.110 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:24 smithi017 systemd[1]: Starting Ceph mon.b for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:15:25.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:25 smithi082 ceph-mon[97889]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.394 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 podman[97433]: 2023-06-23T18:15:25.394 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: set uid:gid to 167:167 (ceph:ceph) 2023-06-23T18:15:25.394 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev), process ceph-mon, pid 7 2023-06-23T18:15:25.395 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pidfile_write: ignore empty --pid-file 2023-06-23T18:15:25.395 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: load: jerasure load: lrc 2023-06-23T18:15:25.395 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: RocksDB version: 7.9.2 2023-06-23T18:15:25.395 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Git sha 0 2023-06-23T18:15:25.395 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Compile date 2023-06-21 18:00:35 2023-06-23T18:15:25.395 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: DB SUMMARY 2023-06-23T18:15:25.396 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: DB Session ID: H4QFY8UGJEGV0W09RXFU 2023-06-23T18:15:25.396 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: CURRENT file: CURRENT 2023-06-23T18:15:25.396 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: IDENTITY file: IDENTITY 2023-06-23T18:15:25.396 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-06-23T18:15:25.396 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-06-23T18:15:25.396 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-06-23T18:15:25.396 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.error_if_exists: 0 2023-06-23T18:15:25.397 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.create_if_missing: 0 2023-06-23T18:15:25.397 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.paranoid_checks: 1 2023-06-23T18:15:25.397 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.flush_verify_memtable_count: 1 2023-06-23T18:15:25.397 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-06-23T18:15:25.397 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-06-23T18:15:25.397 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.env: 0x561a4898e720 2023-06-23T18:15:25.397 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.fs: PosixFileSystem 2023-06-23T18:15:25.398 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.info_log: 0x561a498d9260 2023-06-23T18:15:25.398 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_file_opening_threads: 16 2023-06-23T18:15:25.398 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.statistics: (nil) 2023-06-23T18:15:25.398 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.use_fsync: 0 2023-06-23T18:15:25.398 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_log_file_size: 0 2023-06-23T18:15:25.398 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-06-23T18:15:25.398 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.log_file_time_to_roll: 0 2023-06-23T18:15:25.398 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.keep_log_file_num: 1000 2023-06-23T18:15:25.399 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.recycle_log_file_num: 0 2023-06-23T18:15:25.399 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.allow_fallocate: 1 2023-06-23T18:15:25.399 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.allow_mmap_reads: 0 2023-06-23T18:15:25.399 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.allow_mmap_writes: 0 2023-06-23T18:15:25.399 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.use_direct_reads: 0 2023-06-23T18:15:25.399 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-06-23T18:15:25.399 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.create_missing_column_families: 0 2023-06-23T18:15:25.400 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.db_log_dir: 2023-06-23T18:15:25.400 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.wal_dir: 2023-06-23T18:15:25.400 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.table_cache_numshardbits: 6 2023-06-23T18:15:25.401 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.WAL_ttl_seconds: 0 2023-06-23T18:15:25.401 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 bash[97433]: 23368a99f9d12fdf2a21e47837c6048a11b9b73e273130b41d1748652bb8cbdc 2023-06-23T18:15:25.402 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.WAL_size_limit_MB: 0 2023-06-23T18:15:25.402 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-06-23T18:15:25.402 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-06-23T18:15:25.402 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.is_fd_close_on_exec: 1 2023-06-23T18:15:25.402 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.advise_random_on_open: 1 2023-06-23T18:15:25.402 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.db_write_buffer_size: 0 2023-06-23T18:15:25.402 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.write_buffer_manager: 0x561a497fa3c0 2023-06-23T18:15:25.403 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-06-23T18:15:25.403 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-06-23T18:15:25.403 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.use_adaptive_mutex: 0 2023-06-23T18:15:25.403 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.rate_limiter: (nil) 2023-06-23T18:15:25.403 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-06-23T18:15:25.403 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.wal_recovery_mode: 2 2023-06-23T18:15:25.403 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.enable_thread_tracking: 0 2023-06-23T18:15:25.403 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.enable_pipelined_write: 0 2023-06-23T18:15:25.404 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.unordered_write: 0 2023-06-23T18:15:25.404 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-06-23T18:15:25.404 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-06-23T18:15:25.404 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-06-23T18:15:25.404 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-06-23T18:15:25.404 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.row_cache: None 2023-06-23T18:15:25.404 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.wal_filter: None 2023-06-23T18:15:25.405 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-06-23T18:15:25.405 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.allow_ingest_behind: 0 2023-06-23T18:15:25.405 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.two_write_queues: 0 2023-06-23T18:15:25.405 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.manual_wal_flush: 0 2023-06-23T18:15:25.405 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.wal_compression: 0 2023-06-23T18:15:25.405 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.atomic_flush: 0 2023-06-23T18:15:25.405 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-06-23T18:15:25.405 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.persist_stats_to_disk: 0 2023-06-23T18:15:25.406 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.write_dbid_to_manifest: 0 2023-06-23T18:15:25.406 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.log_readahead_size: 0 2023-06-23T18:15:25.406 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-06-23T18:15:25.406 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.best_efforts_recovery: 0 2023-06-23T18:15:25.406 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-06-23T18:15:25.406 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-06-23T18:15:25.406 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.allow_data_in_errors: 0 2023-06-23T18:15:25.406 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.db_host_id: __hostname__ 2023-06-23T18:15:25.407 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.enforce_single_del_contracts: true 2023-06-23T18:15:25.407 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_background_jobs: 2 2023-06-23T18:15:25.407 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_background_compactions: -1 2023-06-23T18:15:25.407 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_subcompactions: 1 2023-06-23T18:15:25.407 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-06-23T18:15:25.407 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-06-23T18:15:25.407 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.delayed_write_rate : 16777216 2023-06-23T18:15:25.407 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_total_wal_size: 0 2023-06-23T18:15:25.408 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-06-23T18:15:25.408 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.stats_dump_period_sec: 600 2023-06-23T18:15:25.408 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.stats_persist_period_sec: 600 2023-06-23T18:15:25.408 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-06-23T18:15:25.408 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_open_files: -1 2023-06-23T18:15:25.408 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bytes_per_sync: 0 2023-06-23T18:15:25.408 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.wal_bytes_per_sync: 0 2023-06-23T18:15:25.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.strict_bytes_per_sync: 0 2023-06-23T18:15:25.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_readahead_size: 0 2023-06-23T18:15:25.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_background_flushes: -1 2023-06-23T18:15:25.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Compression algorithms supported: 2023-06-23T18:15:25.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-06-23T18:15:25.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: kZSTD supported: 0 2023-06-23T18:15:25.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: kXpressCompression supported: 0 2023-06-23T18:15:25.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: kLZ4HCCompression supported: 1 2023-06-23T18:15:25.410 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: kZlibCompression supported: 1 2023-06-23T18:15:25.410 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: kSnappyCompression supported: 1 2023-06-23T18:15:25.410 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: kLZ4Compression supported: 1 2023-06-23T18:15:25.410 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: kBZip2Compression supported: 0 2023-06-23T18:15:25.410 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Fast CRC32 supported: Supported on x86 2023-06-23T18:15:25.410 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: DMutex implementation: pthread_mutex_t 2023-06-23T18:15:25.410 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-06-23T18:15:25.410 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-06-23T18:15:25.411 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-06-23T18:15:25.411 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.merge_operator: 2023-06-23T18:15:25.411 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_filter: None 2023-06-23T18:15:25.411 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_filter_factory: None 2023-06-23T18:15:25.411 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.sst_partitioner_factory: None 2023-06-23T18:15:25.411 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.memtable_factory: SkipListFactory 2023-06-23T18:15:25.411 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.table_factory: BlockBasedTable 2023-06-23T18:15:25.411 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561a498d9360) 2023-06-23T18:15:25.412 INFO:journalctl@ceph.mon.b.smithi017.stdout: cache_index_and_filter_blocks: 1 2023-06-23T18:15:25.412 INFO:journalctl@ceph.mon.b.smithi017.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-06-23T18:15:25.412 INFO:journalctl@ceph.mon.b.smithi017.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-06-23T18:15:25.412 INFO:journalctl@ceph.mon.b.smithi017.stdout: pin_top_level_index_and_filter: 1 2023-06-23T18:15:25.412 INFO:journalctl@ceph.mon.b.smithi017.stdout: index_type: 0 2023-06-23T18:15:25.412 INFO:journalctl@ceph.mon.b.smithi017.stdout: data_block_index_type: 0 2023-06-23T18:15:25.413 INFO:journalctl@ceph.mon.b.smithi017.stdout: index_shortening: 1 2023-06-23T18:15:25.413 INFO:journalctl@ceph.mon.b.smithi017.stdout: data_block_hash_table_util_ratio: 0.750000 2023-06-23T18:15:25.413 INFO:journalctl@ceph.mon.b.smithi017.stdout: checksum: 4 2023-06-23T18:15:25.413 INFO:journalctl@ceph.mon.b.smithi017.stdout: no_block_cache: 0 2023-06-23T18:15:25.413 INFO:journalctl@ceph.mon.b.smithi017.stdout: block_cache: 0x561a498991f0 2023-06-23T18:15:25.413 INFO:journalctl@ceph.mon.b.smithi017.stdout: block_cache_name: BinnedLRUCache 2023-06-23T18:15:25.413 INFO:journalctl@ceph.mon.b.smithi017.stdout: block_cache_options: 2023-06-23T18:15:25.413 INFO:journalctl@ceph.mon.b.smithi017.stdout: capacity : 536870912 2023-06-23T18:15:25.414 INFO:journalctl@ceph.mon.b.smithi017.stdout: num_shard_bits : 4 2023-06-23T18:15:25.414 INFO:journalctl@ceph.mon.b.smithi017.stdout: strict_capacity_limit : 0 2023-06-23T18:15:25.414 INFO:journalctl@ceph.mon.b.smithi017.stdout: high_pri_pool_ratio: 0.000 2023-06-23T18:15:25.414 INFO:journalctl@ceph.mon.b.smithi017.stdout: block_cache_compressed: (nil) 2023-06-23T18:15:25.414 INFO:journalctl@ceph.mon.b.smithi017.stdout: persistent_cache: (nil) 2023-06-23T18:15:25.414 INFO:journalctl@ceph.mon.b.smithi017.stdout: block_size: 4096 2023-06-23T18:15:25.414 INFO:journalctl@ceph.mon.b.smithi017.stdout: block_size_deviation: 10 2023-06-23T18:15:25.414 INFO:journalctl@ceph.mon.b.smithi017.stdout: block_restart_interval: 16 2023-06-23T18:15:25.415 INFO:journalctl@ceph.mon.b.smithi017.stdout: index_block_restart_interval: 1 2023-06-23T18:15:25.415 INFO:journalctl@ceph.mon.b.smithi017.stdout: metadata_block_size: 4096 2023-06-23T18:15:25.415 INFO:journalctl@ceph.mon.b.smithi017.stdout: partition_filters: 0 2023-06-23T18:15:25.415 INFO:journalctl@ceph.mon.b.smithi017.stdout: use_delta_encoding: 1 2023-06-23T18:15:25.415 INFO:journalctl@ceph.mon.b.smithi017.stdout: filter_policy: bloomfilter 2023-06-23T18:15:25.415 INFO:journalctl@ceph.mon.b.smithi017.stdout: whole_key_filtering: 1 2023-06-23T18:15:25.415 INFO:journalctl@ceph.mon.b.smithi017.stdout: verify_compression: 0 2023-06-23T18:15:25.415 INFO:journalctl@ceph.mon.b.smithi017.stdout: read_amp_bytes_per_bit: 0 2023-06-23T18:15:25.416 INFO:journalctl@ceph.mon.b.smithi017.stdout: format_version: 5 2023-06-23T18:15:25.416 INFO:journalctl@ceph.mon.b.smithi017.stdout: enable_index_compression: 1 2023-06-23T18:15:25.416 INFO:journalctl@ceph.mon.b.smithi017.stdout: block_align: 0 2023-06-23T18:15:25.416 INFO:journalctl@ceph.mon.b.smithi017.stdout: max_auto_readahead_size: 262144 2023-06-23T18:15:25.416 INFO:journalctl@ceph.mon.b.smithi017.stdout: prepopulate_block_cache: 0 2023-06-23T18:15:25.417 INFO:journalctl@ceph.mon.b.smithi017.stdout: initial_auto_readahead_size: 8192 2023-06-23T18:15:25.417 INFO:journalctl@ceph.mon.b.smithi017.stdout: num_file_reads_for_auto_readahead: 2 2023-06-23T18:15:25.417 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.write_buffer_size: 33554432 2023-06-23T18:15:25.417 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_write_buffer_number: 2 2023-06-23T18:15:25.418 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression: NoCompression 2023-06-23T18:15:25.418 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression: Disabled 2023-06-23T18:15:25.418 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.prefix_extractor: nullptr 2023-06-23T18:15:25.419 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-06-23T18:15:25.419 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.num_levels: 7 2023-06-23T18:15:25.419 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-06-23T18:15:25.419 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-06-23T18:15:25.420 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-06-23T18:15:25.420 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-06-23T18:15:25.420 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-06-23T18:15:25.420 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-06-23T18:15:25.421 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-06-23T18:15:25.421 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-06-23T18:15:25.421 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-06-23T18:15:25.422 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-06-23T18:15:25.422 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-06-23T18:15:25.422 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-06-23T18:15:25.422 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.window_bits: -14 2023-06-23T18:15:25.422 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.level: 32767 2023-06-23T18:15:25.422 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.strategy: 0 2023-06-23T18:15:25.423 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-06-23T18:15:25.423 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-06-23T18:15:25.423 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-06-23T18:15:25.424 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-06-23T18:15:25.424 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.enabled: false 2023-06-23T18:15:25.424 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-06-23T18:15:25.424 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-06-23T18:15:25.425 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-06-23T18:15:25.425 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-06-23T18:15:25.425 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.target_file_size_base: 67108864 2023-06-23T18:15:25.425 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.target_file_size_multiplier: 1 2023-06-23T18:15:25.426 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-06-23T18:15:25.426 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-06-23T18:15:25.426 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-06-23T18:15:25.426 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-06-23T18:15:25.427 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-06-23T18:15:25.427 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-06-23T18:15:25.427 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-06-23T18:15:25.428 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-06-23T18:15:25.428 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-06-23T18:15:25.428 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-06-23T18:15:25.428 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-06-23T18:15:25.429 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-06-23T18:15:25.429 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-06-23T18:15:25.429 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.arena_block_size: 1048576 2023-06-23T18:15:25.429 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-06-23T18:15:25.429 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-06-23T18:15:25.429 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.disable_auto_compactions: 0 2023-06-23T18:15:25.430 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-06-23T18:15:25.430 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-06-23T18:15:25.430 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-06-23T18:15:25.430 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-06-23T18:15:25.430 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-06-23T18:15:25.430 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-06-23T18:15:25.430 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-06-23T18:15:25.430 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-06-23T18:15:25.431 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-06-23T18:15:25.431 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-06-23T18:15:25.431 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.table_properties_collectors: 2023-06-23T18:15:25.431 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.inplace_update_support: 0 2023-06-23T18:15:25.431 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.inplace_update_num_locks: 10000 2023-06-23T18:15:25.431 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-06-23T18:15:25.431 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-06-23T18:15:25.431 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.memtable_huge_page_size: 0 2023-06-23T18:15:25.432 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.bloom_locality: 0 2023-06-23T18:15:25.432 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.max_successive_merges: 0 2023-06-23T18:15:25.432 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.optimize_filters_for_hits: 0 2023-06-23T18:15:25.432 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.paranoid_file_checks: 0 2023-06-23T18:15:25.432 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.force_consistency_checks: 1 2023-06-23T18:15:25.432 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.report_bg_io_stats: 0 2023-06-23T18:15:25.432 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.ttl: 2592000 2023-06-23T18:15:25.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.periodic_compaction_seconds: 0 2023-06-23T18:15:25.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-06-23T18:15:25.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-06-23T18:15:25.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.enable_blob_files: false 2023-06-23T18:15:25.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.min_blob_size: 0 2023-06-23T18:15:25.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.blob_file_size: 268435456 2023-06-23T18:15:25.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.blob_compression_type: NoCompression 2023-06-23T18:15:25.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.enable_blob_garbage_collection: false 2023-06-23T18:15:25.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-06-23T18:15:25.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-06-23T18:15:25.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-06-23T18:15:25.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.blob_file_starting_level: 0 2023-06-23T18:15:25.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-06-23T18:15:25.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-06-23T18:15:25.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-06-23T18:15:25.435 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 08db7d3d-0ac3-4f66-abe4-b39ec93b71f6 2023-06-23T18:15:25.435 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687544125318116, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-06-23T18:15:25.435 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-06-23T18:15:25.435 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687544125318739, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1687544125, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "08db7d3d-0ac3-4f66-abe4-b39ec93b71f6", "db_session_id": "H4QFY8UGJEGV0W09RXFU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-06-23T18:15:25.435 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: EVENT_LOG_v1 {"time_micros": 1687544125318838, "job": 1, "event": "recovery_finished"} 2023-06-23T18:15:25.435 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-06-23T18:15:25.435 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-06-23T18:15:25.435 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561a49932000 2023-06-23T18:15:25.436 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: DB pointer 0x561a4991e000 2023-06-23T18:15:25.436 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-06-23T18:15:25.436 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-06-23T18:15:25.436 INFO:journalctl@ceph.mon.b.smithi017.stdout: ** DB Stats ** 2023-06-23T18:15:25.436 INFO:journalctl@ceph.mon.b.smithi017.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T18:15:25.436 INFO:journalctl@ceph.mon.b.smithi017.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-06-23T18:15:25.436 INFO:journalctl@ceph.mon.b.smithi017.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-23T18:15:25.437 INFO:journalctl@ceph.mon.b.smithi017.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-23T18:15:25.437 INFO:journalctl@ceph.mon.b.smithi017.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-06-23T18:15:25.437 INFO:journalctl@ceph.mon.b.smithi017.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-06-23T18:15:25.437 INFO:journalctl@ceph.mon.b.smithi017.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-06-23T18:15:25.438 INFO:journalctl@ceph.mon.b.smithi017.stdout: 2023-06-23T18:15:25.438 INFO:journalctl@ceph.mon.b.smithi017.stdout: ** Compaction Stats [default] ** 2023-06-23T18:15:25.438 INFO:journalctl@ceph.mon.b.smithi017.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-06-23T18:15:25.438 INFO:journalctl@ceph.mon.b.smithi017.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-06-23T18:15:25.439 INFO:journalctl@ceph.mon.b.smithi017.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-23T18:15:25.439 INFO:journalctl@ceph.mon.b.smithi017.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-23T18:15:25.439 INFO:journalctl@ceph.mon.b.smithi017.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-23T18:15:25.439 INFO:journalctl@ceph.mon.b.smithi017.stdout: 2023-06-23T18:15:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout: ** Compaction Stats [default] ** 2023-06-23T18:15:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-06-23T18:15:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-06-23T18:15:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-06-23T18:15:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout: 2023-06-23T18:15:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-06-23T18:15:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout: 2023-06-23T18:15:25.442 INFO:journalctl@ceph.mon.b.smithi017.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-06-23T18:15:25.442 INFO:journalctl@ceph.mon.b.smithi017.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-06-23T18:15:25.442 INFO:journalctl@ceph.mon.b.smithi017.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-06-23T18:15:25.442 INFO:journalctl@ceph.mon.b.smithi017.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-06-23T18:15:25.443 INFO:journalctl@ceph.mon.b.smithi017.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-06-23T18:15:25.443 INFO:journalctl@ceph.mon.b.smithi017.stdout: AddFile(Keys): cumulative 0, interval 0 2023-06-23T18:15:25.443 INFO:journalctl@ceph.mon.b.smithi017.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T18:15:25.443 INFO:journalctl@ceph.mon.b.smithi017.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-06-23T18:15:25.443 INFO:journalctl@ceph.mon.b.smithi017.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-06-23T18:15:25.444 INFO:journalctl@ceph.mon.b.smithi017.stdout: Block cache BinnedLRUCache@0x561a498991f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-06-23T18:15:25.444 INFO:journalctl@ceph.mon.b.smithi017.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-06-23T18:15:25.444 INFO:journalctl@ceph.mon.b.smithi017.stdout: 2023-06-23T18:15:25.445 INFO:journalctl@ceph.mon.b.smithi017.stdout: ** File Read Latency Histogram By Level [default] ** 2023-06-23T18:15:25.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-06-23T18:15:25.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: using public_addr v2:172.21.15.17:0/0 -> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] 2023-06-23T18:15:25.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: starting mon.b rank -1 at public addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] at bind addrs [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:15:25.446 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(???) e0 preinit fsid 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:15:25.446 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).mds e1 new map 2023-06-23T18:15:25.446 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).mds e1 print_map 2023-06-23T18:15:25.446 INFO:journalctl@ceph.mon.b.smithi017.stdout: e1 2023-06-23T18:15:25.447 INFO:journalctl@ceph.mon.b.smithi017.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-06-23T18:15:25.447 INFO:journalctl@ceph.mon.b.smithi017.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-23T18:15:25.447 INFO:journalctl@ceph.mon.b.smithi017.stdout: legacy client fscid: -1 2023-06-23T18:15:25.447 INFO:journalctl@ceph.mon.b.smithi017.stdout: 2023-06-23T18:15:25.448 INFO:journalctl@ceph.mon.b.smithi017.stdout: No filesystems configured 2023-06-23T18:15:25.448 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-06-23T18:15:25.448 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-06-23T18:15:25.448 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-06-23T18:15:25.449 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-06-23T18:15:25.449 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-06-23T18:15:25.449 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-06-23T18:15:25.450 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-06-23T18:15:25.450 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T18:15:25.450 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T18:15:25.450 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-06-23T18:15:25.450 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.450 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.450 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.451 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.451 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.451 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi017", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:15:25.451 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:25.451 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:25.451 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Updating smithi017:/etc/ceph/ceph.conf 2023-06-23T18:15:25.451 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Updating smithi017:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:25.451 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi082", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:25.452 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Updating smithi017:/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:15:25.452 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Updating smithi017:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T18:15:25.452 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.452 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Deploying cephadm binary to smithi082 2023-06-23T18:15:25.452 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.452 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.452 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.453 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.453 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.453 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Added host smithi082 2023-06-23T18:15:25.453 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:25.453 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.453 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.453 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.454 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.454 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:15:25.454 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.454 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.454 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.454 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.454 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.454 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi082", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:15:25.455 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:25.455 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:25.455 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Updating smithi082:/etc/ceph/ceph.conf 2023-06-23T18:15:25.455 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3247693326' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-06-23T18:15:25.455 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Updating smithi082:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:25.455 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Updating smithi082:/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:15:25.455 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Updating smithi082:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.client.admin.keyring 2023-06-23T18:15:25.455 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.456 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.456 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.456 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.456 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3247693326' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-06-23T18:15:25.456 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T18:15:25.456 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.457 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi002:172.21.15.2=a;smithi017:172.21.15.17=b;smithi082:172.21.15.82=c", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:25.457 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Saving service mon spec with placement smithi002:172.21.15.2=a;smithi017:172.21.15.17=b;smithi082:172.21.15.82=c;count:3 2023-06-23T18:15:25.457 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.457 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:25.457 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:25.457 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:25.457 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.458 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:25.458 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:25.458 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Deploying daemon mon.c on smithi082 2023-06-23T18:15:25.458 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.458 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.458 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:15:25.459 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.a calling monitor election 2023-06-23T18:15:25.459 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:25.459 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:25.460 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.460 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:25.460 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.c calling monitor election 2023-06-23T18:15:25.460 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:25.461 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.461 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:25.461 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:25.461 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-06-23T18:15:25.462 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: monmap e2: 2 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-06-23T18:15:25.462 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: fsmap 2023-06-23T18:15:25.462 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T18:15:25.463 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mgrmap e13: x(active, since 45s) 2023-06-23T18:15:25.463 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: overall HEALTH_OK 2023-06-23T18:15:25.463 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.463 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.464 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:25.464 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:25.464 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:25.464 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: Deploying daemon mon.b on smithi017 2023-06-23T18:15:25.465 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.465 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:25.465 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/3994559003' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T18:15:25.465 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:25.465 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-06-23T18:15:25.465 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 ceph-mon[97472]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-06-23T18:15:25.465 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:25 smithi017 systemd[1]: Started Ceph mon.b for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:15:25.466 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:15:25.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:25 smithi002 ceph-mon[95947]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:15:30.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: mon.a calling monitor election 2023-06-23T18:15:30.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: mon.c calling monitor election 2023-06-23T18:15:30.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:30.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:30.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: mon.b calling monitor election 2023-06-23T18:15:30.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:30.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-23T18:15:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-06-23T18:15:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: fsmap 2023-06-23T18:15:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T18:15:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: mgrmap e13: x(active, since 53s) 2023-06-23T18:15:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: overall HEALTH_OK 2023-06-23T18:15:30.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:15:30.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: mon.a calling monitor election 2023-06-23T18:15:30.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: mon.c calling monitor election 2023-06-23T18:15:30.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:30.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:30.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: mon.b calling monitor election 2023-06-23T18:15:30.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:30.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-23T18:15:30.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-06-23T18:15:30.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: fsmap 2023-06-23T18:15:30.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T18:15:30.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: mgrmap e13: x(active, since 53s) 2023-06-23T18:15:30.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: overall HEALTH_OK 2023-06-23T18:15:30.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:15:30.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: mon.a calling monitor election 2023-06-23T18:15:30.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: mon.c calling monitor election 2023-06-23T18:15:30.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:15:30.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:30.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: mon.b calling monitor election 2023-06-23T18:15:30.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:30.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:30.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-06-23T18:15:30.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: monmap e3: 3 mons at {a=[v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0],b=[v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0],c=[v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0]} removed_ranks: {} 2023-06-23T18:15:30.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: fsmap 2023-06-23T18:15:30.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: osdmap e4: 0 total, 0 up, 0 in 2023-06-23T18:15:30.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: mgrmap e13: x(active, since 53s) 2023-06-23T18:15:30.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: overall HEALTH_OK 2023-06-23T18:15:30.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:30.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:31.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 -- 172.21.15.82:0/2021971290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 msgr2=0x7fa4f8005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:31.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 --2- 172.21.15.82:0/2021971290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 0x7fa4f8005a40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa50c068590 tx=0x7fa4fc00fe60 comp rx=0 tx=0).stop 2023-06-23T18:15:31.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 -- 172.21.15.82:0/2021971290 shutdown_connections 2023-06-23T18:15:31.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 --2- 172.21.15.82:0/2021971290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 0x7fa4f8005a40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:31.018 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 -- 172.21.15.82:0/2021971290 >> 172.21.15.82:0/2021971290 conn(0x7fa50c0fa120 msgr2=0x7fa50c0fc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:31.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 -- 172.21.15.82:0/2021971290 shutdown_connections 2023-06-23T18:15:31.019 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 -- 172.21.15.82:0/2021971290 wait complete. 2023-06-23T18:15:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 Processor -- start 2023-06-23T18:15:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.013+0000 7fa513d4a700 1 -- start start 2023-06-23T18:15:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa513d4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 0x7fa50c194d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa513d4a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 0x7fa50c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa513d4a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 0x7fa50c19a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa513d4a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c195920 con 0x7fa4f80035c0 2023-06-23T18:15:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa513d4a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1a1970 con 0x7fa50c195250 2023-06-23T18:15:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa513d4a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50c1a1ab0 con 0x7fa50c199c00 2023-06-23T18:15:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5122e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 0x7fa50c19a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5122e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 0x7fa50c19a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40490/0 (socket says 172.21.15.82:40490) 2023-06-23T18:15:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5122e7700 1 -- 172.21.15.82:0/2634291957 learned_addr learned my addr 172.21.15.82:0/2634291957 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:15:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5122e7700 1 -- 172.21.15.82:0/2634291957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 msgr2=0x7fa50c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-06-23T18:15:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5122e7700 1 -- 172.21.15.82:0/2634291957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 msgr2=0x7fa50c19a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-06-23T18:15:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5122e7700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 0x7fa50c19a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:15:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa511ae6700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 0x7fa50c194d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5122e7700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 0x7fa50c19a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:15:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5112e5700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 0x7fa50c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5112e5700 1 -- 172.21.15.82:0/2634291957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 msgr2=0x7fa50c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-06-23T18:15:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5112e5700 1 -- 172.21.15.82:0/2634291957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 msgr2=0x7fa50c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-06-23T18:15:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5112e5700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 0x7fa50c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-06-23T18:15:31.023 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa5112e5700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 0x7fa50c1996c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-06-23T18:15:31.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa511ae6700 1 -- 172.21.15.82:0/2634291957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 msgr2=0x7fa50c19a0a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:15:31.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa511ae6700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 0x7fa50c19a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:31.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa511ae6700 1 -- 172.21.15.82:0/2634291957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 msgr2=0x7fa50c1996c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:15:31.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa511ae6700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 0x7fa50c1996c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:31.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.014+0000 7fa511ae6700 1 -- 172.21.15.82:0/2634291957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc00f8b0 con 0x7fa4f80035c0 2023-06-23T18:15:31.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.015+0000 7fa511ae6700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 0x7fa50c194d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa50c068590 tx=0x7fa4fc0040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:31.024 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.015+0000 7fa502ffd700 1 -- 172.21.15.82:0/2634291957 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc012d60 con 0x7fa4f80035c0 2023-06-23T18:15:31.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.015+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa50c1a1d00 con 0x7fa4f80035c0 2023-06-23T18:15:31.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.015+0000 7fa502ffd700 1 -- 172.21.15.82:0/2634291957 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa4fc004350 con 0x7fa4f80035c0 2023-06-23T18:15:31.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.015+0000 7fa502ffd700 1 -- 172.21.15.82:0/2634291957 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc027770 con 0x7fa4f80035c0 2023-06-23T18:15:31.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.015+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa50c1a21a0 con 0x7fa4f80035c0 2023-06-23T18:15:31.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.016+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa50c18f110 con 0x7fa4f80035c0 2023-06-23T18:15:31.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.016+0000 7fa502ffd700 1 -- 172.21.15.82:0/2634291957 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fa4fc024050 con 0x7fa4f80035c0 2023-06-23T18:15:31.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.016+0000 7fa502ffd700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4f0038c30 0x7fa4f003b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:31.025 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.016+0000 7fa502ffd700 1 -- 172.21.15.82:0/2634291957 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa4fc000e40 con 0x7fa4f80035c0 2023-06-23T18:15:31.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.016+0000 7fa5112e5700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4f0038c30 0x7fa4f003b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:31.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.017+0000 7fa5112e5700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4f0038c30 0x7fa4f003b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa508009850 tx=0x7fa508008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:31.026 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.021+0000 7fa502ffd700 1 -- 172.21.15.82:0/2634291957 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa4fc01d440 con 0x7fa4f80035c0 2023-06-23T18:15:31.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.297+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa50c18eaf0 con 0x7fa4f80035c0 2023-06-23T18:15:31.302 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.297+0000 7fa502ffd700 1 -- 172.21.15.82:0/2634291957 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1264 (secure 0 0 0) 0x7fa4fc022090 con 0x7fa4f80035c0 2023-06-23T18:15:31.303 INFO:teuthology.orchestra.run.smithi082.stdout: 2023-06-23T18:15:31.303 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","modified":"2023-06-23T18:15:25.345293Z","created":"2023-06-23T18:13:36.370680Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:3300","nonce":0},{"type":"v1","addr":"172.21.15.2:6789","nonce":0}]},"addr":"172.21.15.2:6789/0","public_addr":"172.21.15.2:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:3300","nonce":0},{"type":"v1","addr":"172.21.15.82:6789","nonce":0}]},"addr":"172.21.15.82:6789/0","public_addr":"172.21.15.82:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-06-23T18:15:31.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4f0038c30 msgr2=0x7fa4f003b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:31.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4f0038c30 0x7fa4f003b0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa508009850 tx=0x7fa508008040 comp rx=0 tx=0).stop 2023-06-23T18:15:31.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 msgr2=0x7fa50c194d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:31.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 0x7fa50c194d10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa50c068590 tx=0x7fa4fc0040b0 comp rx=0 tx=0).stop 2023-06-23T18:15:31.305 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 shutdown_connections 2023-06-23T18:15:31.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa50c199c00 0x7fa50c19a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:31.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa4f80035c0 0x7fa50c194d10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:31.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4f0038c30 0x7fa4f003b0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:31.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 --2- 172.21.15.82:0/2634291957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa50c195250 0x7fa50c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:31.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 >> 172.21.15.82:0/2634291957 conn(0x7fa50c0fa120 msgr2=0x7fa50c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:31.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 shutdown_connections 2023-06-23T18:15:31.306 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:31.300+0000 7fa513d4a700 1 -- 172.21.15.82:0/2634291957 wait complete. 2023-06-23T18:15:31.307 INFO:teuthology.orchestra.run.smithi082.stderr:dumped monmap epoch 3 2023-06-23T18:15:31.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:31 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:31.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:31 smithi082 ceph-mon[97889]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:31.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:31 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/2634291957' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T18:15:31.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:31 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:31.679 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-06-23T18:15:31.679 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph config generate-minimal-conf 2023-06-23T18:15:31.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:31 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:31.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:31 smithi002 ceph-mon[95947]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:31.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:31 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/2634291957' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T18:15:31.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:31 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:31.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:31 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:31.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:31 smithi017 ceph-mon[97472]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:31.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:31 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/2634291957' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-06-23T18:15:31.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:31 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:15:32.259 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:15:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: Updating smithi002:/etc/ceph/ceph.conf 2023-06-23T18:15:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: Updating smithi017:/etc/ceph/ceph.conf 2023-06-23T18:15:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: Updating smithi082:/etc/ceph/ceph.conf 2023-06-23T18:15:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: Updating smithi002:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: Updating smithi082:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: Updating smithi017:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:32.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:32 smithi002 ceph-mon[95947]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:33.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:33.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: Updating smithi002:/etc/ceph/ceph.conf 2023-06-23T18:15:33.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: Updating smithi017:/etc/ceph/ceph.conf 2023-06-23T18:15:33.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: Updating smithi082:/etc/ceph/ceph.conf 2023-06-23T18:15:33.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: Updating smithi002:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:33.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: Updating smithi082:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:33.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: Updating smithi017:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:33.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.102 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:32 smithi082 ceph-mon[97889]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:33.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:33.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: Updating smithi002:/etc/ceph/ceph.conf 2023-06-23T18:15:33.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: Updating smithi017:/etc/ceph/ceph.conf 2023-06-23T18:15:33.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: Updating smithi082:/etc/ceph/ceph.conf 2023-06-23T18:15:33.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: Updating smithi002:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:33.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: Updating smithi082:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:33.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: Updating smithi017:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/config/ceph.conf 2023-06-23T18:15:33.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:33.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:32 smithi017 ceph-mon[97472]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:33.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.617+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1718012233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 msgr2=0x7fd028104940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:33.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.617+0000 7fd02eb0f700 1 --2- 172.21.15.2:0/1718012233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 0x7fd028104940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009910 tx=0x7fd01c009c20 comp rx=0 tx=0).stop 2023-06-23T18:15:33.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.617+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1718012233 shutdown_connections 2023-06-23T18:15:33.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.617+0000 7fd02eb0f700 1 --2- 172.21.15.2:0/1718012233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 0x7fd028104940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:33.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.617+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1718012233 >> 172.21.15.2:0/1718012233 conn(0x7fd0280fa050 msgr2=0x7fd0280fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:33.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.617+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1718012233 shutdown_connections 2023-06-23T18:15:33.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.617+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1718012233 wait complete. 2023-06-23T18:15:33.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.617+0000 7fd02eb0f700 1 Processor -- start 2023-06-23T18:15:33.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02eb0f700 1 -- start start 2023-06-23T18:15:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02eb0f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 0x7fd028195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02eb0f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd028195e40 0x7fd0281962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02eb0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd02819e970 0x7fd0281a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02eb0f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0281a1500 con 0x7fd028104570 2023-06-23T18:15:33.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02eb0f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0281a1670 con 0x7fd02819e970 2023-06-23T18:15:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02eb0f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0281a1830 con 0x7fd028195e40 2023-06-23T18:15:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02c8ab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 0x7fd028195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02c8ab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 0x7fd028195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51508/0 (socket says 172.21.15.2:51508) 2023-06-23T18:15:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02c8ab700 1 -- 172.21.15.2:0/1110020703 learned_addr learned my addr 172.21.15.2:0/1110020703 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:15:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02c8ab700 1 -- 172.21.15.2:0/1110020703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd028195e40 msgr2=0x7fd0281962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02d0ac700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd02819e970 0x7fd0281a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:33.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd027fff700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd028195e40 0x7fd0281962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02c8ab700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd028195e40 0x7fd0281962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02c8ab700 1 -- 172.21.15.2:0/1110020703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd02819e970 msgr2=0x7fd0281a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02c8ab700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd02819e970 0x7fd0281a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02c8ab700 1 -- 172.21.15.2:0/1110020703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c0095f0 con 0x7fd028104570 2023-06-23T18:15:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.618+0000 7fd02d0ac700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd02819e970 0x7fd0281a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:15:33.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.619+0000 7fd02c8ab700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 0x7fd028195900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00bcf0 tx=0x7fd01c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.619+0000 7fd025ffb700 1 -- 172.21.15.2:0/1110020703 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c01d070 con 0x7fd028104570 2023-06-23T18:15:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.619+0000 7fd025ffb700 1 -- 172.21.15.2:0/1110020703 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd01c004db0 con 0x7fd028104570 2023-06-23T18:15:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.619+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0281a1b10 con 0x7fd028104570 2023-06-23T18:15:33.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.619+0000 7fd025ffb700 1 -- 172.21.15.2:0/1110020703 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c022d90 con 0x7fd028104570 2023-06-23T18:15:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.619+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0281a2060 con 0x7fd028104570 2023-06-23T18:15:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.620+0000 7fd025ffb700 1 -- 172.21.15.2:0/1110020703 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fd01c00f460 con 0x7fd028104570 2023-06-23T18:15:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.620+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd028061b00 con 0x7fd028104570 2023-06-23T18:15:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.620+0000 7fd025ffb700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd018038c80 0x7fd01803b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.620+0000 7fd025ffb700 1 -- 172.21.15.2:0/1110020703 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd01c04c8c0 con 0x7fd028104570 2023-06-23T18:15:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.620+0000 7fd027fff700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd018038c80 0x7fd01803b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:33.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.621+0000 7fd027fff700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd018038c80 0x7fd01803b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0140078e0 tx=0x7fd014008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:33.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.625+0000 7fd025ffb700 1 -- 172.21.15.2:0/1110020703 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd01c004f20 con 0x7fd028104570 2023-06-23T18:15:33.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.836+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd028068630 con 0x7fd028104570 2023-06-23T18:15:33.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.836+0000 7fd025ffb700 1 -- 172.21.15.2:0/1110020703 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+269 (secure 0 0 0) 0x7fd01c027030 con 0x7fd028104570 2023-06-23T18:15:33.842 INFO:teuthology.orchestra.run.smithi002.stdout:# minimal ceph.conf for 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:15:33.842 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2023-06-23T18:15:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:15:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: mon_host = [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] 2023-06-23T18:15:33.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd018038c80 msgr2=0x7fd01803b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:33.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd018038c80 0x7fd01803b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd0140078e0 tx=0x7fd014008040 comp rx=0 tx=0).stop 2023-06-23T18:15:33.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 msgr2=0x7fd028195900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:33.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 0x7fd028195900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00bcf0 tx=0x7fd01c00bdd0 comp rx=0 tx=0).stop 2023-06-23T18:15:33.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 shutdown_connections 2023-06-23T18:15:33.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd018038c80 0x7fd01803b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:33.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd02819e970 0x7fd0281a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:33.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd028104570 0x7fd028195900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:33.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 --2- 172.21.15.2:0/1110020703 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd028195e40 0x7fd0281962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:33.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 >> 172.21.15.2:0/1110020703 conn(0x7fd0280fa050 msgr2=0x7fd0280fb700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:33.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 shutdown_connections 2023-06-23T18:15:33.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:33.838+0000 7fd02eb0f700 1 -- 172.21.15.2:0/1110020703 wait complete. 2023-06-23T18:15:34.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:33 smithi002 ceph-mon[95947]: Reconfiguring mon.a (unknown last config time)... 2023-06-23T18:15:34.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:34.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:34.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:34.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:33 smithi002 ceph-mon[95947]: Reconfiguring daemon mon.a on smithi002 2023-06-23T18:15:34.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:33 smithi082 ceph-mon[97889]: Reconfiguring mon.a (unknown last config time)... 2023-06-23T18:15:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:33 smithi082 ceph-mon[97889]: Reconfiguring daemon mon.a on smithi002 2023-06-23T18:15:34.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:33 smithi017 ceph-mon[97472]: Reconfiguring mon.a (unknown last config time)... 2023-06-23T18:15:34.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:34.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:34.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:34.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:33 smithi017 ceph-mon[97472]: Reconfiguring daemon mon.a on smithi002 2023-06-23T18:15:34.613 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-06-23T18:15:34.614 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:15:34.614 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.conf 2023-06-23T18:15:34.683 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:15:34.683 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:15:34.751 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:15:34.752 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.conf 2023-06-23T18:15:34.782 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:15:34.782 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:15:34.850 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:15:34.851 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.conf 2023-06-23T18:15:34.887 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:15:34.888 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-06-23T18:15:34.958 INFO:tasks.cephadm:Adding mgr.x on smithi002 2023-06-23T18:15:34.958 INFO:tasks.cephadm:Adding mgr.y on smithi017 2023-06-23T18:15:34.959 INFO:tasks.cephadm:Adding mgr.z on smithi082 2023-06-23T18:15:34.959 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch apply mgr '3;smithi002=x;smithi017=y;smithi082=z' 2023-06-23T18:15:35.024 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1110020703' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:35.024 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:35.024 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:35.025 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: Reconfiguring mon.b (monmap changed)... 2023-06-23T18:15:35.025 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:35.025 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:35.026 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:35.026 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: Reconfiguring daemon mon.b on smithi017 2023-06-23T18:15:35.026 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:34 smithi082 ceph-mon[97889]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1110020703' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: Reconfiguring mon.b (monmap changed)... 2023-06-23T18:15:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: Reconfiguring daemon mon.b on smithi017 2023-06-23T18:15:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:34 smithi002 ceph-mon[95947]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:35.104 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1110020703' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:35.104 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:35.105 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:35.105 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: Reconfiguring mon.b (monmap changed)... 2023-06-23T18:15:35.105 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:35.105 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:35.105 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:35.105 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: Reconfiguring daemon mon.b on smithi017 2023-06-23T18:15:35.105 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:34 smithi017 ceph-mon[97472]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:35.545 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:15:36.717 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:36.717 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:36.717 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:36 smithi082 ceph-mon[97889]: Reconfiguring mon.c (monmap changed)... 2023-06-23T18:15:36.717 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:36.717 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:36.718 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:36.718 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:36 smithi082 ceph-mon[97889]: Reconfiguring daemon mon.c on smithi082 2023-06-23T18:15:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:36.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:36 smithi002 ceph-mon[95947]: Reconfiguring mon.c (monmap changed)... 2023-06-23T18:15:36.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:36.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:36.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:36.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:36 smithi002 ceph-mon[95947]: Reconfiguring daemon mon.c on smithi082 2023-06-23T18:15:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:36 smithi017 ceph-mon[97472]: Reconfiguring mon.c (monmap changed)... 2023-06-23T18:15:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-06-23T18:15:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-06-23T18:15:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:36 smithi017 ceph-mon[97472]: Reconfiguring daemon mon.c on smithi082 2023-06-23T18:15:37.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 -- 172.21.15.82:0/469403550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f41017b0 msgr2=0x7fa7f4108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:37.159 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/469403550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f41017b0 0x7fa7f4108050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00b0a0 tx=0x7fa7ec00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 -- 172.21.15.82:0/469403550 shutdown_connections 2023-06-23T18:15:37.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/469403550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f41017b0 0x7fa7f4108050 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/469403550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7f40ffb60 0x7fa7f40fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/469403550 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7f4100e00 0x7fa7f4101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 -- 172.21.15.82:0/469403550 >> 172.21.15.82:0/469403550 conn(0x7fa7f40fb1b0 msgr2=0x7fa7f40fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:37.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 -- 172.21.15.82:0/469403550 shutdown_connections 2023-06-23T18:15:37.160 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 -- 172.21.15.82:0/469403550 wait complete. 2023-06-23T18:15:37.161 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.154+0000 7fa7f94cb700 1 Processor -- start 2023-06-23T18:15:37.162 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f94cb700 1 -- start start 2023-06-23T18:15:37.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f94cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f40ffb60 0x7fa7f4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:37.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f94cb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7f4100e00 0x7fa7f4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:37.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f94cb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7f41017b0 0x7fa7f419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:37.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f94cb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f419f5f0 con 0x7fa7f41017b0 2023-06-23T18:15:37.163 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f94cb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f419f730 con 0x7fa7f4100e00 2023-06-23T18:15:37.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f94cb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f419f8c0 con 0x7fa7f40ffb60 2023-06-23T18:15:37.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f40ffb60 0x7fa7f4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:37.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f40ffb60 0x7fa7f4197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40500/0 (socket says 172.21.15.82:40500) 2023-06-23T18:15:37.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 -- 172.21.15.82:0/174352419 learned_addr learned my addr 172.21.15.82:0/174352419 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:15:37.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f27fc700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7f4100e00 0x7fa7f4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:37.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f37fe700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7f41017b0 0x7fa7f419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:37.164 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 -- 172.21.15.82:0/174352419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7f4100e00 msgr2=0x7fa7f4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:37.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7f4100e00 0x7fa7f4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 -- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7f41017b0 msgr2=0x7fa7f419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:37.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7f41017b0 0x7fa7f419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 -- 172.21.15.82:0/174352419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ec009d00 con 0x7fa7f40ffb60 2023-06-23T18:15:37.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7f2ffd700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f40ffb60 0x7fa7f4197a60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800c9e0 tx=0x7fa7e800ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:37.165 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.155+0000 7fa7dbfff700 1 -- 172.21.15.82:0/174352419 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8007ac0 con 0x7fa7f40ffb60 2023-06-23T18:15:37.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.156+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f419fba0 con 0x7fa7f40ffb60 2023-06-23T18:15:37.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.156+0000 7fa7dbfff700 1 -- 172.21.15.82:0/174352419 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa7e800ce80 con 0x7fa7f40ffb60 2023-06-23T18:15:37.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.156+0000 7fa7dbfff700 1 -- 172.21.15.82:0/174352419 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e8018400 con 0x7fa7f40ffb60 2023-06-23T18:15:37.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.156+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f41a00f0 con 0x7fa7f40ffb60 2023-06-23T18:15:37.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.156+0000 7fa7dbfff700 1 -- 172.21.15.82:0/174352419 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fa7e800f450 con 0x7fa7f40ffb60 2023-06-23T18:15:37.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.157+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f41920f0 con 0x7fa7f40ffb60 2023-06-23T18:15:37.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.157+0000 7fa7dbfff700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa7dc0389e0 0x7fa7dc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:37.166 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.157+0000 7fa7dbfff700 1 -- 172.21.15.82:0/174352419 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa7e804b170 con 0x7fa7f40ffb60 2023-06-23T18:15:37.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.157+0000 7fa7f27fc700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa7dc0389e0 0x7fa7dc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:37.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.157+0000 7fa7f27fc700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa7dc0389e0 0x7fa7dc03ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4005950 tx=0x7fa7e400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:37.167 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.162+0000 7fa7dbfff700 1 -- 172.21.15.82:0/174352419 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa7e800e7d0 con 0x7fa7f40ffb60 2023-06-23T18:15:37.369 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.364+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi017=y;smithi082=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7f40609a0 con 0x7fa7dc0389e0 2023-06-23T18:15:37.376 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.371+0000 7fa7dbfff700 1 -- 172.21.15.82:0/174352419 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa7f40609a0 con 0x7fa7dc0389e0 2023-06-23T18:15:37.377 INFO:teuthology.orchestra.run.smithi082.stdout:Scheduled mgr update... 2023-06-23T18:15:37.378 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.373+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa7dc0389e0 msgr2=0x7fa7dc03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.373+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa7dc0389e0 0x7fa7dc03ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4005950 tx=0x7fa7e400a300 comp rx=0 tx=0).stop 2023-06-23T18:15:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.373+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f40ffb60 msgr2=0x7fa7f4197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.373+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f40ffb60 0x7fa7f4197a60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa7e800c9e0 tx=0x7fa7e800ccf0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.374+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 shutdown_connections 2023-06-23T18:15:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.374+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7f41017b0 0x7fa7f419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.374+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7f40ffb60 0x7fa7f4197a60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.379 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.374+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa7dc0389e0 0x7fa7dc03ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.374+0000 7fa7f94cb700 1 --2- 172.21.15.82:0/174352419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7f4100e00 0x7fa7f4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.374+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 >> 172.21.15.82:0/174352419 conn(0x7fa7f40fb1b0 msgr2=0x7fa7f41066e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.374+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 shutdown_connections 2023-06-23T18:15:37.380 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:15:37.374+0000 7fa7f94cb700 1 -- 172.21.15.82:0/174352419 wait complete. 2023-06-23T18:15:37.745 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:37.745 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.745 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.746 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:37.746 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.746 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:37.746 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.747 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.747 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:37.747 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.747 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:37.747 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.747 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:37.747 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T18:15:37.748 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:37.748 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:37.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:37.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:37.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T18:15:37.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:37.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.805 DEBUG:teuthology.orchestra.run.smithi017:mgr.y> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.y.service 2023-06-23T18:15:37.808 DEBUG:teuthology.orchestra.run.smithi082:mgr.z> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.z.service 2023-06-23T18:15:37.836 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:37.836 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.836 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.837 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:37.837 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.837 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:37.837 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.837 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.837 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:37.837 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:37.838 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:37.842 INFO:journalctl@ceph.mgr.y.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:15:37.848 INFO:tasks.cephadm:Deploying OSDs... 2023-06-23T18:15:37.849 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:15:37.849 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T18:15:37.871 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T18:15:37.871 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2023-06-23T18:15:37.878 INFO:journalctl@ceph.mgr.z.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:15:37.932 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T18:15:37.932 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:37.932 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 379073 Links: 1 2023-06-23T18:15:37.932 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:37.933 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:37.933 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-06-23 18:14:48.874938260 +0000 2023-06-23T18:15:37.933 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-06-23 18:09:31.744508623 +0000 2023-06-23T18:15:37.933 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-06-23 18:09:31.744508623 +0000 2023-06-23T18:15:37.933 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2023-06-23T18:15:37.933 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T18:15:38.004 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-06-23T18:15:38.005 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-06-23T18:15:38.005 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.00013815 s, 3.7 MB/s 2023-06-23T18:15:38.006 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T18:15:38.067 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2023-06-23T18:15:38.128 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T18:15:38.128 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:38.128 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 375596 Links: 1 2023-06-23T18:15:38.128 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:38.129 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:38.129 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-06-23 18:14:48.874938260 +0000 2023-06-23T18:15:38.129 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-06-23 18:09:32.259497271 +0000 2023-06-23T18:15:38.129 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-06-23 18:09:32.259497271 +0000 2023-06-23T18:15:38.129 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2023-06-23T18:15:38.129 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T18:15:38.198 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-06-23T18:15:38.199 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-06-23T18:15:38.199 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000128406 s, 4.0 MB/s 2023-06-23T18:15:38.200 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T18:15:38.260 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2023-06-23T18:15:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T18:15:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:38.319 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 370436 Links: 1 2023-06-23T18:15:38.320 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:38.320 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:38.320 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-06-23 18:14:48.874938260 +0000 2023-06-23T18:15:38.320 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-06-23 18:09:32.802485301 +0000 2023-06-23T18:15:38.320 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-06-23 18:09:32.802485301 +0000 2023-06-23T18:15:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2023-06-23T18:15:38.320 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T18:15:38.387 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-06-23T18:15:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-06-23T18:15:38.388 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000124061 s, 4.1 MB/s 2023-06-23T18:15:38.389 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T18:15:38.449 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2023-06-23T18:15:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T18:15:38.508 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:38.508 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 379369 Links: 1 2023-06-23T18:15:38.508 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:38.509 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:38.509 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-06-23 18:14:48.874938260 +0000 2023-06-23T18:15:38.509 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-06-23 18:09:33.320473882 +0000 2023-06-23T18:15:38.509 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-06-23 18:09:33.320473882 +0000 2023-06-23T18:15:38.509 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2023-06-23T18:15:38.509 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T18:15:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-06-23T18:15:38.576 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-06-23T18:15:38.577 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000120588 s, 4.2 MB/s 2023-06-23T18:15:38.577 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T18:15:38.636 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2023-06-23T18:15:38.637 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T18:15:38.658 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T18:15:38.658 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2023-06-23T18:15:38.719 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T18:15:38.719 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:38.719 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 375600 Links: 1 2023-06-23T18:15:38.720 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:38.720 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:38.720 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-06-23 18:15:24.746613968 +0000 2023-06-23T18:15:38.720 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-06-23 18:09:33.716608698 +0000 2023-06-23T18:15:38.720 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-06-23 18:09:33.716608698 +0000 2023-06-23T18:15:38.720 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2023-06-23T18:15:38.721 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T18:15:38.788 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:38 smithi017 ceph-mon[97472]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi017=y;smithi082=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:38.788 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:38 smithi017 ceph-mon[97472]: Saving service mgr spec with placement smithi002=x;smithi017=y;smithi082=z;count:3 2023-06-23T18:15:38.789 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:38 smithi017 ceph-mon[97472]: Deploying daemon mgr.z on smithi082 2023-06-23T18:15:38.791 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-06-23T18:15:38.791 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-06-23T18:15:38.791 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000121525 s, 4.2 MB/s 2023-06-23T18:15:38.792 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T18:15:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:38 smithi002 ceph-mon[95947]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi017=y;smithi082=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:38 smithi002 ceph-mon[95947]: Saving service mgr spec with placement smithi002=x;smithi017=y;smithi082=z;count:3 2023-06-23T18:15:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:38 smithi002 ceph-mon[95947]: Deploying daemon mgr.z on smithi082 2023-06-23T18:15:38.824 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:38 smithi082 ceph-mon[97889]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi002=x;smithi017=y;smithi082=z", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:38.825 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:38 smithi082 ceph-mon[97889]: Saving service mgr spec with placement smithi002=x;smithi017=y;smithi082=z;count:3 2023-06-23T18:15:38.825 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:38 smithi082 ceph-mon[97889]: Deploying daemon mgr.z on smithi082 2023-06-23T18:15:38.853 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2023-06-23T18:15:38.913 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T18:15:38.913 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:38.913 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 373661 Links: 1 2023-06-23T18:15:38.913 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:38.913 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:38.914 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-06-23 18:15:24.746613968 +0000 2023-06-23T18:15:38.914 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-06-23 18:09:34.249603654 +0000 2023-06-23T18:15:38.914 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-06-23 18:09:34.249603654 +0000 2023-06-23T18:15:38.914 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2023-06-23T18:15:38.914 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T18:15:38.981 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-06-23T18:15:38.981 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-06-23T18:15:38.982 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000120256 s, 4.3 MB/s 2023-06-23T18:15:38.982 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T18:15:39.042 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2023-06-23T18:15:39.098 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:39 smithi082 systemd[1]: Starting Ceph mgr.z for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:15:39.100 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T18:15:39.100 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:39.101 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 375749 Links: 1 2023-06-23T18:15:39.101 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:39.101 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:39.101 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-06-23 18:15:24.747613958 +0000 2023-06-23T18:15:39.101 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-06-23 18:09:34.763598790 +0000 2023-06-23T18:15:39.101 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-06-23 18:09:34.763598790 +0000 2023-06-23T18:15:39.101 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2023-06-23T18:15:39.102 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T18:15:39.169 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-06-23T18:15:39.169 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-06-23T18:15:39.169 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000120558 s, 4.2 MB/s 2023-06-23T18:15:39.170 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T18:15:39.229 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2023-06-23T18:15:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T18:15:39.288 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:39.289 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 369248 Links: 1 2023-06-23T18:15:39.289 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:39.289 INFO:teuthology.orchestra.run.smithi017.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:39.289 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2023-06-23 18:15:24.747613958 +0000 2023-06-23T18:15:39.289 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2023-06-23 18:09:35.281593888 +0000 2023-06-23T18:15:39.289 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2023-06-23 18:09:35.281593888 +0000 2023-06-23T18:15:39.289 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2023-06-23T18:15:39.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T18:15:39.358 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2023-06-23T18:15:39.359 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2023-06-23T18:15:39.359 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000115439 s, 4.4 MB/s 2023-06-23T18:15:39.360 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T18:15:39.419 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2023-06-23T18:15:39.419 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2023-06-23T18:15:39.438 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-06-23T18:15:39.438 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2023-06-23T18:15:39.497 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-06-23T18:15:39.497 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 370846 Links: 1 2023-06-23T18:15:39.497 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-06-23 18:15:38.995594976 +0000 2023-06-23T18:15:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-06-23 18:09:17.234272125 +0000 2023-06-23T18:15:39.498 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-06-23 18:09:17.234272125 +0000 2023-06-23T18:15:39.498 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-06-23T18:15:39.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-06-23T18:15:39.566 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-06-23T18:15:39.566 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-06-23T18:15:39.566 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000214855 s, 2.4 MB/s 2023-06-23T18:15:39.567 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-06-23T18:15:39.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:39 smithi082 ceph-mon[97889]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:39.599 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:39 smithi082 podman[99774]: 2023-06-23T18:15:39.616 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2023-06-23T18:15:39.673 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-06-23T18:15:39.673 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:39.673 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 380058 Links: 1 2023-06-23T18:15:39.673 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:39.673 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:39.673 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-06-23 18:15:38.995594976 +0000 2023-06-23T18:15:39.674 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-06-23 18:09:17.768262109 +0000 2023-06-23T18:15:39.674 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-06-23 18:09:17.768262109 +0000 2023-06-23T18:15:39.674 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-06-23T18:15:39.674 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-06-23T18:15:39.738 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-06-23T18:15:39.739 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-06-23T18:15:39.739 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000219822 s, 2.3 MB/s 2023-06-23T18:15:39.740 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-06-23T18:15:39.797 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2023-06-23T18:15:39.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:39 smithi002 ceph-mon[95947]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:39.854 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-06-23T18:15:39.855 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:39.855 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 379817 Links: 1 2023-06-23T18:15:39.855 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:39.855 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:39.855 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-06-23 18:15:38.995594976 +0000 2023-06-23T18:15:39.855 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-06-23 18:09:18.287252374 +0000 2023-06-23T18:15:39.855 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-06-23 18:09:18.287252374 +0000 2023-06-23T18:15:39.856 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-06-23T18:15:39.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-06-23T18:15:39.908 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:39 smithi082 bash[99774]: 49520cddde6b51235614c48992bed55b6bafeef5f6eb5b235137ffdf8a335db5 2023-06-23T18:15:39.909 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:39 smithi082 ceph-mgr[99873]: -- 172.21.15.82:0/321155707 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eaef5e11e0 con 0x55eaef5b7000 2023-06-23T18:15:39.909 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:39 smithi082 systemd[1]: Started Ceph mgr.z for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:15:39.909 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:39.728+0000 7fa0be7a3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:15:39.909 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:39.811+0000 7fa0be7a3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:15:39.909 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:39.903+0000 7fa0be7a3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:15:39.921 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-06-23T18:15:39.921 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-06-23T18:15:39.921 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000209709 s, 2.4 MB/s 2023-06-23T18:15:39.922 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-06-23T18:15:39.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:39 smithi017 ceph-mon[97472]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:39.979 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2023-06-23T18:15:40.035 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-06-23T18:15:40.036 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-06-23T18:15:40.036 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 6h/6d Inode: 371005 Links: 1 2023-06-23T18:15:40.036 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-06-23T18:15:40.036 INFO:teuthology.orchestra.run.smithi082.stdout:Context: system_u:object_r:device_t:s0 2023-06-23T18:15:40.036 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2023-06-23 18:15:38.995594976 +0000 2023-06-23T18:15:40.036 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2023-06-23 18:09:18.798242789 +0000 2023-06-23T18:15:40.037 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2023-06-23 18:09:18.798242789 +0000 2023-06-23T18:15:40.037 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2023-06-23T18:15:40.037 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-06-23T18:15:40.102 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2023-06-23T18:15:40.102 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2023-06-23T18:15:40.102 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000229849 s, 2.2 MB/s 2023-06-23T18:15:40.103 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-06-23T18:15:40.161 INFO:tasks.cephadm:Deploying osd.0 on smithi002 with /dev/vg_nvme/lv_4... 2023-06-23T18:15:40.161 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-23T18:15:40.186 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:40.002+0000 7fa0be7a3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:15:40.731 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:15:40.804 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:40.181+0000 7fa0be7a3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:15:40.805 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:40.488+0000 7fa0be7a3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:15:40.805 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:40.576+0000 7fa0be7a3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:15:40.805 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:40.704+0000 7fa0be7a3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:15:40.805 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:40.805 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:40.805 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:40.806 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:40.806 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T18:15:40.806 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:40.806 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:40.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:40.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T18:15:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:41.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:41.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:41.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:41.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-06-23T18:15:41.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:41.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:41.098 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:40.799+0000 7fa0be7a3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:15:41.441 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:41 smithi017 systemd[1]: Starting Ceph mgr.y for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:15:41.599 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:41.217+0000 7fa0be7a3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:15:41.765 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:41 smithi017 ceph-mon[97472]: Deploying daemon mgr.y on smithi017 2023-06-23T18:15:41.765 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:41 smithi017 ceph-mon[97472]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:41.766 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:41 smithi017 podman[98942]: 2023-06-23T18:15:41.766 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:41 smithi017 bash[98942]: 57366a7a68ec2413683b0cf09526e70f0254c1a42691b377165e5dca093ce096 2023-06-23T18:15:41.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:41 smithi002 ceph-mon[95947]: Deploying daemon mgr.y on smithi017 2023-06-23T18:15:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:41 smithi002 ceph-mon[95947]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:41.962 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:41.787+0000 7fa0be7a3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T18:15:41.963 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:41 smithi082 ceph-mon[97889]: Deploying daemon mgr.y on smithi017 2023-06-23T18:15:41.963 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:41 smithi082 ceph-mon[97889]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:42.116 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:41 smithi017 ceph-mgr[98982]: -- 172.21.15.17:0/2662394276 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55573a7dd1e0 con 0x55573a79f000 2023-06-23T18:15:42.116 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:41 smithi017 ceph-mgr[98982]: -- 172.21.15.17:0/2662394276 <== mon.2 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55573a7c91e0 con 0x55573a79f000 2023-06-23T18:15:42.117 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:41 smithi017 systemd[1]: Started Ceph mgr.y for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:15:42.117 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:41 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:41.906+0000 7f61c670c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-06-23T18:15:42.117 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:42.002+0000 7f61c670c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-06-23T18:15:42.117 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:42.111+0000 7f61c670c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-06-23T18:15:42.348 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:41.957+0000 7fa0be7a3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:15:42.440 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:42.224+0000 7f61c670c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-06-23T18:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-23T18:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-23T18:15:42.560 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2023-06-23T18:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2023-06-23T18:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013568 s, 773 MB/s 2023-06-23T18:15:42.561 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2023-06-23T18:15:42.847 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:42.435+0000 7f61c670c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-06-23T18:15:42.992 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi002:vg_nvme/lv_4 2023-06-23T18:15:43.033 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:42.756+0000 7fa0be7a3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:15:43.033 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:42.849+0000 7fa0be7a3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:15:43.033 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.034 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.034 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.034 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.034 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:43.034 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:42 smithi082 ceph-mon[97889]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:43.116 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:42.842+0000 7f61c670c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-06-23T18:15:43.117 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:42.956+0000 7f61c670c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-06-23T18:15:43.118 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.118 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.119 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.119 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.119 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:43.119 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:42 smithi017 ceph-mon[97472]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:43.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:43.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:43.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:42 smithi002 ceph-mon[95947]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:43.327 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:43.025+0000 7fa0be7a3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:15:43.440 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:43 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:43.110+0000 7f61c670c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-06-23T18:15:43.441 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:43 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:43.238+0000 7f61c670c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-06-23T18:15:43.549 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:15:43.598 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:43.322+0000 7fa0be7a3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T18:15:43.599 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:43.556+0000 7fa0be7a3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:15:43.975 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:43.706+0000 7fa0be7a3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:15:44.190 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:43 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:43.778+0000 7f61c670c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-06-23T18:15:44.348 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:43.970+0000 7fa0be7a3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:15:44.747 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.748 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.748 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.748 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.748 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:44.748 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:44.749 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.749 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:44.749 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:44.749 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:44 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:44.940 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:44.475+0000 7f61c670c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-06-23T18:15:44.940 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:44.668+0000 7f61c670c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-06-23T18:15:44.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:44.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:44.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:44.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:44.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:44 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:44.982 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:44.661+0000 7fa0be7a3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:15:44.983 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:44.790+0000 7fa0be7a3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:15:44.983 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.983 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.983 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.983 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:44.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:44.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:44.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-06-23T18:15:44.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-06-23T18:15:44.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:45.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.284+0000 7fd8e759d700 1 -- 172.21.15.2:0/2878685547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e00fe910 msgr2=0x7fd8e00fed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:45.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.284+0000 7fd8e759d700 1 --2- 172.21.15.2:0/2878685547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e00fe910 0x7fd8e00fed20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009990 tx=0x7fd8dc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:15:45.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.284+0000 7fd8e759d700 1 -- 172.21.15.2:0/2878685547 shutdown_connections 2023-06-23T18:15:45.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.284+0000 7fd8e759d700 1 --2- 172.21.15.2:0/2878685547 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e01004a0 0x7fd8e0106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:45.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.284+0000 7fd8e759d700 1 --2- 172.21.15.2:0/2878685547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e00fe910 0x7fd8e00fed20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:45.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.284+0000 7fd8e759d700 1 --2- 172.21.15.2:0/2878685547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e00ffb10 0x7fd8e00fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:45.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.284+0000 7fd8e759d700 1 -- 172.21.15.2:0/2878685547 >> 172.21.15.2:0/2878685547 conn(0x7fd8e00f9f80 msgr2=0x7fd8e00fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:15:45.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.285+0000 7fd8e759d700 1 -- 172.21.15.2:0/2878685547 shutdown_connections 2023-06-23T18:15:45.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.285+0000 7fd8e759d700 1 -- 172.21.15.2:0/2878685547 wait complete. 2023-06-23T18:15:45.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.285+0000 7fd8e759d700 1 Processor -- start 2023-06-23T18:15:45.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.285+0000 7fd8e759d700 1 -- start start 2023-06-23T18:15:45.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e759d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e00fe910 0x7fd8e01944e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:45.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e759d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e00ffb10 0x7fd8e0194a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:45.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e759d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e01004a0 0x7fd8e019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:45.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e759d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e019c070 con 0x7fd8e01004a0 2023-06-23T18:15:45.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e759d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e019c1b0 con 0x7fd8e00ffb10 2023-06-23T18:15:45.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e759d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e019c340 con 0x7fd8e00fe910 2023-06-23T18:15:45.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e00fe910 0x7fd8e01944e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:45.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e6d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e01004a0 0x7fd8e019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:45.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e5d9a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e00ffb10 0x7fd8e0194a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:45.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e00fe910 0x7fd8e01944e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:51860/0 (socket says 172.21.15.2:51860) 2023-06-23T18:15:45.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 -- 172.21.15.2:0/1991709326 learned_addr learned my addr 172.21.15.2:0/1991709326 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:15:45.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e6d9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e01004a0 0x7fd8e019baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51526/0 (socket says 172.21.15.2:51526) 2023-06-23T18:15:45.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 -- 172.21.15.2:0/1991709326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e00ffb10 msgr2=0x7fd8e0194a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:45.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e00ffb10 0x7fd8e0194a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:45.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 -- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e01004a0 msgr2=0x7fd8e019baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:15:45.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e01004a0 0x7fd8e019baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:15:45.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 -- 172.21.15.2:0/1991709326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0095f0 con 0x7fd8e00fe910 2023-06-23T18:15:45.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e6d9c700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e01004a0 0x7fd8e019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:15:45.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e5d9a700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e00ffb10 0x7fd8e0194a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:15:45.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8e659b700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e00fe910 0x7fd8e01944e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc000c00 tx=0x7fd8dc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:45.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.286+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc01d070 con 0x7fd8e00fe910 2023-06-23T18:15:45.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.287+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8dc00bdb0 con 0x7fd8e00fe910 2023-06-23T18:15:45.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.287+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e00680d0 con 0x7fd8e00fe910 2023-06-23T18:15:45.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.287+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc022d70 con 0x7fd8e00fe910 2023-06-23T18:15:45.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.287+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e0068570 con 0x7fd8e00fe910 2023-06-23T18:15:45.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.287+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fd8dc00b920 con 0x7fd8e00fe910 2023-06-23T18:15:45.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.287+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e004ea20 con 0x7fd8e00fe910 2023-06-23T18:15:45.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.288+0000 7fd8d37fe700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8cc0389e0 0x7fd8cc03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:15:45.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.288+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd8dc04c9b0 con 0x7fd8e00fe910 2023-06-23T18:15:45.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.288+0000 7fd8e5d9a700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8cc0389e0 0x7fd8cc03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:15:45.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.288+0000 7fd8e5d9a700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8cc0389e0 0x7fd8cc03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4010420 tx=0x7fd8d40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:15:45.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.293+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd8dc00edb0 con 0x7fd8e00fe910 2023-06-23T18:15:45.307 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:44.977+0000 7fa0be7a3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:15:45.308 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:45.090+0000 7fa0be7a3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:15:45.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:45.500+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8e0068790 con 0x7fd8cc0389e0 2023-06-23T18:15:45.940 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:45.629+0000 7f61c670c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:15:45.941 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:45.743+0000 7f61c670c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-06-23T18:15:45.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: Reconfiguring mgr.x (unknown last config time)... 2023-06-23T18:15:45.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: Reconfiguring daemon mgr.x on smithi002 2023-06-23T18:15:45.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:45.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:15:45.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:15:45.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:45.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:45.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:45.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:45.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:45.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:45.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:46.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: Reconfiguring mgr.x (unknown last config time)... 2023-06-23T18:15:46.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: Reconfiguring daemon mgr.x on smithi002 2023-06-23T18:15:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:15:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:15:46.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:46.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:46.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:46.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:46.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:46.098 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:45.302+0000 7fa0be7a3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:15:46.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: Reconfiguring mgr.x (unknown last config time)... 2023-06-23T18:15:46.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: Reconfiguring daemon mgr.x on smithi002 2023-06-23T18:15:46.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:46.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:15:46.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:15:46.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:46.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:46.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:46.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:46.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:46.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:15:46.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:46.312 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:45 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:45.959+0000 7f61c670c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-06-23T18:15:46.565 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:46.307+0000 7f61c670c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-06-23T18:15:46.598 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:15:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z[99790]: 2023-06-23T18:15:46.151+0000 7fa0be7a3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:15:46.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:46.650+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mgrmap(e 14) v1 ==== 90598+0+0 (secure 0 0 0) 0x7fd8dc00f980 con 0x7fd8e00fe910 2023-06-23T18:15:46.940 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:46.560+0000 7f61c670c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-06-23T18:15:46.940 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:46.741+0000 7f61c670c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-06-23T18:15:46.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-mon[97472]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:46.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-mon[97472]: Standby manager daemon z started 2023-06-23T18:15:46.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-mon[97472]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-23T18:15:46.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-mon[97472]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T18:15:46.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-mon[97472]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-23T18:15:46.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:46 smithi017 ceph-mon[97472]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T18:15:46.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:46 smithi002 ceph-mon[95947]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:46.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:46 smithi002 ceph-mon[95947]: Standby manager daemon z started 2023-06-23T18:15:46.942 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:46 smithi002 ceph-mon[95947]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-23T18:15:46.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:46 smithi002 ceph-mon[95947]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T18:15:46.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:46 smithi002 ceph-mon[95947]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-23T18:15:46.943 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:46 smithi002 ceph-mon[95947]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T18:15:47.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:46 smithi082 ceph-mon[97889]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:15:47.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:46 smithi082 ceph-mon[97889]: Standby manager daemon z started 2023-06-23T18:15:47.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:46 smithi082 ceph-mon[97889]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-06-23T18:15:47.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:46 smithi082 ceph-mon[97889]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T18:15:47.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:46 smithi082 ceph-mon[97889]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-06-23T18:15:47.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:46 smithi082 ceph-mon[97889]: from='mgr.? 172.21.15.82:0/3719457969' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T18:15:47.440 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:47 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:47.068+0000 7f61c670c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-06-23T18:15:47.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:47 smithi017 ceph-mon[97472]: mgrmap e14: x(active, since 69s), standbys: z 2023-06-23T18:15:47.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-23T18:15:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:47 smithi017 ceph-mon[97472]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:47.941 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:47 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:47.886+0000 7f61c670c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-06-23T18:15:48.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:47 smithi002 ceph-mon[95947]: mgrmap e14: x(active, since 69s), standbys: z 2023-06-23T18:15:48.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-23T18:15:48.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:47 smithi002 ceph-mon[95947]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:48.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:48.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:47 smithi082 ceph-mon[97889]: mgrmap e14: x(active, since 69s), standbys: z 2023-06-23T18:15:48.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-06-23T18:15:48.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:47 smithi082 ceph-mon[97889]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:48.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:48.424 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:48.043+0000 7f61c670c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-06-23T18:15:48.425 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:48.282+0000 7f61c670c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-06-23T18:15:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/4027397833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d2907d6-0f38-4387-afe6-dda12616124d"}]: dispatch 2023-06-23T18:15:48.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/4027397833' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d2907d6-0f38-4387-afe6-dda12616124d"}]': finished 2023-06-23T18:15:48.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-mon[97472]: osdmap e5: 1 total, 0 up, 1 in 2023-06-23T18:15:48.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:15:48.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2113174785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:15:48.691 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:48.419+0000 7f61c670c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-06-23T18:15:49.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:48 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/4027397833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d2907d6-0f38-4387-afe6-dda12616124d"}]: dispatch 2023-06-23T18:15:49.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:48 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/4027397833' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d2907d6-0f38-4387-afe6-dda12616124d"}]': finished 2023-06-23T18:15:49.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:48 smithi002 ceph-mon[95947]: osdmap e5: 1 total, 0 up, 1 in 2023-06-23T18:15:49.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:15:49.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:48 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2113174785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:15:49.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:48 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/4027397833' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9d2907d6-0f38-4387-afe6-dda12616124d"}]: dispatch 2023-06-23T18:15:49.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:48 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/4027397833' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d2907d6-0f38-4387-afe6-dda12616124d"}]': finished 2023-06-23T18:15:49.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:48 smithi082 ceph-mon[97889]: osdmap e5: 1 total, 0 up, 1 in 2023-06-23T18:15:49.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:15:49.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:48 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2113174785' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:15:49.190 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:48 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:48.692+0000 7f61c670c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-06-23T18:15:49.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:49 smithi017 ceph-mon[97472]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:49.941 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:15:49 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y[98958]: 2023-06-23T18:15:49.725+0000 7f61c670c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-06-23T18:15:50.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:49 smithi002 ceph-mon[95947]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:50.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:49 smithi082 ceph-mon[97889]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:50.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:15:50.675+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mon.1 v2:172.21.15.82:3300/0 8 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7fd8dc02ccc0 con 0x7fd8e00fe910 2023-06-23T18:15:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:50 smithi017 ceph-mon[97472]: Standby manager daemon y started 2023-06-23T18:15:50.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:50 smithi017 ceph-mon[97472]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-23T18:15:50.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:50 smithi017 ceph-mon[97472]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T18:15:50.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:50 smithi017 ceph-mon[97472]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-23T18:15:50.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:50 smithi017 ceph-mon[97472]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T18:15:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:50 smithi002 ceph-mon[95947]: Standby manager daemon y started 2023-06-23T18:15:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:50 smithi002 ceph-mon[95947]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-23T18:15:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:50 smithi002 ceph-mon[95947]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T18:15:51.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:50 smithi002 ceph-mon[95947]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-23T18:15:51.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:50 smithi002 ceph-mon[95947]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T18:15:51.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:50 smithi082 ceph-mon[97889]: Standby manager daemon y started 2023-06-23T18:15:51.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:50 smithi082 ceph-mon[97889]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-06-23T18:15:51.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:50 smithi082 ceph-mon[97889]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-06-23T18:15:51.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:50 smithi082 ceph-mon[97889]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-06-23T18:15:51.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:50 smithi082 ceph-mon[97889]: from='mgr.? 172.21.15.17:0/988650486' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-06-23T18:15:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:51 smithi017 ceph-mon[97472]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:51 smithi017 ceph-mon[97472]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-06-23T18:15:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-23T18:15:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:51 smithi002 ceph-mon[95947]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:51 smithi002 ceph-mon[95947]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-06-23T18:15:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-23T18:15:52.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:51 smithi082 ceph-mon[97889]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:52.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:51 smithi082 ceph-mon[97889]: mgrmap e15: x(active, since 74s), standbys: z, y 2023-06-23T18:15:52.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-06-23T18:15:53.796 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:53 smithi002 ceph-mon[95947]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:53.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:53 smithi017 ceph-mon[97472]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:54.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:53 smithi082 ceph-mon[97889]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:55.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:54 smithi002 ceph-mon[95947]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:55.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-23T18:15:55.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:55.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:54 smithi082 ceph-mon[97889]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:55.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-23T18:15:55.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:55.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:54 smithi017 ceph-mon[97472]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-06-23T18:15:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:15:56.027 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:55 smithi002 ceph-mon[95947]: Deploying daemon osd.0 on smithi002 2023-06-23T18:15:56.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:55 smithi082 ceph-mon[97889]: Deploying daemon osd.0 on smithi002 2023-06-23T18:15:56.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:55 smithi017 ceph-mon[97472]: Deploying daemon osd.0 on smithi002 2023-06-23T18:15:57.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:56 smithi002 ceph-mon[95947]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:56 smithi082 ceph-mon[97889]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:57.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:56 smithi017 ceph-mon[97472]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:59.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:58 smithi002 ceph-mon[95947]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:58 smithi082 ceph-mon[97889]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:59.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:58 smithi017 ceph-mon[97472]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:15:59.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:15:59.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:15:59.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:15:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:00.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:00.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:00.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:15:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:00.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:00.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:00.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:15:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:01.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:00 smithi002 ceph-mon[95947]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:00 smithi082 ceph-mon[97889]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:00 smithi017 ceph-mon[97472]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.756+0000 7fd8d37fe700 1 -- 172.21.15.2:0/1991709326 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd8e0068790 con 0x7fd8cc0389e0 2023-06-23T18:16:02.762 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 0 on host 'smithi002' 2023-06-23T18:16:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8cc0389e0 msgr2=0x7fd8cc03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8cc0389e0 0x7fd8cc03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8d4010420 tx=0x7fd8d40073d0 comp rx=0 tx=0).stop 2023-06-23T18:16:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e00fe910 msgr2=0x7fd8e01944e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e00fe910 0x7fd8e01944e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc000c00 tx=0x7fd8dc004730 comp rx=0 tx=0).stop 2023-06-23T18:16:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 shutdown_connections 2023-06-23T18:16:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8e01004a0 0x7fd8e019baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8e00fe910 0x7fd8e01944e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8cc0389e0 0x7fd8cc03ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 --2- 172.21.15.2:0/1991709326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8e00ffb10 0x7fd8e0194a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 >> 172.21.15.2:0/1991709326 conn(0x7fd8e00f9f80 msgr2=0x7fd8e0103120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:16:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 shutdown_connections 2023-06-23T18:16:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:02.759+0000 7fd8e759d700 1 -- 172.21.15.2:0/1991709326 wait complete. 2023-06-23T18:16:03.004 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.004 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.005 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:03.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:03.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:03.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:03.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.195 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:03.280 DEBUG:teuthology.orchestra.run.smithi002:osd.0> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.0.service 2023-06-23T18:16:03.283 INFO:tasks.cephadm:Deploying osd.1 on smithi017 with /dev/vg_nvme/lv_4... 2023-06-23T18:16:03.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-23T18:16:03.314 INFO:journalctl@ceph.osd.0.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:16:03.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:03.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:03.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:03.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:03.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:03.551 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-0[108965]: 2023-06-23T18:16:03.466+0000 7f4f4f1b5640 -1 osd.0 0 log_to_monitors true 2023-06-23T18:16:03.855 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:16:04.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:03 smithi017 ceph-mon[97472]: from='osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T18:16:04.251 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:03 smithi017 ceph-mon[97472]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T18:16:04.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:03 smithi002 ceph-mon[95947]: from='osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T18:16:04.252 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:03 smithi002 ceph-mon[95947]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T18:16:04.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:03 smithi082 ceph-mon[97889]: from='osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T18:16:04.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:03 smithi082 ceph-mon[97889]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-06-23T18:16:05.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:04 smithi017 ceph-mon[97472]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-23T18:16:05.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:04 smithi017 ceph-mon[97472]: osdmap e6: 1 total, 0 up, 1 in 2023-06-23T18:16:05.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:04 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:05.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:04 smithi017 ceph-mon[97472]: from='osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:16:05.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:04 smithi017 ceph-mon[97472]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:16:05.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:04 smithi017 ceph-mon[97472]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:05.289 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-23T18:16:05.290 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-23T18:16:05.290 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2023-06-23T18:16:05.290 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2023-06-23T18:16:05.290 INFO:teuthology.orchestra.run.smithi017.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133187 s, 787 MB/s 2023-06-23T18:16:05.291 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 2023-06-23T18:16:05.291 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2023-06-23T18:16:05.301 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-0[108965]: 2023-06-23T18:16:04.912+0000 7f4f4482d700 -1 osd.0 0 waiting for initial osdmap 2023-06-23T18:16:05.302 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-0[108965]: 2023-06-23T18:16:04.916+0000 7f4f3f620700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:16:05.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:04 smithi002 ceph-mon[95947]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-23T18:16:05.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:04 smithi002 ceph-mon[95947]: osdmap e6: 1 total, 0 up, 1 in 2023-06-23T18:16:05.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:04 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:05.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:04 smithi002 ceph-mon[95947]: from='osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:16:05.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:04 smithi002 ceph-mon[95947]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:16:05.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:04 smithi002 ceph-mon[95947]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:05.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:04 smithi082 ceph-mon[97889]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-06-23T18:16:05.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:04 smithi082 ceph-mon[97889]: osdmap e6: 1 total, 0 up, 1 in 2023-06-23T18:16:05.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:04 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:05.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:04 smithi082 ceph-mon[97889]: from='osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:16:05.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:04 smithi082 ceph-mon[97889]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:16:05.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:04 smithi082 ceph-mon[97889]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-06-23T18:16:05.713 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_4 2023-06-23T18:16:06.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:16:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: osdmap e7: 1 total, 0 up, 1 in 2023-06-23T18:16:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:06.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:06.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:06.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:06.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:16:06.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: osdmap e7: 1 total, 0 up, 1 in 2023-06-23T18:16:06.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:06.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:06.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:06.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:06.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:06.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.273 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:16:06.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:16:06.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: osdmap e7: 1 total, 0 up, 1 in 2023-06-23T18:16:06.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:06.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:06.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:06.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:06.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:06.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:06.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:06 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:16:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:06 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:16:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:06 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi002 2023-06-23T18:16:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:06 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi002 to 17173M 2023-06-23T18:16:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:06 smithi017 ceph-mon[97472]: osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] boot 2023-06-23T18:16:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:06 smithi017 ceph-mon[97472]: osdmap e8: 1 total, 1 up, 1 in 2023-06-23T18:16:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:06 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:06 smithi017 ceph-mon[97472]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:07.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:06 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:16:07.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:06 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:16:07.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:06 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi002 2023-06-23T18:16:07.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:06 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi002 to 17173M 2023-06-23T18:16:07.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:06 smithi002 ceph-mon[95947]: osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] boot 2023-06-23T18:16:07.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:06 smithi002 ceph-mon[95947]: osdmap e8: 1 total, 1 up, 1 in 2023-06-23T18:16:07.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:06 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:07.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:06 smithi002 ceph-mon[95947]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:07.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:06 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:16:07.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:06 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:16:07.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:06 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi002 2023-06-23T18:16:07.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:06 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi002 to 17173M 2023-06-23T18:16:07.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:06 smithi082 ceph-mon[97889]: osd.0 [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] boot 2023-06-23T18:16:07.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:06 smithi082 ceph-mon[97889]: osdmap e8: 1 total, 1 up, 1 in 2023-06-23T18:16:07.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:06 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-06-23T18:16:07.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:06 smithi082 ceph-mon[97889]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:07.502 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 -- 172.21.15.17:0/334908376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21341005d0 msgr2=0x7f2134106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:07.503 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 --2- 172.21.15.17:0/334908376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21341005d0 0x7f2134106e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f212c00b0a0 tx=0x7f212c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:16:07.503 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 -- 172.21.15.17:0/334908376 shutdown_connections 2023-06-23T18:16:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 --2- 172.21.15.17:0/334908376 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21341005d0 0x7f2134106e90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 --2- 172.21.15.17:0/334908376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21340fea40 0x7f21340fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 --2- 172.21.15.17:0/334908376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21340ffc40 0x7f2134100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 -- 172.21.15.17:0/334908376 >> 172.21.15.17:0/334908376 conn(0x7f21340fa070 msgr2=0x7f21340fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:16:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 -- 172.21.15.17:0/334908376 shutdown_connections 2023-06-23T18:16:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 -- 172.21.15.17:0/334908376 wait complete. 2023-06-23T18:16:07.504 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.496+0000 7f213a9a8700 1 Processor -- start 2023-06-23T18:16:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f213a9a8700 1 -- start start 2023-06-23T18:16:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f213a9a8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21340fea40 0x7f21341958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f213a9a8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21340ffc40 0x7f2134195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f213a9a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21341005d0 0x7f213419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f213a9a8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213419d440 con 0x7f21341005d0 2023-06-23T18:16:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f213a9a8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213419d580 con 0x7f21340ffc40 2023-06-23T18:16:07.505 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f213a9a8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213419d710 con 0x7f21340fea40 2023-06-23T18:16:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f21337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21340ffc40 0x7f2134195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f21337fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21340ffc40 0x7f2134195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33012/0 (socket says 172.21.15.17:33012) 2023-06-23T18:16:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f21337fe700 1 -- 172.21.15.17:0/3955846753 learned_addr learned my addr 172.21.15.17:0/3955846753 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-06-23T18:16:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f2138f45700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21341005d0 0x7f213419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f2133fff700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21340fea40 0x7f21341958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f21337fe700 1 -- 172.21.15.17:0/3955846753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21340fea40 msgr2=0x7f21341958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f21337fe700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21340fea40 0x7f21341958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:07.506 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f21337fe700 1 -- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21341005d0 msgr2=0x7f213419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:07.507 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f21337fe700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21341005d0 0x7f213419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:07.507 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.497+0000 7f21337fe700 1 -- 172.21.15.17:0/3955846753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c009d00 con 0x7f21340ffc40 2023-06-23T18:16:07.507 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.498+0000 7f21337fe700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21340ffc40 0x7f2134195df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f212400c970 tx=0x7f212400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:16:07.508 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.498+0000 7f21317fa700 1 -- 172.21.15.17:0/3955846753 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124007aa0 con 0x7f21340ffc40 2023-06-23T18:16:07.508 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.498+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213419d9f0 con 0x7f21340ffc40 2023-06-23T18:16:07.509 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.498+0000 7f21317fa700 1 -- 172.21.15.17:0/3955846753 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2124004cc0 con 0x7f21340ffc40 2023-06-23T18:16:07.509 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.498+0000 7f21317fa700 1 -- 172.21.15.17:0/3955846753 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124005440 con 0x7f21340ffc40 2023-06-23T18:16:07.509 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.498+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213419dea0 con 0x7f21340ffc40 2023-06-23T18:16:07.509 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.499+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213410a130 con 0x7f21340ffc40 2023-06-23T18:16:07.509 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.500+0000 7f21317fa700 1 -- 172.21.15.17:0/3955846753 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f21240055a0 con 0x7f21340ffc40 2023-06-23T18:16:07.509 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.501+0000 7f21317fa700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f21200a10c0 0x7f21200a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:07.509 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.501+0000 7f21317fa700 1 -- 172.21.15.17:0/3955846753 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f2124013070 con 0x7f21340ffc40 2023-06-23T18:16:07.510 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.501+0000 7f2133fff700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f21200a10c0 0x7f21200a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:07.510 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.501+0000 7f2133fff700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f21200a10c0 0x7f21200a3570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f211c005fd0 tx=0x7f211c005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:16:07.510 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.504+0000 7f21317fa700 1 -- 172.21.15.17:0/3955846753 <== mon.2 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f21240961c0 con 0x7f21340ffc40 2023-06-23T18:16:07.718 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:07.713+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f21340686c0 con 0x7f21200a10c0 2023-06-23T18:16:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:07 smithi017 ceph-mon[97472]: osdmap e9: 1 total, 1 up, 1 in 2023-06-23T18:16:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:08.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:08.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:08.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:07 smithi002 ceph-mon[95947]: osdmap e9: 1 total, 1 up, 1 in 2023-06-23T18:16:08.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:08.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:08.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:08.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:07 smithi082 ceph-mon[97889]: osdmap e9: 1 total, 1 up, 1 in 2023-06-23T18:16:08.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:08.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:08.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:08 smithi017 ceph-mon[97472]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:09.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:08 smithi017 ceph-mon[97472]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:09.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:08 smithi002 ceph-mon[95947]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:09.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:08 smithi002 ceph-mon[95947]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:09.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:08 smithi082 ceph-mon[97889]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:09.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:08 smithi082 ceph-mon[97889]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:10.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:10 smithi017 ceph-mon[97472]: from='client.? 172.21.15.17:0/3016657042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]: dispatch 2023-06-23T18:16:10.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:10 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]: dispatch 2023-06-23T18:16:10.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:10 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]': finished 2023-06-23T18:16:10.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:10 smithi017 ceph-mon[97472]: osdmap e10: 2 total, 1 up, 2 in 2023-06-23T18:16:10.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:10 smithi002 ceph-mon[95947]: from='client.? 172.21.15.17:0/3016657042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]: dispatch 2023-06-23T18:16:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:10 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]: dispatch 2023-06-23T18:16:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:10 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]': finished 2023-06-23T18:16:10.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:10 smithi002 ceph-mon[95947]: osdmap e10: 2 total, 1 up, 2 in 2023-06-23T18:16:10.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:10.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:10 smithi082 ceph-mon[97889]: from='client.? 172.21.15.17:0/3016657042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]: dispatch 2023-06-23T18:16:10.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:10 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]: dispatch 2023-06-23T18:16:10.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:10 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b3d5d141-29f5-4a3e-b854-4ca05bf85d90"}]': finished 2023-06-23T18:16:10.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:10 smithi082 ceph-mon[97889]: osdmap e10: 2 total, 1 up, 2 in 2023-06-23T18:16:10.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:11.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:11 smithi002 ceph-mon[95947]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:11.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:11 smithi002 ceph-mon[95947]: from='client.? 172.21.15.17:0/1117644047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:11.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:11 smithi082 ceph-mon[97889]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:11.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:11 smithi082 ceph-mon[97889]: from='client.? 172.21.15.17:0/1117644047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:11.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:11 smithi017 ceph-mon[97472]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:11.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:11 smithi017 ceph-mon[97472]: from='client.? 172.21.15.17:0/1117644047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:13.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:12 smithi002 ceph-mon[95947]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:12 smithi082 ceph-mon[97889]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:13.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:12 smithi017 ceph-mon[97472]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:15.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:14 smithi002 ceph-mon[95947]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:15.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:14 smithi082 ceph-mon[97889]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:15.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:14 smithi017 ceph-mon[97472]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:16.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:16 smithi017 ceph-mon[97472]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:17.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:16 smithi002 ceph-mon[95947]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:16 smithi082 ceph-mon[97889]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:18.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-23T18:16:18.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:18.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:17 smithi002 ceph-mon[95947]: Deploying daemon osd.1 on smithi017 2023-06-23T18:16:18.069 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-23T18:16:18.070 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:18.070 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:17 smithi017 ceph-mon[97472]: Deploying daemon osd.1 on smithi017 2023-06-23T18:16:18.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-06-23T18:16:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:17 smithi082 ceph-mon[97889]: Deploying daemon osd.1 on smithi017 2023-06-23T18:16:19.041 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:18 smithi017 ceph-mon[97472]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:18 smithi002 ceph-mon[95947]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:18 smithi082 ceph-mon[97889]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:21.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:20 smithi002 ceph-mon[95947]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:20 smithi082 ceph-mon[97889]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:21.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:20 smithi017 ceph-mon[97472]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:22.816 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:22.817 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:22.817 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:23.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:23.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:23.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:23.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:23 smithi017 ceph-mon[97472]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:24.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:23 smithi002 ceph-mon[95947]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:24.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:23 smithi082 ceph-mon[97889]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:24.389 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.385+0000 7f21317fa700 1 -- 172.21.15.17:0/3955846753 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f21340686c0 con 0x7f21200a10c0 2023-06-23T18:16:24.390 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 1 on host 'smithi017' 2023-06-23T18:16:24.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f21200a10c0 msgr2=0x7f21200a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:24.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f21200a10c0 0x7f21200a3570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f211c005fd0 tx=0x7f211c005dd0 comp rx=0 tx=0).stop 2023-06-23T18:16:24.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21340ffc40 msgr2=0x7f2134195df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:24.392 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21340ffc40 0x7f2134195df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f212400c970 tx=0x7f212400cc80 comp rx=0 tx=0).stop 2023-06-23T18:16:24.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 shutdown_connections 2023-06-23T18:16:24.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21341005d0 0x7f213419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:24.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21340fea40 0x7f21341958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:24.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f21200a10c0 0x7f21200a3570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:24.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 --2- 172.21.15.17:0/3955846753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21340ffc40 0x7f2134195df0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:24.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 >> 172.21.15.17:0/3955846753 conn(0x7f21340fa070 msgr2=0x7f2134067fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:16:24.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 shutdown_connections 2023-06-23T18:16:24.393 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:16:24.387+0000 7f213a9a8700 1 -- 172.21.15.17:0/3955846753 wait complete. 2023-06-23T18:16:24.754 DEBUG:teuthology.orchestra.run.smithi017:osd.1> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.1.service 2023-06-23T18:16:24.757 INFO:tasks.cephadm:Deploying osd.2 on smithi082 with /dev/vg_nvme/lv_4... 2023-06-23T18:16:24.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-06-23T18:16:24.787 INFO:journalctl@ceph.osd.1.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:16:25.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:25.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:25.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:25.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:24 smithi002 ceph-mon[95947]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:25.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:25.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:25.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:24 smithi082 ceph-mon[97889]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:25.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:25.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:25.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:25.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:25.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:24 smithi017 ceph-mon[97472]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:25.318 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:16:25.940 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-1[102600]: 2023-06-23T18:16:25.657+0000 7f02d2ae6640 -1 osd.1 0 log_to_monitors true 2023-06-23T18:16:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:26 smithi082 ceph-mon[97889]: from='osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T18:16:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:26 smithi082 ceph-mon[97889]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T18:16:26.688 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:26 smithi017 ceph-mon[97472]: from='osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T18:16:26.688 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:26 smithi017 ceph-mon[97472]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T18:16:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:26 smithi002 ceph-mon[95947]: from='osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T18:16:26.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:26 smithi002 ceph-mon[95947]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-06-23T18:16:26.817 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-06-23T18:16:26.817 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-06-23T18:16:26.817 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-06-23T18:16:26.817 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-06-23T18:16:26.817 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0206678 s, 507 MB/s 2023-06-23T18:16:26.818 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-06-23T18:16:27.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_4 2023-06-23T18:16:27.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-23T18:16:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: osdmap e11: 2 total, 1 up, 2 in 2023-06-23T18:16:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:27.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:16:27.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:16:27.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:27.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:27.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:27.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:27.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.690 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-1[102600]: 2023-06-23T18:16:27.375+0000 7f02c815e700 -1 osd.1 0 waiting for initial osdmap 2023-06-23T18:16:27.691 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-1[102600]: 2023-06-23T18:16:27.379+0000 7f02c2750700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:16:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-23T18:16:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: osdmap e11: 2 total, 1 up, 2 in 2023-06-23T18:16:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:16:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:16:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:27.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:27.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:16:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-06-23T18:16:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: osdmap e11: 2 total, 1 up, 2 in 2023-06-23T18:16:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:16:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:16:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-06-23T18:16:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:28.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:28 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi017 2023-06-23T18:16:28.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:28 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi017 to 17173M 2023-06-23T18:16:28.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:28 smithi082 ceph-mon[97889]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:16:28.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:28 smithi082 ceph-mon[97889]: osdmap e12: 2 total, 1 up, 2 in 2023-06-23T18:16:28.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:28 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:28.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:28 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:28 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi017 2023-06-23T18:16:28.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:28 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi017 to 17173M 2023-06-23T18:16:28.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:28 smithi017 ceph-mon[97472]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:16:28.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:28 smithi017 ceph-mon[97472]: osdmap e12: 2 total, 1 up, 2 in 2023-06-23T18:16:28.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:28 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:28.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:28 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:28.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:28 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi017 2023-06-23T18:16:28.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:28 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi017 to 17173M 2023-06-23T18:16:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:28 smithi002 ceph-mon[95947]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:16:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:28 smithi002 ceph-mon[95947]: osdmap e12: 2 total, 1 up, 2 in 2023-06-23T18:16:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:28 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:28 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:28.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3275523854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf40ffc00 msgr2=0x7fbcf4100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:28.917 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3275523854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf40ffc00 0x7fbcf4100050 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc009990 tx=0x7fbcdc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:16:28.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3275523854 shutdown_connections 2023-06-23T18:16:28.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3275523854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcf4100590 0x7fbcf4106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:28.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3275523854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf40fea00 0x7fbcf40fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:28.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3275523854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf40ffc00 0x7fbcf4100050 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:28.918 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3275523854 >> 172.21.15.82:0/3275523854 conn(0x7fbcf40fa050 msgr2=0x7fbcf40fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:16:28.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3275523854 shutdown_connections 2023-06-23T18:16:28.919 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.912+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3275523854 wait complete. 2023-06-23T18:16:28.921 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcfa48c700 1 Processor -- start 2023-06-23T18:16:28.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcfa48c700 1 -- start start 2023-06-23T18:16:28.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcfa48c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf40fea00 0x7fbcf4068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:28.922 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcfa48c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcf40ffc00 0x7fbcf419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:28.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcfa48c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf4100590 0x7fbcf419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:28.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcfa48c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419d3b0 con 0x7fbcf40ffc00 2023-06-23T18:16:28.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcfa48c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419d520 con 0x7fbcf40fea00 2023-06-23T18:16:28.923 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcfa48c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf419d6e0 con 0x7fbcf4100590 2023-06-23T18:16:28.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf4100590 0x7fbcf419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:28.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf4100590 0x7fbcf419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40536/0 (socket says 172.21.15.82:40536) 2023-06-23T18:16:28.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 -- 172.21.15.82:0/3245070299 learned_addr learned my addr 172.21.15.82:0/3245070299 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:16:28.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf3fff700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf40fea00 0x7fbcf4068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:28.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcebfff700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcf40ffc00 0x7fbcf419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:28.924 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 -- 172.21.15.82:0/3245070299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf40fea00 msgr2=0x7fbcf4068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:28.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf40fea00 0x7fbcf4068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:28.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 -- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcf40ffc00 msgr2=0x7fbcf419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:28.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcf40ffc00 0x7fbcf419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:28.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 -- 172.21.15.82:0/3245070299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc0095f0 con 0x7fbcf4100590 2023-06-23T18:16:28.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf8a29700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf4100590 0x7fbcf419cc10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00b970 tx=0x7fbcec00bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:16:28.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf1ffb700 1 -- 172.21.15.82:0/3245070299 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00c9b0 con 0x7fbcf4100590 2023-06-23T18:16:28.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.913+0000 7fbcf1ffb700 1 -- 172.21.15.82:0/3245070299 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcec014440 con 0x7fbcf4100590 2023-06-23T18:16:28.925 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.914+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf419d9c0 con 0x7fbcf4100590 2023-06-23T18:16:28.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.914+0000 7fbcf1ffb700 1 -- 172.21.15.82:0/3245070299 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec0075d0 con 0x7fbcf4100590 2023-06-23T18:16:28.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.914+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf419df40 con 0x7fbcf4100590 2023-06-23T18:16:28.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.914+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf410a190 con 0x7fbcf4100590 2023-06-23T18:16:28.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.915+0000 7fbcf1ffb700 1 -- 172.21.15.82:0/3245070299 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7fbcec0145b0 con 0x7fbcf4100590 2023-06-23T18:16:28.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.916+0000 7fbcf1ffb700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbce00a1140 0x7fbce00a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:28.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.916+0000 7fbcf1ffb700 1 -- 172.21.15.82:0/3245070299 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fbcec006e40 con 0x7fbcf4100590 2023-06-23T18:16:28.926 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.916+0000 7fbcf3fff700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbce00a1140 0x7fbce00a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:28.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.917+0000 7fbcf3fff700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbce00a1140 0x7fbce00a35f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbce4006fd0 tx=0x7fbce4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:16:28.927 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:28.921+0000 7fbcf1ffb700 1 -- 172.21.15.82:0/3245070299 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbcec095860 con 0x7fbcf4100590 2023-06-23T18:16:29.118 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:29.113+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcf4060910 con 0x7fbce00a1140 2023-06-23T18:16:29.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:16:29.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:16:29.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] boot 2023-06-23T18:16:29.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: osdmap e13: 2 total, 2 up, 2 in 2023-06-23T18:16:29.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:29.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:29.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:29.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:29.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:29.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:16:29.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:16:29.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] boot 2023-06-23T18:16:29.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: osdmap e13: 2 total, 2 up, 2 in 2023-06-23T18:16:29.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:29.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:29.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:16:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:16:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: osd.1 [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] boot 2023-06-23T18:16:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: osdmap e13: 2 total, 2 up, 2 in 2023-06-23T18:16:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-06-23T18:16:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:30 smithi082 ceph-mon[97889]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:30.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:30 smithi082 ceph-mon[97889]: osdmap e14: 2 total, 2 up, 2 in 2023-06-23T18:16:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:30 smithi017 ceph-mon[97472]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:30.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:30 smithi017 ceph-mon[97472]: osdmap e14: 2 total, 2 up, 2 in 2023-06-23T18:16:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:30 smithi002 ceph-mon[95947]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:30 smithi002 ceph-mon[95947]: osdmap e14: 2 total, 2 up, 2 in 2023-06-23T18:16:31.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:31 smithi017 ceph-mon[97472]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:31.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:31 smithi002 ceph-mon[95947]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:31.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:31 smithi082 ceph-mon[97889]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:32 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/2168496528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]: dispatch 2023-06-23T18:16:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:32 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]: dispatch 2023-06-23T18:16:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:32 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]': finished 2023-06-23T18:16:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:32 smithi017 ceph-mon[97472]: osdmap e15: 3 total, 2 up, 3 in 2023-06-23T18:16:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:32 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/1314202921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:32 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/2168496528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]: dispatch 2023-06-23T18:16:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:32 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]: dispatch 2023-06-23T18:16:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:32 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]': finished 2023-06-23T18:16:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:32 smithi002 ceph-mon[95947]: osdmap e15: 3 total, 2 up, 3 in 2023-06-23T18:16:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:32 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/1314202921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:32 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/2168496528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]: dispatch 2023-06-23T18:16:32.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:32 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]: dispatch 2023-06-23T18:16:32.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:32 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c62ee3f3-654e-4a1f-b47c-6dd9b9585db0"}]': finished 2023-06-23T18:16:32.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:32 smithi082 ceph-mon[97889]: osdmap e15: 3 total, 2 up, 3 in 2023-06-23T18:16:32.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:32.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:32 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/1314202921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:33.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:33 smithi017 ceph-mon[97472]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:33.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:33 smithi002 ceph-mon[95947]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:33.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:33 smithi082 ceph-mon[97889]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:34 smithi002 ceph-mon[95947]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:35.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:34 smithi082 ceph-mon[97889]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:34 smithi017 ceph-mon[97472]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:36.831 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:36 smithi082 ceph-mon[97889]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:37.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:36 smithi002 ceph-mon[95947]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:37.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:36 smithi017 ceph-mon[97472]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:38.102 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-23T18:16:38.102 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:38.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-23T18:16:38.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-06-23T18:16:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:39.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:38 smithi082 ceph-mon[97889]: Deploying daemon osd.2 on smithi082 2023-06-23T18:16:39.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:38 smithi082 ceph-mon[97889]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:38 smithi017 ceph-mon[97472]: Deploying daemon osd.2 on smithi082 2023-06-23T18:16:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:38 smithi017 ceph-mon[97472]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:38 smithi002 ceph-mon[95947]: Deploying daemon osd.2 on smithi082 2023-06-23T18:16:39.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:38 smithi002 ceph-mon[95947]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:40 smithi002 ceph-mon[95947]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:41.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:40 smithi082 ceph-mon[97889]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:40 smithi017 ceph-mon[97472]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:43.417 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:43.417 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:43.417 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:43.417 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:43 smithi082 ceph-mon[97889]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:43.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:43.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:43.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:43.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:43 smithi002 ceph-mon[95947]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:43.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:43.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:43.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:43.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:43 smithi017 ceph-mon[97472]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:45.031 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.026+0000 7fbcf1ffb700 1 -- 172.21.15.82:0/3245070299 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fbcf4060910 con 0x7fbce00a1140 2023-06-23T18:16:45.032 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 2 on host 'smithi082' 2023-06-23T18:16:45.033 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbce00a1140 msgr2=0x7fbce00a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:45.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbce00a1140 0x7fbce00a35f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbce4006fd0 tx=0x7fbce4008040 comp rx=0 tx=0).stop 2023-06-23T18:16:45.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf4100590 msgr2=0x7fbcf419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:45.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf4100590 0x7fbcf419cc10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00b970 tx=0x7fbcec00bd30 comp rx=0 tx=0).stop 2023-06-23T18:16:45.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 shutdown_connections 2023-06-23T18:16:45.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcf4100590 0x7fbcf419cc10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:45.034 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbce00a1140 0x7fbce00a35f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf40fea00 0x7fbcf4068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 --2- 172.21.15.82:0/3245070299 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbcf40ffc00 0x7fbcf419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 >> 172.21.15.82:0/3245070299 conn(0x7fbcf40fa050 msgr2=0x7fbcf40fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:16:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 shutdown_connections 2023-06-23T18:16:45.035 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:16:45.028+0000 7fbcfa48c700 1 -- 172.21.15.82:0/3245070299 wait complete. 2023-06-23T18:16:45.492 DEBUG:teuthology.orchestra.run.smithi082:osd.2> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.2.service 2023-06-23T18:16:45.496 INFO:tasks.cephadm:Deploying osd.3 on smithi002 with /dev/vg_nvme/lv_3... 2023-06-23T18:16:45.497 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-23T18:16:45.525 INFO:journalctl@ceph.osd.2.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:16:45.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:45.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:45.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:45.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:45.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:45.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:45.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:16:46.056 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:16:46.548 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:46 smithi082 ceph-mon[97889]: from='osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T18:16:46.549 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:46 smithi082 ceph-mon[97889]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T18:16:46.549 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-2[103784]: 2023-06-23T18:16:46.198+0000 7f178e201640 -1 osd.2 0 log_to_monitors true 2023-06-23T18:16:46.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:46 smithi002 ceph-mon[95947]: from='osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T18:16:46.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:46 smithi002 ceph-mon[95947]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T18:16:46.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:46 smithi017 ceph-mon[97472]: from='osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T18:16:46.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:46 smithi017 ceph-mon[97472]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-06-23T18:16:47.619 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-23T18:16:47.619 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-23T18:16:47.619 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2023-06-23T18:16:47.619 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2023-06-23T18:16:47.619 INFO:teuthology.orchestra.run.smithi002.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0081225 s, 1.3 GB/s 2023-06-23T18:16:47.619 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2023-06-23T18:16:47.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:47 smithi002 ceph-mon[95947]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-23T18:16:47.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:47 smithi002 ceph-mon[95947]: osdmap e16: 3 total, 2 up, 3 in 2023-06-23T18:16:47.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:47.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:47 smithi002 ceph-mon[95947]: from='osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:16:47.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:47 smithi002 ceph-mon[95947]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:16:47.756 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:47 smithi002 ceph-mon[95947]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:47.819 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-2[103784]: 2023-06-23T18:16:47.455+0000 7f1783879700 -1 osd.2 0 waiting for initial osdmap 2023-06-23T18:16:47.820 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-2[103784]: 2023-06-23T18:16:47.459+0000 7f177d66a700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:16:47.820 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:47 smithi082 ceph-mon[97889]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-23T18:16:47.821 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:47 smithi082 ceph-mon[97889]: osdmap e16: 3 total, 2 up, 3 in 2023-06-23T18:16:47.822 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:47.822 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:47 smithi082 ceph-mon[97889]: from='osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:16:47.822 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:47 smithi082 ceph-mon[97889]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:16:47.823 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:47 smithi082 ceph-mon[97889]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:47 smithi017 ceph-mon[97472]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-06-23T18:16:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:47 smithi017 ceph-mon[97472]: osdmap e16: 3 total, 2 up, 3 in 2023-06-23T18:16:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:47 smithi017 ceph-mon[97472]: from='osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:16:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:47 smithi017 ceph-mon[97472]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:16:47.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:47 smithi017 ceph-mon[97472]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:48.006 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi002:vg_nvme/lv_3 2023-06-23T18:16:48.563 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:16:48.753 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:16:48.753 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: osdmap e17: 3 total, 2 up, 3 in 2023-06-23T18:16:48.753 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:48.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:48.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:48.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:48.754 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:48.755 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:16:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: osdmap e17: 3 total, 2 up, 3 in 2023-06-23T18:16:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:16:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: osdmap e17: 3 total, 2 up, 3 in 2023-06-23T18:16:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:48.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:16:48.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:48.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:48.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:16:48.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:49.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:16:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:16:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi082 2023-06-23T18:16:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi082 to 17168M 2023-06-23T18:16:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] boot 2023-06-23T18:16:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: osdmap e18: 3 total, 3 up, 3 in 2023-06-23T18:16:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:49 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-06-23T18:16:49.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 -- 172.21.15.2:0/908103680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840fea40 msgr2=0x7f10840fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:49.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 --2- 172.21.15.2:0/908103680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840fea40 0x7f10840fee50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1074009990 tx=0x7f1074009ca0 comp rx=0 tx=0).stop 2023-06-23T18:16:49.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 -- 172.21.15.2:0/908103680 shutdown_connections 2023-06-23T18:16:49.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 --2- 172.21.15.2:0/908103680 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10841005d0 0x7f1084106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:49.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 --2- 172.21.15.2:0/908103680 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840fea40 0x7f10840fee50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:49.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 --2- 172.21.15.2:0/908103680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10840ffc40 0x7f1084100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:49.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 -- 172.21.15.2:0/908103680 >> 172.21.15.2:0/908103680 conn(0x7f10840fa070 msgr2=0x7f10840fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:16:49.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 -- 172.21.15.2:0/908103680 shutdown_connections 2023-06-23T18:16:49.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.805+0000 7f1088814700 1 -- 172.21.15.2:0/908103680 wait complete. 2023-06-23T18:16:49.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1088814700 1 Processor -- start 2023-06-23T18:16:49.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1088814700 1 -- start start 2023-06-23T18:16:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1088814700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10840fea40 0x7f10841946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1088814700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840ffc40 0x7f1084194c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1088814700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10841005d0 0x7f108419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1088814700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108419c270 con 0x7f10840ffc40 2023-06-23T18:16:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1088814700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108419c3b0 con 0x7f10841005d0 2023-06-23T18:16:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1088814700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108419c540 con 0x7f10840fea40 2023-06-23T18:16:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f107bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840ffc40 0x7f1084194c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:49.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f107bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840ffc40 0x7f1084194c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51596/0 (socket says 172.21.15.2:51596) 2023-06-23T18:16:49.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f107bfff700 1 -- 172.21.15.2:0/4203248341 learned_addr learned my addr 172.21.15.2:0/4203248341 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:16:49.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f108259c700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10840fea40 0x7f10841946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:49.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.806+0000 7f1082d9d700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10841005d0 0x7f108419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:49.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107bfff700 1 -- 172.21.15.2:0/4203248341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10840fea40 msgr2=0x7f10841946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:49.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107bfff700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10840fea40 0x7f10841946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:49.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107bfff700 1 -- 172.21.15.2:0/4203248341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10841005d0 msgr2=0x7f108419bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:16:49.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107bfff700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10841005d0 0x7f108419bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:16:49.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107bfff700 1 -- 172.21.15.2:0/4203248341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10740095f0 con 0x7f10840ffc40 2023-06-23T18:16:49.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107bfff700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840ffc40 0x7f1084194c20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d9b0 tx=0x7f106c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:16:49.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107b7fe700 1 -- 172.21.15.2:0/4203248341 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00bb60 con 0x7f10840ffc40 2023-06-23T18:16:49.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107b7fe700 1 -- 172.21.15.2:0/4203248341 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f106c007e80 con 0x7f10840ffc40 2023-06-23T18:16:49.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f108419c820 con 0x7f10840ffc40 2023-06-23T18:16:49.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f107b7fe700 1 -- 172.21.15.2:0/4203248341 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c004cf0 con 0x7f10840ffc40 2023-06-23T18:16:49.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.807+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f108419cd20 con 0x7f10840ffc40 2023-06-23T18:16:49.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.808+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1084106150 con 0x7f10840ffc40 2023-06-23T18:16:49.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.809+0000 7f107b7fe700 1 -- 172.21.15.2:0/4203248341 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f106c004020 con 0x7f10840ffc40 2023-06-23T18:16:49.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.810+0000 7f107b7fe700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f10700a10c0 0x7f10700a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:16:49.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.810+0000 7f108259c700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f10700a10c0 0x7f10700a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:16:49.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.810+0000 7f107b7fe700 1 -- 172.21.15.2:0/4203248341 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f106c0cb920 con 0x7f10840ffc40 2023-06-23T18:16:49.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.810+0000 7f108259c700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f10700a10c0 0x7f10700a3570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f107400bfd0 tx=0x7f107400bf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:16:49.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:49.814+0000 7f107b7fe700 1 -- 172.21.15.2:0/4203248341 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f106c096490 con 0x7f10840ffc40 2023-06-23T18:16:49.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:16:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:16:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi082 2023-06-23T18:16:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi082 to 17168M 2023-06-23T18:16:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] boot 2023-06-23T18:16:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: osdmap e18: 3 total, 3 up, 3 in 2023-06-23T18:16:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:49.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:49 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-06-23T18:16:49.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:16:49.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:16:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi082 2023-06-23T18:16:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi082 to 17168M 2023-06-23T18:16:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: osd.2 [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] boot 2023-06-23T18:16:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: osdmap e18: 3 total, 3 up, 3 in 2023-06-23T18:16:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-06-23T18:16:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-06-23T18:16:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:49 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-06-23T18:16:50.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:50.019+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1084060910 con 0x7f10700a10c0 2023-06-23T18:16:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-06-23T18:16:50.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:50 smithi002 ceph-mon[95947]: osdmap e19: 3 total, 3 up, 3 in 2023-06-23T18:16:50.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-23T18:16:50.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:50.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:50.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:50.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-06-23T18:16:50.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:50 smithi082 ceph-mon[97889]: osdmap e19: 3 total, 3 up, 3 in 2023-06-23T18:16:50.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-23T18:16:50.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:50.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:50.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:50.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-06-23T18:16:50.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:50 smithi017 ceph-mon[97472]: osdmap e19: 3 total, 3 up, 3 in 2023-06-23T18:16:50.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-06-23T18:16:50.946 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:16:50.946 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:16:50.947 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:16:51.480 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:51 smithi002 sudo[112141]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T18:16:51.480 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:51 smithi002 sudo[112141]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:51.480 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:51 smithi002 sudo[112141]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:51.480 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:51 smithi002 sudo[112141]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:51.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:16:51.479+0000 7f107b7fe700 1 -- 172.21.15.2:0/4203248341 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f106c0042d0 con 0x7f10840ffc40 2023-06-23T18:16:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:51 smithi002 ceph-mon[95947]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-23T18:16:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:51 smithi002 ceph-mon[95947]: osdmap e20: 3 total, 3 up, 3 in 2023-06-23T18:16:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:51 smithi002 ceph-mon[95947]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:51.802 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:51 smithi002 sudo[112159]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T18:16:51.802 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:51 smithi002 sudo[112159]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:51.802 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:51 smithi002 sudo[112159]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:51.803 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:16:51 smithi002 sudo[112159]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:51 smithi082 ceph-mon[97889]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-23T18:16:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:51 smithi082 ceph-mon[97889]: osdmap e20: 3 total, 3 up, 3 in 2023-06-23T18:16:51.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:51 smithi082 ceph-mon[97889]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:51 smithi017 ceph-mon[97472]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:16:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-06-23T18:16:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:51 smithi017 ceph-mon[97472]: osdmap e20: 3 total, 3 up, 3 in 2023-06-23T18:16:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:51 smithi017 ceph-mon[97472]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:52.440 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:52 smithi017 sudo[104901]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T18:16:52.441 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:52 smithi017 sudo[104901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:52.441 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:52 smithi017 sudo[104901]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:52.441 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:52 smithi017 sudo[104901]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:52 smithi002 ceph-mon[95947]: osdmap e21: 3 total, 3 up, 3 in 2023-06-23T18:16:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:52 smithi002 ceph-mon[95947]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-23T18:16:52.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:52.808 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:52 smithi082 ceph-mon[97889]: osdmap e21: 3 total, 3 up, 3 in 2023-06-23T18:16:52.809 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:52 smithi082 ceph-mon[97889]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-23T18:16:52.809 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:52 smithi017 ceph-mon[97472]: osdmap e21: 3 total, 3 up, 3 in 2023-06-23T18:16:52.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:52 smithi017 ceph-mon[97472]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-06-23T18:16:52.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:52.943 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:52 smithi017 sudo[104904]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T18:16:52.943 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:52 smithi017 sudo[104904]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:52.943 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:52 smithi017 sudo[104904]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:52.943 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:16:52 smithi017 sudo[104904]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:53.062 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:52 smithi082 sudo[106031]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T18:16:53.062 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:52 smithi082 sudo[106031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:53.063 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:52 smithi082 sudo[106031]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:53.063 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:52 smithi082 sudo[106031]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:53.348 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:53 smithi082 sudo[106034]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T18:16:53.349 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:53 smithi082 sudo[106034]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:53.349 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:53 smithi082 sudo[106034]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:53.349 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:16:53 smithi082 sudo[106034]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:53.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 sudo[112325]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T18:16:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 sudo[112325]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 sudo[112325]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 sudo[112325]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:54.247 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:54.247 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:53 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1255818198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0c433bc-2983-4dac-a88c-27d6d0060480"}]: dispatch 2023-06-23T18:16:54.247 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:53 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1255818198' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0c433bc-2983-4dac-a88c-27d6d0060480"}]': finished 2023-06-23T18:16:54.248 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:53 smithi082 ceph-mon[97889]: osdmap e22: 4 total, 3 up, 4 in 2023-06-23T18:16:54.248 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:16:54.248 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:53 smithi082 ceph-mon[97889]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:54.248 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:54.248 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:53 smithi082 ceph-mon[97889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:54.248 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:54 smithi082 sudo[106037]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T18:16:54.248 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:54 smithi082 sudo[106037]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:54.249 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:54 smithi082 sudo[106037]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:54.249 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:54 smithi082 sudo[106037]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:54.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 sudo[112514]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T18:16:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 sudo[112514]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 sudo[112514]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 sudo[112514]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1255818198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0c433bc-2983-4dac-a88c-27d6d0060480"}]: dispatch 2023-06-23T18:16:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1255818198' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0c433bc-2983-4dac-a88c-27d6d0060480"}]': finished 2023-06-23T18:16:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 ceph-mon[95947]: osdmap e22: 4 total, 3 up, 4 in 2023-06-23T18:16:54.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:16:54.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 ceph-mon[95947]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:54.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:54.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:53 smithi002 ceph-mon[95947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:54.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:53 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1255818198' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d0c433bc-2983-4dac-a88c-27d6d0060480"}]: dispatch 2023-06-23T18:16:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:53 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1255818198' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d0c433bc-2983-4dac-a88c-27d6d0060480"}]': finished 2023-06-23T18:16:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:53 smithi017 ceph-mon[97472]: osdmap e22: 4 total, 3 up, 4 in 2023-06-23T18:16:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:16:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:53 smithi017 ceph-mon[97472]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:54.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:16:54.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:53 smithi017 ceph-mon[97472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:54 smithi082 sudo[106040]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T18:16:54.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:54 smithi082 sudo[106040]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:54.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:54 smithi082 sudo[106040]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:54.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:54 smithi082 sudo[106040]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:54 smithi017 sudo[104907]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-06-23T18:16:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:54 smithi017 sudo[104907]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:54 smithi017 sudo[104907]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:54 smithi017 sudo[104907]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:55.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 sudo[104910]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-06-23T18:16:55.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 sudo[104910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-06-23T18:16:55.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 sudo[104910]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-06-23T18:16:55.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 sudo[104910]: pam_unix(sudo:session): session closed for user root 2023-06-23T18:16:55.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2977245298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:55.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:55.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:55.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:55.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.695 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.695 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.695 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:55 smithi017 ceph-mon[97472]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:55.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2977245298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:55.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:55.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:55.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:55.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:55 smithi002 ceph-mon[95947]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:55.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2977245298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:16:55.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:55.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:55.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:55.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-06-23T18:16:55.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-06-23T18:16:55.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-06-23T18:16:55.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:55 smithi082 ceph-mon[97889]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:56 smithi017 ceph-mon[97472]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:56 smithi017 ceph-mon[97472]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:56.765 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:56 smithi002 ceph-mon[95947]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:56.765 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:56 smithi002 ceph-mon[95947]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:56 smithi082 ceph-mon[97889]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-06-23T18:16:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:56 smithi082 ceph-mon[97889]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-06-23T18:16:57.644 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:57 smithi002 ceph-mon[95947]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:57 smithi017 ceph-mon[97472]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:57.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:57 smithi082 ceph-mon[97889]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:16:59.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:58 smithi002 ceph-mon[95947]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-06-23T18:16:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:58 smithi082 ceph-mon[97889]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-06-23T18:16:59.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:58 smithi017 ceph-mon[97472]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2023-06-23T18:17:00.269 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-23T18:17:00.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:00.270 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:16:59 smithi002 ceph-mon[95947]: Deploying daemon osd.3 on smithi002 2023-06-23T18:17:00.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-23T18:17:00.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:00.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:16:59 smithi082 ceph-mon[97889]: Deploying daemon osd.3 on smithi002 2023-06-23T18:17:00.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-06-23T18:17:00.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:00.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:16:59 smithi017 ceph-mon[97472]: Deploying daemon osd.3 on smithi002 2023-06-23T18:17:01.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:00 smithi002 ceph-mon[95947]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-06-23T18:17:01.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:00 smithi082 ceph-mon[97889]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-06-23T18:17:01.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:00 smithi017 ceph-mon[97472]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2023-06-23T18:17:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:02 smithi002 ceph-mon[95947]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-06-23T18:17:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:02 smithi082 ceph-mon[97889]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-06-23T18:17:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:02 smithi017 ceph-mon[97472]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2023-06-23T18:17:05.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:05.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:05.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:05.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:05 smithi002 ceph-mon[95947]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-06-23T18:17:05.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:05.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:05.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:05.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:05 smithi082 ceph-mon[97889]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-06-23T18:17:05.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:05.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:05.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:05.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:05 smithi017 ceph-mon[97472]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2023-06-23T18:17:07.020 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:06 smithi002 ceph-mon[95947]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T18:17:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:06 smithi082 ceph-mon[97889]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T18:17:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:06 smithi017 ceph-mon[97472]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T18:17:07.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.424+0000 7f107b7fe700 1 -- 172.21.15.2:0/4203248341 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1084060910 con 0x7f10700a10c0 2023-06-23T18:17:07.430 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 3 on host 'smithi002' 2023-06-23T18:17:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f10700a10c0 msgr2=0x7f10700a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f10700a10c0 0x7f10700a3570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f107400bfd0 tx=0x7f107400bf60 comp rx=0 tx=0).stop 2023-06-23T18:17:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840ffc40 msgr2=0x7f1084194c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840ffc40 0x7f1084194c20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d9b0 tx=0x7f106c00dd70 comp rx=0 tx=0).stop 2023-06-23T18:17:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 shutdown_connections 2023-06-23T18:17:07.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f10700a10c0 0x7f10700a3570 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:07.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10841005d0 0x7f108419bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:07.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10840fea40 0x7f10841946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:07.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 --2- 172.21.15.2:0/4203248341 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10840ffc40 0x7f1084194c20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:07.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 >> 172.21.15.2:0/4203248341 conn(0x7f10840fa070 msgr2=0x7f1084103290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:17:07.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 shutdown_connections 2023-06-23T18:17:07.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:07.426+0000 7f1088814700 1 -- 172.21.15.2:0/4203248341 wait complete. 2023-06-23T18:17:08.050 DEBUG:teuthology.orchestra.run.smithi002:osd.3> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.3.service 2023-06-23T18:17:08.053 INFO:tasks.cephadm:Deploying osd.4 on smithi017 with /dev/vg_nvme/lv_3... 2023-06-23T18:17:08.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-23T18:17:08.076 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.077 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.077 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:08.077 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:08.077 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.077 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.077 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.077 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:08.082 INFO:journalctl@ceph.osd.3.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:17:08.087 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.087 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.087 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:08.088 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:08.088 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.088 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.088 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.088 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:08.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:08.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:08.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:08.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:08.517 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:17:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3[114478]: 2023-06-23T18:17:08.187+0000 7f85b627f640 -1 osd.3 0 log_to_monitors true 2023-06-23T18:17:08.628 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:17:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:08 smithi082 ceph-mon[97889]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-23T18:17:09.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:08 smithi082 ceph-mon[97889]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T18:17:09.119 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:08 smithi017 ceph-mon[97472]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-23T18:17:09.119 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:08 smithi017 ceph-mon[97472]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T18:17:09.201 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:08 smithi002 ceph-mon[95947]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-06-23T18:17:09.202 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:08 smithi002 ceph-mon[95947]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2023-06-23T18:17:09.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:09 smithi017 ceph-mon[97472]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-23T18:17:09.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:09 smithi017 ceph-mon[97472]: osdmap e23: 4 total, 3 up, 4 in 2023-06-23T18:17:09.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:09 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:09.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:09 smithi017 ceph-mon[97472]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:17:10.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:09 smithi082 ceph-mon[97889]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-23T18:17:10.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:09 smithi082 ceph-mon[97889]: osdmap e23: 4 total, 3 up, 4 in 2023-06-23T18:17:10.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:09 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:10.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:09 smithi082 ceph-mon[97889]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:17:10.146 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-23T18:17:10.147 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-23T18:17:10.147 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2023-06-23T18:17:10.147 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2023-06-23T18:17:10.147 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00825149 s, 1.3 GB/s 2023-06-23T18:17:10.148 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2023-06-23T18:17:10.212 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:17:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3[114478]: 2023-06-23T18:17:09.825+0000 7f85ab8f7700 -1 osd.3 0 waiting for initial osdmap 2023-06-23T18:17:10.212 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:17:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3[114478]: 2023-06-23T18:17:09.832+0000 7f85a5ee9700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:17:10.212 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:09 smithi002 ceph-mon[95947]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-06-23T18:17:10.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:09 smithi002 ceph-mon[95947]: osdmap e23: 4 total, 3 up, 4 in 2023-06-23T18:17:10.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:09 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:10.213 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:09 smithi002 ceph-mon[95947]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:17:10.587 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_3 2023-06-23T18:17:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:17:11.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: osdmap e24: 4 total, 3 up, 4 in 2023-06-23T18:17:11.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:11.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:11.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi002 2023-06-23T18:17:11.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:11.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:11.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi002 to 8586M 2023-06-23T18:17:11.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:11.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:11.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:10 smithi082 ceph-mon[97889]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:17:11.190 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:17:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:17:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: osdmap e24: 4 total, 3 up, 4 in 2023-06-23T18:17:11.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:11.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:11.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi002 2023-06-23T18:17:11.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:11.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:11.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi002 to 8586M 2023-06-23T18:17:11.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:11.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:11.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:10 smithi017 ceph-mon[97472]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:17:11.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:17:11.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: osdmap e24: 4 total, 3 up, 4 in 2023-06-23T18:17:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi002 2023-06-23T18:17:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:11.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi002 to 8586M 2023-06-23T18:17:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:11.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:10 smithi002 ceph-mon[95947]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-06-23T18:17:11.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:11 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:17:11.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:11 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:17:11.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:11 smithi017 ceph-mon[97472]: osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] boot 2023-06-23T18:17:11.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:11 smithi017 ceph-mon[97472]: osdmap e25: 4 total, 4 up, 4 in 2023-06-23T18:17:11.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:11 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:11.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:11 smithi017 ceph-mon[97472]: osdmap e26: 4 total, 4 up, 4 in 2023-06-23T18:17:12.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:11 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:17:12.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:11 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:17:12.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:11 smithi082 ceph-mon[97889]: osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] boot 2023-06-23T18:17:12.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:11 smithi082 ceph-mon[97889]: osdmap e25: 4 total, 4 up, 4 in 2023-06-23T18:17:12.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:11 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:12.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:11 smithi082 ceph-mon[97889]: osdmap e26: 4 total, 4 up, 4 in 2023-06-23T18:17:12.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:11 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:17:12.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:11 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:17:12.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:11 smithi002 ceph-mon[95947]: osd.3 [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] boot 2023-06-23T18:17:12.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:11 smithi002 ceph-mon[95947]: osdmap e25: 4 total, 4 up, 4 in 2023-06-23T18:17:12.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:11 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-06-23T18:17:12.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:11 smithi002 ceph-mon[95947]: osdmap e26: 4 total, 4 up, 4 in 2023-06-23T18:17:12.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 -- 172.21.15.17:0/2062228660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40ffc40 msgr2=0x7fddd4100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:12.550 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/2062228660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40ffc40 0x7fddd4100090 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009990 tx=0x7fddc8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:17:12.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 -- 172.21.15.17:0/2062228660 shutdown_connections 2023-06-23T18:17:12.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/2062228660 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd41005d0 0x7fddd4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:12.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/2062228660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd40fea40 0x7fddd40fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:12.551 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/2062228660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40ffc40 0x7fddd4100090 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:12.552 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 -- 172.21.15.17:0/2062228660 >> 172.21.15.17:0/2062228660 conn(0x7fddd40fa070 msgr2=0x7fddd40fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:17:12.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 -- 172.21.15.17:0/2062228660 shutdown_connections 2023-06-23T18:17:12.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 -- 172.21.15.17:0/2062228660 wait complete. 2023-06-23T18:17:12.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.543+0000 7fddd9aa3700 1 Processor -- start 2023-06-23T18:17:12.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd9aa3700 1 -- start start 2023-06-23T18:17:12.553 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd9aa3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40fea40 0x7fddd41957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:12.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd9aa3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd40ffc40 0x7fddd4195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:12.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd9aa3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd41005d0 0x7fddd419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:12.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd9aa3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd419d340 con 0x7fddd40ffc40 2023-06-23T18:17:12.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd9aa3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd419d480 con 0x7fddd40fea40 2023-06-23T18:17:12.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd9aa3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd419d610 con 0x7fddd41005d0 2023-06-23T18:17:12.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40fea40 0x7fddd41957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:12.554 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40fea40 0x7fddd41957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33086/0 (socket says 172.21.15.17:33086) 2023-06-23T18:17:12.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 -- 172.21.15.17:0/3318935339 learned_addr learned my addr 172.21.15.17:0/3318935339 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-06-23T18:17:12.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd3fff700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd41005d0 0x7fddd419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:12.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd2ffd700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd40ffc40 0x7fddd4195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:12.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 -- 172.21.15.17:0/3318935339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd41005d0 msgr2=0x7fddd419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:12.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd41005d0 0x7fddd419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:12.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 -- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd40ffc40 msgr2=0x7fddd4195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:12.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd40ffc40 0x7fddd4195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:12.555 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 -- 172.21.15.17:0/3318935339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80095f0 con 0x7fddd40fea40 2023-06-23T18:17:12.556 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd37fe700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40fea40 0x7fddd41957b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fddc400e970 tx=0x7fddc400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:17:12.556 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.544+0000 7fddd0ff9700 1 -- 172.21.15.17:0/3318935339 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4009790 con 0x7fddd40fea40 2023-06-23T18:17:12.556 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.545+0000 7fddd0ff9700 1 -- 172.21.15.17:0/3318935339 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddc400eec0 con 0x7fddd40fea40 2023-06-23T18:17:12.556 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.545+0000 7fddd0ff9700 1 -- 172.21.15.17:0/3318935339 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc4003c40 con 0x7fddd40fea40 2023-06-23T18:17:12.556 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.545+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd419d8f0 con 0x7fddd40fea40 2023-06-23T18:17:12.557 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.545+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd419ddf0 con 0x7fddd40fea40 2023-06-23T18:17:12.557 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.545+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd410a100 con 0x7fddd40fea40 2023-06-23T18:17:12.558 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.547+0000 7fddd0ff9700 1 -- 172.21.15.17:0/3318935339 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fddc4003da0 con 0x7fddd40fea40 2023-06-23T18:17:12.558 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.547+0000 7fddd0ff9700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fddbc0a11d0 0x7fddbc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:12.558 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.547+0000 7fddd0ff9700 1 -- 172.21.15.17:0/3318935339 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fddc4014070 con 0x7fddd40fea40 2023-06-23T18:17:12.558 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.548+0000 7fddd2ffd700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fddbc0a11d0 0x7fddbc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:12.558 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.548+0000 7fddd2ffd700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fddbc0a11d0 0x7fddbc0a3680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fddc8006010 tx=0x7fddc800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:17:12.558 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.551+0000 7fddd0ff9700 1 -- 172.21.15.17:0/3318935339 <== mon.2 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fddc4096360 con 0x7fddd40fea40 2023-06-23T18:17:12.763 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:12.758+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd4068760 con 0x7fddbc0a11d0 2023-06-23T18:17:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:12 smithi082 ceph-mon[97889]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:13.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:13.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:13.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:13.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:12 smithi082 ceph-mon[97889]: osdmap e27: 4 total, 4 up, 4 in 2023-06-23T18:17:13.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:12 smithi017 ceph-mon[97472]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:13.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:13.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:13.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:13.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:12 smithi017 ceph-mon[97472]: osdmap e27: 4 total, 4 up, 4 in 2023-06-23T18:17:13.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:12 smithi002 ceph-mon[95947]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:13.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:13.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:13.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:13.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:12 smithi002 ceph-mon[95947]: osdmap e27: 4 total, 4 up, 4 in 2023-06-23T18:17:14.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:13 smithi082 ceph-mon[97889]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:14.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:13 smithi017 ceph-mon[97472]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:14.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:13 smithi002 ceph-mon[95947]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:15.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:14 smithi017 ceph-mon[97472]: osdmap e28: 4 total, 4 up, 4 in 2023-06-23T18:17:15.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:14 smithi017 ceph-mon[97472]: pgmap v97: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:17:15.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:14 smithi002 ceph-mon[95947]: osdmap e28: 4 total, 4 up, 4 in 2023-06-23T18:17:15.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:14 smithi002 ceph-mon[95947]: pgmap v97: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:17:15.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:14 smithi082 ceph-mon[97889]: osdmap e28: 4 total, 4 up, 4 in 2023-06-23T18:17:15.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:14 smithi082 ceph-mon[97889]: pgmap v97: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:17:16.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:15 smithi017 ceph-mon[97472]: osdmap e29: 4 total, 4 up, 4 in 2023-06-23T18:17:16.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:15 smithi017 ceph-mon[97472]: from='client.? 172.21.15.17:0/2971720584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]: dispatch 2023-06-23T18:17:16.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:15 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]: dispatch 2023-06-23T18:17:16.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:15 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]': finished 2023-06-23T18:17:16.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:15 smithi017 ceph-mon[97472]: osdmap e30: 5 total, 4 up, 5 in 2023-06-23T18:17:16.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:16.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:15 smithi002 ceph-mon[95947]: osdmap e29: 4 total, 4 up, 4 in 2023-06-23T18:17:16.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:15 smithi002 ceph-mon[95947]: from='client.? 172.21.15.17:0/2971720584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]: dispatch 2023-06-23T18:17:16.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:15 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]: dispatch 2023-06-23T18:17:16.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:15 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]': finished 2023-06-23T18:17:16.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:15 smithi002 ceph-mon[95947]: osdmap e30: 5 total, 4 up, 5 in 2023-06-23T18:17:16.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:16.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:15 smithi082 ceph-mon[97889]: osdmap e29: 4 total, 4 up, 4 in 2023-06-23T18:17:16.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:15 smithi082 ceph-mon[97889]: from='client.? 172.21.15.17:0/2971720584' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]: dispatch 2023-06-23T18:17:16.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:15 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]: dispatch 2023-06-23T18:17:16.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:15 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "820c84c1-7f7a-4c03-8380-3dffe8d9c3fd"}]': finished 2023-06-23T18:17:16.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:15 smithi082 ceph-mon[97889]: osdmap e30: 5 total, 4 up, 5 in 2023-06-23T18:17:16.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:17.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:16 smithi017 ceph-mon[97472]: from='client.? 172.21.15.17:0/4256536817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:17.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:16 smithi017 ceph-mon[97472]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:17:17.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:16 smithi002 ceph-mon[95947]: from='client.? 172.21.15.17:0/4256536817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:17.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:16 smithi002 ceph-mon[95947]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:17:17.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:16 smithi082 ceph-mon[97889]: from='client.? 172.21.15.17:0/4256536817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:17.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:16 smithi082 ceph-mon[97889]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:17:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:18 smithi002 ceph-mon[95947]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-06-23T18:17:19.068 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:18 smithi017 ceph-mon[97472]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-06-23T18:17:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:18 smithi082 ceph-mon[97889]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 98 KiB/s, 0 objects/s recovering 2023-06-23T18:17:21.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:20 smithi002 ceph-mon[95947]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:20 smithi082 ceph-mon[97889]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:20 smithi017 ceph-mon[97472]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:22.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:21 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-23T18:17:22.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:21 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:22.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:21 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-23T18:17:22.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:21 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:22.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:21 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-06-23T18:17:22.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:21 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:22.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:22 smithi017 ceph-mon[97472]: Deploying daemon osd.4 on smithi017 2023-06-23T18:17:22.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:22 smithi017 ceph-mon[97472]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:22 smithi082 ceph-mon[97889]: Deploying daemon osd.4 on smithi017 2023-06-23T18:17:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:22 smithi082 ceph-mon[97889]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:22 smithi002 ceph-mon[95947]: Deploying daemon osd.4 on smithi017 2023-06-23T18:17:23.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:22 smithi002 ceph-mon[95947]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:25.270 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:24 smithi017 ceph-mon[97472]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:25.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:24 smithi002 ceph-mon[95947]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:25.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:24 smithi082 ceph-mon[97889]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:27.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:27.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:27.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:27.409 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:27 smithi017 ceph-mon[97472]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:27.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:27 smithi082 ceph-mon[97889]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:27.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:27 smithi002 ceph-mon[95947]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:29.079 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.074+0000 7fddd0ff9700 1 -- 172.21.15.17:0/3318935339 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fddd4068760 con 0x7fddbc0a11d0 2023-06-23T18:17:29.080 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 4 on host 'smithi017' 2023-06-23T18:17:29.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fddbc0a11d0 msgr2=0x7fddbc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:29.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fddbc0a11d0 0x7fddbc0a3680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fddc8006010 tx=0x7fddc800b260 comp rx=0 tx=0).stop 2023-06-23T18:17:29.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40fea40 msgr2=0x7fddd41957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:29.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40fea40 0x7fddd41957b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fddc400e970 tx=0x7fddc400ec80 comp rx=0 tx=0).stop 2023-06-23T18:17:29.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 shutdown_connections 2023-06-23T18:17:29.082 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fddd41005d0 0x7fddd419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:29.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fddbc0a11d0 0x7fddbc0a3680 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:29.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fddd40fea40 0x7fddd41957b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:29.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 --2- 172.21.15.17:0/3318935339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fddd40ffc40 0x7fddd4195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:29.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 >> 172.21.15.17:0/3318935339 conn(0x7fddd40fa070 msgr2=0x7fddd4068050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:17:29.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 shutdown_connections 2023-06-23T18:17:29.083 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:17:29.077+0000 7fddd9aa3700 1 -- 172.21.15.17:0/3318935339 wait complete. 2023-06-23T18:17:29.490 DEBUG:teuthology.orchestra.run.smithi017:osd.4> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.4.service 2023-06-23T18:17:29.493 INFO:tasks.cephadm:Deploying osd.5 on smithi082 with /dev/vg_nvme/lv_3... 2023-06-23T18:17:29.493 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-06-23T18:17:29.529 INFO:journalctl@ceph.osd.4.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:17:29.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:29.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:29.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:29.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:29.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:29.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:30.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:30.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:30.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:17:30.284 INFO:journalctl@ceph.osd.4.smithi017.stdout:Jun 23 18:17:30 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-4[108080]: 2023-06-23T18:17:30.233+0000 7f64f1b76640 -1 osd.4 0 log_to_monitors true 2023-06-23T18:17:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:30 smithi082 ceph-mon[97889]: from='osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T18:17:30.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:30 smithi082 ceph-mon[97889]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T18:17:30.878 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:30 smithi017 ceph-mon[97472]: from='osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T18:17:30.878 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:30 smithi017 ceph-mon[97472]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T18:17:31.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:30 smithi002 ceph-mon[95947]: from='osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T18:17:31.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:30 smithi002 ceph-mon[95947]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-06-23T18:17:31.543 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-06-23T18:17:31.543 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-06-23T18:17:31.544 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-06-23T18:17:31.544 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-06-23T18:17:31.544 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0117414 s, 893 MB/s 2023-06-23T18:17:31.544 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 2023-06-23T18:17:31.544 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-06-23T18:17:31.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:31 smithi017 ceph-mon[97472]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-23T18:17:31.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:31 smithi017 ceph-mon[97472]: osdmap e31: 5 total, 4 up, 5 in 2023-06-23T18:17:31.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:31 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:31.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:31 smithi017 ceph-mon[97472]: from='osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:17:31.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:31 smithi017 ceph-mon[97472]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:17:31.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:31 smithi017 ceph-mon[97472]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:31.692 INFO:journalctl@ceph.osd.4.smithi017.stdout:Jun 23 18:17:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-4[108080]: 2023-06-23T18:17:31.634+0000 7f64e71ee700 -1 osd.4 0 waiting for initial osdmap 2023-06-23T18:17:31.692 INFO:journalctl@ceph.osd.4.smithi017.stdout:Jun 23 18:17:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-4[108080]: 2023-06-23T18:17:31.640+0000 7f64df7dc700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:17:31.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:31 smithi082 ceph-mon[97889]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-23T18:17:31.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:31 smithi082 ceph-mon[97889]: osdmap e31: 5 total, 4 up, 5 in 2023-06-23T18:17:31.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:31 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:31.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:31 smithi082 ceph-mon[97889]: from='osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:17:31.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:31 smithi082 ceph-mon[97889]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:17:31.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:31 smithi082 ceph-mon[97889]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:31.907 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_3 2023-06-23T18:17:32.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:31 smithi002 ceph-mon[95947]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-06-23T18:17:32.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:31 smithi002 ceph-mon[95947]: osdmap e31: 5 total, 4 up, 5 in 2023-06-23T18:17:32.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:31 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:32.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:31 smithi002 ceph-mon[95947]: from='osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:17:32.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:31 smithi002 ceph-mon[95947]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:17:32.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:31 smithi002 ceph-mon[95947]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-06-23T18:17:32.458 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:17:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:17:32.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: osdmap e32: 5 total, 4 up, 5 in 2023-06-23T18:17:32.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:32.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:32.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:32.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:32.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:32.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:32.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:32.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:32.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:32.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:32.981 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:17:32.981 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: osdmap e32: 5 total, 4 up, 5 in 2023-06-23T18:17:32.981 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:32.982 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:32.982 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:32.982 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:32.982 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:32.982 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:32.983 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:32.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:32.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:32.984 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:33.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:17:33.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: osdmap e32: 5 total, 4 up, 5 in 2023-06-23T18:17:33.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:33.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:33.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:33.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:33.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:33.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:33.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:33.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:33.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:33.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 -- 172.21.15.82:0/3118343368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd741005d0 msgr2=0x7fcd74106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/3118343368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd741005d0 0x7fcd74106e90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00b0a0 tx=0x7fcd6c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:17:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 -- 172.21.15.82:0/3118343368 shutdown_connections 2023-06-23T18:17:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/3118343368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd741005d0 0x7fcd74106e90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/3118343368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd740fea40 0x7fcd740fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/3118343368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd740ffc40 0x7fcd74100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 -- 172.21.15.82:0/3118343368 >> 172.21.15.82:0/3118343368 conn(0x7fcd740fa070 msgr2=0x7fcd740fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:17:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 -- 172.21.15.82:0/3118343368 shutdown_connections 2023-06-23T18:17:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.655+0000 7fcd7c14d700 1 -- 172.21.15.82:0/3118343368 wait complete. 2023-06-23T18:17:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7c14d700 1 Processor -- start 2023-06-23T18:17:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7c14d700 1 -- start start 2023-06-23T18:17:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7c14d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd740fea40 0x7fcd74068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7c14d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd740ffc40 0x7fcd7419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7c14d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd741005d0 0x7fcd7419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7c14d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7419d3d0 con 0x7fcd740fea40 2023-06-23T18:17:33.666 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7c14d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7419d540 con 0x7fcd741005d0 2023-06-23T18:17:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7c14d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7419d700 con 0x7fcd740ffc40 2023-06-23T18:17:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd740ffc40 0x7fcd7419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd740ffc40 0x7fcd7419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40618/0 (socket says 172.21.15.82:40618) 2023-06-23T18:17:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 -- 172.21.15.82:0/2679077109 learned_addr learned my addr 172.21.15.82:0/2679077109 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:17:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd7a6ea700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd741005d0 0x7fcd7419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:33.667 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd79ee9700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd740fea40 0x7fcd74068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 -- 172.21.15.82:0/2679077109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd741005d0 msgr2=0x7fcd7419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd741005d0 0x7fcd7419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 -- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd740fea40 msgr2=0x7fcd74068af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd740fea40 0x7fcd74068af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 -- 172.21.15.82:0/2679077109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c009d00 con 0x7fcd740ffc40 2023-06-23T18:17:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.656+0000 7fcd796e8700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd740ffc40 0x7fcd7419a5f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000d980 tx=0x7fcd7000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:17:33.668 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.657+0000 7fcd66ffd700 1 -- 172.21.15.82:0/2679077109 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70004220 con 0x7fcd740ffc40 2023-06-23T18:17:33.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.657+0000 7fcd66ffd700 1 -- 172.21.15.82:0/2679077109 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd70007640 con 0x7fcd740ffc40 2023-06-23T18:17:33.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.657+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7419d9e0 con 0x7fcd740ffc40 2023-06-23T18:17:33.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.657+0000 7fcd66ffd700 1 -- 172.21.15.82:0/2679077109 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd700055e0 con 0x7fcd740ffc40 2023-06-23T18:17:33.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.657+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7419dee0 con 0x7fcd740ffc40 2023-06-23T18:17:33.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.658+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd74106090 con 0x7fcd740ffc40 2023-06-23T18:17:33.669 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.659+0000 7fcd66ffd700 1 -- 172.21.15.82:0/2679077109 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcd70004750 con 0x7fcd740ffc40 2023-06-23T18:17:33.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.659+0000 7fcd66ffd700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd600a1150 0x7fcd600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:33.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.659+0000 7fcd66ffd700 1 -- 172.21.15.82:0/2679077109 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fcd700cbba0 con 0x7fcd740ffc40 2023-06-23T18:17:33.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.659+0000 7fcd79ee9700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd600a1150 0x7fcd600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:33.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.660+0000 7fcd79ee9700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd600a1150 0x7fcd600a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005950 tx=0x7fcd680058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:17:33.670 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.663+0000 7fcd66ffd700 1 -- 172.21.15.82:0/2679077109 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcd700cbe60 con 0x7fcd740ffc40 2023-06-23T18:17:33.860 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:33.855+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd74060910 con 0x7fcd600a1150 2023-06-23T18:17:33.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:33 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:17:33.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:33 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:17:33.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:33 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi017 2023-06-23T18:17:33.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:33 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi017 to 8586M 2023-06-23T18:17:33.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:33 smithi017 ceph-mon[97472]: osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] boot 2023-06-23T18:17:33.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:33 smithi017 ceph-mon[97472]: osdmap e33: 5 total, 5 up, 5 in 2023-06-23T18:17:33.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:33.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:33 smithi017 ceph-mon[97472]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:34.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:33 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:17:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:33 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:17:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:33 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi017 2023-06-23T18:17:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:33 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi017 to 8586M 2023-06-23T18:17:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:33 smithi002 ceph-mon[95947]: osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] boot 2023-06-23T18:17:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:33 smithi002 ceph-mon[95947]: osdmap e33: 5 total, 5 up, 5 in 2023-06-23T18:17:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:34.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:33 smithi002 ceph-mon[95947]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:34.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:33 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:17:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:33 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:17:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:33 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi017 2023-06-23T18:17:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:33 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi017 to 8586M 2023-06-23T18:17:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:33 smithi082 ceph-mon[97889]: osd.4 [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] boot 2023-06-23T18:17:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:33 smithi082 ceph-mon[97889]: osdmap e33: 5 total, 5 up, 5 in 2023-06-23T18:17:34.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-06-23T18:17:34.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:33 smithi082 ceph-mon[97889]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:34 smithi017 ceph-mon[97472]: osdmap e34: 5 total, 5 up, 5 in 2023-06-23T18:17:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:34.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:34.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:34.981 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:34 smithi082 ceph-mon[97889]: osdmap e34: 5 total, 5 up, 5 in 2023-06-23T18:17:34.981 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:34.981 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:34.981 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:34 smithi002 ceph-mon[95947]: osdmap e34: 5 total, 5 up, 5 in 2023-06-23T18:17:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:35.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:35 smithi082 ceph-mon[97889]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:35.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:35 smithi082 ceph-mon[97889]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:35.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:35 smithi017 ceph-mon[97472]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:35.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:35 smithi017 ceph-mon[97472]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:36.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:35 smithi002 ceph-mon[95947]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:36.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:35 smithi002 ceph-mon[95947]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:36 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/2536645333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]: dispatch 2023-06-23T18:17:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:36 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]: dispatch 2023-06-23T18:17:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:36 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]': finished 2023-06-23T18:17:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:36 smithi017 ceph-mon[97472]: osdmap e35: 6 total, 5 up, 6 in 2023-06-23T18:17:36.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:37.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:36 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/2536645333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]: dispatch 2023-06-23T18:17:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:36 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]: dispatch 2023-06-23T18:17:37.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:36 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]': finished 2023-06-23T18:17:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:36 smithi002 ceph-mon[95947]: osdmap e35: 6 total, 5 up, 6 in 2023-06-23T18:17:37.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:36 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/2536645333' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]: dispatch 2023-06-23T18:17:37.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:36 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]: dispatch 2023-06-23T18:17:37.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:36 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eac48eb4-7436-4849-bff3-ccd457ee594a"}]': finished 2023-06-23T18:17:37.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:36 smithi082 ceph-mon[97889]: osdmap e35: 6 total, 5 up, 6 in 2023-06-23T18:17:37.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:37.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:37 smithi017 ceph-mon[97472]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:37.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:37 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/664753940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:38.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:37 smithi002 ceph-mon[95947]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:38.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:37 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/664753940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:38.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:37 smithi082 ceph-mon[97889]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:38.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:37 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/664753940' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:39.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:38 smithi002 ceph-mon[95947]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:39.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:38 smithi082 ceph-mon[97889]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:38 smithi017 ceph-mon[97472]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:41.026 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:40 smithi082 ceph-mon[97889]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:40 smithi002 ceph-mon[95947]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:40 smithi017 ceph-mon[97472]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:43.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-23T18:17:43.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-23T18:17:43.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:43.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-06-23T18:17:43.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:43.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:43 smithi082 ceph-mon[97889]: Deploying daemon osd.5 on smithi082 2023-06-23T18:17:43.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:43 smithi082 ceph-mon[97889]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:44.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:43 smithi002 ceph-mon[95947]: Deploying daemon osd.5 on smithi082 2023-06-23T18:17:44.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:43 smithi002 ceph-mon[95947]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:44.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:43 smithi017 ceph-mon[97472]: Deploying daemon osd.5 on smithi082 2023-06-23T18:17:44.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:43 smithi017 ceph-mon[97472]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:45.045 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:45 smithi082 ceph-mon[97889]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:45.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:45 smithi002 ceph-mon[95947]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:45.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:45 smithi017 ceph-mon[97472]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:46.897 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:46 smithi082 ceph-mon[97889]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:47.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:46 smithi002 ceph-mon[95947]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:46 smithi017 ceph-mon[97472]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:48.077 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:48.077 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:48.078 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:48.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:48.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:48.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:48.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:48.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:48.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:49.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:48 smithi082 ceph-mon[97889]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:49.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:48 smithi002 ceph-mon[95947]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:49.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:48 smithi017 ceph-mon[97472]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:49.627 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 5 on host 'smithi082' 2023-06-23T18:17:49.628 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.622+0000 7fcd66ffd700 1 -- 172.21.15.82:0/2679077109 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcd74060910 con 0x7fcd600a1150 2023-06-23T18:17:49.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd600a1150 msgr2=0x7fcd600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:49.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd600a1150 0x7fcd600a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005950 tx=0x7fcd680058e0 comp rx=0 tx=0).stop 2023-06-23T18:17:49.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd740ffc40 msgr2=0x7fcd7419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:49.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd740ffc40 0x7fcd7419a5f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcd7000d980 tx=0x7fcd7000dc90 comp rx=0 tx=0).stop 2023-06-23T18:17:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 shutdown_connections 2023-06-23T18:17:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd600a1150 0x7fcd600a3600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd741005d0 0x7fcd7419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:49.636 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd740fea40 0x7fcd74068af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:49.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 --2- 172.21.15.82:0/2679077109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd740ffc40 0x7fcd7419a5f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:49.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 >> 172.21.15.82:0/2679077109 conn(0x7fcd740fa070 msgr2=0x7fcd741031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:17:49.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 shutdown_connections 2023-06-23T18:17:49.637 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:17:49.624+0000 7fcd7c14d700 1 -- 172.21.15.82:0/2679077109 wait complete. 2023-06-23T18:17:50.047 DEBUG:teuthology.orchestra.run.smithi082:osd.5> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.5.service 2023-06-23T18:17:50.051 INFO:tasks.cephadm:Deploying osd.6 on smithi002 with /dev/vg_nvme/lv_2... 2023-06-23T18:17:50.051 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-23T18:17:50.093 INFO:journalctl@ceph.osd.5.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:17:50.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:50.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:50.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:50.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:50.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:50.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:50 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:50.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:50.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:50.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:50.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:50 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:17:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:17:51.164 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:17:50 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:17:50.929+0000 7f729d219640 -1 osd.5 0 log_to_monitors true 2023-06-23T18:17:51.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:51 smithi017 ceph-mon[97472]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:51.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:51 smithi017 ceph-mon[97472]: from='osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T18:17:51.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:51 smithi017 ceph-mon[97472]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T18:17:51.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:51 smithi002 ceph-mon[95947]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:51.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:51 smithi002 ceph-mon[95947]: from='osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T18:17:51.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:51 smithi002 ceph-mon[95947]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T18:17:51.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:51 smithi082 ceph-mon[97889]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:51.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:51 smithi082 ceph-mon[97889]: from='osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T18:17:51.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:51 smithi082 ceph-mon[97889]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-06-23T18:17:52.357 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-23T18:17:52.358 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-23T18:17:52.358 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2023-06-23T18:17:52.358 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2023-06-23T18:17:52.358 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116959 s, 897 MB/s 2023-06-23T18:17:52.358 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2023-06-23T18:17:52.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:52 smithi017 ceph-mon[97472]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-23T18:17:52.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:52 smithi017 ceph-mon[97472]: osdmap e36: 6 total, 5 up, 6 in 2023-06-23T18:17:52.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:52.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:52 smithi017 ceph-mon[97472]: from='osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:17:52.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:52 smithi017 ceph-mon[97472]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:17:52.495 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:52 smithi002 ceph-mon[95947]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-23T18:17:52.496 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:52 smithi002 ceph-mon[95947]: osdmap e36: 6 total, 5 up, 6 in 2023-06-23T18:17:52.496 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:52.496 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:52 smithi002 ceph-mon[95947]: from='osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:17:52.496 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:52 smithi002 ceph-mon[95947]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:17:52.598 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:17:52 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:17:52.167+0000 7f7292891700 -1 osd.5 0 waiting for initial osdmap 2023-06-23T18:17:52.599 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:17:52 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:17:52.174+0000 7f728c682700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:17:52.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:52 smithi082 ceph-mon[97889]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-06-23T18:17:52.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:52 smithi082 ceph-mon[97889]: osdmap e36: 6 total, 5 up, 6 in 2023-06-23T18:17:52.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:52.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:52 smithi082 ceph-mon[97889]: from='osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:17:52.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:52 smithi082 ceph-mon[97889]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:17:52.886 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi002:vg_nvme/lv_2 2023-06-23T18:17:53.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:17:53.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: osdmap e37: 6 total, 5 up, 6 in 2023-06-23T18:17:53.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:53.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:53.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi082 2023-06-23T18:17:53.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:53.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:53.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi082 to 8584M 2023-06-23T18:17:53.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:53.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:53.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:53 smithi017 ceph-mon[97472]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:53.450 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:17:53.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:17:53.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: osdmap e37: 6 total, 5 up, 6 in 2023-06-23T18:17:53.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:53.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:53.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi082 2023-06-23T18:17:53.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:53.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:53.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi082 to 8584M 2023-06-23T18:17:53.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:53.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:53.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.556 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:53 smithi002 ceph-mon[95947]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:53.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:17:53.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: osdmap e37: 6 total, 5 up, 6 in 2023-06-23T18:17:53.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:53.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:53.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi082 2023-06-23T18:17:53.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:17:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi082 to 8584M 2023-06-23T18:17:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:17:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:17:53.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:53 smithi082 ceph-mon[97889]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-06-23T18:17:54.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:54 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:17:54.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:54 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:17:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:54 smithi002 ceph-mon[95947]: osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] boot 2023-06-23T18:17:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:54 smithi002 ceph-mon[95947]: osdmap e38: 6 total, 6 up, 6 in 2023-06-23T18:17:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:54.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:54 smithi002 ceph-mon[95947]: osdmap e39: 6 total, 6 up, 6 in 2023-06-23T18:17:54.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:54 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:17:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:54 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:17:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:54 smithi017 ceph-mon[97472]: osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] boot 2023-06-23T18:17:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:54 smithi017 ceph-mon[97472]: osdmap e38: 6 total, 6 up, 6 in 2023-06-23T18:17:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:54.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:54 smithi017 ceph-mon[97472]: osdmap e39: 6 total, 6 up, 6 in 2023-06-23T18:17:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:54 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:17:54.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:54 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:17:54.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:54 smithi082 ceph-mon[97889]: osd.5 [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] boot 2023-06-23T18:17:54.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:54 smithi082 ceph-mon[97889]: osdmap e38: 6 total, 6 up, 6 in 2023-06-23T18:17:54.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-06-23T18:17:54.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:54 smithi082 ceph-mon[97889]: osdmap e39: 6 total, 6 up, 6 in 2023-06-23T18:17:54.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 -- 172.21.15.2:0/1391926797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b80ffb80 msgr2=0x7f47b80fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:54.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 --2- 172.21.15.2:0/1391926797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b80ffb80 0x7f47b80fff90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f47ac009990 tx=0x7f47ac009ca0 comp rx=0 tx=0).stop 2023-06-23T18:17:54.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 -- 172.21.15.2:0/1391926797 shutdown_connections 2023-06-23T18:17:54.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 --2- 172.21.15.2:0/1391926797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b8101710 0x7f47b8107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:54.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 --2- 172.21.15.2:0/1391926797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b80ffb80 0x7f47b80fff90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:54.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 --2- 172.21.15.2:0/1391926797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47b8100d80 0x7f47b81011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:54.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 -- 172.21.15.2:0/1391926797 >> 172.21.15.2:0/1391926797 conn(0x7f47b80fb1f0 msgr2=0x7f47b80fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:17:54.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 -- 172.21.15.2:0/1391926797 shutdown_connections 2023-06-23T18:17:54.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 -- 172.21.15.2:0/1391926797 wait complete. 2023-06-23T18:17:54.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 Processor -- start 2023-06-23T18:17:54.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.773+0000 7f47beac1700 1 -- start start 2023-06-23T18:17:54.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47beac1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b80ffb80 0x7f47b81957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:54.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47beac1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47b8100d80 0x7f47b8195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:54.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47beac1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b8101710 0x7f47b819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:54.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47beac1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819d340 con 0x7f47b8101710 2023-06-23T18:17:54.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47beac1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819d480 con 0x7f47b8100d80 2023-06-23T18:17:54.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47beac1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b819d610 con 0x7f47b80ffb80 2023-06-23T18:17:54.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b8101710 0x7f47b819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:54.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b8101710 0x7f47b819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51728/0 (socket says 172.21.15.2:51728) 2023-06-23T18:17:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 -- 172.21.15.2:0/1888965295 learned_addr learned my addr 172.21.15.2:0/1888965295 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:17:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47b7fff700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47b8100d80 0x7f47b8195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bc85d700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b80ffb80 0x7f47b81957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 -- 172.21.15.2:0/1888965295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b80ffb80 msgr2=0x7f47b81957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b80ffb80 0x7f47b81957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 -- 172.21.15.2:0/1888965295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47b8100d80 msgr2=0x7f47b8195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:17:54.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47b8100d80 0x7f47b8195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:17:54.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 -- 172.21.15.2:0/1888965295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0095f0 con 0x7f47b8101710 2023-06-23T18:17:54.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bc85d700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b80ffb80 0x7f47b81957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:17:54.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47bd05e700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b8101710 0x7f47b819cd70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47b0007f10 tx=0x7f47b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:17:54.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47b5ffb700 1 -- 172.21.15.2:0/1888965295 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b000ddd0 con 0x7f47b8101710 2023-06-23T18:17:54.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47b5ffb700 1 -- 172.21.15.2:0/1888965295 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f47b000f040 con 0x7f47b8101710 2023-06-23T18:17:54.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47b5ffb700 1 -- 172.21.15.2:0/1888965295 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b0004870 con 0x7f47b8101710 2023-06-23T18:17:54.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.774+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b819d8f0 con 0x7f47b8101710 2023-06-23T18:17:54.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.775+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b819ddf0 con 0x7f47b8101710 2023-06-23T18:17:54.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.776+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8107220 con 0x7f47b8101710 2023-06-23T18:17:54.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.777+0000 7f47b5ffb700 1 -- 172.21.15.2:0/1888965295 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f47b0003a00 con 0x7f47b8101710 2023-06-23T18:17:54.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.777+0000 7f47b5ffb700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47a80a11d0 0x7f47a80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:17:54.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.777+0000 7f47bc85d700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47a80a11d0 0x7f47a80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:17:54.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.777+0000 7f47b5ffb700 1 -- 172.21.15.2:0/1888965295 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f47b00cb300 con 0x7f47b8101710 2023-06-23T18:17:54.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.778+0000 7f47bc85d700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47a80a11d0 0x7f47a80a3680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00b5c0 tx=0x7f47ac005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:17:54.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.782+0000 7f47b5ffb700 1 -- 172.21.15.2:0/1888965295 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f47b004e770 con 0x7f47b8101710 2023-06-23T18:17:54.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:17:54.993+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f47b8060910 con 0x7f47a80a11d0 2023-06-23T18:17:55.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:55 smithi017 ceph-mon[97472]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:17:55.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:55.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:55.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:55.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:55 smithi017 ceph-mon[97472]: osdmap e40: 6 total, 6 up, 6 in 2023-06-23T18:17:55.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:55 smithi002 ceph-mon[95947]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:17:55.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:55.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:55.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:55.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:55 smithi002 ceph-mon[95947]: osdmap e40: 6 total, 6 up, 6 in 2023-06-23T18:17:55.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:55 smithi082 ceph-mon[97889]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:17:55.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:17:55.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:17:55.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:17:55.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:55 smithi082 ceph-mon[97889]: osdmap e40: 6 total, 6 up, 6 in 2023-06-23T18:17:56.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:56 smithi017 ceph-mon[97472]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:56.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:56 smithi002 ceph-mon[95947]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:56.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:56 smithi082 ceph-mon[97889]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:17:57.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:57 smithi002 ceph-mon[95947]: osdmap e41: 6 total, 6 up, 6 in 2023-06-23T18:17:57.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:57 smithi002 ceph-mon[95947]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:17:57.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:57 smithi082 ceph-mon[97889]: osdmap e41: 6 total, 6 up, 6 in 2023-06-23T18:17:57.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:57 smithi082 ceph-mon[97889]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:17:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:57 smithi017 ceph-mon[97472]: osdmap e41: 6 total, 6 up, 6 in 2023-06-23T18:17:57.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:57 smithi017 ceph-mon[97472]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:17:58.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:58 smithi002 ceph-mon[95947]: osdmap e42: 6 total, 6 up, 6 in 2023-06-23T18:17:58.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:58 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/4142608176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e320af6-3d7b-4934-9f7a-2860876ee5a0"}]: dispatch 2023-06-23T18:17:58.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:58 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/4142608176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e320af6-3d7b-4934-9f7a-2860876ee5a0"}]': finished 2023-06-23T18:17:58.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:58 smithi002 ceph-mon[95947]: osdmap e43: 7 total, 6 up, 7 in 2023-06-23T18:17:58.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:17:58.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:58 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2269197305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:58.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:58 smithi082 ceph-mon[97889]: osdmap e42: 6 total, 6 up, 6 in 2023-06-23T18:17:58.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:58 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/4142608176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e320af6-3d7b-4934-9f7a-2860876ee5a0"}]: dispatch 2023-06-23T18:17:58.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:58 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/4142608176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e320af6-3d7b-4934-9f7a-2860876ee5a0"}]': finished 2023-06-23T18:17:58.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:58 smithi082 ceph-mon[97889]: osdmap e43: 7 total, 6 up, 7 in 2023-06-23T18:17:58.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:17:58.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:58 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2269197305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:58 smithi017 ceph-mon[97472]: osdmap e42: 6 total, 6 up, 6 in 2023-06-23T18:17:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:58 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/4142608176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e320af6-3d7b-4934-9f7a-2860876ee5a0"}]: dispatch 2023-06-23T18:17:58.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:58 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/4142608176' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e320af6-3d7b-4934-9f7a-2860876ee5a0"}]': finished 2023-06-23T18:17:58.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:58 smithi017 ceph-mon[97472]: osdmap e43: 7 total, 6 up, 7 in 2023-06-23T18:17:58.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:17:58.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:58 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2269197305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:17:59.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:17:59 smithi002 ceph-mon[95947]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:17:59.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:17:59 smithi082 ceph-mon[97889]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:17:59.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:17:59 smithi017 ceph-mon[97472]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:18:01.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:00 smithi002 ceph-mon[95947]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-06-23T18:18:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:00 smithi082 ceph-mon[97889]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-06-23T18:18:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:00 smithi017 ceph-mon[97472]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2023-06-23T18:18:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:02 smithi002 ceph-mon[95947]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-06-23T18:18:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:02 smithi082 ceph-mon[97889]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-06-23T18:18:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:02 smithi017 ceph-mon[97472]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2023-06-23T18:18:05.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-23T18:18:05.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:05.011 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:05 smithi002 ceph-mon[95947]: Deploying daemon osd.6 on smithi002 2023-06-23T18:18:05.012 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:05 smithi002 ceph-mon[95947]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T18:18:05.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-23T18:18:05.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:05.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:05 smithi082 ceph-mon[97889]: Deploying daemon osd.6 on smithi002 2023-06-23T18:18:05.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:05 smithi082 ceph-mon[97889]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T18:18:05.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-06-23T18:18:05.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:05.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:05 smithi017 ceph-mon[97472]: Deploying daemon osd.6 on smithi002 2023-06-23T18:18:05.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:05 smithi017 ceph-mon[97472]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T18:18:07.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:06 smithi002 ceph-mon[95947]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-06-23T18:18:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:06 smithi082 ceph-mon[97889]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-06-23T18:18:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:06 smithi017 ceph-mon[97472]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2023-06-23T18:18:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:08 smithi002 ceph-mon[95947]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-06-23T18:18:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:08 smithi082 ceph-mon[97889]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-06-23T18:18:09.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:08 smithi017 ceph-mon[97472]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2023-06-23T18:18:10.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:09 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:10.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:09 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:10.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:09 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:10.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:09 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:10.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:09 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:10.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:09 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:10.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:09 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:10.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:09 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:10.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:09 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:10 smithi082 ceph-mon[97889]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:11.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:10 smithi017 ceph-mon[97472]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:11.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:10 smithi002 ceph-mon[95947]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.853+0000 7f47b5ffb700 1 -- 172.21.15.2:0/1888965295 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f47b8060910 con 0x7f47a80a11d0 2023-06-23T18:18:11.858 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 6 on host 'smithi002' 2023-06-23T18:18:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47a80a11d0 msgr2=0x7f47a80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47a80a11d0 0x7f47a80a3680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00b5c0 tx=0x7f47ac005fd0 comp rx=0 tx=0).stop 2023-06-23T18:18:11.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b8101710 msgr2=0x7f47b819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b8101710 0x7f47b819cd70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f47b0007f10 tx=0x7f47b000d3b0 comp rx=0 tx=0).stop 2023-06-23T18:18:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 shutdown_connections 2023-06-23T18:18:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f47b8101710 0x7f47b819cd70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47b80ffb80 0x7f47b81957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47a80a11d0 0x7f47a80a3680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 --2- 172.21.15.2:0/1888965295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47b8100d80 0x7f47b8195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:11.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 >> 172.21.15.2:0/1888965295 conn(0x7f47b80fb1f0 msgr2=0x7f47b8104360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:18:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 shutdown_connections 2023-06-23T18:18:11.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:11.855+0000 7f47beac1700 1 -- 172.21.15.2:0/1888965295 wait complete. 2023-06-23T18:18:12.337 DEBUG:teuthology.orchestra.run.smithi002:osd.6> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.6.service 2023-06-23T18:18:12.340 INFO:tasks.cephadm:Deploying osd.7 on smithi017 with /dev/vg_nvme/lv_2... 2023-06-23T18:18:12.341 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-23T18:18:12.371 INFO:journalctl@ceph.osd.6.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:18:12.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:12.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:12.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:12.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:12.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:12.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:12 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:12.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:12.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:12.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:12 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:12.926 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:18:12.983 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:18:12 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:18:12.706+0000 7fbb18ec3640 -1 osd.6 0 log_to_monitors true 2023-06-23T18:18:13.412 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:13 smithi017 ceph-mon[97472]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:13.413 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:13 smithi017 ceph-mon[97472]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-23T18:18:13.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:13 smithi082 ceph-mon[97889]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:13.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:13 smithi082 ceph-mon[97889]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-23T18:18:13.713 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:13 smithi002 ceph-mon[95947]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:13.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:13 smithi002 ceph-mon[95947]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-06-23T18:18:14.511 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-23T18:18:14.511 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-23T18:18:14.511 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2023-06-23T18:18:14.511 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2023-06-23T18:18:14.511 INFO:teuthology.orchestra.run.smithi017.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0082567 s, 1.3 GB/s 2023-06-23T18:18:14.511 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2023-06-23T18:18:14.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:18:14 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:18:14.352+0000 7fbb0e53b700 -1 osd.6 0 waiting for initial osdmap 2023-06-23T18:18:14.552 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:18:14 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:18:14.360+0000 7fbb0832c700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:18:14.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:14 smithi002 ceph-mon[95947]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-23T18:18:14.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:14 smithi002 ceph-mon[95947]: osdmap e44: 7 total, 6 up, 7 in 2023-06-23T18:18:14.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:14.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:14 smithi002 ceph-mon[95947]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:18:14.669 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:14 smithi017 ceph-mon[97472]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-23T18:18:14.670 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:14 smithi017 ceph-mon[97472]: osdmap e44: 7 total, 6 up, 7 in 2023-06-23T18:18:14.670 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:14 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:14.670 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:14 smithi017 ceph-mon[97472]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:18:14.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:14 smithi082 ceph-mon[97889]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-06-23T18:18:14.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:14 smithi082 ceph-mon[97889]: osdmap e44: 7 total, 6 up, 7 in 2023-06-23T18:18:14.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:14 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:14.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:14 smithi082 ceph-mon[97889]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:18:14.987 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_2 2023-06-23T18:18:15.543 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:18:15.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:18:15.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: osdmap e45: 7 total, 6 up, 7 in 2023-06-23T18:18:15.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:15.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:15.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi002 2023-06-23T18:18:15.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi002 to 5724M 2023-06-23T18:18:15.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:15.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:15.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:15.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:18:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: osdmap e45: 7 total, 6 up, 7 in 2023-06-23T18:18:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi002 2023-06-23T18:18:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi002 to 5724M 2023-06-23T18:18:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:15.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:15.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:18:15.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: osdmap e45: 7 total, 6 up, 7 in 2023-06-23T18:18:15.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:15.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:15.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi002 2023-06-23T18:18:15.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:15.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi002 to 5724M 2023-06-23T18:18:15.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:15.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:15.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-06-23T18:18:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:15.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:16.631 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:16 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:18:16.631 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:16 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:18:16.631 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:16 smithi017 ceph-mon[97472]: osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] boot 2023-06-23T18:18:16.631 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:16 smithi017 ceph-mon[97472]: osdmap e46: 7 total, 7 up, 7 in 2023-06-23T18:18:16.631 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:16 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:18:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:16 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:18:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:16 smithi002 ceph-mon[95947]: osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] boot 2023-06-23T18:18:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:16 smithi002 ceph-mon[95947]: osdmap e46: 7 total, 7 up, 7 in 2023-06-23T18:18:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:16.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:16 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:18:16.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:16 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:18:16.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:16 smithi082 ceph-mon[97889]: osd.6 [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] boot 2023-06-23T18:18:16.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:16 smithi082 ceph-mon[97889]: osdmap e46: 7 total, 7 up, 7 in 2023-06-23T18:18:16.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-06-23T18:18:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 -- 172.21.15.17:0/521480267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0574101710 msgr2=0x7f0574107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:16.884 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 --2- 172.21.15.17:0/521480267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0574101710 0x7f0574107fd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f056c00b0a0 tx=0x7f056c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:18:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 -- 172.21.15.17:0/521480267 shutdown_connections 2023-06-23T18:18:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 --2- 172.21.15.17:0/521480267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0574101710 0x7f0574107fd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 --2- 172.21.15.17:0/521480267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05740ffb80 0x7f05740fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 --2- 172.21.15.17:0/521480267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0574100d80 0x7f05741011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 -- 172.21.15.17:0/521480267 >> 172.21.15.17:0/521480267 conn(0x7f05740fb1f0 msgr2=0x7f05740fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:18:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 -- 172.21.15.17:0/521480267 shutdown_connections 2023-06-23T18:18:16.885 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 -- 172.21.15.17:0/521480267 wait complete. 2023-06-23T18:18:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.879+0000 7f057a8cc700 1 Processor -- start 2023-06-23T18:18:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f057a8cc700 1 -- start start 2023-06-23T18:18:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f057a8cc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05740ffb80 0x7f0574195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f057a8cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0574100d80 0x7f0574195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f057a8cc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0574101710 0x7f057419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f057a8cc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057419d3e0 con 0x7f0574101710 2023-06-23T18:18:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f057a8cc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057419d520 con 0x7f05740ffb80 2023-06-23T18:18:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f057a8cc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057419d6b0 con 0x7f0574100d80 2023-06-23T18:18:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f0573fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05740ffb80 0x7f0574195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f0573fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05740ffb80 0x7f0574195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33198/0 (socket says 172.21.15.17:33198) 2023-06-23T18:18:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f0573fff700 1 -- 172.21.15.17:0/38540826 learned_addr learned my addr 172.21.15.17:0/38540826 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-06-23T18:18:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f0578e69700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0574101710 0x7f057419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f05737fe700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0574100d80 0x7f0574195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f0573fff700 1 -- 172.21.15.17:0/38540826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0574100d80 msgr2=0x7f0574195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f0573fff700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0574100d80 0x7f0574195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f0573fff700 1 -- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0574101710 msgr2=0x7f057419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.880+0000 7f0573fff700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0574101710 0x7f057419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.881+0000 7f0573fff700 1 -- 172.21.15.17:0/38540826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c009d00 con 0x7f05740ffb80 2023-06-23T18:18:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.881+0000 7f0573fff700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05740ffb80 0x7f0574195850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f056400c970 tx=0x7f056400cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:18:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.881+0000 7f05717fa700 1 -- 172.21.15.17:0/38540826 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564004220 con 0x7f05740ffb80 2023-06-23T18:18:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.881+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057419d990 con 0x7f05740ffb80 2023-06-23T18:18:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.881+0000 7f05717fa700 1 -- 172.21.15.17:0/38540826 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0564007d30 con 0x7f05740ffb80 2023-06-23T18:18:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.881+0000 7f05717fa700 1 -- 172.21.15.17:0/38540826 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564003bf0 con 0x7f05740ffb80 2023-06-23T18:18:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.881+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057419de90 con 0x7f05740ffb80 2023-06-23T18:18:16.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.882+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057410b200 con 0x7f05740ffb80 2023-06-23T18:18:16.892 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.884+0000 7f05717fa700 1 -- 172.21.15.17:0/38540826 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0564003d50 con 0x7f05740ffb80 2023-06-23T18:18:16.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.885+0000 7f05717fa700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f055c0a1150 0x7f055c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:16.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.885+0000 7f05717fa700 1 -- 172.21.15.17:0/38540826 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f05640cbc70 con 0x7f05740ffb80 2023-06-23T18:18:16.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.885+0000 7f05737fe700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f055c0a1150 0x7f055c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:16.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.886+0000 7f05737fe700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f055c0a1150 0x7f055c0a3600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0568005fd0 tx=0x7f056800a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:18:16.893 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:16.888+0000 7f05717fa700 1 -- 172.21.15.17:0/38540826 <== mon.2 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0564096030 con 0x7f05740ffb80 2023-06-23T18:18:17.097 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:17.092+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0574060910 con 0x7f055c0a1150 2023-06-23T18:18:17.973 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:17 smithi017 ceph-mon[97472]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:17.974 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:17 smithi017 ceph-mon[97472]: osdmap e47: 7 total, 7 up, 7 in 2023-06-23T18:18:17.974 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:18:17.974 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:18:17.975 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:18.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:17 smithi002 ceph-mon[95947]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:18.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:17 smithi002 ceph-mon[95947]: osdmap e47: 7 total, 7 up, 7 in 2023-06-23T18:18:18.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:18:18.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:18:18.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:18.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:17 smithi082 ceph-mon[97889]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:17 smithi082 ceph-mon[97889]: osdmap e47: 7 total, 7 up, 7 in 2023-06-23T18:18:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:18:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:18:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:18 smithi002 ceph-mon[95947]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:18:19.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:18 smithi082 ceph-mon[97889]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:18:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:18 smithi017 ceph-mon[97472]: from='client.24278 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:18:20.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:19 smithi002 ceph-mon[95947]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:20.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:19 smithi002 ceph-mon[95947]: from='client.? 172.21.15.17:0/521315313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]: dispatch 2023-06-23T18:18:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:19 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]: dispatch 2023-06-23T18:18:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:19 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]': finished 2023-06-23T18:18:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:19 smithi002 ceph-mon[95947]: osdmap e48: 8 total, 7 up, 8 in 2023-06-23T18:18:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:20.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:19 smithi082 ceph-mon[97889]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:19 smithi082 ceph-mon[97889]: from='client.? 172.21.15.17:0/521315313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]: dispatch 2023-06-23T18:18:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:19 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]: dispatch 2023-06-23T18:18:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:19 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]': finished 2023-06-23T18:18:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:19 smithi082 ceph-mon[97889]: osdmap e48: 8 total, 7 up, 8 in 2023-06-23T18:18:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:20.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:19 smithi017 ceph-mon[97472]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:20.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:19 smithi017 ceph-mon[97472]: from='client.? 172.21.15.17:0/521315313' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]: dispatch 2023-06-23T18:18:20.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:19 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]: dispatch 2023-06-23T18:18:20.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:19 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d5b16bb7-b550-4400-846a-0d321e8af4c7"}]': finished 2023-06-23T18:18:20.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:19 smithi017 ceph-mon[97472]: osdmap e48: 8 total, 7 up, 8 in 2023-06-23T18:18:20.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:21.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:20 smithi002 ceph-mon[95947]: from='client.? 172.21.15.17:0/2821265317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:18:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:20 smithi082 ceph-mon[97889]: from='client.? 172.21.15.17:0/2821265317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:18:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:20 smithi017 ceph-mon[97472]: from='client.? 172.21.15.17:0/2821265317' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:18:22.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:21 smithi002 ceph-mon[95947]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:22.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:21 smithi082 ceph-mon[97889]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:22.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:21 smithi017 ceph-mon[97472]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:23.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:23 smithi017 ceph-mon[97472]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:24.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:23 smithi002 ceph-mon[95947]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:24.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:23 smithi082 ceph-mon[97889]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:25.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:25 smithi002 ceph-mon[95947]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:25.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:25 smithi082 ceph-mon[97889]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:25 smithi017 ceph-mon[97472]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:26.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:26 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-23T18:18:26.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:26 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:26.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:26 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-23T18:18:26.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:26 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:26.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:26 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-06-23T18:18:26.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:26 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:27 smithi017 ceph-mon[97472]: Deploying daemon osd.7 on smithi017 2023-06-23T18:18:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:27 smithi017 ceph-mon[97472]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:27.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:27 smithi002 ceph-mon[95947]: Deploying daemon osd.7 on smithi017 2023-06-23T18:18:27.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:27 smithi002 ceph-mon[95947]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:27.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:27 smithi082 ceph-mon[97889]: Deploying daemon osd.7 on smithi017 2023-06-23T18:18:27.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:27 smithi082 ceph-mon[97889]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:29.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:28 smithi002 ceph-mon[95947]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:28 smithi082 ceph-mon[97889]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:29.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:28 smithi017 ceph-mon[97472]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:31.433 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:31 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:31.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:31 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:31.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:31 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:31.434 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:31 smithi017 ceph-mon[97472]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:31.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:31 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:31.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:31 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:31.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:31 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:31.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:31 smithi082 ceph-mon[97889]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:31.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:31 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:31.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:31 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:31.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:31 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:31.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:31 smithi002 ceph-mon[95947]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:33.063 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.057+0000 7f05717fa700 1 -- 172.21.15.17:0/38540826 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0574060910 con 0x7f055c0a1150 2023-06-23T18:18:33.063 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 7 on host 'smithi017' 2023-06-23T18:18:33.064 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f055c0a1150 msgr2=0x7f055c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:33.064 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f055c0a1150 0x7f055c0a3600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0568005fd0 tx=0x7f056800a560 comp rx=0 tx=0).stop 2023-06-23T18:18:33.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05740ffb80 msgr2=0x7f0574195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:33.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05740ffb80 0x7f0574195850 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f056400c970 tx=0x7f056400cc80 comp rx=0 tx=0).stop 2023-06-23T18:18:33.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 shutdown_connections 2023-06-23T18:18:33.065 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0574101710 0x7f057419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:33.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f055c0a1150 0x7f055c0a3600 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:33.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05740ffb80 0x7f0574195850 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:33.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 --2- 172.21.15.17:0/38540826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0574100d80 0x7f0574195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:33.066 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 >> 172.21.15.17:0/38540826 conn(0x7f05740fb1f0 msgr2=0x7f05740fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:18:33.067 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 shutdown_connections 2023-06-23T18:18:33.067 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:18:33.059+0000 7f057a8cc700 1 -- 172.21.15.17:0/38540826 wait complete. 2023-06-23T18:18:33.424 DEBUG:teuthology.orchestra.run.smithi017:osd.7> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.7.service 2023-06-23T18:18:33.427 INFO:tasks.cephadm:Deploying osd.8 on smithi082 with /dev/vg_nvme/lv_2... 2023-06-23T18:18:33.427 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-06-23T18:18:33.457 INFO:journalctl@ceph.osd.7.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:18:33.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:33.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:33.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:33.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:33 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:33.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:33.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:33.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:33.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:33.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:33 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:33.989 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:18:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:34.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:34.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:34.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:34.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:34.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:34.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:33 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:34.690 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:18:34 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:18:34.291+0000 7eff2f971640 -1 osd.7 0 log_to_monitors true 2023-06-23T18:18:35.019 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:35 smithi082 ceph-mon[97889]: from='osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T18:18:35.019 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:35 smithi082 ceph-mon[97889]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T18:18:35.020 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:35 smithi082 ceph-mon[97889]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:35.020 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:35 smithi017 ceph-mon[97472]: from='osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T18:18:35.020 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:35 smithi017 ceph-mon[97472]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T18:18:35.020 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:35 smithi017 ceph-mon[97472]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:35.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:35 smithi002 ceph-mon[95947]: from='osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T18:18:35.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:35 smithi002 ceph-mon[95947]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-06-23T18:18:35.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:35 smithi002 ceph-mon[95947]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:35.440 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-06-23T18:18:35.440 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-06-23T18:18:35.440 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-06-23T18:18:35.440 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-06-23T18:18:35.440 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0109532 s, 957 MB/s 2023-06-23T18:18:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-06-23T18:18:35.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_2 2023-06-23T18:18:36.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-23T18:18:36.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: osdmap e49: 8 total, 7 up, 8 in 2023-06-23T18:18:36.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:36.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:18:36.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:18:36.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:36.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:36.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-23T18:18:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: osdmap e49: 8 total, 7 up, 8 in 2023-06-23T18:18:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:18:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:18:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:36.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:36.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.381 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:18:36.442 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:18:36.022+0000 7eff24fe9700 -1 osd.7 0 waiting for initial osdmap 2023-06-23T18:18:36.442 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:18:36.032+0000 7eff1f5db700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:18:36.443 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-06-23T18:18:36.443 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: osdmap e49: 8 total, 7 up, 8 in 2023-06-23T18:18:36.443 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:36.443 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:18:36.443 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:18:36.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:36.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:36.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:36.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:36.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:37.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:37 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi017 2023-06-23T18:18:37.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:37 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi017 to 5724M 2023-06-23T18:18:37.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:37 smithi002 ceph-mon[95947]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:18:37.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:37 smithi002 ceph-mon[95947]: osdmap e50: 8 total, 7 up, 8 in 2023-06-23T18:18:37.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:37.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:37.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:37 smithi002 ceph-mon[95947]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:37 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi017 2023-06-23T18:18:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:37 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi017 to 5724M 2023-06-23T18:18:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:37 smithi082 ceph-mon[97889]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:18:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:37 smithi082 ceph-mon[97889]: osdmap e50: 8 total, 7 up, 8 in 2023-06-23T18:18:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:37 smithi082 ceph-mon[97889]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:37.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:37 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi017 2023-06-23T18:18:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:37 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi017 to 5724M 2023-06-23T18:18:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:37 smithi017 ceph-mon[97472]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:18:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:37 smithi017 ceph-mon[97472]: osdmap e50: 8 total, 7 up, 8 in 2023-06-23T18:18:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:37 smithi017 ceph-mon[97472]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-06-23T18:18:37.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 -- 172.21.15.82:0/2473001659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10102f50 msgr2=0x7ffb10105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:37.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 --2- 172.21.15.82:0/2473001659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10102f50 0x7ffb10105330 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c009990 tx=0x7ffb0c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:18:37.562 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 -- 172.21.15.82:0/2473001659 shutdown_connections 2023-06-23T18:18:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 --2- 172.21.15.82:0/2473001659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb10105870 0x7ffb10107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 --2- 172.21.15.82:0/2473001659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb10100630 0x7ffb10102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 --2- 172.21.15.82:0/2473001659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10102f50 0x7ffb10105330 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 -- 172.21.15.82:0/2473001659 >> 172.21.15.82:0/2473001659 conn(0x7ffb100fa070 msgr2=0x7ffb100fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:18:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 -- 172.21.15.82:0/2473001659 shutdown_connections 2023-06-23T18:18:37.563 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.557+0000 7ffb17520700 1 -- 172.21.15.82:0/2473001659 wait complete. 2023-06-23T18:18:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb17520700 1 Processor -- start 2023-06-23T18:18:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb17520700 1 -- start start 2023-06-23T18:18:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb17520700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10100630 0x7ffb10195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb17520700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb10102f50 0x7ffb10195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:37.565 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb17520700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb10105870 0x7ffb1019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:37.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb17520700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1019d310 con 0x7ffb10105870 2023-06-23T18:18:37.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb17520700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1019d450 con 0x7ffb10102f50 2023-06-23T18:18:37.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb17520700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1019d5e0 con 0x7ffb10100630 2023-06-23T18:18:37.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10100630 0x7ffb10195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:37.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10100630 0x7ffb10195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40724/0 (socket says 172.21.15.82:40724) 2023-06-23T18:18:37.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 -- 172.21.15.82:0/1726393804 learned_addr learned my addr 172.21.15.82:0/1726393804 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:18:37.566 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb14abb700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb10102f50 0x7ffb10195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:37.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb15abd700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb10105870 0x7ffb1019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:37.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 -- 172.21.15.82:0/1726393804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb10102f50 msgr2=0x7ffb10195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:37.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb10102f50 0x7ffb10195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:37.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 -- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb10105870 msgr2=0x7ffb1019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:37.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb10105870 0x7ffb1019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:37.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 -- 172.21.15.82:0/1726393804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c0095f0 con 0x7ffb10100630 2023-06-23T18:18:37.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb15abd700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb10105870 0x7ffb1019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:18:37.567 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb152bc700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10100630 0x7ffb10195780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000b560 tx=0x7ffb0000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:18:37.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb067fc700 1 -- 172.21.15.82:0/1726393804 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00011830 con 0x7ffb10100630 2023-06-23T18:18:37.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.558+0000 7ffb067fc700 1 -- 172.21.15.82:0/1726393804 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb00011e20 con 0x7ffb10100630 2023-06-23T18:18:37.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.559+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1019d8c0 con 0x7ffb10100630 2023-06-23T18:18:37.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.559+0000 7ffb067fc700 1 -- 172.21.15.82:0/1726393804 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00010dd0 con 0x7ffb10100630 2023-06-23T18:18:37.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.559+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1019de90 con 0x7ffb10100630 2023-06-23T18:18:37.568 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.559+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb100fbbe0 con 0x7ffb10100630 2023-06-23T18:18:37.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.561+0000 7ffb067fc700 1 -- 172.21.15.82:0/1726393804 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ffb00011990 con 0x7ffb10100630 2023-06-23T18:18:37.569 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.561+0000 7ffb067fc700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffafc0a11d0 0x7ffafc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:37.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.561+0000 7ffb067fc700 1 -- 172.21.15.82:0/1726393804 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5430+0+0 (secure 0 0 0) 0x7ffb000cbb90 con 0x7ffb10100630 2023-06-23T18:18:37.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.561+0000 7ffb14abb700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffafc0a11d0 0x7ffafc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:37.570 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.562+0000 7ffb14abb700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffafc0a11d0 0x7ffafc0a3680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00b5c0 tx=0x7ffb0c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:18:37.571 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.565+0000 7ffb067fc700 1 -- 172.21.15.82:0/1726393804 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ffb00095e20 con 0x7ffb10100630 2023-06-23T18:18:37.764 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:37.759+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb10060910 con 0x7ffafc0a11d0 2023-06-23T18:18:38.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:38 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:18:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:38 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:18:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:38 smithi002 ceph-mon[95947]: osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] boot 2023-06-23T18:18:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:38 smithi002 ceph-mon[95947]: osdmap e51: 8 total, 8 up, 8 in 2023-06-23T18:18:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:18:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:18:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:38.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:38 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:18:38.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:38 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:18:38.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:38 smithi082 ceph-mon[97889]: osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] boot 2023-06-23T18:18:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:38 smithi082 ceph-mon[97889]: osdmap e51: 8 total, 8 up, 8 in 2023-06-23T18:18:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:18:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:18:38.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:38.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:38 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:18:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:38 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:18:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:38 smithi017 ceph-mon[97472]: osd.7 [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] boot 2023-06-23T18:18:38.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:38 smithi017 ceph-mon[97472]: osdmap e51: 8 total, 8 up, 8 in 2023-06-23T18:18:38.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-06-23T18:18:38.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:18:38.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:18:38.443 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:39.270 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:39 smithi082 ceph-mon[97889]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:18:39.270 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:39 smithi082 ceph-mon[97889]: osdmap e52: 8 total, 8 up, 8 in 2023-06-23T18:18:39.270 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:39 smithi082 ceph-mon[97889]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:39 smithi002 ceph-mon[95947]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:18:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:39 smithi002 ceph-mon[95947]: osdmap e52: 8 total, 8 up, 8 in 2023-06-23T18:18:39.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:39 smithi002 ceph-mon[95947]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:39.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:39 smithi017 ceph-mon[97472]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:18:39.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:39 smithi017 ceph-mon[97472]: osdmap e52: 8 total, 8 up, 8 in 2023-06-23T18:18:39.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:39 smithi017 ceph-mon[97472]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:40.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:40 smithi002 ceph-mon[95947]: osdmap e53: 8 total, 8 up, 8 in 2023-06-23T18:18:40.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:40 smithi082 ceph-mon[97889]: osdmap e53: 8 total, 8 up, 8 in 2023-06-23T18:18:40.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:40 smithi017 ceph-mon[97472]: osdmap e53: 8 total, 8 up, 8 in 2023-06-23T18:18:41.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:41 smithi002 ceph-mon[95947]: osdmap e54: 8 total, 8 up, 8 in 2023-06-23T18:18:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:41 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/3739809648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]: dispatch 2023-06-23T18:18:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:41 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]: dispatch 2023-06-23T18:18:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:41 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]': finished 2023-06-23T18:18:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:41 smithi002 ceph-mon[95947]: osdmap e55: 9 total, 8 up, 9 in 2023-06-23T18:18:41.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:41.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:41 smithi002 ceph-mon[95947]: pgmap v167: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:41.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:41 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/2305914035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:18:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:41 smithi082 ceph-mon[97889]: osdmap e54: 8 total, 8 up, 8 in 2023-06-23T18:18:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:41 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/3739809648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]: dispatch 2023-06-23T18:18:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:41 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]: dispatch 2023-06-23T18:18:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:41 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]': finished 2023-06-23T18:18:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:41 smithi082 ceph-mon[97889]: osdmap e55: 9 total, 8 up, 9 in 2023-06-23T18:18:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:41.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:41 smithi082 ceph-mon[97889]: pgmap v167: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:41.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:41 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/2305914035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:18:41.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:41 smithi017 ceph-mon[97472]: osdmap e54: 8 total, 8 up, 8 in 2023-06-23T18:18:41.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:41 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/3739809648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]: dispatch 2023-06-23T18:18:41.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:41 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]: dispatch 2023-06-23T18:18:41.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:41 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e7ac9f9-4c02-4639-ba72-7af18127d2eb"}]': finished 2023-06-23T18:18:41.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:41 smithi017 ceph-mon[97472]: osdmap e55: 9 total, 8 up, 9 in 2023-06-23T18:18:41.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:41.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:41 smithi017 ceph-mon[97472]: pgmap v167: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:41.446 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:41 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/2305914035' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:18:43.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:42 smithi002 ceph-mon[95947]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:42 smithi082 ceph-mon[97889]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:43.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:42 smithi017 ceph-mon[97472]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:45.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:45 smithi002 ceph-mon[95947]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:45.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:45 smithi082 ceph-mon[97889]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:45.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:45 smithi017 ceph-mon[97472]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:47.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:46 smithi002 ceph-mon[95947]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:47.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:46 smithi082 ceph-mon[97889]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:46 smithi017 ceph-mon[97472]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:48.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-23T18:18:48.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:48.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:47 smithi002 ceph-mon[95947]: Deploying daemon osd.8 on smithi082 2023-06-23T18:18:48.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-23T18:18:48.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:48.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:47 smithi082 ceph-mon[97889]: Deploying daemon osd.8 on smithi082 2023-06-23T18:18:48.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-06-23T18:18:48.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:48.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:47 smithi017 ceph-mon[97472]: Deploying daemon osd.8 on smithi082 2023-06-23T18:18:48.985 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:48 smithi082 ceph-mon[97889]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:49.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:48 smithi002 ceph-mon[95947]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:49.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:48 smithi017 ceph-mon[97472]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:50 smithi002 ceph-mon[95947]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:51.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:50 smithi082 ceph-mon[97889]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:51.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:50 smithi017 ceph-mon[97472]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:52.338 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:52.338 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:52.338 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:52.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:52.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:52.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:52.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:52.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:53.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:53 smithi082 ceph-mon[97889]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:53.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:53 smithi002 ceph-mon[95947]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:53.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:53 smithi017 ceph-mon[97472]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:53.981 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.976+0000 7ffb067fc700 1 -- 172.21.15.82:0/1726393804 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffb10060910 con 0x7ffafc0a11d0 2023-06-23T18:18:53.982 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 8 on host 'smithi082' 2023-06-23T18:18:53.983 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.978+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffafc0a11d0 msgr2=0x7ffafc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.978+0000 7ffb17520700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffafc0a11d0 0x7ffafc0a3680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00b5c0 tx=0x7ffb0c005fb0 comp rx=0 tx=0).stop 2023-06-23T18:18:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.978+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10100630 msgr2=0x7ffb10195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.978+0000 7ffb17520700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10100630 0x7ffb10195780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000b560 tx=0x7ffb0000b870 comp rx=0 tx=0).stop 2023-06-23T18:18:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.979+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 shutdown_connections 2023-06-23T18:18:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.979+0000 7ffb17520700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb10105870 0x7ffb1019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.979+0000 7ffb17520700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10100630 0x7ffb10195780 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:53.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.979+0000 7ffb17520700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffafc0a11d0 0x7ffafc0a3680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:53.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.979+0000 7ffb17520700 1 --2- 172.21.15.82:0/1726393804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb10102f50 0x7ffb10195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:53.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.979+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 >> 172.21.15.82:0/1726393804 conn(0x7ffb100fa070 msgr2=0x7ffb10101290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:18:53.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.979+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 shutdown_connections 2023-06-23T18:18:53.985 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:18:53.979+0000 7ffb17520700 1 -- 172.21.15.82:0/1726393804 wait complete. 2023-06-23T18:18:54.465 DEBUG:teuthology.orchestra.run.smithi082:osd.8> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.8.service 2023-06-23T18:18:54.468 INFO:tasks.cephadm:Deploying osd.9 on smithi002 with /dev/vg_nvme/lv_1... 2023-06-23T18:18:54.468 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-23T18:18:54.499 INFO:journalctl@ceph.osd.8.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:18:54.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:54.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:54.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:54.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:54.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:54.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:54.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:54.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:54 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:55.045 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:18:55.066 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.066 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.067 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:55.067 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:55.067 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.068 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.068 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.068 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:54 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:55.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:55.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:54 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:18:55.348 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:18:55 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:18:55.227+0000 7f0f0b1c2640 -1 osd.8 0 log_to_monitors true 2023-06-23T18:18:56.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:55 smithi002 ceph-mon[95947]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:56.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:55 smithi002 ceph-mon[95947]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T18:18:56.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:55 smithi082 ceph-mon[97889]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:56.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:55 smithi082 ceph-mon[97889]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T18:18:56.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:55 smithi017 ceph-mon[97472]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:56.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:55 smithi017 ceph-mon[97472]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-06-23T18:18:56.601 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-23T18:18:56.601 INFO:teuthology.orchestra.run.smithi002.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-23T18:18:56.602 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10+0 records in 2023-06-23T18:18:56.602 INFO:teuthology.orchestra.run.smithi002.stderr:10+0 records out 2023-06-23T18:18:56.602 INFO:teuthology.orchestra.run.smithi002.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134078 s, 782 MB/s 2023-06-23T18:18:56.602 INFO:teuthology.orchestra.run.smithi002.stderr:--> Zapping successful for: 2023-06-23T18:18:56.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:18:56.717+0000 7f0f0083a700 -1 osd.8 0 waiting for initial osdmap 2023-06-23T18:18:56.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:18:56.729+0000 7f0ef8e28700 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:18:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-23T18:18:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: osdmap e56: 9 total, 8 up, 9 in 2023-06-23T18:18:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:18:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:56.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:56.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:56.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:56.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:56.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:56.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:56.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:56.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:56.999 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-23T18:18:56.999 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: osdmap e56: 9 total, 8 up, 9 in 2023-06-23T18:18:56.999 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:56.999 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:18:57.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:57.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:57.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:57.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:57.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:57.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:57.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:57.000 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:57.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:57.010 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi002:vg_nvme/lv_1 2023-06-23T18:18:57.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-06-23T18:18:57.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: osdmap e56: 9 total, 8 up, 9 in 2023-06-23T18:18:57.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:57.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:18:57.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:57.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:57.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:57.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:57.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:18:57.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:57.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:18:57.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:18:57.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:18:57.567 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:18:58.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:57 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi082 2023-06-23T18:18:58.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:57 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi082 to 5722M 2023-06-23T18:18:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:57 smithi002 ceph-mon[95947]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:57 smithi002 ceph-mon[95947]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:18:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:57 smithi002 ceph-mon[95947]: osdmap e57: 9 total, 8 up, 9 in 2023-06-23T18:18:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:58.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:58.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:57 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi082 2023-06-23T18:18:58.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:57 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi082 to 5722M 2023-06-23T18:18:58.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:57 smithi082 ceph-mon[97889]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:58.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:57 smithi082 ceph-mon[97889]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:18:58.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:57 smithi082 ceph-mon[97889]: osdmap e57: 9 total, 8 up, 9 in 2023-06-23T18:18:58.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:57 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:58.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:57 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:58.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:57 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi082 2023-06-23T18:18:58.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:57 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi082 to 5722M 2023-06-23T18:18:58.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:57 smithi017 ceph-mon[97472]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:18:58.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:57 smithi017 ceph-mon[97472]: from='osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760]' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:18:58.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:57 smithi017 ceph-mon[97472]: osdmap e57: 9 total, 8 up, 9 in 2023-06-23T18:18:58.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:57 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:58.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:57 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:58.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.880+0000 7fab961a2700 1 -- 172.21.15.2:0/2463069685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab901005d0 msgr2=0x7fab90106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:58.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.880+0000 7fab961a2700 1 --2- 172.21.15.2:0/2463069685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab901005d0 0x7fab90106e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fab8800b0a0 tx=0x7fab8800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:18:58.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.880+0000 7fab961a2700 1 -- 172.21.15.2:0/2463069685 shutdown_connections 2023-06-23T18:18:58.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.881+0000 7fab961a2700 1 --2- 172.21.15.2:0/2463069685 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab901005d0 0x7fab90106e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:58.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.881+0000 7fab961a2700 1 --2- 172.21.15.2:0/2463069685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab900fea40 0x7fab900fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:58.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.881+0000 7fab961a2700 1 --2- 172.21.15.2:0/2463069685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab900ffc40 0x7fab90100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:58.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.881+0000 7fab961a2700 1 -- 172.21.15.2:0/2463069685 >> 172.21.15.2:0/2463069685 conn(0x7fab900fa070 msgr2=0x7fab900fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:18:58.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.881+0000 7fab961a2700 1 -- 172.21.15.2:0/2463069685 shutdown_connections 2023-06-23T18:18:58.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.881+0000 7fab961a2700 1 -- 172.21.15.2:0/2463069685 wait complete. 2023-06-23T18:18:58.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.881+0000 7fab961a2700 1 Processor -- start 2023-06-23T18:18:58.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab961a2700 1 -- start start 2023-06-23T18:18:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab961a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab900fea40 0x7fab90194760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:58.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab961a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab900ffc40 0x7fab90194ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:58.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab961a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab901005d0 0x7fab9019bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:58.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab961a2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9019c2f0 con 0x7fab900ffc40 2023-06-23T18:18:58.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab961a2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9019c430 con 0x7fab900fea40 2023-06-23T18:18:58.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab961a2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9019c5c0 con 0x7fab901005d0 2023-06-23T18:18:58.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab900ffc40 0x7fab90194ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:58.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab900ffc40 0x7fab90194ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:51868/0 (socket says 172.21.15.2:51868) 2023-06-23T18:18:58.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 -- 172.21.15.2:0/788536192 learned_addr learned my addr 172.21.15.2:0/788536192 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:18:58.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8f7fe700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab900fea40 0x7fab90194760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:58.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8ffff700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab901005d0 0x7fab9019bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:58.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 -- 172.21.15.2:0/788536192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab901005d0 msgr2=0x7fab9019bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:58.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab901005d0 0x7fab9019bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:58.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 -- 172.21.15.2:0/788536192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab900fea40 msgr2=0x7fab90194760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:18:58.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab900fea40 0x7fab90194760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:18:58.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 -- 172.21.15.2:0/788536192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab88009d00 con 0x7fab900ffc40 2023-06-23T18:18:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8f7fe700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab900fea40 0x7fab90194760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:18:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.882+0000 7fab8effd700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab900ffc40 0x7fab90194ca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab840099a0 tx=0x7fab84009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:18:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.883+0000 7fab8cff9700 1 -- 172.21.15.2:0/788536192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab84012070 con 0x7fab900ffc40 2023-06-23T18:18:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.883+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab90067ff0 con 0x7fab900ffc40 2023-06-23T18:18:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.883+0000 7fab8cff9700 1 -- 172.21.15.2:0/788536192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab8400d260 con 0x7fab900ffc40 2023-06-23T18:18:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.883+0000 7fab8cff9700 1 -- 172.21.15.2:0/788536192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8400ca20 con 0x7fab900ffc40 2023-06-23T18:18:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.883+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab900684f0 con 0x7fab900ffc40 2023-06-23T18:18:58.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.884+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9010a110 con 0x7fab900ffc40 2023-06-23T18:18:58.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.885+0000 7fab8cff9700 1 -- 172.21.15.2:0/788536192 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fab8400cb80 con 0x7fab900ffc40 2023-06-23T18:18:58.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.885+0000 7fab8cff9700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fab780a1150 0x7fab780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:18:58.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.886+0000 7fab8f7fe700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fab780a1150 0x7fab780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:18:58.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.886+0000 7fab8cff9700 1 -- 172.21.15.2:0/788536192 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5858+0+0 (secure 0 0 0) 0x7fab840cd070 con 0x7fab900ffc40 2023-06-23T18:18:58.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.886+0000 7fab8f7fe700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fab780a1150 0x7fab780a3600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab900ffaa0 tx=0x7fab8000b670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:18:58.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:58.889+0000 7fab8cff9700 1 -- 172.21.15.2:0/788536192 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fab840970b0 con 0x7fab900ffc40 2023-06-23T18:18:59.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:58 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:18:59.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:58 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:18:59.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:58 smithi002 ceph-mon[95947]: osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] boot 2023-06-23T18:18:59.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:58 smithi002 ceph-mon[95947]: osdmap e58: 9 total, 9 up, 9 in 2023-06-23T18:18:59.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:58 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:18:59.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:58 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:18:59.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:58 smithi082 ceph-mon[97889]: osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] boot 2023-06-23T18:18:59.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:58 smithi082 ceph-mon[97889]: osdmap e58: 9 total, 9 up, 9 in 2023-06-23T18:18:59.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:18:59.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:18:59.097+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fab90060910 con 0x7fab780a1150 2023-06-23T18:18:59.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:58 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:18:59.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:58 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:18:59.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:58 smithi017 ceph-mon[97472]: osd.8 [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] boot 2023-06-23T18:18:59.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:58 smithi017 ceph-mon[97472]: osdmap e58: 9 total, 9 up, 9 in 2023-06-23T18:18:59.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-06-23T18:19:00.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:59 smithi002 ceph-mon[95947]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:19:00.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:59 smithi002 ceph-mon[95947]: osdmap e59: 9 total, 9 up, 9 in 2023-06-23T18:19:00.001 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:00.002 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:00.002 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:18:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:00.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:59 smithi082 ceph-mon[97889]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:19:00.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:59 smithi082 ceph-mon[97889]: osdmap e59: 9 total, 9 up, 9 in 2023-06-23T18:19:00.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:00.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:00.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:18:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:00.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:59 smithi017 ceph-mon[97472]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-06-23T18:19:00.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:59 smithi017 ceph-mon[97472]: osdmap e59: 9 total, 9 up, 9 in 2023-06-23T18:19:00.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:00.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:00.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:18:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:01.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:00 smithi002 ceph-mon[95947]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:01.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:00 smithi002 ceph-mon[95947]: osdmap e60: 9 total, 9 up, 9 in 2023-06-23T18:19:01.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:00 smithi002 ceph-mon[95947]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:00 smithi082 ceph-mon[97889]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:01.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:00 smithi082 ceph-mon[97889]: osdmap e60: 9 total, 9 up, 9 in 2023-06-23T18:19:01.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:00 smithi082 ceph-mon[97889]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:00 smithi017 ceph-mon[97472]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi002:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:00 smithi017 ceph-mon[97472]: osdmap e60: 9 total, 9 up, 9 in 2023-06-23T18:19:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:00 smithi017 ceph-mon[97472]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:02.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:01 smithi002 ceph-mon[95947]: osdmap e61: 9 total, 9 up, 9 in 2023-06-23T18:19:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:01 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3454867776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9013ccc7-bc53-477d-8a23-2806313db552"}]: dispatch 2023-06-23T18:19:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:01 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3454867776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9013ccc7-bc53-477d-8a23-2806313db552"}]': finished 2023-06-23T18:19:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:01 smithi002 ceph-mon[95947]: osdmap e62: 10 total, 9 up, 10 in 2023-06-23T18:19:02.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:02.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:01 smithi082 ceph-mon[97889]: osdmap e61: 9 total, 9 up, 9 in 2023-06-23T18:19:02.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:01 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3454867776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9013ccc7-bc53-477d-8a23-2806313db552"}]: dispatch 2023-06-23T18:19:02.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:01 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3454867776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9013ccc7-bc53-477d-8a23-2806313db552"}]': finished 2023-06-23T18:19:02.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:01 smithi082 ceph-mon[97889]: osdmap e62: 10 total, 9 up, 10 in 2023-06-23T18:19:02.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:02.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:01 smithi017 ceph-mon[97472]: osdmap e61: 9 total, 9 up, 9 in 2023-06-23T18:19:02.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:01 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3454867776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9013ccc7-bc53-477d-8a23-2806313db552"}]: dispatch 2023-06-23T18:19:02.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:01 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3454867776' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9013ccc7-bc53-477d-8a23-2806313db552"}]': finished 2023-06-23T18:19:02.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:01 smithi017 ceph-mon[97472]: osdmap e62: 10 total, 9 up, 10 in 2023-06-23T18:19:02.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:02 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3828159475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:02 smithi002 ceph-mon[95947]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:19:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:02 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3828159475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:03.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:02 smithi082 ceph-mon[97889]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:19:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:02 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3828159475' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:03.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:02 smithi017 ceph-mon[97472]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-06-23T18:19:05.297 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:05 smithi002 ceph-mon[95947]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T18:19:05.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:05 smithi082 ceph-mon[97889]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T18:19:05.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:05 smithi017 ceph-mon[97472]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T18:19:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:06 smithi002 ceph-mon[95947]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-06-23T18:19:07.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:06 smithi082 ceph-mon[97889]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-06-23T18:19:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:06 smithi017 ceph-mon[97472]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2023-06-23T18:19:08.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:08 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-23T18:19:08.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:08 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:08.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:08 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-23T18:19:08.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:08 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:08.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:08 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-06-23T18:19:08.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:08 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:09.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:09 smithi002 ceph-mon[95947]: Deploying daemon osd.9 on smithi002 2023-06-23T18:19:09.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:09 smithi002 ceph-mon[95947]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T18:19:09.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:09 smithi082 ceph-mon[97889]: Deploying daemon osd.9 on smithi002 2023-06-23T18:19:09.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:09 smithi082 ceph-mon[97889]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T18:19:09.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:09 smithi017 ceph-mon[97472]: Deploying daemon osd.9 on smithi002 2023-06-23T18:19:09.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:09 smithi017 ceph-mon[97472]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-06-23T18:19:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:10 smithi002 ceph-mon[95947]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T18:19:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:10 smithi082 ceph-mon[97889]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T18:19:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:10 smithi017 ceph-mon[97472]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T18:19:12.867 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:12 smithi002 ceph-mon[95947]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-06-23T18:19:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:12 smithi082 ceph-mon[97889]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-06-23T18:19:13.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:12 smithi017 ceph-mon[97472]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2023-06-23T18:19:14.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:14.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:14.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:14.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:14.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:14.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:14.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:14.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:14.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:15.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:15 smithi002 ceph-mon[95947]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:15.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:15 smithi082 ceph-mon[97889]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:15.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:15 smithi017 ceph-mon[97472]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:16.061 INFO:teuthology.orchestra.run.smithi002.stdout:Created osd(s) 9 on host 'smithi002' 2023-06-23T18:19:16.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.056+0000 7fab8cff9700 1 -- 172.21.15.2:0/788536192 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fab90060910 con 0x7fab780a1150 2023-06-23T18:19:16.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fab780a1150 msgr2=0x7fab780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fab780a1150 0x7fab780a3600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fab900ffaa0 tx=0x7fab8000b670 comp rx=0 tx=0).stop 2023-06-23T18:19:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab900ffc40 msgr2=0x7fab90194ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab900ffc40 0x7fab90194ca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fab840099a0 tx=0x7fab84009d60 comp rx=0 tx=0).stop 2023-06-23T18:19:16.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 shutdown_connections 2023-06-23T18:19:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab901005d0 0x7fab9019bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fab780a1150 0x7fab780a3600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:16.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab900fea40 0x7fab90194760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 --2- 172.21.15.2:0/788536192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fab900ffc40 0x7fab90194ca0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 >> 172.21.15.2:0/788536192 conn(0x7fab900fa070 msgr2=0x7fab900fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 shutdown_connections 2023-06-23T18:19:16.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:19:16.058+0000 7fab961a2700 1 -- 172.21.15.2:0/788536192 wait complete. 2023-06-23T18:19:16.451 DEBUG:teuthology.orchestra.run.smithi002:osd.9> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.9.service 2023-06-23T18:19:16.453 INFO:tasks.cephadm:Deploying osd.10 on smithi017 with /dev/vg_nvme/lv_1... 2023-06-23T18:19:16.454 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-23T18:19:16.488 INFO:journalctl@ceph.osd.9.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:19:16.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:19:16.540+0000 7f7e0f60d640 -1 osd.9 0 log_to_monitors true 2023-06-23T18:19:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:16.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:16 smithi002 ceph-mon[95947]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-23T18:19:16.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:16.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:16.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:16.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:16.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:16 smithi017 ceph-mon[97472]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-23T18:19:17.026 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:19:17.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:17.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:17.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:17.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:17.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:17.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:17.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:17.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:17.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:16 smithi082 ceph-mon[97889]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-06-23T18:19:17.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:17 smithi002 ceph-mon[95947]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:17.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:17 smithi002 ceph-mon[95947]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-23T18:19:17.772 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:17 smithi002 ceph-mon[95947]: osdmap e63: 10 total, 9 up, 10 in 2023-06-23T18:19:17.773 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:17.773 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:17 smithi002 ceph-mon[95947]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:19:17.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:17 smithi017 ceph-mon[97472]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:17.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:17 smithi017 ceph-mon[97472]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-23T18:19:17.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:17 smithi017 ceph-mon[97472]: osdmap e63: 10 total, 9 up, 10 in 2023-06-23T18:19:17.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:17.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:17 smithi017 ceph-mon[97472]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:19:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:17 smithi082 ceph-mon[97889]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:18.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:17 smithi082 ceph-mon[97889]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-06-23T18:19:18.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:17 smithi082 ceph-mon[97889]: osdmap e63: 10 total, 9 up, 10 in 2023-06-23T18:19:18.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:18.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:17 smithi082 ceph-mon[97889]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]: dispatch 2023-06-23T18:19:18.401 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:19:18 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:19:18.058+0000 7f7e04c85700 -1 osd.9 0 waiting for initial osdmap 2023-06-23T18:19:18.401 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:19:18 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:19:18.070+0000 7f7dfea76700 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:19:18.544 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-23T18:19:18.545 INFO:teuthology.orchestra.run.smithi017.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-23T18:19:18.545 INFO:teuthology.orchestra.run.smithi017.stderr: stderr: 10+0 records in 2023-06-23T18:19:18.545 INFO:teuthology.orchestra.run.smithi017.stderr:10+0 records out 2023-06-23T18:19:18.545 INFO:teuthology.orchestra.run.smithi017.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00792403 s, 1.3 GB/s 2023-06-23T18:19:18.545 INFO:teuthology.orchestra.run.smithi017.stderr:--> Zapping successful for: 2023-06-23T18:19:19.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi017:vg_nvme/lv_1 2023-06-23T18:19:19.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:19:19.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: osdmap e64: 10 total, 9 up, 10 in 2023-06-23T18:19:19.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:19.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:19.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:19.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.352 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.352 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.353 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:19.353 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:19.353 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:19:19.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: osdmap e64: 10 total, 9 up, 10 in 2023-06-23T18:19:19.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:19.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:19.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:19.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:19.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:19.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi002", "root=default"]}]': finished 2023-06-23T18:19:19.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: osdmap e64: 10 total, 9 up, 10 in 2023-06-23T18:19:19.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:19.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-06-23T18:19:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:19.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:19.555 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:19.634 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:19:20.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:20 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:19:20.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:20 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:19:20.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:20 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi002 2023-06-23T18:19:20.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:20 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi002 to 4293M 2023-06-23T18:19:20.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:20 smithi017 ceph-mon[97472]: osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] boot 2023-06-23T18:19:20.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:20 smithi017 ceph-mon[97472]: osdmap e65: 10 total, 10 up, 10 in 2023-06-23T18:19:20.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:20 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:20.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:20 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:19:20.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:20 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:19:20.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:20 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi002 2023-06-23T18:19:20.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:20 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi002 to 4293M 2023-06-23T18:19:20.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:20 smithi082 ceph-mon[97889]: osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] boot 2023-06-23T18:19:20.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:20 smithi082 ceph-mon[97889]: osdmap e65: 10 total, 10 up, 10 in 2023-06-23T18:19:20.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:20 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:20.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:20 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:19:20.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:20 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:19:20.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:20 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi002 2023-06-23T18:19:20.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:20 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi002 to 4293M 2023-06-23T18:19:20.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:20 smithi002 ceph-mon[95947]: osd.9 [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] boot 2023-06-23T18:19:20.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:20 smithi002 ceph-mon[95947]: osdmap e65: 10 total, 10 up, 10 in 2023-06-23T18:19:20.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:20 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-06-23T18:19:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 -- 172.21.15.17:0/2636268332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 msgr2=0x7fe23c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 --2- 172.21.15.17:0/2636268332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 0x7fe23c100090 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe230009990 tx=0x7fe230009ca0 comp rx=0 tx=0).stop 2023-06-23T18:19:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 -- 172.21.15.17:0/2636268332 shutdown_connections 2023-06-23T18:19:20.974 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 --2- 172.21.15.17:0/2636268332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe23c1005d0 0x7fe23c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 --2- 172.21.15.17:0/2636268332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c0fea40 0x7fe23c0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 --2- 172.21.15.17:0/2636268332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 0x7fe23c100090 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 -- 172.21.15.17:0/2636268332 >> 172.21.15.17:0/2636268332 conn(0x7fe23c0fa070 msgr2=0x7fe23c0fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:19:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 -- 172.21.15.17:0/2636268332 shutdown_connections 2023-06-23T18:19:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 -- 172.21.15.17:0/2636268332 wait complete. 2023-06-23T18:19:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.968+0000 7fe24219d700 1 Processor -- start 2023-06-23T18:19:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe24219d700 1 -- start start 2023-06-23T18:19:20.975 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe24219d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe23c0fea40 0x7fe23c1956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:19:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe24219d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 0x7fe23c195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:19:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe24219d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1005d0 0x7fe23c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:19:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe24219d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c19d250 con 0x7fe23c0fea40 2023-06-23T18:19:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe24219d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c19d390 con 0x7fe23c0ffc40 2023-06-23T18:19:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe24219d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23c19d520 con 0x7fe23c1005d0 2023-06-23T18:19:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 0x7fe23c195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:19:20.976 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 0x7fe23c195c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33336/0 (socket says 172.21.15.17:33336) 2023-06-23T18:19:20.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 -- 172.21.15.17:0/2896054042 learned_addr learned my addr 172.21.15.17:0/2896054042 (peer_addr_for_me v2:172.21.15.17:0/0) 2023-06-23T18:19:20.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23bfff700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1005d0 0x7fe23c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:19:20.977 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23b7fe700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe23c0fea40 0x7fe23c1956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:19:20.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 -- 172.21.15.17:0/2896054042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1005d0 msgr2=0x7fe23c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:20.978 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1005d0 0x7fe23c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:20.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 -- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe23c0fea40 msgr2=0x7fe23c1956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:20.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe23c0fea40 0x7fe23c1956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:20.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 -- 172.21.15.17:0/2896054042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2300095f0 con 0x7fe23c0ffc40 2023-06-23T18:19:20.979 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23bfff700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1005d0 0x7fe23c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:19:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23b7fe700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe23c0fea40 0x7fe23c1956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:19:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe23affd700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 0x7fe23c195c00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe230006010 tx=0x7fe230004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:19:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.969+0000 7fe238ff9700 1 -- 172.21.15.17:0/2896054042 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23001d070 con 0x7fe23c0ffc40 2023-06-23T18:19:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.970+0000 7fe238ff9700 1 -- 172.21.15.17:0/2896054042 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2300041b0 con 0x7fe23c0ffc40 2023-06-23T18:19:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.970+0000 7fe238ff9700 1 -- 172.21.15.17:0/2896054042 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe230022da0 con 0x7fe23c0ffc40 2023-06-23T18:19:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.970+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23c19d7a0 con 0x7fe23c0ffc40 2023-06-23T18:19:20.980 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.970+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23c19dbb0 con 0x7fe23c0ffc40 2023-06-23T18:19:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.970+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23c105fd0 con 0x7fe23c0ffc40 2023-06-23T18:19:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.972+0000 7fe238ff9700 1 -- 172.21.15.17:0/2896054042 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe230022570 con 0x7fe23c0ffc40 2023-06-23T18:19:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.972+0000 7fe238ff9700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe2240a11d0 0x7fe2240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:19:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.972+0000 7fe238ff9700 1 -- 172.21.15.17:0/2896054042 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fe2300cd980 con 0x7fe23c0ffc40 2023-06-23T18:19:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.973+0000 7fe23b7fe700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe2240a11d0 0x7fe2240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:19:20.981 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.973+0000 7fe23b7fe700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe2240a11d0 0x7fe2240a3680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0ffaa0 tx=0x7fe22c00a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:19:20.982 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:20.976+0000 7fe238ff9700 1 -- 172.21.15.17:0/2896054042 <== mon.2 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe230097960 con 0x7fe23c0ffc40 2023-06-23T18:19:21.188 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:21.182+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe23c060910 con 0x7fe2240a11d0 2023-06-23T18:19:21.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:21 smithi082 ceph-mon[97889]: osdmap e66: 10 total, 10 up, 10 in 2023-06-23T18:19:21.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:21 smithi082 ceph-mon[97889]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:21.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:21 smithi017 ceph-mon[97472]: osdmap e66: 10 total, 10 up, 10 in 2023-06-23T18:19:21.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:21 smithi017 ceph-mon[97472]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:21.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:21 smithi002 ceph-mon[95947]: osdmap e66: 10 total, 10 up, 10 in 2023-06-23T18:19:21.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:21 smithi002 ceph-mon[95947]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:22.311 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:22 smithi017 ceph-mon[97472]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:22.311 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:22.312 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:22.312 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:22.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:22 smithi082 ceph-mon[97889]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:22.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:22.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:22.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:22 smithi002 ceph-mon[95947]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi017:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:22.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:23.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:23 smithi082 ceph-mon[97889]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:23.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:23 smithi017 ceph-mon[97472]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:23.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:23 smithi002 ceph-mon[95947]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:24.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:24 smithi082 ceph-mon[97889]: from='client.? 172.21.15.17:0/2434029740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]: dispatch 2023-06-23T18:19:24.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:24 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]: dispatch 2023-06-23T18:19:24.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:24 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]': finished 2023-06-23T18:19:24.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:24 smithi082 ceph-mon[97889]: osdmap e67: 11 total, 10 up, 11 in 2023-06-23T18:19:24.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:24.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:24 smithi017 ceph-mon[97472]: from='client.? 172.21.15.17:0/2434029740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]: dispatch 2023-06-23T18:19:24.444 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:24 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]: dispatch 2023-06-23T18:19:24.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:24 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]': finished 2023-06-23T18:19:24.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:24 smithi017 ceph-mon[97472]: osdmap e67: 11 total, 10 up, 11 in 2023-06-23T18:19:24.445 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:24.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:24 smithi002 ceph-mon[95947]: from='client.? 172.21.15.17:0/2434029740' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]: dispatch 2023-06-23T18:19:24.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:24 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]: dispatch 2023-06-23T18:19:24.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:24 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9ab48541-33f0-44f9-8d30-727fd67ecbe4"}]': finished 2023-06-23T18:19:24.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:24 smithi002 ceph-mon[95947]: osdmap e67: 11 total, 10 up, 11 in 2023-06-23T18:19:24.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:25.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:25 smithi082 ceph-mon[97889]: from='client.? 172.21.15.17:0/597026644' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:25.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:25 smithi082 ceph-mon[97889]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:25 smithi017 ceph-mon[97472]: from='client.? 172.21.15.17:0/597026644' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:25 smithi017 ceph-mon[97472]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:25.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:25 smithi002 ceph-mon[95947]: from='client.? 172.21.15.17:0/597026644' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:25.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:25 smithi002 ceph-mon[95947]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:27.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:26 smithi002 ceph-mon[95947]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:26 smithi082 ceph-mon[97889]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:26 smithi017 ceph-mon[97472]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:28 smithi017 ceph-mon[97472]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:29.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:28 smithi002 ceph-mon[95947]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:28 smithi082 ceph-mon[97889]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-23T18:19:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-23T18:19:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-06-23T18:19:30.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:31.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:31 smithi017 ceph-mon[97472]: Deploying daemon osd.10 on smithi017 2023-06-23T18:19:31.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:31 smithi017 ceph-mon[97472]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:31.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:31 smithi002 ceph-mon[95947]: Deploying daemon osd.10 on smithi017 2023-06-23T18:19:31.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:31 smithi002 ceph-mon[95947]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:31.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:31 smithi082 ceph-mon[97889]: Deploying daemon osd.10 on smithi017 2023-06-23T18:19:31.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:31 smithi082 ceph-mon[97889]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:32.832 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:32 smithi017 ceph-mon[97472]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:33.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:32 smithi002 ceph-mon[95947]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:33.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:32 smithi082 ceph-mon[97889]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:35.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:35 smithi002 ceph-mon[95947]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:35.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:35 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:35.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:35 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:35.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:35 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:35.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:35 smithi082 ceph-mon[97889]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:35.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:35 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:35.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:35 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:35.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:35 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:35.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:35 smithi017 ceph-mon[97472]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:35.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:35 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:35.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:35 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:35.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:35 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:37.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:36 smithi002 ceph-mon[95947]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:36 smithi082 ceph-mon[97889]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:37.151 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:36 smithi017 ceph-mon[97472]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:37.462 INFO:teuthology.orchestra.run.smithi017.stdout:Created osd(s) 10 on host 'smithi017' 2023-06-23T18:19:37.462 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.457+0000 7fe238ff9700 1 -- 172.21.15.17:0/2896054042 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fe23c060910 con 0x7fe2240a11d0 2023-06-23T18:19:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe2240a11d0 msgr2=0x7fe2240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:37.464 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe2240a11d0 0x7fe2240a3680 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0ffaa0 tx=0x7fe22c00a680 comp rx=0 tx=0).stop 2023-06-23T18:19:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 msgr2=0x7fe23c195c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 0x7fe23c195c00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe230006010 tx=0x7fe230004730 comp rx=0 tx=0).stop 2023-06-23T18:19:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 shutdown_connections 2023-06-23T18:19:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe23c1005d0 0x7fe23c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe23c0fea40 0x7fe23c1956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe2240a11d0 0x7fe2240a3680 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 --2- 172.21.15.17:0/2896054042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe23c0ffc40 0x7fe23c195c00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 >> 172.21.15.17:0/2896054042 conn(0x7fe23c0fa070 msgr2=0x7fe23c103110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:19:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 shutdown_connections 2023-06-23T18:19:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2023-06-23T18:19:37.459+0000 7fe24219d700 1 -- 172.21.15.17:0/2896054042 wait complete. 2023-06-23T18:19:37.934 DEBUG:teuthology.orchestra.run.smithi017:osd.10> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.10.service 2023-06-23T18:19:37.937 INFO:tasks.cephadm:Deploying osd.11 on smithi082 with /dev/vg_nvme/lv_1... 2023-06-23T18:19:37.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-06-23T18:19:37.969 INFO:journalctl@ceph.osd.10.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:19:38.502 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:19:38.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.751 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.752 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.752 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.752 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:38.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:39.100 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:19:38 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:19:38.789+0000 7f16c24d2640 -1 osd.10 0 log_to_monitors true 2023-06-23T18:19:39.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:39 smithi002 ceph-mon[95947]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:39.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:39 smithi002 ceph-mon[95947]: from='osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T18:19:39.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:39 smithi002 ceph-mon[95947]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T18:19:39.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:39 smithi082 ceph-mon[97889]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:39.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:39 smithi082 ceph-mon[97889]: from='osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T18:19:39.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:39 smithi082 ceph-mon[97889]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T18:19:39.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:39 smithi017 ceph-mon[97472]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:39.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:39 smithi017 ceph-mon[97472]: from='osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T18:19:39.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:39 smithi017 ceph-mon[97472]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-06-23T18:19:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-06-23T18:19:40.075 INFO:teuthology.orchestra.run.smithi082.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-06-23T18:19:40.075 INFO:teuthology.orchestra.run.smithi082.stderr: stderr: 10+0 records in 2023-06-23T18:19:40.076 INFO:teuthology.orchestra.run.smithi082.stderr:10+0 records out 2023-06-23T18:19:40.076 INFO:teuthology.orchestra.run.smithi082.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150012 s, 699 MB/s 2023-06-23T18:19:40.076 INFO:teuthology.orchestra.run.smithi082.stderr:--> Zapping successful for: 2023-06-23T18:19:40.462 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch daemon add osd smithi082:vg_nvme/lv_1 2023-06-23T18:19:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:40 smithi002 ceph-mon[95947]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-23T18:19:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:40 smithi002 ceph-mon[95947]: osdmap e68: 11 total, 10 up, 11 in 2023-06-23T18:19:40.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:40.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:40 smithi002 ceph-mon[95947]: from='osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:19:40.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:40 smithi002 ceph-mon[95947]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:19:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:40 smithi082 ceph-mon[97889]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-23T18:19:40.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:40 smithi082 ceph-mon[97889]: osdmap e68: 11 total, 10 up, 11 in 2023-06-23T18:19:40.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:40.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:40 smithi082 ceph-mon[97889]: from='osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:19:40.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:40 smithi082 ceph-mon[97889]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:19:40.940 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:19:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:19:40.451+0000 7f16b7b4a700 -1 osd.10 0 waiting for initial osdmap 2023-06-23T18:19:40.941 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:19:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:19:40.461+0000 7f16af937700 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:19:40.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:40 smithi017 ceph-mon[97472]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-06-23T18:19:40.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:40 smithi017 ceph-mon[97472]: osdmap e68: 11 total, 10 up, 11 in 2023-06-23T18:19:40.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:40.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:40 smithi017 ceph-mon[97472]: from='osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:19:40.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:40 smithi017 ceph-mon[97472]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]: dispatch 2023-06-23T18:19:41.016 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:19:41.526 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:19:41.526 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: osdmap e69: 11 total, 10 up, 11 in 2023-06-23T18:19:41.527 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:41.527 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:41.527 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi017 2023-06-23T18:19:41.527 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.527 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.527 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.527 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi017 to 4293M 2023-06-23T18:19:41.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:41.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:41.528 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:41.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:41.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:41.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:41.529 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:19:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: osdmap e69: 11 total, 10 up, 11 in 2023-06-23T18:19:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi017 2023-06-23T18:19:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi017 to 4293M 2023-06-23T18:19:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:41.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:41.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:41.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:41.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:41.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:41.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi017", "root=default"]}]': finished 2023-06-23T18:19:41.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: osdmap e69: 11 total, 10 up, 11 in 2023-06-23T18:19:41.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:41.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:41.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi017 2023-06-23T18:19:41.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:19:41.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi017 to 4293M 2023-06-23T18:19:41.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:41.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:41.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:41.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-06-23T18:19:41.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:41.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:41.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:41.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:42.209 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 -- 172.21.15.82:0/3140781811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 msgr2=0x7fa8701052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:42.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 --2- 172.21.15.82:0/3140781811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 0x7fa8701052d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa860009990 tx=0x7fa860009ca0 comp rx=0 tx=0).stop 2023-06-23T18:19:42.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 -- 172.21.15.82:0/3140781811 shutdown_connections 2023-06-23T18:19:42.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 --2- 172.21.15.82:0/3140781811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870105810 0x7fa870107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:42.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 --2- 172.21.15.82:0/3140781811 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8701005d0 0x7fa8701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:42.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 --2- 172.21.15.82:0/3140781811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 0x7fa8701052d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:42.210 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 -- 172.21.15.82:0/3140781811 >> 172.21.15.82:0/3140781811 conn(0x7fa8700fa070 msgr2=0x7fa8700fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:19:42.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 -- 172.21.15.82:0/3140781811 shutdown_connections 2023-06-23T18:19:42.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.205+0000 7fa8759a4700 1 -- 172.21.15.82:0/3140781811 wait complete. 2023-06-23T18:19:42.211 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa8759a4700 1 Processor -- start 2023-06-23T18:19:42.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa8759a4700 1 -- start start 2023-06-23T18:19:42.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa8759a4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8701005d0 0x7fa870195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:19:42.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa8759a4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 0x7fa870195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:19:42.213 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa8759a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870105810 0x7fa87019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:19:42.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa8759a4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019d290 con 0x7fa870105810 2023-06-23T18:19:42.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa8759a4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019d3d0 con 0x7fa8701005d0 2023-06-23T18:19:42.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa8759a4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019d560 con 0x7fa870102ef0 2023-06-23T18:19:42.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 0x7fa870195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:19:42.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 0x7fa870195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40876/0 (socket says 172.21.15.82:40876) 2023-06-23T18:19:42.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86e7fc700 1 -- 172.21.15.82:0/1956089439 learned_addr learned my addr 172.21.15.82:0/1956089439 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:19:42.214 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86f7fe700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870105810 0x7fa87019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:19:42.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86effd700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8701005d0 0x7fa870195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:19:42.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86e7fc700 1 -- 172.21.15.82:0/1956089439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8701005d0 msgr2=0x7fa870195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:42.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86e7fc700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8701005d0 0x7fa870195700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:42.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86e7fc700 1 -- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870105810 msgr2=0x7fa87019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:42.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86e7fc700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870105810 0x7fa87019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:42.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.206+0000 7fa86e7fc700 1 -- 172.21.15.82:0/1956089439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600095f0 con 0x7fa870102ef0 2023-06-23T18:19:42.215 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.207+0000 7fa86e7fc700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 0x7fa870195c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8600059d0 tx=0x7fa860004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:19:42.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.207+0000 7fa8749a2700 1 -- 172.21.15.82:0/1956089439 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86001d070 con 0x7fa870102ef0 2023-06-23T18:19:42.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.207+0000 7fa8749a2700 1 -- 172.21.15.82:0/1956089439 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa860004080 con 0x7fa870102ef0 2023-06-23T18:19:42.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.207+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87019d7e0 con 0x7fa870102ef0 2023-06-23T18:19:42.216 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.207+0000 7fa8749a2700 1 -- 172.21.15.82:0/1956089439 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860022c50 con 0x7fa870102ef0 2023-06-23T18:19:42.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.207+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87019dbf0 con 0x7fa870102ef0 2023-06-23T18:19:42.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.208+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8700fbbe0 con 0x7fa870102ef0 2023-06-23T18:19:42.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.209+0000 7fa8749a2700 1 -- 172.21.15.82:0/1956089439 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa860022470 con 0x7fa870102ef0 2023-06-23T18:19:42.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.209+0000 7fa8749a2700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa8580a1150 0x7fa8580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:19:42.217 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.209+0000 7fa8749a2700 1 -- 172.21.15.82:0/1956089439 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6727+0+0 (secure 0 0 0) 0x7fa8600cda20 con 0x7fa870102ef0 2023-06-23T18:19:42.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.210+0000 7fa86effd700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa8580a1150 0x7fa8580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:19:42.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.210+0000 7fa86effd700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa8580a1150 0x7fa8580a3600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa864005950 tx=0x7fa8640058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:19:42.218 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.213+0000 7fa8749a2700 1 -- 172.21.15.82:0/1956089439 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa860097830 con 0x7fa870102ef0 2023-06-23T18:19:42.412 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:42.408+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa870060910 con 0x7fa8580a1150 2023-06-23T18:19:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:19:42.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:19:42.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:42.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] boot 2023-06-23T18:19:42.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: osdmap e70: 11 total, 11 up, 11 in 2023-06-23T18:19:42.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:42.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:42.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:42.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:19:42.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:19:42.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:42.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] boot 2023-06-23T18:19:42.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: osdmap e70: 11 total, 11 up, 11 in 2023-06-23T18:19:42.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:42.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:42.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:42.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:19:42.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:19:42.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:42.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: osd.10 [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] boot 2023-06-23T18:19:42.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: osdmap e70: 11 total, 11 up, 11 in 2023-06-23T18:19:42.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-06-23T18:19:42.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-06-23T18:19:42.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-06-23T18:19:42.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:43.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:43 smithi002 ceph-mon[95947]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:43 smithi002 ceph-mon[95947]: osdmap e71: 11 total, 11 up, 11 in 2023-06-23T18:19:43.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:43 smithi002 ceph-mon[95947]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:43.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:43 smithi082 ceph-mon[97889]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:43.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:43 smithi082 ceph-mon[97889]: osdmap e71: 11 total, 11 up, 11 in 2023-06-23T18:19:43.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:43 smithi082 ceph-mon[97889]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:43.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:43 smithi017 ceph-mon[97472]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi082:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:19:43.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:43 smithi017 ceph-mon[97472]: osdmap e71: 11 total, 11 up, 11 in 2023-06-23T18:19:43.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:43 smithi017 ceph-mon[97472]: pgmap v214: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:44 smithi002 ceph-mon[95947]: osdmap e72: 11 total, 11 up, 11 in 2023-06-23T18:19:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:44 smithi002 ceph-mon[95947]: osdmap e73: 11 total, 11 up, 11 in 2023-06-23T18:19:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:44 smithi082 ceph-mon[97889]: osdmap e72: 11 total, 11 up, 11 in 2023-06-23T18:19:44.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:44 smithi082 ceph-mon[97889]: osdmap e73: 11 total, 11 up, 11 in 2023-06-23T18:19:44.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:44 smithi017 ceph-mon[97472]: osdmap e72: 11 total, 11 up, 11 in 2023-06-23T18:19:44.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:44 smithi017 ceph-mon[97472]: osdmap e73: 11 total, 11 up, 11 in 2023-06-23T18:19:45.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:45 smithi002 ceph-mon[95947]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:45 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/3659077428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]: dispatch 2023-06-23T18:19:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:45 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]: dispatch 2023-06-23T18:19:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:45 smithi002 ceph-mon[95947]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]': finished 2023-06-23T18:19:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:45 smithi002 ceph-mon[95947]: osdmap e74: 12 total, 11 up, 12 in 2023-06-23T18:19:45.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:19:45.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:45 smithi082 ceph-mon[97889]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:45 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/3659077428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]: dispatch 2023-06-23T18:19:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:45 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]: dispatch 2023-06-23T18:19:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:45 smithi082 ceph-mon[97889]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]': finished 2023-06-23T18:19:45.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:45 smithi082 ceph-mon[97889]: osdmap e74: 12 total, 11 up, 12 in 2023-06-23T18:19:45.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:19:45.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:45 smithi017 ceph-mon[97472]: pgmap v217: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:45.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:45 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/3659077428' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]: dispatch 2023-06-23T18:19:45.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:45 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]: dispatch 2023-06-23T18:19:45.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:45 smithi017 ceph-mon[97472]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77"}]': finished 2023-06-23T18:19:45.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:45 smithi017 ceph-mon[97472]: osdmap e74: 12 total, 11 up, 12 in 2023-06-23T18:19:45.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:19:46.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:46 smithi002 ceph-mon[95947]: from='client.? 172.21.15.82:0/2613821298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:46 smithi082 ceph-mon[97889]: from='client.? 172.21.15.82:0/2613821298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:46.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:46 smithi017 ceph-mon[97472]: from='client.? 172.21.15.82:0/2613821298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-06-23T18:19:47.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:47 smithi002 ceph-mon[95947]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:47.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:47 smithi082 ceph-mon[97889]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:47.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:47 smithi017 ceph-mon[97472]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:19:49.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:48 smithi002 ceph-mon[95947]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T18:19:49.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:48 smithi082 ceph-mon[97889]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T18:19:49.195 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:48 smithi017 ceph-mon[97472]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-06-23T18:19:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:50 smithi002 ceph-mon[95947]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-06-23T18:19:51.081 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:50 smithi082 ceph-mon[97889]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-06-23T18:19:51.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:50 smithi017 ceph-mon[97472]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-06-23T18:19:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-23T18:19:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:52.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:51 smithi002 ceph-mon[95947]: Deploying daemon osd.11 on smithi082 2023-06-23T18:19:52.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-23T18:19:52.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:52.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:51 smithi082 ceph-mon[97889]: Deploying daemon osd.11 on smithi082 2023-06-23T18:19:52.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-06-23T18:19:52.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:52.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:51 smithi017 ceph-mon[97472]: Deploying daemon osd.11 on smithi082 2023-06-23T18:19:52.987 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:52 smithi082 ceph-mon[97889]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-06-23T18:19:53.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:52 smithi017 ceph-mon[97472]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-06-23T18:19:53.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:52 smithi002 ceph-mon[95947]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-06-23T18:19:55.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:55 smithi002 ceph-mon[95947]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T18:19:55.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:55 smithi082 ceph-mon[97889]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T18:19:55.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:55 smithi017 ceph-mon[97472]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-06-23T18:19:57.090 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:57.090 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:57.090 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:57.091 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:56 smithi082 ceph-mon[97889]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-06-23T18:19:57.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:57.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:57.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:57.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:56 smithi002 ceph-mon[95947]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-06-23T18:19:57.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:57.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:57.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:57.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:56 smithi017 ceph-mon[97472]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-06-23T18:19:58.779 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.773+0000 7fa8749a2700 1 -- 172.21.15.82:0/1956089439 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fa870060910 con 0x7fa8580a1150 2023-06-23T18:19:58.779 INFO:teuthology.orchestra.run.smithi082.stdout:Created osd(s) 11 on host 'smithi082' 2023-06-23T18:19:58.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa8580a1150 msgr2=0x7fa8580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:58.780 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa8580a1150 0x7fa8580a3600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa864005950 tx=0x7fa8640058e0 comp rx=0 tx=0).stop 2023-06-23T18:19:58.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 msgr2=0x7fa870195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:19:58.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 0x7fa870195c40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8600059d0 tx=0x7fa860004730 comp rx=0 tx=0).stop 2023-06-23T18:19:58.781 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 shutdown_connections 2023-06-23T18:19:58.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa870105810 0x7fa87019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:58.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa8580a1150 0x7fa8580a3600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:58.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8701005d0 0x7fa870195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:58.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 --2- 172.21.15.82:0/1956089439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa870102ef0 0x7fa870195c40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:19:58.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 >> 172.21.15.82:0/1956089439 conn(0x7fa8700fa070 msgr2=0x7fa870101230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:19:58.782 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 shutdown_connections 2023-06-23T18:19:58.783 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:19:58.775+0000 7fa8759a4700 1 -- 172.21.15.82:0/1956089439 wait complete. 2023-06-23T18:19:59.213 DEBUG:teuthology.orchestra.run.smithi082:osd.11> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.11.service 2023-06-23T18:19:59.217 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-06-23T18:19:59.217 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-23T18:19:59.245 INFO:journalctl@ceph.osd.11.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:19:59.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:59.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:59.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-06-23T18:19:59.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:19:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:59.758 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.758 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.758 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:59.758 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:59.759 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.759 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-06-23T18:19:59.759 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.759 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.759 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:19:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:19:59.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:19:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:19:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-06-23T18:19:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:19:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:19:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:00.039 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:19:59 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:19:59.876+0000 7fc979d03640 -1 osd.11 0 log_to_monitors true 2023-06-23T18:20:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:00 smithi002 ceph-mon[95947]: from='osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T18:20:00.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:00 smithi002 ceph-mon[95947]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T18:20:00.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:00 smithi002 ceph-mon[95947]: overall HEALTH_OK 2023-06-23T18:20:00.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:00 smithi082 ceph-mon[97889]: from='osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T18:20:00.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:00 smithi082 ceph-mon[97889]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T18:20:00.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:00 smithi082 ceph-mon[97889]: overall HEALTH_OK 2023-06-23T18:20:00.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:00 smithi017 ceph-mon[97472]: from='osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T18:20:00.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:00 smithi017 ceph-mon[97472]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-06-23T18:20:00.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:00 smithi017 ceph-mon[97472]: overall HEALTH_OK 2023-06-23T18:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 -- 172.21.15.2:0/3967506881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c0ffc40 msgr2=0x7fea5c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 --2- 172.21.15.2:0/3967506881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c0ffc40 0x7fea5c100090 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fea58009990 tx=0x7fea58009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 -- 172.21.15.2:0/3967506881 shutdown_connections 2023-06-23T18:20:01.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 --2- 172.21.15.2:0/3967506881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea5c1005d0 0x7fea5c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 --2- 172.21.15.2:0/3967506881 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea5c0fea40 0x7fea5c0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 --2- 172.21.15.2:0/3967506881 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c0ffc40 0x7fea5c100090 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 -- 172.21.15.2:0/3967506881 >> 172.21.15.2:0/3967506881 conn(0x7fea5c0fa070 msgr2=0x7fea5c0fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 -- 172.21.15.2:0/3967506881 shutdown_connections 2023-06-23T18:20:01.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.097+0000 7fea64514700 1 -- 172.21.15.2:0/3967506881 wait complete. 2023-06-23T18:20:01.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea64514700 1 Processor -- start 2023-06-23T18:20:01.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea64514700 1 -- start start 2023-06-23T18:20:01.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea64514700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea5c0fea40 0x7fea5c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:01.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea64514700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea5c0ffc40 0x7fea5c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:01.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea64514700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c1005d0 0x7fea5c19ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:01.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea64514700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c19f4d0 con 0x7fea5c1005d0 2023-06-23T18:20:01.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea64514700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c19f640 con 0x7fea5c0ffc40 2023-06-23T18:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea64514700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c19f800 con 0x7fea5c0fea40 2023-06-23T18:20:01.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c1005d0 0x7fea5c19ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:01.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea622b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea5c0fea40 0x7fea5c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:01.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c1005d0 0x7fea5c19ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52024/0 (socket says 172.21.15.2:52024) 2023-06-23T18:20:01.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea622b0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea5c0fea40 0x7fea5c069380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:52358/0 (socket says 172.21.15.2:52358) 2023-06-23T18:20:01.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 -- 172.21.15.2:0/3902746183 learned_addr learned my addr 172.21.15.2:0/3902746183 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:01.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea61aaf700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea5c0ffc40 0x7fea5c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:01.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 -- 172.21.15.2:0/3902746183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea5c0fea40 msgr2=0x7fea5c069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:01.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea5c0fea40 0x7fea5c069380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 -- 172.21.15.2:0/3902746183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea5c0ffc40 msgr2=0x7fea5c19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea5c0ffc40 0x7fea5c19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 -- 172.21.15.2:0/3902746183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea580095f0 con 0x7fea5c1005d0 2023-06-23T18:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea62ab1700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c1005d0 0x7fea5c19ee20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea54007ee0 tx=0x7fea5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.098+0000 7fea4f7fe700 1 -- 172.21.15.2:0/3902746183 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54017070 con 0x7fea5c1005d0 2023-06-23T18:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.099+0000 7fea4f7fe700 1 -- 172.21.15.2:0/3902746183 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea5400f040 con 0x7fea5c1005d0 2023-06-23T18:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.099+0000 7fea4f7fe700 1 -- 172.21.15.2:0/3902746183 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54005570 con 0x7fea5c1005d0 2023-06-23T18:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.099+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c19fae0 con 0x7fea5c1005d0 2023-06-23T18:20:01.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.099+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c19ff60 con 0x7fea5c1005d0 2023-06-23T18:20:01.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.100+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5c106070 con 0x7fea5c1005d0 2023-06-23T18:20:01.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.101+0000 7fea4f7fe700 1 -- 172.21.15.2:0/3902746183 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fea5400db20 con 0x7fea5c1005d0 2023-06-23T18:20:01.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.101+0000 7fea4f7fe700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea480a1150 0x7fea480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:01.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.101+0000 7fea4f7fe700 1 -- 172.21.15.2:0/3902746183 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6831+0+0 (secure 0 0 0) 0x7fea540cbaf0 con 0x7fea5c1005d0 2023-06-23T18:20:01.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.101+0000 7fea622b0700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea480a1150 0x7fea480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:01.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.102+0000 7fea622b0700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea480a1150 0x7fea480a3600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0ffaa0 tx=0x7fea50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:01.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.106+0000 7fea4f7fe700 1 -- 172.21.15.2:0/3902746183 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fea54095810 con 0x7fea5c1005d0 2023-06-23T18:20:01.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.303+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fea5c02c9f0 con 0x7fea5c1005d0 2023-06-23T18:20:01.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.303+0000 7fea4f7fe700 1 -- 172.21.15.2:0/3902746183 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7fea54098e30 con 0x7fea5c1005d0 2023-06-23T18:20:01.309 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:01.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea480a1150 msgr2=0x7fea480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:01.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea480a1150 0x7fea480a3600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0ffaa0 tx=0x7fea50009260 comp rx=0 tx=0).stop 2023-06-23T18:20:01.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c1005d0 msgr2=0x7fea5c19ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:01.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c1005d0 0x7fea5c19ee20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fea54007ee0 tx=0x7fea5400d3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 shutdown_connections 2023-06-23T18:20:01.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea5c1005d0 0x7fea5c19ee20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea5c0fea40 0x7fea5c069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea480a1150 0x7fea480a3600 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 --2- 172.21.15.2:0/3902746183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea5c0ffc40 0x7fea5c19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:01.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 >> 172.21.15.2:0/3902746183 conn(0x7fea5c0fa070 msgr2=0x7fea5c1031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:01.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 shutdown_connections 2023-06-23T18:20:01.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:01.305+0000 7fea64514700 1 -- 172.21.15.2:0/3902746183 wait complete. 2023-06-23T18:20:01.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-23T18:20:01.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: osdmap e75: 12 total, 11 up, 12 in 2023-06-23T18:20:01.727 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:01.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:20:01.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:20:01.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:20:01.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3902746183' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T18:20:01.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:01.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:01.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.743 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1687544381,"num_in_osds":12,"osd_in_since":1687544384,"num_remapped_pgs":0} 2023-06-23T18:20:01.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:20:01.514+0000 7fc96f37b700 -1 osd.11 0 waiting for initial osdmap 2023-06-23T18:20:01.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:20:01.525+0000 7fc967969700 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-06-23T18:20:01.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-23T18:20:01.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: osdmap e75: 12 total, 11 up, 12 in 2023-06-23T18:20:01.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:01.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:20:01.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:20:01.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:20:01.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3902746183' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T18:20:01.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:01.852 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-06-23T18:20:01.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: osdmap e75: 12 total, 11 up, 12 in 2023-06-23T18:20:01.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:01.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:20:01.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]: dispatch 2023-06-23T18:20:01.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-06-23T18:20:01.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3902746183' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T18:20:01.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:20:01.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:01.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:01.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:01.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:02.745 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd stat -f json 2023-06-23T18:20:02.778 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:02 smithi002 ceph-mon[95947]: Detected new or changed devices on smithi082 2023-06-23T18:20:02.779 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:02 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi082 to 4292M 2023-06-23T18:20:02.779 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:02 smithi002 ceph-mon[95947]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:20:02.779 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:02 smithi002 ceph-mon[95947]: osdmap e76: 12 total, 11 up, 12 in 2023-06-23T18:20:02.779 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:02.779 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:02.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:02 smithi082 ceph-mon[97889]: Detected new or changed devices on smithi082 2023-06-23T18:20:02.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:02 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi082 to 4292M 2023-06-23T18:20:02.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:02 smithi082 ceph-mon[97889]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:20:02.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:02 smithi082 ceph-mon[97889]: osdmap e76: 12 total, 11 up, 12 in 2023-06-23T18:20:02.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:02.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:02.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:02 smithi017 ceph-mon[97472]: Detected new or changed devices on smithi082 2023-06-23T18:20:02.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:02 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi082 to 4292M 2023-06-23T18:20:02.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:02 smithi017 ceph-mon[97472]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi082", "root=default"]}]': finished 2023-06-23T18:20:02.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:02 smithi017 ceph-mon[97472]: osdmap e76: 12 total, 11 up, 12 in 2023-06-23T18:20:02.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:02.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:03.323 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:20:03.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:03 smithi002 ceph-mon[95947]: purged_snaps scrub starts 2023-06-23T18:20:03.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:03 smithi002 ceph-mon[95947]: purged_snaps scrub ok 2023-06-23T18:20:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:03 smithi002 ceph-mon[95947]: osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] boot 2023-06-23T18:20:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:03 smithi002 ceph-mon[95947]: osdmap e77: 12 total, 12 up, 12 in 2023-06-23T18:20:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:03 smithi002 ceph-mon[95947]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:03.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:03 smithi082 ceph-mon[97889]: purged_snaps scrub starts 2023-06-23T18:20:03.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:03 smithi082 ceph-mon[97889]: purged_snaps scrub ok 2023-06-23T18:20:03.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:03 smithi082 ceph-mon[97889]: osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] boot 2023-06-23T18:20:03.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:03 smithi082 ceph-mon[97889]: osdmap e77: 12 total, 12 up, 12 in 2023-06-23T18:20:03.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:03.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:03 smithi082 ceph-mon[97889]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:03.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:03 smithi017 ceph-mon[97472]: purged_snaps scrub starts 2023-06-23T18:20:03.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:03 smithi017 ceph-mon[97472]: purged_snaps scrub ok 2023-06-23T18:20:03.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:03 smithi017 ceph-mon[97472]: osd.11 [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] boot 2023-06-23T18:20:03.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:03 smithi017 ceph-mon[97472]: osdmap e77: 12 total, 12 up, 12 in 2023-06-23T18:20:03.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-06-23T18:20:03.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:03 smithi017 ceph-mon[97472]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:04.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 -- 172.21.15.2:0/536913850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 msgr2=0x7fce34106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 --2- 172.21.15.2:0/536913850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 0x7fce34106e90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00b0a0 tx=0x7fce2c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 -- 172.21.15.2:0/536913850 shutdown_connections 2023-06-23T18:20:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 --2- 172.21.15.2:0/536913850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 0x7fce34106e90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 --2- 172.21.15.2:0/536913850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce340fea40 0x7fce340fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 --2- 172.21.15.2:0/536913850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce340ffc40 0x7fce34100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 -- 172.21.15.2:0/536913850 >> 172.21.15.2:0/536913850 conn(0x7fce340fa070 msgr2=0x7fce340fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 -- 172.21.15.2:0/536913850 shutdown_connections 2023-06-23T18:20:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 -- 172.21.15.2:0/536913850 wait complete. 2023-06-23T18:20:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.643+0000 7fce38f3c700 1 Processor -- start 2023-06-23T18:20:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce38f3c700 1 -- start start 2023-06-23T18:20:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce38f3c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce340fea40 0x7fce34102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce38f3c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce340ffc40 0x7fce34101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:04.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce38f3c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 0x7fce341018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:04.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce38f3c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34102040 con 0x7fce341005d0 2023-06-23T18:20:04.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce38f3c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce341021b0 con 0x7fce340fea40 2023-06-23T18:20:04.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce38f3c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3419f8f0 con 0x7fce340ffc40 2023-06-23T18:20:04.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 0x7fce341018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:04.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 0x7fce341018c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52050/0 (socket says 172.21.15.2:52050) 2023-06-23T18:20:04.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 -- 172.21.15.2:0/89660093 learned_addr learned my addr 172.21.15.2:0/89660093 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:04.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce31d9b700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce340ffc40 0x7fce34101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:04.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce3259c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce340fea40 0x7fce34102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:04.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 -- 172.21.15.2:0/89660093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce340ffc40 msgr2=0x7fce34101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:04.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce340ffc40 0x7fce34101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 -- 172.21.15.2:0/89660093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce340fea40 msgr2=0x7fce34102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:04.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce340fea40 0x7fce34102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 -- 172.21.15.2:0/89660093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c009d00 con 0x7fce341005d0 2023-06-23T18:20:04.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.644+0000 7fce32d9d700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 0x7fce341018c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0062a0 tx=0x7fce2c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:04.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.645+0000 7fce237fe700 1 -- 172.21.15.2:0/89660093 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c00e040 con 0x7fce341005d0 2023-06-23T18:20:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.645+0000 7fce237fe700 1 -- 172.21.15.2:0/89660093 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce2c01db20 con 0x7fce341005d0 2023-06-23T18:20:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.645+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3419fb70 con 0x7fce341005d0 2023-06-23T18:20:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.645+0000 7fce237fe700 1 -- 172.21.15.2:0/89660093 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c01bac0 con 0x7fce341005d0 2023-06-23T18:20:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.645+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3419ffe0 con 0x7fce341005d0 2023-06-23T18:20:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.646+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce34106b80 con 0x7fce341005d0 2023-06-23T18:20:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.647+0000 7fce237fe700 1 -- 172.21.15.2:0/89660093 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fce2c01dc90 con 0x7fce341005d0 2023-06-23T18:20:04.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.647+0000 7fce237fe700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce1c0a1150 0x7fce1c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:04.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.647+0000 7fce3259c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce1c0a1150 0x7fce1c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:04.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.647+0000 7fce237fe700 1 -- 172.21.15.2:0/89660093 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fce2c0ce2a0 con 0x7fce341005d0 2023-06-23T18:20:04.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.648+0000 7fce3259c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce1c0a1150 0x7fce1c0a3600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce340ffaa0 tx=0x7fce2400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:04.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.652+0000 7fce237fe700 1 -- 172.21.15.2:0/89660093 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fce2c097f20 con 0x7fce341005d0 2023-06-23T18:20:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:04 smithi002 ceph-mon[95947]: osdmap e78: 12 total, 12 up, 12 in 2023-06-23T18:20:04.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:04 smithi082 ceph-mon[97889]: osdmap e78: 12 total, 12 up, 12 in 2023-06-23T18:20:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.850+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fce3402ca20 con 0x7fce341005d0 2023-06-23T18:20:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.850+0000 7fce237fe700 1 -- 172.21.15.2:0/89660093 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) v1 ==== 74+0+133 (secure 0 0 0) 0x7fce2c09b540 con 0x7fce341005d0 2023-06-23T18:20:04.861 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce1c0a1150 msgr2=0x7fce1c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce1c0a1150 0x7fce1c0a3600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fce340ffaa0 tx=0x7fce2400b410 comp rx=0 tx=0).stop 2023-06-23T18:20:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 msgr2=0x7fce341018c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 0x7fce341018c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fce2c0062a0 tx=0x7fce2c004d00 comp rx=0 tx=0).stop 2023-06-23T18:20:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 shutdown_connections 2023-06-23T18:20:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fce341005d0 0x7fce341018c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce1c0a1150 0x7fce1c0a3600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce340fea40 0x7fce34102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 --2- 172.21.15.2:0/89660093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce340ffc40 0x7fce34101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 >> 172.21.15.2:0/89660093 conn(0x7fce340fa070 msgr2=0x7fce340fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 shutdown_connections 2023-06-23T18:20:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:04.852+0000 7fce38f3c700 1 -- 172.21.15.2:0/89660093 wait complete. 2023-06-23T18:20:04.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:04 smithi017 ceph-mon[97472]: osdmap e78: 12 total, 12 up, 12 in 2023-06-23T18:20:05.272 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1687544402,"num_in_osds":12,"osd_in_since":1687544384,"num_remapped_pgs":0} 2023-06-23T18:20:05.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd dump --format=json 2023-06-23T18:20:05.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:05 smithi002 ceph-mon[95947]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:05.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:05 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/89660093' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T18:20:05.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:05 smithi082 ceph-mon[97889]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:05.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:05 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/89660093' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T18:20:05.869 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:20:05.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:05 smithi017 ceph-mon[97472]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:05.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:05 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/89660093' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-06-23T18:20:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:06 smithi002 ceph-mon[95947]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:06 smithi082 ceph-mon[97889]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:06 smithi017 ceph-mon[97472]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:07.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/280751750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f01005d0 msgr2=0x7ff4f0100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:07.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/280751750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f01005d0 0x7ff4f0100a40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc009920 tx=0x7ff4dc009c30 comp rx=0 tx=0).stop 2023-06-23T18:20:07.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/280751750 shutdown_connections 2023-06-23T18:20:07.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/280751750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f0100f80 0x7ff4f010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/280751750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f00fee40 0x7ff4f00ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/280751750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f01005d0 0x7ff4f0100a40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/280751750 >> 172.21.15.2:0/280751750 conn(0x7ff4f0075b20 msgr2=0x7ff4f0075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:07.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/280751750 shutdown_connections 2023-06-23T18:20:07.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.371+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/280751750 wait complete. 2023-06-23T18:20:07.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f6c7b700 1 Processor -- start 2023-06-23T18:20:07.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f6c7b700 1 -- start start 2023-06-23T18:20:07.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f6c7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f00fee40 0x7ff4f01017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:07.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f6c7b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f01005d0 0x7ff4f0101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:07.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f6c7b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f0100f80 0x7ff4f0102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:07.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f6c7b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f01a40f0 con 0x7ff4f00fee40 2023-06-23T18:20:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f6c7b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f01a4260 con 0x7ff4f0100f80 2023-06-23T18:20:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f6c7b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f01a4420 con 0x7ff4f01005d0 2023-06-23T18:20:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f4a17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f00fee40 0x7ff4f01017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f4a17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f00fee40 0x7ff4f01017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52060/0 (socket says 172.21.15.2:52060) 2023-06-23T18:20:07.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f4a17700 1 -- 172.21.15.2:0/1487562432 learned_addr learned my addr 172.21.15.2:0/1487562432 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:07.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4effff700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f01005d0 0x7ff4f0101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:07.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f5218700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f0100f80 0x7ff4f0102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:07.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f4a17700 1 -- 172.21.15.2:0/1487562432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f01005d0 msgr2=0x7ff4f0101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f4a17700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f01005d0 0x7ff4f0101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f4a17700 1 -- 172.21.15.2:0/1487562432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f0100f80 msgr2=0x7ff4f0102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f4a17700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f0100f80 0x7ff4f0102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f4a17700 1 -- 172.21.15.2:0/1487562432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e8009d00 con 0x7ff4f00fee40 2023-06-23T18:20:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.372+0000 7ff4f5218700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f0100f80 0x7ff4f0102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.373+0000 7ff4f4a17700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f00fee40 0x7ff4f01017b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400e9a0 tx=0x7ff4e400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.373+0000 7ff4edffb700 1 -- 172.21.15.2:0/1487562432 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4009790 con 0x7ff4f00fee40 2023-06-23T18:20:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.373+0000 7ff4edffb700 1 -- 172.21.15.2:0/1487562432 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4e400cbd0 con 0x7ff4f00fee40 2023-06-23T18:20:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.373+0000 7ff4edffb700 1 -- 172.21.15.2:0/1487562432 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4018680 con 0x7ff4f00fee40 2023-06-23T18:20:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.373+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc009520 con 0x7ff4f00fee40 2023-06-23T18:20:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.373+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f01a4a70 con 0x7ff4f00fee40 2023-06-23T18:20:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.374+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f004e1d0 con 0x7ff4f00fee40 2023-06-23T18:20:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.375+0000 7ff4edffb700 1 -- 172.21.15.2:0/1487562432 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff4e400cd40 con 0x7ff4f00fee40 2023-06-23T18:20:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.375+0000 7ff4edffb700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4e00a11d0 0x7ff4e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.376+0000 7ff4effff700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4e00a11d0 0x7ff4e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.376+0000 7ff4edffb700 1 -- 172.21.15.2:0/1487562432 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff4e4014070 con 0x7ff4f00fee40 2023-06-23T18:20:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.376+0000 7ff4effff700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4e00a11d0 0x7ff4e00a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc005e90 tx=0x7ff4dc019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.380+0000 7ff4edffb700 1 -- 172.21.15.2:0/1487562432 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff4e4051770 con 0x7ff4f00fee40 2023-06-23T18:20:07.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.579+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff4f01a4c90 con 0x7ff4f00fee40 2023-06-23T18:20:07.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.580+0000 7ff4edffb700 1 -- 172.21.15.2:0/1487562432 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18243 (secure 0 0 0) 0x7ff4e4099e40 con 0x7ff4f00fee40 2023-06-23T18:20:07.585 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:07.585 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":78,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:03.515638+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:20:07.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4e00a11d0 msgr2=0x7ff4e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:07.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4e00a11d0 0x7ff4e00a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc005e90 tx=0x7ff4dc019040 comp rx=0 tx=0).stop 2023-06-23T18:20:07.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f00fee40 msgr2=0x7ff4f01017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:07.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f00fee40 0x7ff4f01017b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400e9a0 tx=0x7ff4e400ed60 comp rx=0 tx=0).stop 2023-06-23T18:20:07.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 shutdown_connections 2023-06-23T18:20:07.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4e00a11d0 0x7ff4e00a3680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4f0100f80 0x7ff4f0102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4f00fee40 0x7ff4f01017b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 --2- 172.21.15.2:0/1487562432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f01005d0 0x7ff4f0101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:07.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 >> 172.21.15.2:0/1487562432 conn(0x7ff4f0075b20 msgr2=0x7ff4f0068920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:07.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 shutdown_connections 2023-06-23T18:20:07.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:07.582+0000 7ff4f6c7b700 1 -- 172.21.15.2:0/1487562432 wait complete. 2023-06-23T18:20:07.950 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:07 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1487562432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:08.021 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:20:08.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-06-23T18:20:08.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:07 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1487562432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:07 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1487562432' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:08.584 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:20:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:08 smithi002 ceph-mon[95947]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:08 smithi082 ceph-mon[97889]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:08 smithi017 ceph-mon[97472]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:09.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.877+0000 7fc4eb96c700 1 -- 172.21.15.2:0/2882824981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e4100f80 msgr2=0x7fc4e410b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:09.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.877+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/2882824981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e4100f80 0x7fc4e410b9d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00b0d0 tx=0x7fc4dc00b3e0 comp rx=0 tx=0).stop 2023-06-23T18:20:09.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.877+0000 7fc4eb96c700 1 -- 172.21.15.2:0/2882824981 shutdown_connections 2023-06-23T18:20:09.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.877+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/2882824981 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e4100f80 0x7fc4e410b9d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:09.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.877+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/2882824981 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e40fee40 0x7fc4e40ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:09.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/2882824981 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e41005d0 0x7fc4e4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:09.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 -- 172.21.15.2:0/2882824981 >> 172.21.15.2:0/2882824981 conn(0x7fc4e4075b20 msgr2=0x7fc4e4075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:09.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 -- 172.21.15.2:0/2882824981 shutdown_connections 2023-06-23T18:20:09.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 -- 172.21.15.2:0/2882824981 wait complete. 2023-06-23T18:20:09.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 Processor -- start 2023-06-23T18:20:09.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 -- start start 2023-06-23T18:20:09.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e40fee40 0x7fc4e41017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:09.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41005d0 0x7fc4e4101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:09.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e4100f80 0x7fc4e4102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:09.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e41a4150 con 0x7fc4e40fee40 2023-06-23T18:20:09.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e41a42c0 con 0x7fc4e41005d0 2023-06-23T18:20:09.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.878+0000 7fc4eb96c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e41a4480 con 0x7fc4e4100f80 2023-06-23T18:20:09.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e40fee40 0x7fc4e41017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:09.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e40fee40 0x7fc4e41017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52072/0 (socket says 172.21.15.2:52072) 2023-06-23T18:20:09.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 -- 172.21.15.2:0/3388673918 learned_addr learned my addr 172.21.15.2:0/3388673918 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:09.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e8f07700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41005d0 0x7fc4e4101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:09.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 -- 172.21.15.2:0/3388673918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e4100f80 msgr2=0x7fc4e4102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:09.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9f09700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e4100f80 0x7fc4e4102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:09.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e4100f80 0x7fc4e4102270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:09.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 -- 172.21.15.2:0/3388673918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41005d0 msgr2=0x7fc4e4101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:09.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41005d0 0x7fc4e4101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:09.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 -- 172.21.15.2:0/3388673918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc009d00 con 0x7fc4e40fee40 2023-06-23T18:20:09.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4e9708700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e40fee40 0x7fc4e41017b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9a0 tx=0x7fc4d400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:09.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4da7fc700 1 -- 172.21.15.2:0/3388673918 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4009790 con 0x7fc4e40fee40 2023-06-23T18:20:09.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4da7fc700 1 -- 172.21.15.2:0/3388673918 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4d400cbd0 con 0x7fc4e40fee40 2023-06-23T18:20:09.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e41a4760 con 0x7fc4e40fee40 2023-06-23T18:20:09.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4da7fc700 1 -- 172.21.15.2:0/3388673918 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4018680 con 0x7fc4e40fee40 2023-06-23T18:20:09.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.879+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e41a4cb0 con 0x7fc4e40fee40 2023-06-23T18:20:09.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.880+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e404e1d0 con 0x7fc4e40fee40 2023-06-23T18:20:09.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.881+0000 7fc4da7fc700 1 -- 172.21.15.2:0/3388673918 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc4d400cd40 con 0x7fc4e40fee40 2023-06-23T18:20:09.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.882+0000 7fc4da7fc700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc4d00a11d0 0x7fc4d00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:09.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.882+0000 7fc4e8f07700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc4d00a11d0 0x7fc4d00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:09.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.882+0000 7fc4da7fc700 1 -- 172.21.15.2:0/3388673918 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc4d4014070 con 0x7fc4e40fee40 2023-06-23T18:20:09.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.882+0000 7fc4e8f07700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc4d00a11d0 0x7fc4d00a3680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4102fa0 tx=0x7fc4e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:09.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:09.886+0000 7fc4da7fc700 1 -- 172.21.15.2:0/3388673918 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc4d4051770 con 0x7fc4e40fee40 2023-06-23T18:20:10.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.083+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4e4102ce0 con 0x7fc4e40fee40 2023-06-23T18:20:10.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.083+0000 7fc4da7fc700 1 -- 172.21.15.2:0/3388673918 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4d4099e40 con 0x7fc4e40fee40 2023-06-23T18:20:10.088 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:20:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.085+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc4d00a11d0 msgr2=0x7fc4d00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.085+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc4d00a11d0 0x7fc4d00a3680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4102fa0 tx=0x7fc4e0009260 comp rx=0 tx=0).stop 2023-06-23T18:20:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.085+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e40fee40 msgr2=0x7fc4e41017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.085+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e40fee40 0x7fc4e41017b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9a0 tx=0x7fc4d400ed60 comp rx=0 tx=0).stop 2023-06-23T18:20:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.086+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 shutdown_connections 2023-06-23T18:20:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.086+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4e4100f80 0x7fc4e4102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.086+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc4e40fee40 0x7fc4e41017b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.086+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc4d00a11d0 0x7fc4d00a3680 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.086+0000 7fc4eb96c700 1 --2- 172.21.15.2:0/3388673918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4e41005d0 0x7fc4e4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.086+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 >> 172.21.15.2:0/3388673918 conn(0x7fc4e4075b20 msgr2=0x7fc4e4068880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.086+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 shutdown_connections 2023-06-23T18:20:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:10.086+0000 7fc4eb96c700 1 -- 172.21.15.2:0/3388673918 wait complete. 2023-06-23T18:20:10.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:10 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3388673918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:10.499 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:10 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3388673918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:10.543 INFO:tasks.cephadm:Adding mds.a on smithi002 2023-06-23T18:20:10.544 INFO:tasks.cephadm:Adding mds.d on smithi002 2023-06-23T18:20:10.544 INFO:tasks.cephadm:Adding mds.g on smithi002 2023-06-23T18:20:10.544 INFO:tasks.cephadm:Adding mds.j on smithi002 2023-06-23T18:20:10.544 INFO:tasks.cephadm:Adding mds.b on smithi017 2023-06-23T18:20:10.545 INFO:tasks.cephadm:Adding mds.e on smithi017 2023-06-23T18:20:10.545 INFO:tasks.cephadm:Adding mds.h on smithi017 2023-06-23T18:20:10.545 INFO:tasks.cephadm:Adding mds.k on smithi017 2023-06-23T18:20:10.546 INFO:tasks.cephadm:Adding mds.c on smithi082 2023-06-23T18:20:10.546 INFO:tasks.cephadm:Adding mds.f on smithi082 2023-06-23T18:20:10.546 INFO:tasks.cephadm:Adding mds.i on smithi082 2023-06-23T18:20:10.547 INFO:tasks.cephadm:Adding mds.l on smithi082 2023-06-23T18:20:10.547 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph orch apply mds all '12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l' 2023-06-23T18:20:10.580 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:10 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3388673918' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:11.117 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:20:11.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:11 smithi017 ceph-mon[97472]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:11.513 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:11 smithi082 ceph-mon[97889]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:11.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:11 smithi002 ceph-mon[95947]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:12.418 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 -- 172.21.15.82:0/4168196298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c40ffc40 msgr2=0x7f55c4100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 --2- 172.21.15.82:0/4168196298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c40ffc40 0x7f55c4100090 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009990 tx=0x7f55c0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 -- 172.21.15.82:0/4168196298 shutdown_connections 2023-06-23T18:20:12.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 --2- 172.21.15.82:0/4168196298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c41005d0 0x7f55c4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 --2- 172.21.15.82:0/4168196298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55c40fea40 0x7f55c40fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 --2- 172.21.15.82:0/4168196298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c40ffc40 0x7f55c4100090 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 -- 172.21.15.82:0/4168196298 >> 172.21.15.82:0/4168196298 conn(0x7f55c40fa070 msgr2=0x7f55c40fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:12.420 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 -- 172.21.15.82:0/4168196298 shutdown_connections 2023-06-23T18:20:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.410+0000 7f55cbe41700 1 -- 172.21.15.82:0/4168196298 wait complete. 2023-06-23T18:20:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55cbe41700 1 Processor -- start 2023-06-23T18:20:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55cbe41700 1 -- start start 2023-06-23T18:20:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55cbe41700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40fea40 0x7f55c41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55cbe41700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55c40ffc40 0x7f55c4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55cbe41700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c41005d0 0x7f55c419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55cbe41700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c419f480 con 0x7f55c40fea40 2023-06-23T18:20:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55cbe41700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c419f5c0 con 0x7f55c40ffc40 2023-06-23T18:20:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55cbe41700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55c419f750 con 0x7f55c41005d0 2023-06-23T18:20:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55ca3de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c41005d0 0x7f55c419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:12.423 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55ca3de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c41005d0 0x7f55c419eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41004/0 (socket says 172.21.15.82:41004) 2023-06-23T18:20:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55ca3de700 1 -- 172.21.15.82:0/3602601724 learned_addr learned my addr 172.21.15.82:0/3602601724 (peer_addr_for_me v2:172.21.15.82:0/0) 2023-06-23T18:20:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55c93dc700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55c40ffc40 0x7f55c4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55c9bdd700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40fea40 0x7f55c41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55ca3de700 1 -- 172.21.15.82:0/3602601724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55c40ffc40 msgr2=0x7f55c4197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55ca3de700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55c40ffc40 0x7f55c4197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55ca3de700 1 -- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40fea40 msgr2=0x7f55c41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55ca3de700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40fea40 0x7f55c41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55ca3de700 1 -- 172.21.15.82:0/3602601724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00095f0 con 0x7f55c41005d0 2023-06-23T18:20:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.411+0000 7f55c93dc700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55c40ffc40 0x7f55c4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.412+0000 7f55ca3de700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c41005d0 0x7f55c419eeb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f55bc007d40 tx=0x7f55bc00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.412+0000 7f55baffd700 1 -- 172.21.15.82:0/3602601724 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00ad60 con 0x7f55c41005d0 2023-06-23T18:20:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.412+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c419fa30 con 0x7f55c41005d0 2023-06-23T18:20:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.412+0000 7f55baffd700 1 -- 172.21.15.82:0/3602601724 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55bc0128b0 con 0x7f55c41005d0 2023-06-23T18:20:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.412+0000 7f55baffd700 1 -- 172.21.15.82:0/3602601724 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc011760 con 0x7f55c41005d0 2023-06-23T18:20:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.412+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55c419ff80 con 0x7f55c41005d0 2023-06-23T18:20:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.413+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55c410a100 con 0x7f55c41005d0 2023-06-23T18:20:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.414+0000 7f55baffd700 1 -- 172.21.15.82:0/3602601724 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f55bc012a20 con 0x7f55c41005d0 2023-06-23T18:20:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.414+0000 7f55baffd700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f55b00a1150 0x7f55b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.415+0000 7f55baffd700 1 -- 172.21.15.82:0/3602601724 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f55bc0ccbc0 con 0x7f55c41005d0 2023-06-23T18:20:12.428 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.415+0000 7f55c9bdd700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f55b00a1150 0x7f55b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.415+0000 7f55c9bdd700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f55b00a1150 0x7f55b00a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005fd0 tx=0x7f55b4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:12.429 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.418+0000 7f55baffd700 1 -- 172.21.15.82:0/3602601724 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f55bc096710 con 0x7f55c41005d0 2023-06-23T18:20:12.620 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.615+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f55c40690b0 con 0x7f55b00a1150 2023-06-23T18:20:12.630 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.624+0000 7f55baffd700 1 -- 172.21.15.82:0/3602601724 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f55c40690b0 con 0x7f55b00a1150 2023-06-23T18:20:12.630 INFO:teuthology.orchestra.run.smithi082.stdout:Scheduled mds.all update... 2023-06-23T18:20:12.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f55b00a1150 msgr2=0x7f55b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:12.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f55b00a1150 0x7f55b00a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f55b4005fd0 tx=0x7f55b4005e50 comp rx=0 tx=0).stop 2023-06-23T18:20:12.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c41005d0 msgr2=0x7f55c419eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:12.632 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c41005d0 0x7f55c419eeb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f55bc007d40 tx=0x7f55bc00a340 comp rx=0 tx=0).stop 2023-06-23T18:20:12.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 shutdown_connections 2023-06-23T18:20:12.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55c41005d0 0x7f55c419eeb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f55c40fea40 0x7f55c41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.633 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f55b00a1150 0x7f55b00a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 --2- 172.21.15.82:0/3602601724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f55c40ffc40 0x7f55c4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:12.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.626+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 >> 172.21.15.82:0/3602601724 conn(0x7f55c40fa070 msgr2=0x7f55c40688a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:12.634 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.627+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 shutdown_connections 2023-06-23T18:20:12.635 INFO:teuthology.orchestra.run.smithi082.stderr:2023-06-23T18:20:12.627+0000 7f55cbe41700 1 -- 172.21.15.82:0/3602601724 wait complete. 2023-06-23T18:20:13.068 DEBUG:teuthology.orchestra.run.smithi002:mds.a> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.a.service 2023-06-23T18:20:13.071 DEBUG:teuthology.orchestra.run.smithi002:mds.d> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.d.service 2023-06-23T18:20:13.110 INFO:journalctl@ceph.mds.a.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:20:13.111 DEBUG:teuthology.orchestra.run.smithi002:mds.g> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.g.service 2023-06-23T18:20:13.141 INFO:journalctl@ceph.mds.d.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:20:13.143 DEBUG:teuthology.orchestra.run.smithi002:mds.j> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.j.service 2023-06-23T18:20:13.172 INFO:journalctl@ceph.mds.g.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:20:13.173 DEBUG:teuthology.orchestra.run.smithi017:mds.b> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.b.service 2023-06-23T18:20:13.176 DEBUG:teuthology.orchestra.run.smithi017:mds.e> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.e.service 2023-06-23T18:20:13.200 INFO:journalctl@ceph.mds.j.smithi002.stdout:-- Logs begin at Fri 2023-06-23 17:59:53 UTC. -- 2023-06-23T18:20:13.214 INFO:journalctl@ceph.mds.b.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:20:13.216 DEBUG:teuthology.orchestra.run.smithi017:mds.h> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.h.service 2023-06-23T18:20:13.246 INFO:journalctl@ceph.mds.e.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:20:13.247 DEBUG:teuthology.orchestra.run.smithi017:mds.k> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.k.service 2023-06-23T18:20:13.276 INFO:journalctl@ceph.mds.h.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:20:13.277 DEBUG:teuthology.orchestra.run.smithi082:mds.c> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.c.service 2023-06-23T18:20:13.279 DEBUG:teuthology.orchestra.run.smithi082:mds.f> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.f.service 2023-06-23T18:20:13.304 INFO:journalctl@ceph.mds.k.smithi017.stdout:-- Logs begin at Fri 2023-06-23 17:59:57 UTC. -- 2023-06-23T18:20:13.307 INFO:journalctl@ceph.mds.c.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:20:13.308 DEBUG:teuthology.orchestra.run.smithi082:mds.i> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.i.service 2023-06-23T18:20:13.335 INFO:journalctl@ceph.mds.f.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:20:13.336 DEBUG:teuthology.orchestra.run.smithi082:mds.l> sudo journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.l.service 2023-06-23T18:20:13.361 INFO:journalctl@ceph.mds.i.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:20:13.362 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-06-23T18:20:13.363 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:20:13.388 INFO:journalctl@ceph.mds.l.smithi082.stdout:-- Logs begin at Fri 2023-06-23 18:00:15 UTC. -- 2023-06-23T18:20:13.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 -- 172.21.15.2:0/3214202197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c102f00 msgr2=0x7f664c1052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:13.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 --2- 172.21.15.2:0/3214202197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c102f00 0x7f664c1052e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6648009990 tx=0x7f6648009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 -- 172.21.15.2:0/3214202197 shutdown_connections 2023-06-23T18:20:13.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 --2- 172.21.15.2:0/3214202197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f664c105820 0x7f664c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 --2- 172.21.15.2:0/3214202197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f664c1005e0 0x7f664c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 --2- 172.21.15.2:0/3214202197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c102f00 0x7f664c1052e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 -- 172.21.15.2:0/3214202197 >> 172.21.15.2:0/3214202197 conn(0x7f664c0fa0e0 msgr2=0x7f664c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:13.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 -- 172.21.15.2:0/3214202197 shutdown_connections 2023-06-23T18:20:13.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.494+0000 7f665451c700 1 -- 172.21.15.2:0/3214202197 wait complete. 2023-06-23T18:20:13.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f665451c700 1 Processor -- start 2023-06-23T18:20:13.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f665451c700 1 -- start start 2023-06-23T18:20:13.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f665451c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c1005e0 0x7f664c0fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:13.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f665451c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f664c102f00 0x7f664c0fe590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:13.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f665451c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f664c105820 0x7f664c0fead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:13.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f665451c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0ff270 con 0x7f664c1005e0 2023-06-23T18:20:13.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f665451c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0ff3e0 con 0x7f664c105820 2023-06-23T18:20:13.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f665451c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c19f870 con 0x7f664c102f00 2023-06-23T18:20:13.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f66522b8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c1005e0 0x7f664c0fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:13.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f66522b8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c1005e0 0x7f664c0fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52084/0 (socket says 172.21.15.2:52084) 2023-06-23T18:20:13.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f66522b8700 1 -- 172.21.15.2:0/3477476290 learned_addr learned my addr 172.21.15.2:0/3477476290 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:13.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.495+0000 7f6651ab7700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f664c102f00 0x7f664c0fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:13.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f6652ab9700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f664c105820 0x7f664c0fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:13.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f66522b8700 1 -- 172.21.15.2:0/3477476290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f664c102f00 msgr2=0x7f664c0fe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:13.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f66522b8700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f664c102f00 0x7f664c0fe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f66522b8700 1 -- 172.21.15.2:0/3477476290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f664c105820 msgr2=0x7f664c0fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:13.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f66522b8700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f664c105820 0x7f664c0fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f66522b8700 1 -- 172.21.15.2:0/3477476290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66480095f0 con 0x7f664c1005e0 2023-06-23T18:20:13.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f66522b8700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c1005e0 0x7f664c0fff50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f664000d950 tx=0x7f664000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:13.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f663f7fe700 1 -- 172.21.15.2:0/3477476290 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000f890 con 0x7f664c1005e0 2023-06-23T18:20:13.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f663f7fe700 1 -- 172.21.15.2:0/3477476290 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f664000fe80 con 0x7f664c1005e0 2023-06-23T18:20:13.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664c19fb50 con 0x7f664c1005e0 2023-06-23T18:20:13.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f663f7fe700 1 -- 172.21.15.2:0/3477476290 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000e3b0 con 0x7f664c1005e0 2023-06-23T18:20:13.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.496+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c1a00b0 con 0x7f664c1005e0 2023-06-23T18:20:13.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.497+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c0fcc60 con 0x7f664c1005e0 2023-06-23T18:20:13.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.499+0000 7f663f7fe700 1 -- 172.21.15.2:0/3477476290 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6640010460 con 0x7f664c1005e0 2023-06-23T18:20:13.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.499+0000 7f663f7fe700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66380a11d0 0x7f66380a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:13.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.499+0000 7f663f7fe700 1 -- 172.21.15.2:0/3477476290 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66400cc160 con 0x7f664c1005e0 2023-06-23T18:20:13.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.499+0000 7f6651ab7700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66380a11d0 0x7f66380a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:13.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.500+0000 7f6651ab7700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66380a11d0 0x7f66380a3680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f664800b5c0 tx=0x7f6648009f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:13.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.503+0000 7f663f7fe700 1 -- 172.21.15.2:0/3477476290 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6640095d50 con 0x7f664c1005e0 2023-06-23T18:20:13.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.706+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f664c02c810 con 0x7f664c1005e0 2023-06-23T18:20:13.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.707+0000 7f663f7fe700 1 -- 172.21.15.2:0/3477476290 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18243 (secure 0 0 0) 0x7f6640099370 con 0x7f664c1005e0 2023-06-23T18:20:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:13.713 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":78,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:03.515638+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:20:13.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66380a11d0 msgr2=0x7f66380a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:13.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66380a11d0 0x7f66380a3680 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f664800b5c0 tx=0x7f6648009f00 comp rx=0 tx=0).stop 2023-06-23T18:20:13.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c1005e0 msgr2=0x7f664c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:13.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c1005e0 0x7f664c0fff50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f664000d950 tx=0x7f664000dc60 comp rx=0 tx=0).stop 2023-06-23T18:20:13.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 shutdown_connections 2023-06-23T18:20:13.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66380a11d0 0x7f66380a3680 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f664c105820 0x7f664c0fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f664c1005e0 0x7f664c0fff50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 --2- 172.21.15.2:0/3477476290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f664c102f00 0x7f664c0fe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.709+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 >> 172.21.15.2:0/3477476290 conn(0x7f664c0fa0e0 msgr2=0x7f664c101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:13.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.710+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 shutdown_connections 2023-06-23T18:20:13.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.710+0000 7f665451c700 1 -- 172.21.15.2:0/3477476290 wait complete. 2023-06-23T18:20:13.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:13.728 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l;count:12 2023-06-23T18:20:13.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:13.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:13.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:13.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:13.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:13.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:13.729 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:13.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:13.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: Deploying daemon mds.l on smithi082 2023-06-23T18:20:13.730 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:13 smithi002 ceph-mon[95947]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:13.731 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:20:13.732 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:20:13.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 -- 172.21.15.2:0/2829405546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b44100600 msgr2=0x7f5b44106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:13.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2829405546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b44100600 0x7f5b44106ec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b0a0 tx=0x7f5b3c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 -- 172.21.15.2:0/2829405546 shutdown_connections 2023-06-23T18:20:13.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2829405546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b44100600 0x7f5b44106ec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2829405546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b440fea80 0x7f5b440fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2829405546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b440ffc70 0x7f5b441000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 -- 172.21.15.2:0/2829405546 >> 172.21.15.2:0/2829405546 conn(0x7f5b440fa0e0 msgr2=0x7f5b440fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:13.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 -- 172.21.15.2:0/2829405546 shutdown_connections 2023-06-23T18:20:13.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.857+0000 7f5b4baab700 1 -- 172.21.15.2:0/2829405546 wait complete. 2023-06-23T18:20:13.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4baab700 1 Processor -- start 2023-06-23T18:20:13.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4baab700 1 -- start start 2023-06-23T18:20:13.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4baab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b440fea80 0x7f5b44102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:13.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4baab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b440ffc70 0x7f5b44101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:13.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4baab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b44100600 0x7f5b44101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:13.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4baab700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44101d30 con 0x7f5b440ffc70 2023-06-23T18:20:13.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4baab700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44101ea0 con 0x7f5b440fea80 2023-06-23T18:20:13.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4baab700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4419f970 con 0x7f5b44100600 2023-06-23T18:20:13.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49046700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b440ffc70 0x7f5b44101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:13.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49046700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b440ffc70 0x7f5b44101050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52100/0 (socket says 172.21.15.2:52100) 2023-06-23T18:20:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49046700 1 -- 172.21.15.2:0/2016332403 learned_addr learned my addr 172.21.15.2:0/2016332403 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49847700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b440fea80 0x7f5b44102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b4a048700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b44100600 0x7f5b44101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:13.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49046700 1 -- 172.21.15.2:0/2016332403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b44100600 msgr2=0x7f5b44101590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49046700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b44100600 0x7f5b44101590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49046700 1 -- 172.21.15.2:0/2016332403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b440fea80 msgr2=0x7f5b44102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49046700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b440fea80 0x7f5b44102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.858+0000 7f5b49046700 1 -- 172.21.15.2:0/2016332403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c009d00 con 0x7f5b440ffc70 2023-06-23T18:20:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.859+0000 7f5b49046700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b440ffc70 0x7f5b44101050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000e9a0 tx=0x7f5b4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.859+0000 7f5b3affd700 1 -- 172.21.15.2:0/2016332403 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40009790 con 0x7f5b440ffc70 2023-06-23T18:20:13.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.859+0000 7f5b3affd700 1 -- 172.21.15.2:0/2016332403 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b4000cbd0 con 0x7f5b440ffc70 2023-06-23T18:20:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.859+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4419fc50 con 0x7f5b440ffc70 2023-06-23T18:20:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.859+0000 7f5b3affd700 1 -- 172.21.15.2:0/2016332403 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40018680 con 0x7f5b440ffc70 2023-06-23T18:20:13.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.859+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b441a01b0 con 0x7f5b440ffc70 2023-06-23T18:20:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.860+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4410a160 con 0x7f5b440ffc70 2023-06-23T18:20:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.862+0000 7f5b3affd700 1 -- 172.21.15.2:0/2016332403 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5b4000cd40 con 0x7f5b440ffc70 2023-06-23T18:20:13.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.862+0000 7f5b3affd700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5b300a11d0 0x7f5b300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.862+0000 7f5b3affd700 1 -- 172.21.15.2:0/2016332403 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b40014070 con 0x7f5b440ffc70 2023-06-23T18:20:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.862+0000 7f5b49847700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5b300a11d0 0x7f5b300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.863+0000 7f5b49847700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5b300a11d0 0x7f5b300a3680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b440ffad0 tx=0x7f5b3400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:13.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:13.866+0000 7f5b3affd700 1 -- 172.21.15.2:0/2016332403 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5b400965c0 con 0x7f5b440ffc70 2023-06-23T18:20:13.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:13.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l;count:12 2023-06-23T18:20:13.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:13.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:13.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:13.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:13.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:13.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:13.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:13.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:13.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: Deploying daemon mds.l on smithi082 2023-06-23T18:20:13.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:13 smithi017 ceph-mon[97472]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:13.957 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:13.958 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: Saving service mds.all spec with placement smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l;count:12 2023-06-23T18:20:13.958 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:13.958 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:13.958 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:13.959 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:13.959 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:13.960 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:13.960 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:13.961 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:13.961 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: Deploying daemon mds.l on smithi082 2023-06-23T18:20:13.961 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:13 smithi082 ceph-mon[97889]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:14.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.066+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b441a0760 con 0x7f5b440ffc70 2023-06-23T18:20:14.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.066+0000 7f5b3affd700 1 -- 172.21.15.2:0/2016332403 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b40026020 con 0x7f5b440ffc70 2023-06-23T18:20:14.072 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:20:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5b300a11d0 msgr2=0x7f5b300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5b300a11d0 0x7f5b300a3680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5b440ffad0 tx=0x7f5b3400b410 comp rx=0 tx=0).stop 2023-06-23T18:20:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b440ffc70 msgr2=0x7f5b44101050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b440ffc70 0x7f5b44101050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b4000e9a0 tx=0x7f5b4000ecb0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 shutdown_connections 2023-06-23T18:20:14.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b44100600 0x7f5b44101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5b300a11d0 0x7f5b300a3680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b440fea80 0x7f5b44102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 --2- 172.21.15.2:0/2016332403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b440ffc70 0x7f5b44101050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.068+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 >> 172.21.15.2:0/2016332403 conn(0x7f5b440fa0e0 msgr2=0x7f5b44067840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:14.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.069+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 shutdown_connections 2023-06-23T18:20:14.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.069+0000 7f5b4baab700 1 -- 172.21.15.2:0/2016332403 wait complete. 2023-06-23T18:20:14.086 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 -- 172.21.15.2:0/2114821958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e40fecb0 msgr2=0x7f13e40ff120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 --2- 172.21.15.2:0/2114821958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e40fecb0 0x7f13e40ff120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009990 tx=0x7f13d8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 -- 172.21.15.2:0/2114821958 shutdown_connections 2023-06-23T18:20:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 --2- 172.21.15.2:0/2114821958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e40ff660 0x7f13e4106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 --2- 172.21.15.2:0/2114821958 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13e41000f0 0x7f13e4100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 --2- 172.21.15.2:0/2114821958 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e40fecb0 0x7f13e40ff120 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 -- 172.21.15.2:0/2114821958 >> 172.21.15.2:0/2114821958 conn(0x7f13e40f9810 msgr2=0x7f13e40fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 -- 172.21.15.2:0/2114821958 shutdown_connections 2023-06-23T18:20:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.207+0000 7f13ea545700 1 -- 172.21.15.2:0/2114821958 wait complete. 2023-06-23T18:20:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13ea545700 1 Processor -- start 2023-06-23T18:20:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13ea545700 1 -- start start 2023-06-23T18:20:14.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13ea545700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13e40fecb0 0x7f13e41946b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:14.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13ea545700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e40ff660 0x7f13e4194bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:14.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13ea545700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e41000f0 0x7f13e419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:14.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13ea545700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419c230 con 0x7f13e41000f0 2023-06-23T18:20:14.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13ea545700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419c370 con 0x7f13e40fecb0 2023-06-23T18:20:14.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13ea545700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e419c500 con 0x7f13e40ff660 2023-06-23T18:20:14.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e8ae2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e41000f0 0x7f13e419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e8ae2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e41000f0 0x7f13e419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52116/0 (socket says 172.21.15.2:52116) 2023-06-23T18:20:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e8ae2700 1 -- 172.21.15.2:0/1343554237 learned_addr learned my addr 172.21.15.2:0/1343554237 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e37fe700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e40ff660 0x7f13e4194bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e3fff700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13e40fecb0 0x7f13e41946b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e8ae2700 1 -- 172.21.15.2:0/1343554237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e40ff660 msgr2=0x7f13e4194bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e8ae2700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e40ff660 0x7f13e4194bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e8ae2700 1 -- 172.21.15.2:0/1343554237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13e40fecb0 msgr2=0x7f13e41946b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e8ae2700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13e40fecb0 0x7f13e41946b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.208+0000 7f13e8ae2700 1 -- 172.21.15.2:0/1343554237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80095f0 con 0x7f13e41000f0 2023-06-23T18:20:14.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.209+0000 7f13e8ae2700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e41000f0 0x7f13e419bc60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f13dc007f10 tx=0x7f13dc00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:14.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.209+0000 7f13e17fa700 1 -- 172.21.15.2:0/1343554237 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc003a00 con 0x7f13e41000f0 2023-06-23T18:20:14.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.209+0000 7f13e17fa700 1 -- 172.21.15.2:0/1343554237 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13dc004020 con 0x7f13e41000f0 2023-06-23T18:20:14.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.209+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e40fdc00 con 0x7f13e41000f0 2023-06-23T18:20:14.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.209+0000 7f13e17fa700 1 -- 172.21.15.2:0/1343554237 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13dc0126a0 con 0x7f13e41000f0 2023-06-23T18:20:14.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.209+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e40fe110 con 0x7f13e41000f0 2023-06-23T18:20:14.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.210+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e4066ac0 con 0x7f13e41000f0 2023-06-23T18:20:14.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.211+0000 7f13e17fa700 1 -- 172.21.15.2:0/1343554237 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f13dc004190 con 0x7f13e41000f0 2023-06-23T18:20:14.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.211+0000 7f13e17fa700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f13cc0a1150 0x7f13cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:14.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.212+0000 7f13e17fa700 1 -- 172.21.15.2:0/1343554237 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f13dc0ce9f0 con 0x7f13e41000f0 2023-06-23T18:20:14.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.212+0000 7f13e3fff700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f13cc0a1150 0x7f13cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:14.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.212+0000 7f13e3fff700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f13cc0a1150 0x7f13cc0a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13d4005950 tx=0x7f13d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:14.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.216+0000 7f13e17fa700 1 -- 172.21.15.2:0/1343554237 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f13dc098540 con 0x7f13e41000f0 2023-06-23T18:20:14.311 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:20:14 smithi082 systemd[1]: Starting Ceph mds.l for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:14.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.460+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f13e4105f80 con 0x7f13e41000f0 2023-06-23T18:20:14.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.461+0000 7f13e17fa700 1 -- 172.21.15.2:0/1343554237 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f13dc09bb60 con 0x7f13e41000f0 2023-06-23T18:20:14.467 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:14.467 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-06-23T18:20:14.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f13cc0a1150 msgr2=0x7f13cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:14.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f13cc0a1150 0x7f13cc0a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13d4005950 tx=0x7f13d400b410 comp rx=0 tx=0).stop 2023-06-23T18:20:14.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e41000f0 msgr2=0x7f13e419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:14.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e41000f0 0x7f13e419bc60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f13dc007f10 tx=0x7f13dc00f5c0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 shutdown_connections 2023-06-23T18:20:14.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f13e41000f0 0x7f13e419bc60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f13cc0a1150 0x7f13cc0a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13e40fecb0 0x7f13e41946b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 --2- 172.21.15.2:0/1343554237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13e40ff660 0x7f13e4194bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:14.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 >> 172.21.15.2:0/1343554237 conn(0x7f13e40f9810 msgr2=0x7f13e41031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:14.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 shutdown_connections 2023-06-23T18:20:14.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:14.463+0000 7f13ea545700 1 -- 172.21.15.2:0/1343554237 wait complete. 2023-06-23T18:20:14.471 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 1 2023-06-23T18:20:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:14 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3477476290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:14 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2016332403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:14 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1343554237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:14.848 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:20:14 smithi082 podman[123061]: 2023-06-23T18:20:14.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:14 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3477476290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:14.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:14 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2016332403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:14.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:14 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1343554237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:14.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:14 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3477476290' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:14.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:14 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2016332403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:14.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:14 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1343554237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:15.348 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:20:14 smithi082 ceph-mds[123101]: -- 172.21.15.82:0/3379293772 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56117fbe91e0 con 0x56117fbbe800 2023-06-23T18:20:15.349 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:20:14 smithi082 bash[123061]: eec95ac973566aab0bb3c3020e611fe0a4439e2e9130da9a2e70094755be3aae 2023-06-23T18:20:15.349 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:20:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-l[123077]: starting mds.l at 2023-06-23T18:20:15.349 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:20:14 smithi082 systemd[1]: Started Ceph mds.l for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:15.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:15.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.82:6832/2827087691,v1:172.21.15.82:6833/2827087691] up:boot 2023-06-23T18:20:15.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: fsmap 1 up:standby 2023-06-23T18:20:15.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-23T18:20:15.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:15.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:15.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:15.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:15.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:15.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:16.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:16.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.82:6832/2827087691,v1:172.21.15.82:6833/2827087691] up:boot 2023-06-23T18:20:16.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: fsmap 1 up:standby 2023-06-23T18:20:16.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-23T18:20:16.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:16.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:16.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:16.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:16.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:16.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:16.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:16.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.82:6832/2827087691,v1:172.21.15.82:6833/2827087691] up:boot 2023-06-23T18:20:16.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: fsmap 1 up:standby 2023-06-23T18:20:16.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-06-23T18:20:16.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:16.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:16.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:16.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:16.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:16.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:16.481 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:16.592 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:20:16 smithi017 systemd[1]: Starting Ceph mds.k for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:16.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 -- 172.21.15.2:0/2738320229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f80ffc50 msgr2=0x7f30f81000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 --2- 172.21.15.2:0/2738320229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f80ffc50 0x7f30f81000a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009990 tx=0x7f30f4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 -- 172.21.15.2:0/2738320229 shutdown_connections 2023-06-23T18:20:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 --2- 172.21.15.2:0/2738320229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f81005e0 0x7f30f8106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 --2- 172.21.15.2:0/2738320229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f80fea60 0x7f30f80fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 --2- 172.21.15.2:0/2738320229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f80ffc50 0x7f30f81000a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 -- 172.21.15.2:0/2738320229 >> 172.21.15.2:0/2738320229 conn(0x7f30f80fa0e0 msgr2=0x7f30f80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 -- 172.21.15.2:0/2738320229 shutdown_connections 2023-06-23T18:20:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.617+0000 7f30ff534700 1 -- 172.21.15.2:0/2738320229 wait complete. 2023-06-23T18:20:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30ff534700 1 Processor -- start 2023-06-23T18:20:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30ff534700 1 -- start start 2023-06-23T18:20:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30ff534700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f80fea60 0x7f30f81957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30ff534700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80ffc50 0x7f30f8195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:16.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30ff534700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f81005e0 0x7f30f819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30ff534700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f819d350 con 0x7f30f81005e0 2023-06-23T18:20:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30ff534700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f819d490 con 0x7f30f80ffc50 2023-06-23T18:20:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30ff534700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f819d620 con 0x7f30f80fea60 2023-06-23T18:20:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fdad1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f81005e0 0x7f30f819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fcacf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80ffc50 0x7f30f8195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fdad1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f81005e0 0x7f30f819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52130/0 (socket says 172.21.15.2:52130) 2023-06-23T18:20:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fdad1700 1 -- 172.21.15.2:0/486439877 learned_addr learned my addr 172.21.15.2:0/486439877 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:16.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fd2d0700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f80fea60 0x7f30f81957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fcacf700 1 -- 172.21.15.2:0/486439877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f80fea60 msgr2=0x7f30f81957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fcacf700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f80fea60 0x7f30f81957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fcacf700 1 -- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f81005e0 msgr2=0x7f30f819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fcacf700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f81005e0 0x7f30f819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fcacf700 1 -- 172.21.15.2:0/486439877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f40095f0 con 0x7f30f80ffc50 2023-06-23T18:20:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.618+0000 7f30fdad1700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f81005e0 0x7f30f819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:20:16.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.619+0000 7f30fcacf700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80ffc50 0x7f30f8195d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f30f4000c00 tx=0x7f30f4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.619+0000 7f30ea7fc700 1 -- 172.21.15.2:0/486439877 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f401d070 con 0x7f30f80ffc50 2023-06-23T18:20:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.619+0000 7f30ea7fc700 1 -- 172.21.15.2:0/486439877 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30f400bdd0 con 0x7f30f80ffc50 2023-06-23T18:20:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.619+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f819d8a0 con 0x7f30f80ffc50 2023-06-23T18:20:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.619+0000 7f30ea7fc700 1 -- 172.21.15.2:0/486439877 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4022db0 con 0x7f30f80ffc50 2023-06-23T18:20:16.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.619+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f819dd20 con 0x7f30f80ffc50 2023-06-23T18:20:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.620+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f810a130 con 0x7f30f80ffc50 2023-06-23T18:20:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.621+0000 7f30ea7fc700 1 -- 172.21.15.2:0/486439877 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f30f4022470 con 0x7f30f80ffc50 2023-06-23T18:20:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.622+0000 7f30ea7fc700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f30e40a11d0 0x7f30e40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.622+0000 7f30fd2d0700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f30e40a11d0 0x7f30e40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:16.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.622+0000 7f30ea7fc700 1 -- 172.21.15.2:0/486439877 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f30f40ce050 con 0x7f30f80ffc50 2023-06-23T18:20:16.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.622+0000 7f30fd2d0700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f30e40a11d0 0x7f30e40a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005950 tx=0x7f30ec0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:16.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.626+0000 7f30ea7fc700 1 -- 172.21.15.2:0/486439877 <== mon.2 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f30f4097c30 con 0x7f30f80ffc50 2023-06-23T18:20:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:16 smithi002 ceph-mon[95947]: Deploying daemon mds.k on smithi017 2023-06-23T18:20:16.859 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:16 smithi017 ceph-mon[97472]: Deploying daemon mds.k on smithi017 2023-06-23T18:20:16.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.869+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30f819e330 con 0x7f30f80ffc50 2023-06-23T18:20:16.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.870+0000 7f30ea7fc700 1 -- 172.21.15.2:0/486439877 <== mon.2 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1257 (secure 0 0 0) 0x7f30f409b250 con 0x7f30f80ffc50 2023-06-23T18:20:16.877 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:16.877 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-06-23T18:20:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f30e40a11d0 msgr2=0x7f30e40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f30e40a11d0 0x7f30e40a3680 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f30ec005950 tx=0x7f30ec0058e0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80ffc50 msgr2=0x7f30f8195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80ffc50 0x7f30f8195d10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f30f4000c00 tx=0x7f30f4004730 comp rx=0 tx=0).stop 2023-06-23T18:20:16.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 shutdown_connections 2023-06-23T18:20:16.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30f81005e0 0x7f30f819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30f80fea60 0x7f30f81957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f30e40a11d0 0x7f30e40a3680 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 --2- 172.21.15.2:0/486439877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30f80ffc50 0x7f30f8195d10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:16.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 >> 172.21.15.2:0/486439877 conn(0x7f30f80fa0e0 msgr2=0x7f30f8067860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 shutdown_connections 2023-06-23T18:20:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:16.872+0000 7f30ff534700 1 -- 172.21.15.2:0/486439877 wait complete. 2023-06-23T18:20:16.883 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 2 2023-06-23T18:20:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:16 smithi082 ceph-mon[97889]: Deploying daemon mds.k on smithi017 2023-06-23T18:20:17.128 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:20:16 smithi017 podman[121840]: 2023-06-23T18:20:17.440 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:20:17 smithi017 bash[121840]: 465a2a6199a4d341931c7fb701d3c8de629d5ae2c71482562cb87eddd96627ad 2023-06-23T18:20:17.440 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mds[121880]: -- 172.21.15.17:0/4143602500 <== mon.2 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5567cec4d1e0 con 0x5567cec22c00 2023-06-23T18:20:17.441 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-k[121856]: starting mds.k at 2023-06-23T18:20:17.441 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:20:17 smithi017 systemd[1]: Started Ceph mds.k for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:17.928 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:17 smithi002 ceph-mon[95947]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:17.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:17 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/486439877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:17.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:17.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:17.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:17.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:17.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:17.929 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:17.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mon[97472]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:17.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/486439877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:17.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:17.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:17.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:17.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:17.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:17.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:18.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:17 smithi082 ceph-mon[97889]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:17 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/486439877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:18.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:18.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:18.737 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:18 smithi002 ceph-mon[95947]: Deploying daemon mds.j on smithi002 2023-06-23T18:20:18.738 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:18 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.17:6832/719153877,v1:172.21.15.17:6833/719153877] up:boot 2023-06-23T18:20:18.738 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:18 smithi002 ceph-mon[95947]: fsmap 2 up:standby 2023-06-23T18:20:18.738 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:18 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-23T18:20:18.893 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:18.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:18 smithi017 ceph-mon[97472]: Deploying daemon mds.j on smithi002 2023-06-23T18:20:18.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:18 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.17:6832/719153877,v1:172.21.15.17:6833/719153877] up:boot 2023-06-23T18:20:18.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:18 smithi017 ceph-mon[97472]: fsmap 2 up:standby 2023-06-23T18:20:18.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:18 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-23T18:20:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 -- 172.21.15.2:0/2204780557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 msgr2=0x7fd6601000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 --2- 172.21.15.2:0/2204780557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 0x7fd6601000a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009990 tx=0x7fd65c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 -- 172.21.15.2:0/2204780557 shutdown_connections 2023-06-23T18:20:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 --2- 172.21.15.2:0/2204780557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6601005e0 0x7fd660106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 --2- 172.21.15.2:0/2204780557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6600fea60 0x7fd6600fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 --2- 172.21.15.2:0/2204780557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 0x7fd6601000a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 -- 172.21.15.2:0/2204780557 >> 172.21.15.2:0/2204780557 conn(0x7fd6600fa0e0 msgr2=0x7fd6600fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:19.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 -- 172.21.15.2:0/2204780557 shutdown_connections 2023-06-23T18:20:19.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.023+0000 7fd667b88700 1 -- 172.21.15.2:0/2204780557 wait complete. 2023-06-23T18:20:19.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.024+0000 7fd667b88700 1 Processor -- start 2023-06-23T18:20:19.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.024+0000 7fd667b88700 1 -- start start 2023-06-23T18:20:19.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.024+0000 7fd667b88700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6600fea60 0x7fd660068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:19.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.024+0000 7fd667b88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 0x7fd66019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:19.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.024+0000 7fd667b88700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6601005e0 0x7fd66019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:19.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.024+0000 7fd667b88700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66019f550 con 0x7fd6600ffc50 2023-06-23T18:20:19.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.024+0000 7fd667b88700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66019f6c0 con 0x7fd6600fea60 2023-06-23T18:20:19.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.024+0000 7fd667b88700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd66019f880 con 0x7fd6601005e0 2023-06-23T18:20:19.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 0x7fd66019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:19.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 0x7fd66019c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52142/0 (socket says 172.21.15.2:52142) 2023-06-23T18:20:19.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 -- 172.21.15.2:0/1272414315 learned_addr learned my addr 172.21.15.2:0/1272414315 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:19.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665924700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6600fea60 0x7fd660068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:19.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd666125700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6601005e0 0x7fd66019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:19.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 -- 172.21.15.2:0/1272414315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6601005e0 msgr2=0x7fd66019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:19.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6601005e0 0x7fd66019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 -- 172.21.15.2:0/1272414315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6600fea60 msgr2=0x7fd660068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6600fea60 0x7fd660068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 -- 172.21.15.2:0/1272414315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd65c0095f0 con 0x7fd6600ffc50 2023-06-23T18:20:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665924700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6600fea60 0x7fd660068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd665123700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 0x7fd66019c950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd65c003a50 tx=0x7fd65c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd656ffd700 1 -- 172.21.15.2:0/1272414315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c01d070 con 0x7fd6600ffc50 2023-06-23T18:20:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd656ffd700 1 -- 172.21.15.2:0/1272414315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd65c005a00 con 0x7fd6600ffc50 2023-06-23T18:20:19.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66019fb00 con 0x7fd6600ffc50 2023-06-23T18:20:19.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd656ffd700 1 -- 172.21.15.2:0/1272414315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65c005280 con 0x7fd6600ffc50 2023-06-23T18:20:19.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.025+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66019ffd0 con 0x7fd6600ffc50 2023-06-23T18:20:19.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.026+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd660105990 con 0x7fd6600ffc50 2023-06-23T18:20:19.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.027+0000 7fd656ffd700 1 -- 172.21.15.2:0/1272414315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd65c0053e0 con 0x7fd6600ffc50 2023-06-23T18:20:19.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.028+0000 7fd656ffd700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd64c0a1150 0x7fd64c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:19.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.028+0000 7fd665924700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd64c0a1150 0x7fd64c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:19.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.028+0000 7fd656ffd700 1 -- 172.21.15.2:0/1272414315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd65c0cd5e0 con 0x7fd6600ffc50 2023-06-23T18:20:19.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.028+0000 7fd665924700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd64c0a1150 0x7fd64c0a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd6600ffab0 tx=0x7fd650008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:19.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.032+0000 7fd656ffd700 1 -- 172.21.15.2:0/1272414315 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd65c05b770 con 0x7fd6600ffc50 2023-06-23T18:20:19.051 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:20:18 smithi002 systemd[1]: Starting Ceph mds.j for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:19.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:18 smithi082 ceph-mon[97889]: Deploying daemon mds.j on smithi002 2023-06-23T18:20:19.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:18 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.17:6832/719153877,v1:172.21.15.17:6833/719153877] up:boot 2023-06-23T18:20:19.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:18 smithi082 ceph-mon[97889]: fsmap 2 up:standby 2023-06-23T18:20:19.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:18 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-06-23T18:20:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.284+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd66002c810 con 0x7fd6600ffc50 2023-06-23T18:20:19.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.284+0000 7fd656ffd700 1 -- 172.21.15.2:0/1272414315 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2011 (secure 0 0 0) 0x7fd65c097260 con 0x7fd6600ffc50 2023-06-23T18:20:19.290 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:19.291 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-23T18:20:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd64c0a1150 msgr2=0x7fd64c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd64c0a1150 0x7fd64c0a3600 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd6600ffab0 tx=0x7fd650008040 comp rx=0 tx=0).stop 2023-06-23T18:20:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 msgr2=0x7fd66019c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 0x7fd66019c950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd65c003a50 tx=0x7fd65c009580 comp rx=0 tx=0).stop 2023-06-23T18:20:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 shutdown_connections 2023-06-23T18:20:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6601005e0 0x7fd66019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd64c0a1150 0x7fd64c0a3600 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6600fea60 0x7fd660068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 --2- 172.21.15.2:0/1272414315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd6600ffc50 0x7fd66019c950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 >> 172.21.15.2:0/1272414315 conn(0x7fd6600fa0e0 msgr2=0x7fd6600fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 shutdown_connections 2023-06-23T18:20:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:19.286+0000 7fd667b88700 1 -- 172.21.15.2:0/1272414315 wait complete. 2023-06-23T18:20:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2023-06-23T18:20:19.340 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:20:19 smithi002 podman[129375]: 2023-06-23T18:20:19.670 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mds[129460]: -- 172.21.15.2:0/686536488 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5563606891e0 con 0x55636065ec00 2023-06-23T18:20:19.671 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-j[129434]: starting mds.j at 2023-06-23T18:20:19.671 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:20:19 smithi002 bash[129375]: 82c5d9966d7d4a9361674da7bbb1638811162b4288ae869fd10127a840330112 2023-06-23T18:20:19.671 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:20:19 smithi002 systemd[1]: Started Ceph mds.j for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:19.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:19 smithi017 ceph-mon[97472]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:19.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:19 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1272414315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:19.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:19.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:19.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:19.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:19.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:19.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:20.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mon[95947]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:20.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1272414315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:20.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:19 smithi082 ceph-mon[97889]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:19 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1272414315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:20.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:20.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:20 smithi017 ceph-mon[97472]: Deploying daemon mds.d on smithi002 2023-06-23T18:20:20.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:20 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.2:6834/1508985058,v1:172.21.15.2:6835/1508985058] up:boot 2023-06-23T18:20:20.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:20 smithi017 ceph-mon[97472]: fsmap 3 up:standby 2023-06-23T18:20:20.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:20 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-23T18:20:20.978 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:20 smithi002 ceph-mon[95947]: Deploying daemon mds.d on smithi002 2023-06-23T18:20:20.978 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:20 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.2:6834/1508985058,v1:172.21.15.2:6835/1508985058] up:boot 2023-06-23T18:20:20.978 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:20 smithi002 ceph-mon[95947]: fsmap 3 up:standby 2023-06-23T18:20:20.978 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:20 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-23T18:20:21.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:20 smithi082 ceph-mon[97889]: Deploying daemon mds.d on smithi002 2023-06-23T18:20:21.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:20 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.2:6834/1508985058,v1:172.21.15.2:6835/1508985058] up:boot 2023-06-23T18:20:21.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:20 smithi082 ceph-mon[97889]: fsmap 3 up:standby 2023-06-23T18:20:21.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:20 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-06-23T18:20:21.273 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:20:20 smithi002 systemd[1]: Starting Ceph mds.d for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:21.305 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:21.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.438+0000 7fec7dfd8700 1 -- 172.21.15.2:0/1686751648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec781017c0 msgr2=0x7fec78108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:21.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.438+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/1686751648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec781017c0 0x7fec78108060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fec7000b0a0 tx=0x7fec7000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.438+0000 7fec7dfd8700 1 -- 172.21.15.2:0/1686751648 shutdown_connections 2023-06-23T18:20:21.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.438+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/1686751648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec781017c0 0x7fec78108060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.438+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/1686751648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec780ffb80 0x7fec780fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.438+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/1686751648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec78100e10 0x7fec78101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.438+0000 7fec7dfd8700 1 -- 172.21.15.2:0/1686751648 >> 172.21.15.2:0/1686751648 conn(0x7fec780fb1c0 msgr2=0x7fec780fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:21.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.439+0000 7fec7dfd8700 1 -- 172.21.15.2:0/1686751648 shutdown_connections 2023-06-23T18:20:21.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.439+0000 7fec7dfd8700 1 -- 172.21.15.2:0/1686751648 wait complete. 2023-06-23T18:20:21.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.439+0000 7fec7dfd8700 1 Processor -- start 2023-06-23T18:20:21.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec7dfd8700 1 -- start start 2023-06-23T18:20:21.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec7dfd8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec780ffb80 0x7fec78195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:21.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec7dfd8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec78100e10 0x7fec78195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:21.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec7dfd8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec781017c0 0x7fec7819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:21.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec7dfd8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7819d3f0 con 0x7fec78100e10 2023-06-23T18:20:21.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec7dfd8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7819d530 con 0x7fec781017c0 2023-06-23T18:20:21.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec7dfd8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7819d6c0 con 0x7fec780ffb80 2023-06-23T18:20:21.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec78100e10 0x7fec78195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:21.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec78100e10 0x7fec78195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52164/0 (socket says 172.21.15.2:52164) 2023-06-23T18:20:21.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 -- 172.21.15.2:0/2377440450 learned_addr learned my addr 172.21.15.2:0/2377440450 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:21.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec777fe700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec780ffb80 0x7fec78195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:21.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec77fff700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec781017c0 0x7fec7819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:21.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 -- 172.21.15.2:0/2377440450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec780ffb80 msgr2=0x7fec78195870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec780ffb80 0x7fec78195870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 -- 172.21.15.2:0/2377440450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec781017c0 msgr2=0x7fec7819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec781017c0 0x7fec7819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 -- 172.21.15.2:0/2377440450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec70009d00 con 0x7fec78100e10 2023-06-23T18:20:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec77fff700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec781017c0 0x7fec7819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.440+0000 7fec76ffd700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec78100e10 0x7fec78195db0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fec6800d950 tx=0x7fec6800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.441+0000 7fec74ff9700 1 -- 172.21.15.2:0/2377440450 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68009940 con 0x7fec78100e10 2023-06-23T18:20:21.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.441+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7819d9a0 con 0x7fec78100e10 2023-06-23T18:20:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.441+0000 7fec74ff9700 1 -- 172.21.15.2:0/2377440450 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec68010460 con 0x7fec78100e10 2023-06-23T18:20:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.441+0000 7fec74ff9700 1 -- 172.21.15.2:0/2377440450 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6800f620 con 0x7fec78100e10 2023-06-23T18:20:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.441+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7819deb0 con 0x7fec78100e10 2023-06-23T18:20:21.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.442+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7810b1b0 con 0x7fec78100e10 2023-06-23T18:20:21.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.443+0000 7fec74ff9700 1 -- 172.21.15.2:0/2377440450 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fec680105d0 con 0x7fec78100e10 2023-06-23T18:20:21.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.443+0000 7fec74ff9700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec600a1150 0x7fec600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:21.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.443+0000 7fec777fe700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec600a1150 0x7fec600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:21.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.444+0000 7fec74ff9700 1 -- 172.21.15.2:0/2377440450 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fec680cc280 con 0x7fec78100e10 2023-06-23T18:20:21.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.444+0000 7fec777fe700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec600a1150 0x7fec600a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009c60 tx=0x7fec6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:21.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.447+0000 7fec74ff9700 1 -- 172.21.15.2:0/2377440450 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fec68095e70 con 0x7fec78100e10 2023-06-23T18:20:21.551 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:20:21 smithi002 podman[129757]: 2023-06-23T18:20:21.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.716+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec7819e520 con 0x7fec78100e10 2023-06-23T18:20:21.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.718+0000 7fec74ff9700 1 -- 172.21.15.2:0/2377440450 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3519 (secure 0 0 0) 0x7fec68016dd0 con 0x7fec78100e10 2023-06-23T18:20:21.724 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:21.725 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-23T18:20:21.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec600a1150 msgr2=0x7fec600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:21.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec600a1150 0x7fec600a3600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009c60 tx=0x7fec6c009260 comp rx=0 tx=0).stop 2023-06-23T18:20:21.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec78100e10 msgr2=0x7fec78195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec78100e10 0x7fec78195db0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fec6800d950 tx=0x7fec6800dc60 comp rx=0 tx=0).stop 2023-06-23T18:20:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 shutdown_connections 2023-06-23T18:20:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec600a1150 0x7fec600a3600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec781017c0 0x7fec7819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec780ffb80 0x7fec78195870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.720+0000 7fec7dfd8700 1 --2- 172.21.15.2:0/2377440450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec78100e10 0x7fec78195db0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.721+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 >> 172.21.15.2:0/2377440450 conn(0x7fec780fb1c0 msgr2=0x7fec780fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.721+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 shutdown_connections 2023-06-23T18:20:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:21.721+0000 7fec7dfd8700 1 -- 172.21.15.2:0/2377440450 wait complete. 2023-06-23T18:20:21.728 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 5 2023-06-23T18:20:21.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:21 smithi017 ceph-mon[97472]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:22.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:21 smithi002 ceph-mon[95947]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:22.052 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:20:21 smithi002 ceph-mds[129840]: -- 172.21.15.2:0/2532753130 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5565e0f931e0 con 0x5565e0f68800 2023-06-23T18:20:22.052 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:20:21 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-d[129796]: starting mds.d at 2023-06-23T18:20:22.052 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:20:21 smithi002 bash[129757]: 83dec644cca9e40d68ef1fdf0b1dc8bc52ced7da866e6fed2e185f5875c5502c 2023-06-23T18:20:22.052 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:20:21 smithi002 systemd[1]: Started Ceph mds.d for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:22.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:21 smithi082 ceph-mon[97889]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.2:6836/3317629497,v1:172.21.15.2:6837/3317629497] up:boot 2023-06-23T18:20:22.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: fsmap 4 up:standby 2023-06-23T18:20:22.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-23T18:20:22.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:22.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2377440450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:22.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:22.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:22.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:22.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:22.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:23.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.2:6836/3317629497,v1:172.21.15.2:6837/3317629497] up:boot 2023-06-23T18:20:23.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: fsmap 4 up:standby 2023-06-23T18:20:23.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-23T18:20:23.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:23.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2377440450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:23.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:23.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:23.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:23.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.2:6836/3317629497,v1:172.21.15.2:6837/3317629497] up:boot 2023-06-23T18:20:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: fsmap 4 up:standby 2023-06-23T18:20:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-06-23T18:20:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2377440450' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:23.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:23.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:23.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:23.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:23.296 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:20:23 smithi017 systemd[1]: Starting Ceph mds.e for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:23.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:23.857 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:20:23 smithi017 podman[122185]: 2023-06-23T18:20:23.857 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:23 smithi017 ceph-mon[97472]: Deploying daemon mds.e on smithi017 2023-06-23T18:20:23.858 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:23 smithi017 ceph-mon[97472]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:23.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 -- 172.21.15.2:0/1564334370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 msgr2=0x7f2a70102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:23.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/1564334370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 0x7f2a70102a60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009990 tx=0x7f2a64009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:23.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 -- 172.21.15.2:0/1564334370 shutdown_connections 2023-06-23T18:20:23.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/1564334370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a701058c0 0x7f2a70107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:23.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/1564334370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 0x7f2a70102a60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:23.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/1564334370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a70102fa0 0x7f2a70105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:23.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 -- 172.21.15.2:0/1564334370 >> 172.21.15.2:0/1564334370 conn(0x7f2a700fa0e0 msgr2=0x7f2a700fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:23.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 -- 172.21.15.2:0/1564334370 shutdown_connections 2023-06-23T18:20:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.879+0000 7f2a77e8c700 1 -- 172.21.15.2:0/1564334370 wait complete. 2023-06-23T18:20:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a77e8c700 1 Processor -- start 2023-06-23T18:20:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a77e8c700 1 -- start start 2023-06-23T18:20:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a77e8c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 0x7f2a70195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:23.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a77e8c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a70102fa0 0x7f2a70195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a77e8c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a701058c0 0x7f2a7019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a77e8c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7019d380 con 0x7f2a70100680 2023-06-23T18:20:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a77e8c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7019d4c0 con 0x7f2a70102fa0 2023-06-23T18:20:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a77e8c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7019d650 con 0x7f2a701058c0 2023-06-23T18:20:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75c28700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 0x7f2a70195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:23.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75c28700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 0x7f2a70195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52188/0 (socket says 172.21.15.2:52188) 2023-06-23T18:20:23.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75c28700 1 -- 172.21.15.2:0/2953599766 learned_addr learned my addr 172.21.15.2:0/2953599766 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:23.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75427700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a70102fa0 0x7f2a70195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:23.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a76429700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a701058c0 0x7f2a7019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:23.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75c28700 1 -- 172.21.15.2:0/2953599766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a701058c0 msgr2=0x7f2a7019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:23.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75c28700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a701058c0 0x7f2a7019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:23.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75c28700 1 -- 172.21.15.2:0/2953599766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a70102fa0 msgr2=0x7f2a70195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:23.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75c28700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a70102fa0 0x7f2a70195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.880+0000 7f2a75c28700 1 -- 172.21.15.2:0/2953599766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a640095f0 con 0x7f2a70100680 2023-06-23T18:20:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.881+0000 7f2a75c28700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 0x7f2a70195800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2a640098e0 tx=0x7f2a64004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.881+0000 7f2a62ffd700 1 -- 172.21.15.2:0/2953599766 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6401d070 con 0x7f2a70100680 2023-06-23T18:20:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.881+0000 7f2a62ffd700 1 -- 172.21.15.2:0/2953599766 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a6400be30 con 0x7f2a70100680 2023-06-23T18:20:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.881+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7019d8d0 con 0x7f2a70100680 2023-06-23T18:20:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.881+0000 7f2a62ffd700 1 -- 172.21.15.2:0/2953599766 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a64022e00 con 0x7f2a70100680 2023-06-23T18:20:23.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.881+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7019dcf0 con 0x7f2a70100680 2023-06-23T18:20:23.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.882+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a700fcc60 con 0x7f2a70100680 2023-06-23T18:20:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.883+0000 7f2a62ffd700 1 -- 172.21.15.2:0/2953599766 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2a6400b9a0 con 0x7f2a70100680 2023-06-23T18:20:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.884+0000 7f2a62ffd700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2a5c0a1150 0x7f2a5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.884+0000 7f2a75427700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2a5c0a1150 0x7f2a5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.884+0000 7f2a62ffd700 1 -- 172.21.15.2:0/2953599766 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a640cdde0 con 0x7f2a70100680 2023-06-23T18:20:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.884+0000 7f2a75427700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2a5c0a1150 0x7f2a5c0a3600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005950 tx=0x7f2a6c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:23.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:23.887+0000 7f2a62ffd700 1 -- 172.21.15.2:0/2953599766 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2a640979c0 con 0x7f2a70100680 2023-06-23T18:20:24.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:23 smithi002 ceph-mon[95947]: Deploying daemon mds.e on smithi017 2023-06-23T18:20:24.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:23 smithi002 ceph-mon[95947]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:24.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:23 smithi082 ceph-mon[97889]: Deploying daemon mds.e on smithi017 2023-06-23T18:20:24.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:23 smithi082 ceph-mon[97889]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:24.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.136+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a7002c810 con 0x7f2a70100680 2023-06-23T18:20:24.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.136+0000 7f2a62ffd700 1 -- 172.21.15.2:0/2953599766 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3519 (secure 0 0 0) 0x7f2a64027dd0 con 0x7f2a70100680 2023-06-23T18:20:24.143 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:24.143 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-06-23T18:20:24.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2a5c0a1150 msgr2=0x7f2a5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:24.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2a5c0a1150 0x7f2a5c0a3600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c005950 tx=0x7f2a6c00b410 comp rx=0 tx=0).stop 2023-06-23T18:20:24.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 msgr2=0x7f2a70195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:24.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 0x7f2a70195800 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2a640098e0 tx=0x7f2a64004730 comp rx=0 tx=0).stop 2023-06-23T18:20:24.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 shutdown_connections 2023-06-23T18:20:24.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a701058c0 0x7f2a7019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:24.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2a70100680 0x7f2a70195800 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:24.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2a5c0a1150 0x7f2a5c0a3600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:24.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 --2- 172.21.15.2:0/2953599766 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a70102fa0 0x7f2a70195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:24.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 >> 172.21.15.2:0/2953599766 conn(0x7f2a700fa0e0 msgr2=0x7f2a701020c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:24.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 shutdown_connections 2023-06-23T18:20:24.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:24.139+0000 7f2a77e8c700 1 -- 172.21.15.2:0/2953599766 wait complete. 2023-06-23T18:20:24.147 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 5 2023-06-23T18:20:24.190 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:20:23 smithi017 ceph-mds[122225]: -- 172.21.15.17:0/2262126636 <== mon.2 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ba9836d1e0 con 0x55ba98342c00 2023-06-23T18:20:24.190 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:20:23 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-e[122201]: starting mds.e at 2023-06-23T18:20:24.190 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:20:23 smithi017 bash[122185]: 469345b6da2880ac271e5d348818ac89d60344c773909d4ce2d0800a6c410c02 2023-06-23T18:20:24.191 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:20:24 smithi017 systemd[1]: Started Ceph mds.e for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:25.330 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.331 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.331 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2953599766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:25.331 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.331 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:25.331 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:25.331 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:25.332 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: Deploying daemon mds.f on smithi082 2023-06-23T18:20:25.332 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.17:6834/2425660103,v1:172.21.15.17:6835/2425660103] up:boot 2023-06-23T18:20:25.332 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: fsmap 5 up:standby 2023-06-23T18:20:25.332 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-23T18:20:25.332 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:25 smithi082 ceph-mon[97889]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2953599766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: Deploying daemon mds.f on smithi082 2023-06-23T18:20:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.17:6834/2425660103,v1:172.21.15.17:6835/2425660103] up:boot 2023-06-23T18:20:25.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: fsmap 5 up:standby 2023-06-23T18:20:25.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-23T18:20:25.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:25 smithi017 ceph-mon[97472]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:25.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2953599766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:25.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:25.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:25.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:25.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:25.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: Deploying daemon mds.f on smithi082 2023-06-23T18:20:25.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.17:6834/2425660103,v1:172.21.15.17:6835/2425660103] up:boot 2023-06-23T18:20:25.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: fsmap 5 up:standby 2023-06-23T18:20:25.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-06-23T18:20:25.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:25 smithi002 ceph-mon[95947]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:25.598 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:20:25 smithi082 systemd[1]: Starting Ceph mds.f for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:26.157 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:26.164 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:20:25 smithi082 podman[123408]: 2023-06-23T18:20:26.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 -- 172.21.15.2:0/1219671849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d081058c0 msgr2=0x7f4d08107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:26.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 --2- 172.21.15.2:0/1219671849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d081058c0 0x7f4d08107ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000b0a0 tx=0x7f4d0000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 -- 172.21.15.2:0/1219671849 shutdown_connections 2023-06-23T18:20:26.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 --2- 172.21.15.2:0/1219671849 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d081058c0 0x7f4d08107ca0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 --2- 172.21.15.2:0/1219671849 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d08100680 0x7f4d08102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 --2- 172.21.15.2:0/1219671849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d08102fa0 0x7f4d08105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 -- 172.21.15.2:0/1219671849 >> 172.21.15.2:0/1219671849 conn(0x7f4d080fa0e0 msgr2=0x7f4d080fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:26.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 -- 172.21.15.2:0/1219671849 shutdown_connections 2023-06-23T18:20:26.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.293+0000 7f4d102d8700 1 -- 172.21.15.2:0/1219671849 wait complete. 2023-06-23T18:20:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.294+0000 7f4d102d8700 1 Processor -- start 2023-06-23T18:20:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.294+0000 7f4d102d8700 1 -- start start 2023-06-23T18:20:26.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d102d8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d08100680 0x7f4d081958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d102d8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d08102fa0 0x7f4d08195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d102d8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d081058c0 0x7f4d0819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d102d8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0819d430 con 0x7f4d08100680 2023-06-23T18:20:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d102d8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0819d570 con 0x7f4d081058c0 2023-06-23T18:20:26.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d102d8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0819d700 con 0x7f4d08102fa0 2023-06-23T18:20:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d08100680 0x7f4d081958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d08100680 0x7f4d081958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52202/0 (socket says 172.21.15.2:52202) 2023-06-23T18:20:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 -- 172.21.15.2:0/739848890 learned_addr learned my addr 172.21.15.2:0/739848890 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:26.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0d873700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d08102fa0 0x7f4d08195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:26.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e875700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d081058c0 0x7f4d0819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:26.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 -- 172.21.15.2:0/739848890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d08102fa0 msgr2=0x7f4d08195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:26.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d08102fa0 0x7f4d08195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 -- 172.21.15.2:0/739848890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d081058c0 msgr2=0x7f4d0819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:26.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d081058c0 0x7f4d0819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 -- 172.21.15.2:0/739848890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00009d00 con 0x7f4d08100680 2023-06-23T18:20:26.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d0e074700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d08100680 0x7f4d081958b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400d980 tx=0x7f4d0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:26.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4cff7fe700 1 -- 172.21.15.2:0/739848890 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04004220 con 0x7f4d08100680 2023-06-23T18:20:26.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0819d9e0 con 0x7f4d08100680 2023-06-23T18:20:26.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.295+0000 7f4cff7fe700 1 -- 172.21.15.2:0/739848890 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d04007640 con 0x7f4d08100680 2023-06-23T18:20:26.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.296+0000 7f4cff7fe700 1 -- 172.21.15.2:0/739848890 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d040055e0 con 0x7f4d08100680 2023-06-23T18:20:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.296+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0819df40 con 0x7f4d08100680 2023-06-23T18:20:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.296+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d080fcc60 con 0x7f4d08100680 2023-06-23T18:20:26.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.298+0000 7f4cff7fe700 1 -- 172.21.15.2:0/739848890 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4d04004750 con 0x7f4d08100680 2023-06-23T18:20:26.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.298+0000 7f4cff7fe700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4cf40a11d0 0x7f4cf40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.298+0000 7f4d0d873700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4cf40a11d0 0x7f4cf40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.299+0000 7f4cff7fe700 1 -- 172.21.15.2:0/739848890 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d040cc5d0 con 0x7f4d08100680 2023-06-23T18:20:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.299+0000 7f4d0d873700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4cf40a11d0 0x7f4cf40a3680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8005950 tx=0x7f4cf800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:26.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.302+0000 7f4cff7fe700 1 -- 172.21.15.2:0/739848890 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4d040961c0 con 0x7f4d08100680 2023-06-23T18:20:26.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.550+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d0802caa0 con 0x7f4d08100680 2023-06-23T18:20:26.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.550+0000 7f4cff7fe700 1 -- 172.21.15.2:0/739848890 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4276 (secure 0 0 0) 0x7f4d04005740 con 0x7f4d08100680 2023-06-23T18:20:26.557 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:26.557 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-06-23T18:20:26.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4cf40a11d0 msgr2=0x7f4cf40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4cf40a11d0 0x7f4cf40a3680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8005950 tx=0x7f4cf800b410 comp rx=0 tx=0).stop 2023-06-23T18:20:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d08100680 msgr2=0x7f4d081958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d08100680 0x7f4d081958b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d0400d980 tx=0x7f4d0400dc90 comp rx=0 tx=0).stop 2023-06-23T18:20:26.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 shutdown_connections 2023-06-23T18:20:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4cf40a11d0 0x7f4cf40a3680 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d081058c0 0x7f4d0819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d08100680 0x7f4d081958b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 --2- 172.21.15.2:0/739848890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d08102fa0 0x7f4d08195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 >> 172.21.15.2:0/739848890 conn(0x7f4d080fa0e0 msgr2=0x7f4d08101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 shutdown_connections 2023-06-23T18:20:26.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:26.553+0000 7f4d102d8700 1 -- 172.21.15.2:0/739848890 wait complete. 2023-06-23T18:20:26.561 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 6 2023-06-23T18:20:26.599 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:20:26 smithi082 ceph-mds[123449]: -- 172.21.15.82:0/3439597745 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5650bc8ff1e0 con 0x5650bc8d4c00 2023-06-23T18:20:26.599 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:20:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-f[123424]: starting mds.f at 2023-06-23T18:20:26.599 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:20:26 smithi082 bash[123408]: 03d890d4768be5bd5507db57c0ed8bc8a40ac0b396ac0acc1c9445ebbaa5eac9 2023-06-23T18:20:26.599 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:20:26 smithi082 systemd[1]: Started Ceph mds.f for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:27.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:27.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: Deploying daemon mds.c on smithi082 2023-06-23T18:20:27.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/739848890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:27.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.82:6834/2287539900,v1:172.21.15.82:6835/2287539900] up:boot 2023-06-23T18:20:27.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: fsmap 6 up:standby 2023-06-23T18:20:27.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-23T18:20:27.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:27 smithi082 ceph-mon[97889]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: Deploying daemon mds.c on smithi082 2023-06-23T18:20:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/739848890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.82:6834/2287539900,v1:172.21.15.82:6835/2287539900] up:boot 2023-06-23T18:20:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: fsmap 6 up:standby 2023-06-23T18:20:27.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-23T18:20:27.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:27 smithi017 ceph-mon[97472]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:27.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: Deploying daemon mds.c on smithi082 2023-06-23T18:20:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/739848890' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.82:6834/2287539900,v1:172.21.15.82:6835/2287539900] up:boot 2023-06-23T18:20:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: fsmap 6 up:standby 2023-06-23T18:20:27.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-06-23T18:20:27.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:27 smithi002 ceph-mon[95947]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:27.956 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:20:27 smithi082 systemd[1]: Starting Ceph mds.c for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:28.522 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:20:28 smithi082 podman[123747]: 2023-06-23T18:20:28.572 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 -- 172.21.15.2:0/41685088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d4c102f60 msgr2=0x7f8d4c105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/41685088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d4c102f60 0x7f8d4c105340 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009990 tx=0x7f8d40009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 -- 172.21.15.2:0/41685088 shutdown_connections 2023-06-23T18:20:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/41685088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d4c105880 0x7f8d4c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/41685088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d4c100640 0x7f8d4c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/41685088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d4c102f60 0x7f8d4c105340 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 -- 172.21.15.2:0/41685088 >> 172.21.15.2:0/41685088 conn(0x7f8d4c0fa0e0 msgr2=0x7f8d4c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 -- 172.21.15.2:0/41685088 shutdown_connections 2023-06-23T18:20:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.710+0000 7f8d51cd0700 1 -- 172.21.15.2:0/41685088 wait complete. 2023-06-23T18:20:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d51cd0700 1 Processor -- start 2023-06-23T18:20:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d51cd0700 1 -- start start 2023-06-23T18:20:28.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d51cd0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d4c100640 0x7f8d4c1957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:28.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d51cd0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d4c102f60 0x7f8d4c195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:28.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d51cd0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d4c105880 0x7f8d4c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:28.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d51cd0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c19d330 con 0x7f8d4c100640 2023-06-23T18:20:28.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d51cd0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c19d470 con 0x7f8d4c102f60 2023-06-23T18:20:28.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d51cd0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c19d600 con 0x7f8d4c105880 2023-06-23T18:20:28.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d4c100640 0x7f8d4c1957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:28.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d4c100640 0x7f8d4c1957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52216/0 (socket says 172.21.15.2:52216) 2023-06-23T18:20:28.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4b7fe700 1 -- 172.21.15.2:0/3946554553 learned_addr learned my addr 172.21.15.2:0/3946554553 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:28.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4affd700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d4c102f60 0x7f8d4c195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:28.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4bfff700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d4c105880 0x7f8d4c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:28.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4b7fe700 1 -- 172.21.15.2:0/3946554553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d4c105880 msgr2=0x7f8d4c19cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4b7fe700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d4c105880 0x7f8d4c19cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4b7fe700 1 -- 172.21.15.2:0/3946554553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d4c102f60 msgr2=0x7f8d4c195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.711+0000 7f8d4b7fe700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d4c102f60 0x7f8d4c195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.712+0000 7f8d4b7fe700 1 -- 172.21.15.2:0/3946554553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400095f0 con 0x7f8d4c100640 2023-06-23T18:20:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.712+0000 7f8d4bfff700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d4c105880 0x7f8d4c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.712+0000 7f8d4b7fe700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d4c100640 0x7f8d4c1957b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00e9a0 tx=0x7f8d3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:28.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.712+0000 7f8d48ff9700 1 -- 172.21.15.2:0/3946554553 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c009790 con 0x7f8d4c100640 2023-06-23T18:20:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.712+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c19d8e0 con 0x7f8d4c100640 2023-06-23T18:20:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.712+0000 7f8d48ff9700 1 -- 172.21.15.2:0/3946554553 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d3c00cbd0 con 0x7f8d4c100640 2023-06-23T18:20:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.712+0000 7f8d48ff9700 1 -- 172.21.15.2:0/3946554553 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c018680 con 0x7f8d4c100640 2023-06-23T18:20:28.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.712+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c19ddf0 con 0x7f8d4c100640 2023-06-23T18:20:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.713+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c0fcc60 con 0x7f8d4c100640 2023-06-23T18:20:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.714+0000 7f8d48ff9700 1 -- 172.21.15.2:0/3946554553 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8d3c00cd40 con 0x7f8d4c100640 2023-06-23T18:20:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.715+0000 7f8d48ff9700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8d340a1150 0x7f8d340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:28.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.715+0000 7f8d4affd700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8d340a1150 0x7f8d340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:28.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.715+0000 7f8d48ff9700 1 -- 172.21.15.2:0/3946554553 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d3c014070 con 0x7f8d4c100640 2023-06-23T18:20:28.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.715+0000 7f8d4affd700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8d340a1150 0x7f8d340a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006010 tx=0x7f8d40005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:28.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.720+0000 7f8d48ff9700 1 -- 172.21.15.2:0/3946554553 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8d3c051770 con 0x7f8d4c100640 2023-06-23T18:20:28.848 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:20:28 smithi082 ceph-mds[123787]: -- 172.21.15.82:0/2880949303 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561a3be851e0 con 0x561a3be5b000 2023-06-23T18:20:28.849 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:20:28 smithi082 bash[123747]: 6e552866e880f1a6077d0bcbfdae3f71e2a428d8dab79984d2269a0bdf1fb766 2023-06-23T18:20:28.849 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:20:28 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-c[123763]: starting mds.c at 2023-06-23T18:20:28.849 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:20:28 smithi082 systemd[1]: Started Ceph mds.c for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:28.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.963+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d4c02c810 con 0x7f8d4c100640 2023-06-23T18:20:28.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.964+0000 7f8d48ff9700 1 -- 172.21.15.2:0/3946554553 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5790 (secure 0 0 0) 0x7f8d3c096300 con 0x7f8d4c100640 2023-06-23T18:20:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:28.970 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-06-23T18:20:28.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8d340a1150 msgr2=0x7f8d340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8d340a1150 0x7f8d340a3600 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d40006010 tx=0x7f8d40005960 comp rx=0 tx=0).stop 2023-06-23T18:20:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d4c100640 msgr2=0x7f8d4c1957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d4c100640 0x7f8d4c1957b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00e9a0 tx=0x7f8d3c00ecb0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 shutdown_connections 2023-06-23T18:20:28.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d4c105880 0x7f8d4c19cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d4c100640 0x7f8d4c1957b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8d340a1150 0x7f8d340a3600 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 --2- 172.21.15.2:0/3946554553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8d4c102f60 0x7f8d4c195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 >> 172.21.15.2:0/3946554553 conn(0x7f8d4c0fa0e0 msgr2=0x7f8d4c101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 shutdown_connections 2023-06-23T18:20:28.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:28.966+0000 7f8d51cd0700 1 -- 172.21.15.2:0/3946554553 wait complete. 2023-06-23T18:20:28.974 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-06-23T18:20:29.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: Deploying daemon mds.h on smithi017 2023-06-23T18:20:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] up:boot 2023-06-23T18:20:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: fsmap 7 up:standby 2023-06-23T18:20:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-23T18:20:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:29.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:29 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3946554553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:29.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:29.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: Deploying daemon mds.h on smithi017 2023-06-23T18:20:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] up:boot 2023-06-23T18:20:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: fsmap 7 up:standby 2023-06-23T18:20:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-23T18:20:29.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:29.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:29 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3946554553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:30.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:30.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:30.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:30.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:30.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:30.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:30.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: Deploying daemon mds.h on smithi017 2023-06-23T18:20:30.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] up:boot 2023-06-23T18:20:30.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: fsmap 7 up:standby 2023-06-23T18:20:30.057 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-06-23T18:20:30.057 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:30.057 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:29 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3946554553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:30.765 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:20:30 smithi017 systemd[1]: Starting Ceph mds.h for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:30.987 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:30 smithi082 ceph-mon[97889]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:31.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.121+0000 7fd08b7b7700 1 -- 172.21.15.2:0/1000600658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0840fea60 msgr2=0x7fd0840fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:31.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.121+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/1000600658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0840fea60 0x7fd0840fee70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd080009990 tx=0x7fd080009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.121+0000 7fd08b7b7700 1 -- 172.21.15.2:0/1000600658 shutdown_connections 2023-06-23T18:20:31.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/1000600658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0841005e0 0x7fd084106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/1000600658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0840fea60 0x7fd0840fee70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/1000600658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0840ffc50 0x7fd0841000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 -- 172.21.15.2:0/1000600658 >> 172.21.15.2:0/1000600658 conn(0x7fd0840fa0e0 msgr2=0x7fd0840fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 -- 172.21.15.2:0/1000600658 shutdown_connections 2023-06-23T18:20:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 -- 172.21.15.2:0/1000600658 wait complete. 2023-06-23T18:20:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 Processor -- start 2023-06-23T18:20:31.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 -- start start 2023-06-23T18:20:31.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0840fea60 0x7fd084197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:31.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.122+0000 7fd08b7b7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0840ffc50 0x7fd084197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:31.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd08b7b7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0841005e0 0x7fd08419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:31.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd08b7b7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08419f5e0 con 0x7fd0841005e0 2023-06-23T18:20:31.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd08b7b7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08419f720 con 0x7fd0840fea60 2023-06-23T18:20:31.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd08b7b7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08419f8b0 con 0x7fd0840ffc50 2023-06-23T18:20:31.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0841005e0 0x7fd08419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:31.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089553700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0840fea60 0x7fd084197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:31.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0841005e0 0x7fd08419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52234/0 (socket says 172.21.15.2:52234) 2023-06-23T18:20:31.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 -- 172.21.15.2:0/3766220452 learned_addr learned my addr 172.21.15.2:0/3766220452 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:31.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd088d52700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0840ffc50 0x7fd084197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:31.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 -- 172.21.15.2:0/3766220452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0840ffc50 msgr2=0x7fd084197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:31.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0840ffc50 0x7fd084197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 -- 172.21.15.2:0/3766220452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0840fea60 msgr2=0x7fd084197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:31.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0840fea60 0x7fd084197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 -- 172.21.15.2:0/3766220452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0800095f0 con 0x7fd0841005e0 2023-06-23T18:20:31.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd088d52700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0840ffc50 0x7fd084197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:31.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd089d54700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0841005e0 0x7fd08419f010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00ada0 tx=0x7fd07c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:31.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd07a7fc700 1 -- 172.21.15.2:0/3766220452 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00f070 con 0x7fd0841005e0 2023-06-23T18:20:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd07a7fc700 1 -- 172.21.15.2:0/3766220452 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd07c007b30 con 0x7fd0841005e0 2023-06-23T18:20:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd07a7fc700 1 -- 172.21.15.2:0/3766220452 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c005280 con 0x7fd0841005e0 2023-06-23T18:20:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08419fb90 con 0x7fd0841005e0 2023-06-23T18:20:31.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.123+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0841a0050 con 0x7fd0841005e0 2023-06-23T18:20:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.124+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08410a150 con 0x7fd0841005e0 2023-06-23T18:20:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.126+0000 7fd07a7fc700 1 -- 172.21.15.2:0/3766220452 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd07c00c980 con 0x7fd0841005e0 2023-06-23T18:20:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.126+0000 7fd07a7fc700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0700a1150 0x7fd0700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:31.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.126+0000 7fd07a7fc700 1 -- 172.21.15.2:0/3766220452 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd07c0cc0c0 con 0x7fd0841005e0 2023-06-23T18:20:31.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.126+0000 7fd089553700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0700a1150 0x7fd0700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:31.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.127+0000 7fd089553700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0700a1150 0x7fd0700a3600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c010 tx=0x7fd080005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:31.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.130+0000 7fd07a7fc700 1 -- 172.21.15.2:0/3766220452 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd07c095d40 con 0x7fd0841005e0 2023-06-23T18:20:31.170 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:20:30 smithi017 podman[122526]: 2023-06-23T18:20:31.170 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:30 smithi017 ceph-mon[97472]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:31.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:30 smithi002 ceph-mon[95947]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:31.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.375+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd08410a340 con 0x7fd0841005e0 2023-06-23T18:20:31.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.375+0000 7fd07a7fc700 1 -- 172.21.15.2:0/3766220452 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5790 (secure 0 0 0) 0x7fd07c09ae60 con 0x7fd0841005e0 2023-06-23T18:20:31.381 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:31.381 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-06-23T18:20:31.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0700a1150 msgr2=0x7fd0700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:31.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0700a1150 0x7fd0700a3600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd08000c010 tx=0x7fd080005fb0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0841005e0 msgr2=0x7fd08419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:31.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0841005e0 0x7fd08419f010 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00ada0 tx=0x7fd07c00c330 comp rx=0 tx=0).stop 2023-06-23T18:20:31.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 shutdown_connections 2023-06-23T18:20:31.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0841005e0 0x7fd08419f010 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0700a1150 0x7fd0700a3600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0840fea60 0x7fd084197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 --2- 172.21.15.2:0/3766220452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0840ffc50 0x7fd084197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:31.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 >> 172.21.15.2:0/3766220452 conn(0x7fd0840fa0e0 msgr2=0x7fd084067860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:31.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 shutdown_connections 2023-06-23T18:20:31.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:31.377+0000 7fd08b7b7700 1 -- 172.21.15.2:0/3766220452 wait complete. 2023-06-23T18:20:31.385 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-06-23T18:20:31.440 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:20:31 smithi017 ceph-mds[122566]: -- 172.21.15.17:0/3666398100 <== mon.2 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b9d87131e0 con 0x55b9d86e9000 2023-06-23T18:20:31.440 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:20:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-h[122542]: starting mds.h at 2023-06-23T18:20:31.440 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:20:31 smithi017 bash[122526]: f0778750c8dd7f9fe3bfd3ad6aa2f802a1242864aab69293b37cfca2bcc6648b 2023-06-23T18:20:31.441 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:20:31 smithi017 systemd[1]: Started Ceph mds.h for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: Deploying daemon mds.i on smithi082 2023-06-23T18:20:32.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3766220452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:32.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.17:6836/2388271800,v1:172.21.15.17:6837/2388271800] up:boot 2023-06-23T18:20:32.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: fsmap 8 up:standby 2023-06-23T18:20:32.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-23T18:20:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: Deploying daemon mds.i on smithi082 2023-06-23T18:20:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3766220452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.17:6836/2388271800,v1:172.21.15.17:6837/2388271800] up:boot 2023-06-23T18:20:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: fsmap 8 up:standby 2023-06-23T18:20:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-23T18:20:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: Deploying daemon mds.i on smithi082 2023-06-23T18:20:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3766220452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.17:6836/2388271800,v1:172.21.15.17:6837/2388271800] up:boot 2023-06-23T18:20:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: fsmap 8 up:standby 2023-06-23T18:20:32.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-06-23T18:20:33.098 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:20:32 smithi082 systemd[1]: Starting Ceph mds.i for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:33.396 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:33.412 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:20:33 smithi082 podman[124088]: 2023-06-23T18:20:33.413 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:33 smithi082 ceph-mon[97889]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:33.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3369230228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d581058c0 msgr2=0x7f4d58107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:33.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3369230228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d581058c0 0x7f4d58107ca0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000b0a0 tx=0x7f4d5000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3369230228 shutdown_connections 2023-06-23T18:20:33.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3369230228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d581058c0 0x7f4d58107ca0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3369230228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d58100680 0x7f4d58102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3369230228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d58102fa0 0x7f4d58105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3369230228 >> 172.21.15.2:0/3369230228 conn(0x7f4d580fa0e0 msgr2=0x7f4d580fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:33.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3369230228 shutdown_connections 2023-06-23T18:20:33.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.536+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3369230228 wait complete. 2023-06-23T18:20:33.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5cd0f700 1 Processor -- start 2023-06-23T18:20:33.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5cd0f700 1 -- start start 2023-06-23T18:20:33.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5cd0f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d58100680 0x7f4d580ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:33.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5cd0f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d58102fa0 0x7f4d580fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:33.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5cd0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d581058c0 0x7f4d580feb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:33.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5cd0f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d580ff310 con 0x7f4d58102fa0 2023-06-23T18:20:33.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5cd0f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d580ff480 con 0x7f4d581058c0 2023-06-23T18:20:33.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5cd0f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5819f8d0 con 0x7f4d58100680 2023-06-23T18:20:33.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d4ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d58102fa0 0x7f4d580fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:33.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d5659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d58100680 0x7f4d580ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:33.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d4ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d58102fa0 0x7f4d580fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52246/0 (socket says 172.21.15.2:52246) 2023-06-23T18:20:33.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d4ffff700 1 -- 172.21.15.2:0/3576691403 learned_addr learned my addr 172.21.15.2:0/3576691403 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:33.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.537+0000 7f4d56d9d700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d581058c0 0x7f4d580feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:33.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d5659c700 1 -- 172.21.15.2:0/3576691403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d581058c0 msgr2=0x7f4d580feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:33.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d5659c700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d581058c0 0x7f4d580feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d5659c700 1 -- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d58102fa0 msgr2=0x7f4d580fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:33.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d5659c700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d58102fa0 0x7f4d580fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d5659c700 1 -- 172.21.15.2:0/3576691403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48009520 con 0x7f4d58100680 2023-06-23T18:20:33.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d4ffff700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d58102fa0 0x7f4d580fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:20:33.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d5659c700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d58100680 0x7f4d580ffff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e9f0 tx=0x7f4d4800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:33.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d4f7fe700 1 -- 172.21.15.2:0/3576691403 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4800cb70 con 0x7f4d58100680 2023-06-23T18:20:33.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d4f7fe700 1 -- 172.21.15.2:0/3576691403 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d4800ee90 con 0x7f4d58100680 2023-06-23T18:20:33.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d50009d00 con 0x7f4d58100680 2023-06-23T18:20:33.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d4f7fe700 1 -- 172.21.15.2:0/3576691403 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d48003bc0 con 0x7f4d58100680 2023-06-23T18:20:33.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.538+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5819ff50 con 0x7f4d58100680 2023-06-23T18:20:33.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.539+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d580fcc60 con 0x7f4d58100680 2023-06-23T18:20:33.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.540+0000 7f4d4f7fe700 1 -- 172.21.15.2:0/3576691403 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4d48004020 con 0x7f4d58100680 2023-06-23T18:20:33.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.541+0000 7f4d4f7fe700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4d440a11d0 0x7f4d440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:33.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.541+0000 7f4d4f7fe700 1 -- 172.21.15.2:0/3576691403 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d48014070 con 0x7f4d58100680 2023-06-23T18:20:33.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.541+0000 7f4d4ffff700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4d440a11d0 0x7f4d440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:33.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.541+0000 7f4d4ffff700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4d440a11d0 0x7f4d440a3680 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d40011550 tx=0x7f4d40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:33.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.545+0000 7f4d4f7fe700 1 -- 172.21.15.2:0/3576691403 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4d48096280 con 0x7f4d58100680 2023-06-23T18:20:33.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:33 smithi017 ceph-mon[97472]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:33.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:33 smithi002 ceph-mon[95947]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.796+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d5802c810 con 0x7f4d58100680 2023-06-23T18:20:33.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.797+0000 7f4d4f7fe700 1 -- 172.21.15.2:0/3576691403 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6547 (secure 0 0 0) 0x7f4d5802c810 con 0x7f4d58100680 2023-06-23T18:20:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:33.803 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-06-23T18:20:33.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4d440a11d0 msgr2=0x7f4d440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:33.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4d440a11d0 0x7f4d440a3680 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d40011550 tx=0x7f4d40009260 comp rx=0 tx=0).stop 2023-06-23T18:20:33.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d58100680 msgr2=0x7f4d580ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:33.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d58100680 0x7f4d580ffff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4d4800e9f0 tx=0x7f4d4800ed00 comp rx=0 tx=0).stop 2023-06-23T18:20:33.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 shutdown_connections 2023-06-23T18:20:33.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4d440a11d0 0x7f4d440a3680 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d581058c0 0x7f4d580feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d58100680 0x7f4d580ffff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 --2- 172.21.15.2:0/3576691403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d58102fa0 0x7f4d580fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 >> 172.21.15.2:0/3576691403 conn(0x7f4d580fa0e0 msgr2=0x7f4d58101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 shutdown_connections 2023-06-23T18:20:33.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:33.799+0000 7f4d5cd0f700 1 -- 172.21.15.2:0/3576691403 wait complete. 2023-06-23T18:20:33.807 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 9 2023-06-23T18:20:33.848 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:20:33 smithi082 ceph-mds[124128]: -- 172.21.15.82:0/692117941 <== mon.1 v2:172.21.15.82:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a46a43d1e0 con 0x55a46a412c00 2023-06-23T18:20:33.848 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:20:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-i[124104]: starting mds.i at 2023-06-23T18:20:33.849 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:20:33 smithi082 bash[124088]: 85759418013a48e238cb1b5bc5ac4f03cba5334b93adcc6c32a0f172b6b8a9b7 2023-06-23T18:20:33.849 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:20:33 smithi082 systemd[1]: Started Ceph mds.i for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:34.476 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:34.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:34.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:34.477 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: Deploying daemon mds.g on smithi002 2023-06-23T18:20:34.478 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3576691403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:34.478 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] up:boot 2023-06-23T18:20:34.478 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: fsmap 9 up:standby 2023-06-23T18:20:34.478 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-23T18:20:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:34.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:34.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:34.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: Deploying daemon mds.g on smithi002 2023-06-23T18:20:34.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3576691403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:34.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] up:boot 2023-06-23T18:20:34.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: fsmap 9 up:standby 2023-06-23T18:20:34.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-23T18:20:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:34.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:34.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:34.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:34.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: Deploying daemon mds.g on smithi002 2023-06-23T18:20:34.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3576691403' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:34.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] up:boot 2023-06-23T18:20:34.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: fsmap 9 up:standby 2023-06-23T18:20:34.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-06-23T18:20:35.480 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:20:35 smithi002 systemd[1]: Starting Ceph mds.g for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:35.481 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:35 smithi002 ceph-mon[95947]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:35.801 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:20:35 smithi002 podman[130370]: 2023-06-23T18:20:35.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:35.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:35 smithi082 ceph-mon[97889]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:35.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:35 smithi017 ceph-mon[97472]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 -- 172.21.15.2:0/3527721543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 msgr2=0x7f6754105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 --2- 172.21.15.2:0/3527721543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 0x7f6754105320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6744009990 tx=0x7f6744009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 -- 172.21.15.2:0/3527721543 shutdown_connections 2023-06-23T18:20:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 --2- 172.21.15.2:0/3527721543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6754105860 0x7f6754107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:35.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 --2- 172.21.15.2:0/3527721543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6754100620 0x7f6754102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:35.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 --2- 172.21.15.2:0/3527721543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 0x7f6754105320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:35.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 -- 172.21.15.2:0/3527721543 >> 172.21.15.2:0/3527721543 conn(0x7f67540fa0e0 msgr2=0x7f67540fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:35.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 -- 172.21.15.2:0/3527721543 shutdown_connections 2023-06-23T18:20:35.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 -- 172.21.15.2:0/3527721543 wait complete. 2023-06-23T18:20:35.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 Processor -- start 2023-06-23T18:20:35.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.959+0000 7f675c074700 1 -- start start 2023-06-23T18:20:35.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675c074700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6754100620 0x7f67540fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:35.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675c074700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 0x7f67540fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:35.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675c074700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6754105860 0x7f67540feb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:35.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675c074700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540ff2b0 con 0x7f6754102f40 2023-06-23T18:20:35.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675c074700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540ff420 con 0x7f6754100620 2023-06-23T18:20:35.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675c074700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675419f850 con 0x7f6754105860 2023-06-23T18:20:35.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 0x7f67540fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:35.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 0x7f67540fe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52270/0 (socket says 172.21.15.2:52270) 2023-06-23T18:20:35.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 -- 172.21.15.2:0/1346260736 learned_addr learned my addr 172.21.15.2:0/1346260736 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:35.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675a611700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6754105860 0x7f67540feb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:35.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f6759e10700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6754100620 0x7f67540fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 -- 172.21.15.2:0/1346260736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6754105860 msgr2=0x7f67540feb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6754105860 0x7f67540feb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 -- 172.21.15.2:0/1346260736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6754100620 msgr2=0x7f67540fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6754100620 0x7f67540fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 -- 172.21.15.2:0/1346260736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440095f0 con 0x7f6754102f40 2023-06-23T18:20:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675960f700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 0x7f67540fe5d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f67440098e0 tx=0x7f6744004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f674affd700 1 -- 172.21.15.2:0/1346260736 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674401d070 con 0x7f6754102f40 2023-06-23T18:20:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f674affd700 1 -- 172.21.15.2:0/1346260736 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f674400be30 con 0x7f6754102f40 2023-06-23T18:20:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675419fad0 con 0x7f6754102f40 2023-06-23T18:20:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f674affd700 1 -- 172.21.15.2:0/1346260736 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6744022e00 con 0x7f6754102f40 2023-06-23T18:20:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.960+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675419fef0 con 0x7f6754102f40 2023-06-23T18:20:35.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.961+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67540fcc60 con 0x7f6754102f40 2023-06-23T18:20:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.963+0000 7f674affd700 1 -- 172.21.15.2:0/1346260736 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f674400b9a0 con 0x7f6754102f40 2023-06-23T18:20:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.963+0000 7f674affd700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67400a1150 0x7f67400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.963+0000 7f6759e10700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67400a1150 0x7f67400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.963+0000 7f674affd700 1 -- 172.21.15.2:0/1346260736 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f67440cdd40 con 0x7f6754102f40 2023-06-23T18:20:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.964+0000 7f6759e10700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67400a1150 0x7f67400a3600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6750005950 tx=0x7f67500058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:35.968+0000 7f674affd700 1 -- 172.21.15.2:0/1346260736 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f67440979c0 con 0x7f6754102f40 2023-06-23T18:20:36.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.220+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f675402c810 con 0x7f6754102f40 2023-06-23T18:20:36.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.221+0000 7f674affd700 1 -- 172.21.15.2:0/1346260736 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7306 (secure 0 0 0) 0x7f674409afe0 con 0x7f6754102f40 2023-06-23T18:20:36.226 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:36.226 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-23T18:20:36.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67400a1150 msgr2=0x7f67400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:36.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67400a1150 0x7f67400a3600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6750005950 tx=0x7f67500058e0 comp rx=0 tx=0).stop 2023-06-23T18:20:36.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 msgr2=0x7f67540fe5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:36.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 0x7f67540fe5d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f67440098e0 tx=0x7f6744004730 comp rx=0 tx=0).stop 2023-06-23T18:20:36.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 shutdown_connections 2023-06-23T18:20:36.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6754105860 0x7f67540feb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:36.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67400a1150 0x7f67400a3600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:36.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6754100620 0x7f67540fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:36.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 --2- 172.21.15.2:0/1346260736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6754102f40 0x7f67540fe5d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:36.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 >> 172.21.15.2:0/1346260736 conn(0x7f67540fa0e0 msgr2=0x7f6754101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:36.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 shutdown_connections 2023-06-23T18:20:36.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:36.223+0000 7f675c074700 1 -- 172.21.15.2:0/1346260736 wait complete. 2023-06-23T18:20:36.231 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 10 2023-06-23T18:20:36.239 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:20:35 smithi002 ceph-mds[130410]: -- 172.21.15.2:0/358984450 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580c85231e0 con 0x5580c84f8800 2023-06-23T18:20:36.240 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:20:35 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-g[130386]: starting mds.g at 2023-06-23T18:20:36.240 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:20:35 smithi002 bash[130370]: b015140c5f564a2cb9e4f63e7aed60aced383fe3625c752975266f357bdb40bc 2023-06-23T18:20:36.240 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:20:35 smithi002 systemd[1]: Started Ceph mds.g for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:37.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:37.193 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:37.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:37.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: Deploying daemon mds.a on smithi002 2023-06-23T18:20:37.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1346260736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:37.195 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.2:6838/3063991639,v1:172.21.15.2:6839/3063991639] up:boot 2023-06-23T18:20:37.195 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: fsmap 10 up:standby 2023-06-23T18:20:37.195 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-23T18:20:37.196 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:36 smithi017 ceph-mon[97472]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:37.226 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.227 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.227 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.227 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:37.227 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:37.227 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:37.227 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: Deploying daemon mds.a on smithi002 2023-06-23T18:20:37.228 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1346260736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:37.228 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.2:6838/3063991639,v1:172.21.15.2:6839/3063991639] up:boot 2023-06-23T18:20:37.228 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: fsmap 10 up:standby 2023-06-23T18:20:37.228 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-23T18:20:37.228 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:36 smithi002 ceph-mon[95947]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: Deploying daemon mds.a on smithi002 2023-06-23T18:20:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1346260736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:37.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.2:6838/3063991639,v1:172.21.15.2:6839/3063991639] up:boot 2023-06-23T18:20:37.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: fsmap 10 up:standby 2023-06-23T18:20:37.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-06-23T18:20:37.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:36 smithi082 ceph-mon[97889]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:37.755 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:20:37 smithi002 systemd[1]: Starting Ceph mds.a for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:38.052 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:20:37 smithi002 podman[130753]: 2023-06-23T18:20:38.242 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:38.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 -- 172.21.15.2:0/1714326638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00fea60 msgr2=0x7f19b00fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:38.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 --2- 172.21.15.2:0/1714326638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00fea60 0x7f19b00fee70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1998009990 tx=0x7f1998009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 -- 172.21.15.2:0/1714326638 shutdown_connections 2023-06-23T18:20:38.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 --2- 172.21.15.2:0/1714326638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19b01005e0 0x7f19b0106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 --2- 172.21.15.2:0/1714326638 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00fea60 0x7f19b00fee70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 --2- 172.21.15.2:0/1714326638 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b00ffc50 0x7f19b01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 -- 172.21.15.2:0/1714326638 >> 172.21.15.2:0/1714326638 conn(0x7f19b00fa0e0 msgr2=0x7f19b00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:38.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 -- 172.21.15.2:0/1714326638 shutdown_connections 2023-06-23T18:20:38.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.370+0000 7f19b60e7700 1 -- 172.21.15.2:0/1714326638 wait complete. 2023-06-23T18:20:38.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19b60e7700 1 Processor -- start 2023-06-23T18:20:38.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19b60e7700 1 -- start start 2023-06-23T18:20:38.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19b60e7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19b00fea60 0x7f19b0102d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:38.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19b60e7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00ffc50 0x7f19b0101390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:38.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19b60e7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b01005e0 0x7f19b01018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:38.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19b60e7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0102070 con 0x7f19b00ffc50 2023-06-23T18:20:38.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19b60e7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b01021e0 con 0x7f19b01005e0 2023-06-23T18:20:38.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19b60e7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b019f890 con 0x7f19b00fea60 2023-06-23T18:20:38.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19aeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00ffc50 0x7f19b0101390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:38.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19aeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00ffc50 0x7f19b0101390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52294/0 (socket says 172.21.15.2:52294) 2023-06-23T18:20:38.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19aeffd700 1 -- 172.21.15.2:0/2308638527 learned_addr learned my addr 172.21.15.2:0/2308638527 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:38.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19affff700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b01005e0 0x7f19b01018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:38.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.371+0000 7f19af7fe700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19b00fea60 0x7f19b0102d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:38.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19aeffd700 1 -- 172.21.15.2:0/2308638527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19b00fea60 msgr2=0x7f19b0102d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:38.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19aeffd700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19b00fea60 0x7f19b0102d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19aeffd700 1 -- 172.21.15.2:0/2308638527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b01005e0 msgr2=0x7f19b01018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:38.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19aeffd700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b01005e0 0x7f19b01018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19aeffd700 1 -- 172.21.15.2:0/2308638527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19980095f0 con 0x7f19b00ffc50 2023-06-23T18:20:38.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19aeffd700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00ffc50 0x7f19b0101390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f19a000d950 tx=0x7f19a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:38.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19acff9700 1 -- 172.21.15.2:0/2308638527 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000f890 con 0x7f19b00ffc50 2023-06-23T18:20:38.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b019fb70 con 0x7f19b00ffc50 2023-06-23T18:20:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19acff9700 1 -- 172.21.15.2:0/2308638527 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f19a000fe80 con 0x7f19b00ffc50 2023-06-23T18:20:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19acff9700 1 -- 172.21.15.2:0/2308638527 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a000e3b0 con 0x7f19b00ffc50 2023-06-23T18:20:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.372+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b01a00d0 con 0x7f19b00ffc50 2023-06-23T18:20:38.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.373+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b0106b90 con 0x7f19b00ffc50 2023-06-23T18:20:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.374+0000 7f19acff9700 1 -- 172.21.15.2:0/2308638527 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f19a0010460 con 0x7f19b00ffc50 2023-06-23T18:20:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.375+0000 7f19acff9700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f199c0a1150 0x7f199c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.375+0000 7f19acff9700 1 -- 172.21.15.2:0/2308638527 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f19a00cc540 con 0x7f19b00ffc50 2023-06-23T18:20:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.375+0000 7f19af7fe700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f199c0a1150 0x7f199c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:38.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.375+0000 7f19af7fe700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f199c0a1150 0x7f199c0a3600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f199800c010 tx=0x7f1998005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:38.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.378+0000 7f19acff9700 1 -- 172.21.15.2:0/2308638527 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f19a0096130 con 0x7f19b00ffc50 2023-06-23T18:20:38.551 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:20:38 smithi002 ceph-mds[130793]: -- 172.21.15.2:0/3659251755 <== mon.0 v2:172.21.15.2:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d65ced1e0 con 0x555d65cc2c00 2023-06-23T18:20:38.551 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:20:38 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-a[130769]: starting mds.a at 2023-06-23T18:20:38.552 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:20:38 smithi002 bash[130753]: 6c4b0e04ae42ecc0efb89ed4e3410de0c2357a04248bb71ed59c81bf20a30264 2023-06-23T18:20:38.552 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:20:38 smithi002 systemd[1]: Started Ceph mds.a for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:38.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.627+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19b002c810 con 0x7f19b00ffc50 2023-06-23T18:20:38.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.628+0000 7f19acff9700 1 -- 172.21.15.2:0/2308638527 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8816 (secure 0 0 0) 0x7f19a0099750 con 0x7f19b00ffc50 2023-06-23T18:20:38.632 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:38.633 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-06-23T18:20:38.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f199c0a1150 msgr2=0x7f199c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:38.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f199c0a1150 0x7f199c0a3600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f199800c010 tx=0x7f1998005fb0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00ffc50 msgr2=0x7f19b0101390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:38.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00ffc50 0x7f19b0101390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f19a000d950 tx=0x7f19a000dc60 comp rx=0 tx=0).stop 2023-06-23T18:20:38.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 shutdown_connections 2023-06-23T18:20:38.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f199c0a1150 0x7f199c0a3600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19b01005e0 0x7f19b01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19b00fea60 0x7f19b0102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 --2- 172.21.15.2:0/2308638527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19b00ffc50 0x7f19b0101390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:38.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 >> 172.21.15.2:0/2308638527 conn(0x7f19b00fa0e0 msgr2=0x7f19b00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:38.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 shutdown_connections 2023-06-23T18:20:38.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:38.630+0000 7f19b60e7700 1 -- 172.21.15.2:0/2308638527 wait complete. 2023-06-23T18:20:38.638 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 12 2023-06-23T18:20:39.509 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.509 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.509 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.509 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:39.509 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:39.510 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:39.510 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: Deploying daemon mds.b on smithi017 2023-06-23T18:20:39.510 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.2:6840/2758814728,v1:172.21.15.2:6841/2758814728] up:boot 2023-06-23T18:20:39.510 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: fsmap 11 up:standby 2023-06-23T18:20:39.510 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-23T18:20:39.510 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2308638527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:39.510 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:39 smithi017 ceph-mon[97472]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:39.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:39.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:39.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:39.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: Deploying daemon mds.b on smithi017 2023-06-23T18:20:39.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.2:6840/2758814728,v1:172.21.15.2:6841/2758814728] up:boot 2023-06-23T18:20:39.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: fsmap 11 up:standby 2023-06-23T18:20:39.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-23T18:20:39.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2308638527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:39.554 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:39 smithi002 ceph-mon[95947]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:39.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:39.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-06-23T18:20:39.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-06-23T18:20:39.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:39.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: Deploying daemon mds.b on smithi017 2023-06-23T18:20:39.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.2:6840/2758814728,v1:172.21.15.2:6841/2758814728] up:boot 2023-06-23T18:20:39.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: fsmap 11 up:standby 2023-06-23T18:20:39.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-06-23T18:20:39.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2308638527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:39.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:39 smithi082 ceph-mon[97889]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:40.190 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:20:39 smithi017 systemd[1]: Starting Ceph mds.b for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:20:40.526 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:20:40 smithi017 podman[122866]: 2023-06-23T18:20:40.647 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:40.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2605034107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf34100690 msgr2=0x7fbf34106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:40.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2605034107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf34100690 0x7fbf34106f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00b0a0 tx=0x7fbf2c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:40.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2605034107 shutdown_connections 2023-06-23T18:20:40.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2605034107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf34100690 0x7fbf34106f30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:40.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2605034107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf340fea50 0x7fbf340fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:40.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2605034107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf340ffce0 0x7fbf34100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:40.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2605034107 >> 172.21.15.2:0/2605034107 conn(0x7fbf340fa090 msgr2=0x7fbf340fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:40.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2605034107 shutdown_connections 2023-06-23T18:20:40.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.782+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2605034107 wait complete. 2023-06-23T18:20:40.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3bed1700 1 Processor -- start 2023-06-23T18:20:40.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3bed1700 1 -- start start 2023-06-23T18:20:40.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3bed1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf340fea50 0x7fbf341947c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:40.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3bed1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf340ffce0 0x7fbf34194d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:40.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3bed1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34100690 0x7fbf3419bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:40.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3bed1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3419c340 con 0x7fbf340ffce0 2023-06-23T18:20:40.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3bed1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3419c480 con 0x7fbf34100690 2023-06-23T18:20:40.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3bed1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3419c610 con 0x7fbf340fea50 2023-06-23T18:20:40.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3946c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf340ffce0 0x7fbf34194d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:40.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3946c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf340ffce0 0x7fbf34194d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52310/0 (socket says 172.21.15.2:52310) 2023-06-23T18:20:40.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3946c700 1 -- 172.21.15.2:0/2561373427 learned_addr learned my addr 172.21.15.2:0/2561373427 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:40.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf39c6d700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf340fea50 0x7fbf341947c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:40.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3a46e700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34100690 0x7fbf3419bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:40.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3946c700 1 -- 172.21.15.2:0/2561373427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf340fea50 msgr2=0x7fbf341947c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:40.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3946c700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf340fea50 0x7fbf341947c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:40.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3946c700 1 -- 172.21.15.2:0/2561373427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34100690 msgr2=0x7fbf3419bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:40.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3946c700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34100690 0x7fbf3419bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:40.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.783+0000 7fbf3946c700 1 -- 172.21.15.2:0/2561373427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c009d00 con 0x7fbf340ffce0 2023-06-23T18:20:40.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.784+0000 7fbf3946c700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf340ffce0 0x7fbf34194d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400e9a0 tx=0x7fbf2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:40.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.784+0000 7fbf2affd700 1 -- 172.21.15.2:0/2561373427 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24009790 con 0x7fbf340ffce0 2023-06-23T18:20:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.784+0000 7fbf2affd700 1 -- 172.21.15.2:0/2561373427 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf2400cbd0 con 0x7fbf340ffce0 2023-06-23T18:20:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.784+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3419c8f0 con 0x7fbf340ffce0 2023-06-23T18:20:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.784+0000 7fbf2affd700 1 -- 172.21.15.2:0/2561373427 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24018680 con 0x7fbf340ffce0 2023-06-23T18:20:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.784+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3419ce00 con 0x7fbf340ffce0 2023-06-23T18:20:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.785+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3410a080 con 0x7fbf340ffce0 2023-06-23T18:20:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.786+0000 7fbf2affd700 1 -- 172.21.15.2:0/2561373427 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbf2400cd40 con 0x7fbf340ffce0 2023-06-23T18:20:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.786+0000 7fbf2affd700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf200a1150 0x7fbf200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.787+0000 7fbf39c6d700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf200a1150 0x7fbf200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.787+0000 7fbf2affd700 1 -- 172.21.15.2:0/2561373427 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbf24014070 con 0x7fbf340ffce0 2023-06-23T18:20:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.787+0000 7fbf39c6d700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf200a1150 0x7fbf200a3600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009c60 tx=0x7fbf30009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:40.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:40.790+0000 7fbf2affd700 1 -- 172.21.15.2:0/2561373427 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbf240966a0 con 0x7fbf340ffce0 2023-06-23T18:20:40.940 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:20:40 smithi017 ceph-mds[122906]: -- 172.21.15.17:0/4277602607 <== mon.2 v2:172.21.15.17:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b3237ab1e0 con 0x55b323780800 2023-06-23T18:20:40.941 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:20:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-b[122882]: starting mds.b at 2023-06-23T18:20:40.941 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:20:40 smithi017 bash[122866]: d32e7bd8abcdbcd1e4453453abbf4f63dda9b2321f15dfb242a80c2c08bfd9ef 2023-06-23T18:20:40.942 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:20:40 smithi017 systemd[1]: Started Ceph mds.b for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:20:41.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.048+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf34105a40 con 0x7fbf340ffce0 2023-06-23T18:20:41.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.049+0000 7fbf2affd700 1 -- 172.21.15.2:0/2561373427 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9574 (secure 0 0 0) 0x7fbf24099cc0 con 0x7fbf340ffce0 2023-06-23T18:20:41.054 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:41.054 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24467,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24493,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24497,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24526,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24533,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-06-23T18:20:41.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf200a1150 msgr2=0x7fbf200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf200a1150 0x7fbf200a3600 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009c60 tx=0x7fbf30009260 comp rx=0 tx=0).stop 2023-06-23T18:20:41.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf340ffce0 msgr2=0x7fbf34194d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf340ffce0 0x7fbf34194d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400e9a0 tx=0x7fbf2400ecb0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 shutdown_connections 2023-06-23T18:20:41.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf200a1150 0x7fbf200a3600 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf34100690 0x7fbf3419bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf340fea50 0x7fbf341947c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 --2- 172.21.15.2:0/2561373427 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbf340ffce0 0x7fbf34194d00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 >> 172.21.15.2:0/2561373427 conn(0x7fbf340fa090 msgr2=0x7fbf340fc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:41.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 shutdown_connections 2023-06-23T18:20:41.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.051+0000 7fbf3bed1700 1 -- 172.21.15.2:0/2561373427 wait complete. 2023-06-23T18:20:41.058 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 13 2023-06-23T18:20:41.070 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:20:41.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 -- 172.21.15.2:0/1990490538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80fea60 msgr2=0x7fa6e80fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 --2- 172.21.15.2:0/1990490538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80fea60 0x7fa6e80fee70 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800b0a0 tx=0x7fa6d800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 -- 172.21.15.2:0/1990490538 shutdown_connections 2023-06-23T18:20:41.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 --2- 172.21.15.2:0/1990490538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e81005e0 0x7fa6e8106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 --2- 172.21.15.2:0/1990490538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80fea60 0x7fa6e80fee70 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 --2- 172.21.15.2:0/1990490538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e80ffc50 0x7fa6e81000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 -- 172.21.15.2:0/1990490538 >> 172.21.15.2:0/1990490538 conn(0x7fa6e80fa0e0 msgr2=0x7fa6e80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:41.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 -- 172.21.15.2:0/1990490538 shutdown_connections 2023-06-23T18:20:41.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.197+0000 7fa6ee962700 1 -- 172.21.15.2:0/1990490538 wait complete. 2023-06-23T18:20:41.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6ee962700 1 Processor -- start 2023-06-23T18:20:41.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6ee962700 1 -- start start 2023-06-23T18:20:41.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6ee962700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e80fea60 0x7fa6e8195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6ee962700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80ffc50 0x7fa6e8195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6ee962700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e81005e0 0x7fa6e819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6ee962700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e819d4b0 con 0x7fa6e80ffc50 2023-06-23T18:20:41.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6ee962700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e819d5f0 con 0x7fa6e80fea60 2023-06-23T18:20:41.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6ee962700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e819d780 con 0x7fa6e81005e0 2023-06-23T18:20:41.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6e77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80ffc50 0x7fa6e8195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6e77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80ffc50 0x7fa6e8195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52322/0 (socket says 172.21.15.2:52322) 2023-06-23T18:20:41.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6e77fe700 1 -- 172.21.15.2:0/3645675757 learned_addr learned my addr 172.21.15.2:0/3645675757 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:41.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6eceff700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e81005e0 0x7fa6e819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6e77fe700 1 -- 172.21.15.2:0/3645675757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e81005e0 msgr2=0x7fa6e819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6e77fe700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e81005e0 0x7fa6e819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6e77fe700 1 -- 172.21.15.2:0/3645675757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e80fea60 msgr2=0x7fa6e8195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6e77fe700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e80fea60 0x7fa6e8195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.198+0000 7fa6e77fe700 1 -- 172.21.15.2:0/3645675757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8009d00 con 0x7fa6e80ffc50 2023-06-23T18:20:41.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.199+0000 7fa6e77fe700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80ffc50 0x7fa6e8195e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c920 tx=0x7fa6d000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:41.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.199+0000 7fa6e57fa700 1 -- 172.21.15.2:0/3645675757 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0007b00 con 0x7fa6e80ffc50 2023-06-23T18:20:41.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.199+0000 7fa6e57fa700 1 -- 172.21.15.2:0/3645675757 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6d000ce70 con 0x7fa6e80ffc50 2023-06-23T18:20:41.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.199+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e819da60 con 0x7fa6e80ffc50 2023-06-23T18:20:41.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.199+0000 7fa6e57fa700 1 -- 172.21.15.2:0/3645675757 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0005440 con 0x7fa6e80ffc50 2023-06-23T18:20:41.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.199+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e819df70 con 0x7fa6e80ffc50 2023-06-23T18:20:41.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.200+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e8106be0 con 0x7fa6e80ffc50 2023-06-23T18:20:41.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.202+0000 7fa6e57fa700 1 -- 172.21.15.2:0/3645675757 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa6d00055a0 con 0x7fa6e80ffc50 2023-06-23T18:20:41.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.203+0000 7fa6e57fa700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6d40a1150 0x7fa6d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.203+0000 7fa6e57fa700 1 -- 172.21.15.2:0/3645675757 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa6d00cc440 con 0x7fa6e80ffc50 2023-06-23T18:20:41.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.203+0000 7fa6e7fff700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6d40a1150 0x7fa6d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.203+0000 7fa6e7fff700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6d40a1150 0x7fa6d40a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800b780 tx=0x7fa6d8009be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:41.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.208+0000 7fa6e57fa700 1 -- 172.21.15.2:0/3645675757 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa6d0096030 con 0x7fa6e80ffc50 2023-06-23T18:20:41.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.413+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6e802c810 con 0x7fa6e80ffc50 2023-06-23T18:20:41.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.414+0000 7fa6e57fa700 1 -- 172.21.15.2:0/3645675757 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+18243 (secure 0 0 0) 0x7fa6d0099650 con 0x7fa6e80ffc50 2023-06-23T18:20:41.419 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:41.419 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":78,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:03.515638+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:20:41.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6d40a1150 msgr2=0x7fa6d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6d40a1150 0x7fa6d40a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6d800b780 tx=0x7fa6d8009be0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80ffc50 msgr2=0x7fa6e8195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80ffc50 0x7fa6e8195e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c920 tx=0x7fa6d000cc30 comp rx=0 tx=0).stop 2023-06-23T18:20:41.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 shutdown_connections 2023-06-23T18:20:41.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e81005e0 0x7fa6e819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6d40a1150 0x7fa6d40a3600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6e80fea60 0x7fa6e8195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 --2- 172.21.15.2:0/3645675757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6e80ffc50 0x7fa6e8195e70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 >> 172.21.15.2:0/3645675757 conn(0x7fa6e80fa0e0 msgr2=0x7fa6e80fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:41.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 shutdown_connections 2023-06-23T18:20:41.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.417+0000 7fa6ee962700 1 -- 172.21.15.2:0/3645675757 wait complete. 2023-06-23T18:20:41.435 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:20:41.436 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:20:41.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 -- 172.21.15.2:0/665686188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 msgr2=0x7fb6d00fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/665686188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 0x7fb6d00fff70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009990 tx=0x7fb6c0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 -- 172.21.15.2:0/665686188 shutdown_connections 2023-06-23T18:20:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/665686188 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d01017a0 0x7fb6d0108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/665686188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 0x7fb6d00fff70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/665686188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0100df0 0x7fb6d0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 -- 172.21.15.2:0/665686188 >> 172.21.15.2:0/665686188 conn(0x7fb6d00fb180 msgr2=0x7fb6d00fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 -- 172.21.15.2:0/665686188 shutdown_connections 2023-06-23T18:20:41.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.567+0000 7fb6d68b8700 1 -- 172.21.15.2:0/665686188 wait complete. 2023-06-23T18:20:41.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6d68b8700 1 Processor -- start 2023-06-23T18:20:41.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6d68b8700 1 -- start start 2023-06-23T18:20:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6d68b8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 0x7fb6d01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6d68b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0100df0 0x7fb6d0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6d68b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d01017a0 0x7fb6d019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6d68b8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d019f540 con 0x7fb6d00ffb60 2023-06-23T18:20:41.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6d68b8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d019f680 con 0x7fb6d0100df0 2023-06-23T18:20:41.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6d68b8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d019f810 con 0x7fb6d01017a0 2023-06-23T18:20:41.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 0x7fb6d01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 0x7fb6d01979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52334/0 (socket says 172.21.15.2:52334) 2023-06-23T18:20:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6cffff700 1 -- 172.21.15.2:0/3836472775 learned_addr learned my addr 172.21.15.2:0/3836472775 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.568+0000 7fb6cf7fe700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0100df0 0x7fb6d0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6d4e55700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d01017a0 0x7fb6d019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cffff700 1 -- 172.21.15.2:0/3836472775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d01017a0 msgr2=0x7fb6d019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cffff700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d01017a0 0x7fb6d019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cffff700 1 -- 172.21.15.2:0/3836472775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0100df0 msgr2=0x7fb6d0197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cffff700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0100df0 0x7fb6d0197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cffff700 1 -- 172.21.15.2:0/3836472775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c00095f0 con 0x7fb6d00ffb60 2023-06-23T18:20:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6d4e55700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d01017a0 0x7fb6d019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cffff700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 0x7fb6d01979c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000c010 tx=0x7fb6c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cd7fa700 1 -- 172.21.15.2:0/3836472775 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c001d070 con 0x7fb6d00ffb60 2023-06-23T18:20:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cd7fa700 1 -- 172.21.15.2:0/3836472775 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb6c0004160 con 0x7fb6d00ffb60 2023-06-23T18:20:41.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d019fa90 con 0x7fb6d00ffb60 2023-06-23T18:20:41.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6cd7fa700 1 -- 172.21.15.2:0/3836472775 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0022e10 con 0x7fb6d00ffb60 2023-06-23T18:20:41.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.569+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d019feb0 con 0x7fb6d00ffb60 2023-06-23T18:20:41.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.570+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d010b230 con 0x7fb6d00ffb60 2023-06-23T18:20:41.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.571+0000 7fb6cd7fa700 1 -- 172.21.15.2:0/3836472775 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb6c0022570 con 0x7fb6d00ffb60 2023-06-23T18:20:41.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.572+0000 7fb6cd7fa700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb6b80a1150 0x7fb6b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.572+0000 7fb6cf7fe700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb6b80a1150 0x7fb6b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.572+0000 7fb6cd7fa700 1 -- 172.21.15.2:0/3836472775 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb6c00cd9f0 con 0x7fb6d00ffb60 2023-06-23T18:20:41.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.572+0000 7fb6cf7fe700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb6b80a1150 0x7fb6b80a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400ac50 tx=0x7fb6c400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:41.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.576+0000 7fb6cd7fa700 1 -- 172.21.15.2:0/3836472775 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb6c0097670 con 0x7fb6d00ffb60 2023-06-23T18:20:41.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.783+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6d01a0460 con 0x7fb6d00ffb60 2023-06-23T18:20:41.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.783+0000 7fb6cd7fa700 1 -- 172.21.15.2:0/3836472775 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6c009ac90 con 0x7fb6d00ffb60 2023-06-23T18:20:41.789 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:20:41.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb6b80a1150 msgr2=0x7fb6b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb6b80a1150 0x7fb6b80a3600 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400ac50 tx=0x7fb6c400a380 comp rx=0 tx=0).stop 2023-06-23T18:20:41.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 msgr2=0x7fb6d01979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 0x7fb6d01979c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000c010 tx=0x7fb6c0004730 comp rx=0 tx=0).stop 2023-06-23T18:20:41.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 shutdown_connections 2023-06-23T18:20:41.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6d01017a0 0x7fb6d019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6d00ffb60 0x7fb6d01979c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb6b80a1150 0x7fb6b80a3600 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 --2- 172.21.15.2:0/3836472775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0100df0 0x7fb6d0197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 >> 172.21.15.2:0/3836472775 conn(0x7fb6d00fb180 msgr2=0x7fb6d00fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:41.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 shutdown_connections 2023-06-23T18:20:41.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.785+0000 7fb6d68b8700 1 -- 172.21.15.2:0/3836472775 wait complete. 2023-06-23T18:20:41.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] up:boot 2023-06-23T18:20:41.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: fsmap 12 up:standby 2023-06-23T18:20:41.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-23T18:20:41.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2561373427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:41 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3645675757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.804 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-06-23T18:20:41.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] up:boot 2023-06-23T18:20:41.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: fsmap 12 up:standby 2023-06-23T18:20:41.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-23T18:20:41.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:41.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2561373427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:41 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3645675757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 -- 172.21.15.2:0/769651776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40ff710 msgr2=0x7fdfb40ffb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/769651776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40ff710 0x7fdfb40ffb80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00b0a0 tx=0x7fdfac00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 -- 172.21.15.2:0/769651776 shutdown_connections 2023-06-23T18:20:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/769651776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40ff710 0x7fdfb40ffb80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/769651776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfb40fe410 0x7fdfb40fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/769651776 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb40fed60 0x7fdfb40ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 -- 172.21.15.2:0/769651776 >> 172.21.15.2:0/769651776 conn(0x7fdfb40fa0c0 msgr2=0x7fdfb40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:41.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 -- 172.21.15.2:0/769651776 shutdown_connections 2023-06-23T18:20:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.932+0000 7fdfbaea0700 1 -- 172.21.15.2:0/769651776 wait complete. 2023-06-23T18:20:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfbaea0700 1 Processor -- start 2023-06-23T18:20:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfbaea0700 1 -- start start 2023-06-23T18:20:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfbaea0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40fe410 0x7fdfb4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfbaea0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfb40fed60 0x7fdfb4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfbaea0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb40ff710 0x7fdfb419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfbaea0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb419f610 con 0x7fdfb40fe410 2023-06-23T18:20:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfbaea0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb419f750 con 0x7fdfb40ff710 2023-06-23T18:20:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfbaea0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb419f8e0 con 0x7fdfb40fed60 2023-06-23T18:20:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfb8c3c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40fe410 0x7fdfb4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfb3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfb40fed60 0x7fdfb4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfb3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfb40fed60 0x7fdfb4198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:52688/0 (socket says 172.21.15.2:52688) 2023-06-23T18:20:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfb8c3c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40fe410 0x7fdfb4197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52348/0 (socket says 172.21.15.2:52348) 2023-06-23T18:20:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfb3fff700 1 -- 172.21.15.2:0/1345197342 learned_addr learned my addr 172.21.15.2:0/1345197342 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:41.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.933+0000 7fdfb943d700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb40ff710 0x7fdfb419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb3fff700 1 -- 172.21.15.2:0/1345197342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb40ff710 msgr2=0x7fdfb419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb3fff700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb40ff710 0x7fdfb419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb3fff700 1 -- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40fe410 msgr2=0x7fdfb4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb3fff700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40fe410 0x7fdfb4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb3fff700 1 -- 172.21.15.2:0/1345197342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac009d00 con 0x7fdfb40fed60 2023-06-23T18:20:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb943d700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb40ff710 0x7fdfb419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:20:41.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb3fff700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfb40fed60 0x7fdfb4198020 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000e9a0 tx=0x7fdfa000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb1ffb700 1 -- 172.21.15.2:0/1345197342 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0009790 con 0x7fdfb40fed60 2023-06-23T18:20:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb419fbc0 con 0x7fdfb40fed60 2023-06-23T18:20:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb1ffb700 1 -- 172.21.15.2:0/1345197342 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdfa0010430 con 0x7fdfb40fed60 2023-06-23T18:20:41.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfb1ffb700 1 -- 172.21.15.2:0/1345197342 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000cea0 con 0x7fdfb40fed60 2023-06-23T18:20:41.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] up:boot 2023-06-23T18:20:41.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: fsmap 12 up:standby 2023-06-23T18:20:41.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-06-23T18:20:41.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.944 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:41.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:41.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2561373427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:41 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3645675757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.934+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb41a00d0 con 0x7fdfb40fed60 2023-06-23T18:20:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.935+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb41091f0 con 0x7fdfb40fed60 2023-06-23T18:20:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.936+0000 7fdfb1ffb700 1 -- 172.21.15.2:0/1345197342 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdfa00105a0 con 0x7fdfb40fed60 2023-06-23T18:20:41.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.937+0000 7fdfb1ffb700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfa40a1150 0x7fdfa40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.937+0000 7fdfb1ffb700 1 -- 172.21.15.2:0/1345197342 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdfa0014070 con 0x7fdfb40fed60 2023-06-23T18:20:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.937+0000 7fdfb8c3c700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfa40a1150 0x7fdfa40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.937+0000 7fdfb8c3c700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfa40a1150 0x7fdfa40a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009c60 tx=0x7fdfa8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:41.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:41.940+0000 7fdfb1ffb700 1 -- 172.21.15.2:0/1345197342 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdfa00961b0 con 0x7fdfb40fed60 2023-06-23T18:20:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.149+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdfb402ca60 con 0x7fdfb40fed60 2023-06-23T18:20:42.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.149+0000 7fdfb1ffb700 1 -- 172.21.15.2:0/1345197342 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdfa00997d0 con 0x7fdfb40fed60 2023-06-23T18:20:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:42.154 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2023-06-23T18:20:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:20:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2023-06-23T18:20:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2023-06-23T18:20:42.154 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:20:42.155 INFO:teuthology.orchestra.run.smithi002.stdout:] 2023-06-23T18:20:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfa40a1150 msgr2=0x7fdfa40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfa40a1150 0x7fdfa40a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009c60 tx=0x7fdfa8009260 comp rx=0 tx=0).stop 2023-06-23T18:20:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfb40fed60 msgr2=0x7fdfb4198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfb40fed60 0x7fdfb4198020 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000e9a0 tx=0x7fdfa000ecb0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 shutdown_connections 2023-06-23T18:20:42.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfa40a1150 0x7fdfa40a3600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfb40ff710 0x7fdfb419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdfb40fe410 0x7fdfb4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.151+0000 7fdfbaea0700 1 --2- 172.21.15.2:0/1345197342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdfb40fed60 0x7fdfb4198020 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.152+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 >> 172.21.15.2:0/1345197342 conn(0x7fdfb40fa0c0 msgr2=0x7fdfb40fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.152+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 shutdown_connections 2023-06-23T18:20:42.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.152+0000 7fdfbaea0700 1 -- 172.21.15.2:0/1345197342 wait complete. 2023-06-23T18:20:42.168 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-06-23T18:20:42.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-06-23T18:20:42.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/2351272389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf4074470 msgr2=0x7f1cf4072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:42.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/2351272389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf4074470 0x7f1cf4072960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009990 tx=0x7f1ce4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/2351272389 shutdown_connections 2023-06-23T18:20:42.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/2351272389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cf41079d0 0x7f1cf4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/2351272389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf4074470 0x7f1cf4072960 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/2351272389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cf4072ea0 0x7f1cf4073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/2351272389 >> 172.21.15.2:0/2351272389 conn(0x7f1cf40fb150 msgr2=0x7f1cf40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:42.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/2351272389 shutdown_connections 2023-06-23T18:20:42.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.297+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/2351272389 wait complete. 2023-06-23T18:20:42.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.298+0000 7f1cfa8fd700 1 Processor -- start 2023-06-23T18:20:42.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.298+0000 7f1cfa8fd700 1 -- start start 2023-06-23T18:20:42.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cfa8fd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cf4072ea0 0x7f1cf4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:42.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cfa8fd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cf4074470 0x7f1cf4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:42.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cfa8fd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf41079d0 0x7f1cf419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:42.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cfa8fd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf419f5e0 con 0x7f1cf41079d0 2023-06-23T18:20:42.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cfa8fd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf419f720 con 0x7f1cf4072ea0 2023-06-23T18:20:42.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cfa8fd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf419f8b0 con 0x7f1cf4074470 2023-06-23T18:20:42.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf41079d0 0x7f1cf419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:42.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf41079d0 0x7f1cf419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52366/0 (socket says 172.21.15.2:52366) 2023-06-23T18:20:42.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 -- 172.21.15.2:0/495213362 learned_addr learned my addr 172.21.15.2:0/495213362 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:42.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf3fff700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cf4072ea0 0x7f1cf4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:42.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf37fe700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cf4074470 0x7f1cf4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:42.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 -- 172.21.15.2:0/495213362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cf4074470 msgr2=0x7f1cf4197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:42.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cf4074470 0x7f1cf4197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 -- 172.21.15.2:0/495213362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cf4072ea0 msgr2=0x7f1cf4197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:42.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cf4072ea0 0x7f1cf4197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:42.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 -- 172.21.15.2:0/495213362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce8009520 con 0x7f1cf41079d0 2023-06-23T18:20:42.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf3fff700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cf4072ea0 0x7f1cf4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:20:42.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf8e9a700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf41079d0 0x7f1cf419f010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00ef30 tx=0x7f1cec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:42.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf17fa700 1 -- 172.21.15.2:0/495213362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec018070 con 0x7f1cf41079d0 2023-06-23T18:20:42.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ce40095f0 con 0x7f1cf41079d0 2023-06-23T18:20:42.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf17fa700 1 -- 172.21.15.2:0/495213362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1cec00f040 con 0x7f1cf41079d0 2023-06-23T18:20:42.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.299+0000 7f1cf17fa700 1 -- 172.21.15.2:0/495213362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec00cbc0 con 0x7f1cf41079d0 2023-06-23T18:20:42.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.300+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf419fe60 con 0x7f1cf41079d0 2023-06-23T18:20:42.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.300+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf4106160 con 0x7f1cf41079d0 2023-06-23T18:20:42.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.302+0000 7f1cf17fa700 1 -- 172.21.15.2:0/495213362 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1cec00cd60 con 0x7f1cf41079d0 2023-06-23T18:20:42.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.302+0000 7f1cf17fa700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1cdc0a1150 0x7f1cdc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:42.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.303+0000 7f1cf17fa700 1 -- 172.21.15.2:0/495213362 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1cec0cc600 con 0x7f1cf41079d0 2023-06-23T18:20:42.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.303+0000 7f1cf3fff700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1cdc0a1150 0x7f1cdc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:42.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.303+0000 7f1cf3fff700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1cdc0a1150 0x7f1cdc0a3600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005910 tx=0x7f1ce400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:42.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.309+0000 7f1cf17fa700 1 -- 172.21.15.2:0/495213362 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1cec0961e0 con 0x7f1cf41079d0 2023-06-23T18:20:42.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:42.524+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1cf402c810 con 0x7f1cf41079d0 2023-06-23T18:20:43.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.018+0000 7f1cf17fa700 1 -- 172.21.15.2:0/495213362 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1cec099800 con 0x7f1cf41079d0 2023-06-23T18:20:43.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.022+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1cdc0a1150 msgr2=0x7f1cdc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.022+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1cdc0a1150 0x7f1cdc0a3600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005910 tx=0x7f1ce400b260 comp rx=0 tx=0).stop 2023-06-23T18:20:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.022+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf41079d0 msgr2=0x7f1cf419f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:43.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.022+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf41079d0 0x7f1cf419f010 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1cec00ef30 tx=0x7f1cec00c3a0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.023+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 shutdown_connections 2023-06-23T18:20:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.023+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1cf41079d0 0x7f1cf419f010 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.023+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1cdc0a1150 0x7f1cdc0a3600 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.023+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cf4072ea0 0x7f1cf4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.023+0000 7f1cfa8fd700 1 --2- 172.21.15.2:0/495213362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cf4074470 0x7f1cf4197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.023+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 >> 172.21.15.2:0/495213362 conn(0x7f1cf40fb150 msgr2=0x7f1cf40fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.023+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 shutdown_connections 2023-06-23T18:20:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.023+0000 7f1cfa8fd700 1 -- 172.21.15.2:0/495213362 wait complete. 2023-06-23T18:20:43.029 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2023-06-23T18:20:43.040 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-06-23T18:20:43.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 -- 172.21.15.2:0/640908234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 msgr2=0x7f87200feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:43.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 --2- 172.21.15.2:0/640908234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 0x7f87200feea0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8710009990 tx=0x7f8710009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 -- 172.21.15.2:0/640908234 shutdown_connections 2023-06-23T18:20:43.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 --2- 172.21.15.2:0/640908234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8720100610 0x7f8720106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 --2- 172.21.15.2:0/640908234 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 0x7f87200feea0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 --2- 172.21.15.2:0/640908234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87200ffc80 0x7f87201000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 -- 172.21.15.2:0/640908234 >> 172.21.15.2:0/640908234 conn(0x7f87200fa090 msgr2=0x7f87200fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:43.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 -- 172.21.15.2:0/640908234 shutdown_connections 2023-06-23T18:20:43.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 -- 172.21.15.2:0/640908234 wait complete. 2023-06-23T18:20:43.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.176+0000 7f8725339700 1 Processor -- start 2023-06-23T18:20:43.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f8725339700 1 -- start start 2023-06-23T18:20:43.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f8725339700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 0x7f8720194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:43.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f8725339700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87200ffc80 0x7f8720194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:43.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f8725339700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8720100610 0x7f872019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:43.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f8725339700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872019c2c0 con 0x7f87200fea90 2023-06-23T18:20:43.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f8725339700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872019c400 con 0x7f8720100610 2023-06-23T18:20:43.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f8725339700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872019c590 con 0x7f87200ffc80 2023-06-23T18:20:43.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 0x7f8720194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:43.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 0x7f8720194740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52376/0 (socket says 172.21.15.2:52376) 2023-06-23T18:20:43.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 -- 172.21.15.2:0/2395277102 learned_addr learned my addr 172.21.15.2:0/2395277102 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:43.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f8717fff700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87200ffc80 0x7f8720194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:43.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871f7fe700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8720100610 0x7f872019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:43.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 -- 172.21.15.2:0/2395277102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87200ffc80 msgr2=0x7f8720194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:43.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87200ffc80 0x7f8720194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 -- 172.21.15.2:0/2395277102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8720100610 msgr2=0x7f872019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:43.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8720100610 0x7f872019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:43.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 -- 172.21.15.2:0/2395277102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100095f0 con 0x7f87200fea90 2023-06-23T18:20:43.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.177+0000 7f871effd700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 0x7f8720194740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f87100098e0 tx=0x7f8710004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:43.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.178+0000 7f871cff9700 1 -- 172.21.15.2:0/2395277102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871001d070 con 0x7f87200fea90 2023-06-23T18:20:43.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.178+0000 7f871cff9700 1 -- 172.21.15.2:0/2395277102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f871000bdd0 con 0x7f87200fea90 2023-06-23T18:20:43.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.178+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f872019c810 con 0x7f87200fea90 2023-06-23T18:20:43.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.178+0000 7f871cff9700 1 -- 172.21.15.2:0/2395277102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710022d70 con 0x7f87200fea90 2023-06-23T18:20:43.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.178+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872019cc30 con 0x7f87200fea90 2023-06-23T18:20:43.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.179+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87201060a0 con 0x7f87200fea90 2023-06-23T18:20:43.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.180+0000 7f871cff9700 1 -- 172.21.15.2:0/2395277102 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8710022470 con 0x7f87200fea90 2023-06-23T18:20:43.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.180+0000 7f871cff9700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f870c0a1150 0x7f870c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:43.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.180+0000 7f8717fff700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f870c0a1150 0x7f870c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:43.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.180+0000 7f871cff9700 1 -- 172.21.15.2:0/2395277102 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f87100cdee0 con 0x7f87200fea90 2023-06-23T18:20:43.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.181+0000 7f8717fff700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f870c0a1150 0x7f870c0a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8708009c60 tx=0x7f8708009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:43.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.184+0000 7f871cff9700 1 -- 172.21.15.2:0/2395277102 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f87100979f0 con 0x7f87200fea90 2023-06-23T18:20:43.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:42 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:43.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:42 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3836472775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:43.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:42 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1345197342' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-23T18:20:43.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:42 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:43.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:42 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3836472775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:43.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:42 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1345197342' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-23T18:20:43.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:43.401+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f872002c810 con 0x7f87200fea90 2023-06-23T18:20:43.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:42 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:43.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:42 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3836472775' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:20:43.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:42 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1345197342' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-06-23T18:20:44.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.020+0000 7f871cff9700 1 -- 172.21.15.2:0/2395277102 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f871009b010 con 0x7f87200fea90 2023-06-23T18:20:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.026+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f870c0a1150 msgr2=0x7f870c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.026+0000 7f8725339700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f870c0a1150 0x7f870c0a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8708009c60 tx=0x7f8708009260 comp rx=0 tx=0).stop 2023-06-23T18:20:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.026+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 msgr2=0x7f8720194740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.026+0000 7f8725339700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 0x7f8720194740 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f87100098e0 tx=0x7f8710004730 comp rx=0 tx=0).stop 2023-06-23T18:20:44.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.032+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 shutdown_connections 2023-06-23T18:20:44.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.032+0000 7f8725339700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f870c0a1150 0x7f870c0a3600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.032+0000 7f8725339700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8720100610 0x7f872019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.032+0000 7f8725339700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f87200fea90 0x7f8720194740 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.032+0000 7f8725339700 1 --2- 172.21.15.2:0/2395277102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f87200ffc80 0x7f8720194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.032+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 >> 172.21.15.2:0/2395277102 conn(0x7f87200fa090 msgr2=0x7f87201031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:44.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.032+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 shutdown_connections 2023-06-23T18:20:44.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.032+0000 7f8725339700 1 -- 172.21.15.2:0/2395277102 wait complete. 2023-06-23T18:20:44.038 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2023-06-23T18:20:44.054 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-06-23T18:20:44.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 -- 172.21.15.2:0/2836925633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680fe220 msgr2=0x7f74680fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 --2- 172.21.15.2:0/2836925633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680fe220 0x7f74680fe630 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f745c009990 tx=0x7f745c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 -- 172.21.15.2:0/2836925633 shutdown_connections 2023-06-23T18:20:44.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 --2- 172.21.15.2:0/2836925633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74680ffe60 0x7f7468106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 --2- 172.21.15.2:0/2836925633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680fe220 0x7f74680fe630 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 --2- 172.21.15.2:0/2836925633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680ff4b0 0x7f74680ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 -- 172.21.15.2:0/2836925633 >> 172.21.15.2:0/2836925633 conn(0x7f74680f9820 msgr2=0x7f74680fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:44.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 -- 172.21.15.2:0/2836925633 shutdown_connections 2023-06-23T18:20:44.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 -- 172.21.15.2:0/2836925633 wait complete. 2023-06-23T18:20:44.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.179+0000 7f746d7ef700 1 Processor -- start 2023-06-23T18:20:44.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f746d7ef700 1 -- start start 2023-06-23T18:20:44.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f746d7ef700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680fe220 0x7f7468197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f746d7ef700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74680ff4b0 0x7f7468197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f746d7ef700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680ffe60 0x7f746819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f746d7ef700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819f600 con 0x7f74680ffe60 2023-06-23T18:20:44.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f746d7ef700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819f740 con 0x7f74680ff4b0 2023-06-23T18:20:44.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f746d7ef700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746819f8d0 con 0x7f74680fe220 2023-06-23T18:20:44.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680ffe60 0x7f746819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680ffe60 0x7f746819f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52394/0 (socket says 172.21.15.2:52394) 2023-06-23T18:20:44.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f7466ffd700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680fe220 0x7f7468197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 -- 172.21.15.2:0/929176791 learned_addr learned my addr 172.21.15.2:0/929176791 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:44.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74667fc700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74680ff4b0 0x7f7468197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 -- 172.21.15.2:0/929176791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680fe220 msgr2=0x7f7468197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680fe220 0x7f7468197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 -- 172.21.15.2:0/929176791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74680ff4b0 msgr2=0x7f7468197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74680ff4b0 0x7f7468197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 -- 172.21.15.2:0/929176791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7458009520 con 0x7f74680ffe60 2023-06-23T18:20:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74667fc700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74680ff4b0 0x7f7468197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f7466ffd700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680fe220 0x7f7468197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:20:44.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f74677fe700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680ffe60 0x7f746819f030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7460007ee0 tx=0x7f746000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f744ffff700 1 -- 172.21.15.2:0/929176791 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460017070 con 0x7f74680ffe60 2023-06-23T18:20:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f744ffff700 1 -- 172.21.15.2:0/929176791 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f746000f040 con 0x7f74680ffe60 2023-06-23T18:20:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.180+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f745c0095f0 con 0x7f74680ffe60 2023-06-23T18:20:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.181+0000 7f744ffff700 1 -- 172.21.15.2:0/929176791 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460005710 con 0x7f74680ffe60 2023-06-23T18:20:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.181+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746819fee0 con 0x7f74680ffe60 2023-06-23T18:20:44.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.181+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7468104290 con 0x7f74680ffe60 2023-06-23T18:20:44.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.183+0000 7f744ffff700 1 -- 172.21.15.2:0/929176791 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7460005870 con 0x7f74680ffe60 2023-06-23T18:20:44.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.183+0000 7f744ffff700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74500a1150 0x7f74500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.183+0000 7f7466ffd700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74500a1150 0x7f74500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.183+0000 7f744ffff700 1 -- 172.21.15.2:0/929176791 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f74600cc5c0 con 0x7f74680ffe60 2023-06-23T18:20:44.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.184+0000 7f7466ffd700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74500a1150 0x7f74500a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f745c006010 tx=0x7f745c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:44.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.187+0000 7f744ffff700 1 -- 172.21.15.2:0/929176791 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7460095ec0 con 0x7f74680ffe60 2023-06-23T18:20:44.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/495213362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T18:20:44.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:44.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/495213362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-23T18:20:44.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: osdmap e79: 12 total, 12 up, 12 in 2023-06-23T18:20:44.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2395277102' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T18:20:44.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:43 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/495213362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T18:20:44.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:44.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/495213362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-23T18:20:44.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: osdmap e79: 12 total, 12 up, 12 in 2023-06-23T18:20:44.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2395277102' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T18:20:44.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:43 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.433+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f746802ca20 con 0x7f74680ffe60 2023-06-23T18:20:44.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/495213362' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-06-23T18:20:44.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail 2023-06-23T18:20:44.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/495213362' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-06-23T18:20:44.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: osdmap e79: 12 total, 12 up, 12 in 2023-06-23T18:20:44.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2395277102' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-06-23T18:20:44.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:43 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:44.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.440+0000 7f744ffff700 1 -- 172.21.15.2:0/929176791 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f74600994e0 con 0x7f74680ffe60 2023-06-23T18:20:44.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.444+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74500a1150 msgr2=0x7f74500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.444+0000 7f746d7ef700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74500a1150 0x7f74500a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f745c006010 tx=0x7f745c00b260 comp rx=0 tx=0).stop 2023-06-23T18:20:44.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.444+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680ffe60 msgr2=0x7f746819f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.444+0000 7f746d7ef700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680ffe60 0x7f746819f030 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7460007ee0 tx=0x7f746000d3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.445+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 shutdown_connections 2023-06-23T18:20:44.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.445+0000 7f746d7ef700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74680ffe60 0x7f746819f030 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.445+0000 7f746d7ef700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74680fe220 0x7f7468197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.445+0000 7f746d7ef700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f74500a1150 0x7f74500a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.445+0000 7f746d7ef700 1 --2- 172.21.15.2:0/929176791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f74680ff4b0 0x7f7468197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.445+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 >> 172.21.15.2:0/929176791 conn(0x7f74680f9820 msgr2=0x7f74680fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:44.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.446+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 shutdown_connections 2023-06-23T18:20:44.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.446+0000 7f746d7ef700 1 -- 172.21.15.2:0/929176791 wait complete. 2023-06-23T18:20:44.452 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 2 and data pool 3 2023-06-23T18:20:44.475 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:20:44.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 -- 172.21.15.2:0/961045053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c0ffc50 msgr2=0x7fa13c1000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 --2- 172.21.15.2:0/961045053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c0ffc50 0x7fa13c1000a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa12c009990 tx=0x7fa12c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 -- 172.21.15.2:0/961045053 shutdown_connections 2023-06-23T18:20:44.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 --2- 172.21.15.2:0/961045053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c1005e0 0x7fa13c106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 --2- 172.21.15.2:0/961045053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c0fea60 0x7fa13c0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 --2- 172.21.15.2:0/961045053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c0ffc50 0x7fa13c1000a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 -- 172.21.15.2:0/961045053 >> 172.21.15.2:0/961045053 conn(0x7fa13c0fa0e0 msgr2=0x7fa13c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:44.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 -- 172.21.15.2:0/961045053 shutdown_connections 2023-06-23T18:20:44.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.599+0000 7fa143efe700 1 -- 172.21.15.2:0/961045053 wait complete. 2023-06-23T18:20:44.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa143efe700 1 Processor -- start 2023-06-23T18:20:44.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa143efe700 1 -- start start 2023-06-23T18:20:44.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa143efe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c0fea60 0x7fa13c1029f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa143efe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c0ffc50 0x7fa13c101030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa143efe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c1005e0 0x7fa13c101570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa143efe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c101d10 con 0x7fa13c1005e0 2023-06-23T18:20:44.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa143efe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c101e80 con 0x7fa13c0fea60 2023-06-23T18:20:44.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa143efe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13c19f880 con 0x7fa13c0ffc50 2023-06-23T18:20:44.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa14249b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c1005e0 0x7fa13c101570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa14249b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c1005e0 0x7fa13c101570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52416/0 (socket says 172.21.15.2:52416) 2023-06-23T18:20:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa14249b700 1 -- 172.21.15.2:0/821145901 learned_addr learned my addr 172.21.15.2:0/821145901 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa141499700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c0ffc50 0x7fa13c101030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa141c9a700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c0fea60 0x7fa13c1029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa14249b700 1 -- 172.21.15.2:0/821145901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c0ffc50 msgr2=0x7fa13c101030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa14249b700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c0ffc50 0x7fa13c101030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa14249b700 1 -- 172.21.15.2:0/821145901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c0fea60 msgr2=0x7fa13c1029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa14249b700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c0fea60 0x7fa13c1029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa14249b700 1 -- 172.21.15.2:0/821145901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0095f0 con 0x7fa13c1005e0 2023-06-23T18:20:44.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.600+0000 7fa141c9a700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c0fea60 0x7fa13c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:44.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.601+0000 7fa14249b700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c1005e0 0x7fa13c101570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa13400ef30 tx=0x7fa13400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:44.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.601+0000 7fa132ffd700 1 -- 172.21.15.2:0/821145901 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134018070 con 0x7fa13c1005e0 2023-06-23T18:20:44.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.601+0000 7fa132ffd700 1 -- 172.21.15.2:0/821145901 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa13400f040 con 0x7fa13c1005e0 2023-06-23T18:20:44.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.601+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa13c19fb00 con 0x7fa13c1005e0 2023-06-23T18:20:44.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.601+0000 7fa132ffd700 1 -- 172.21.15.2:0/821145901 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13400cc70 con 0x7fa13c1005e0 2023-06-23T18:20:44.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.601+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa13c1a0010 con 0x7fa13c1005e0 2023-06-23T18:20:44.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.602+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa13c10a130 con 0x7fa13c1005e0 2023-06-23T18:20:44.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.603+0000 7fa132ffd700 1 -- 172.21.15.2:0/821145901 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa134007500 con 0x7fa13c1005e0 2023-06-23T18:20:44.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.603+0000 7fa132ffd700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa1280a11d0 0x7fa1280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.604+0000 7fa141c9a700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa1280a11d0 0x7fa1280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.604+0000 7fa132ffd700 1 -- 172.21.15.2:0/821145901 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa1340cc240 con 0x7fa13c1005e0 2023-06-23T18:20:44.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.604+0000 7fa141c9a700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa1280a11d0 0x7fa1280a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0ffab0 tx=0x7fa138009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:44.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.608+0000 7fa132ffd700 1 -- 172.21.15.2:0/821145901 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa134095b70 con 0x7fa13c1005e0 2023-06-23T18:20:44.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.821+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa13c1a08c0 con 0x7fa13c1005e0 2023-06-23T18:20:44.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.823+0000 7fa132ffd700 1 -- 172.21.15.2:0/821145901 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+21953 (secure 0 0 0) 0x7fa134099190 con 0x7fa13c1005e0 2023-06-23T18:20:44.828 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:44.828 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":81,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:44.434202+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:20:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa1280a11d0 msgr2=0x7fa1280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa1280a11d0 0x7fa1280a3680 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0ffab0 tx=0x7fa138009670 comp rx=0 tx=0).stop 2023-06-23T18:20:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c1005e0 msgr2=0x7fa13c101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c1005e0 0x7fa13c101570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa13400ef30 tx=0x7fa13400c3a0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 shutdown_connections 2023-06-23T18:20:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa13c1005e0 0x7fa13c101570 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa1280a11d0 0x7fa1280a3680 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa13c0fea60 0x7fa13c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 --2- 172.21.15.2:0/821145901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa13c0ffc50 0x7fa13c101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 >> 172.21.15.2:0/821145901 conn(0x7fa13c0fa0e0 msgr2=0x7fa13c0677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 shutdown_connections 2023-06-23T18:20:44.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.825+0000 7fa143efe700 1 -- 172.21.15.2:0/821145901 wait complete. 2023-06-23T18:20:44.842 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:20:44.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 -- 172.21.15.2:0/1435997671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 msgr2=0x7f5aac106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 --2- 172.21.15.2:0/1435997671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 0x7f5aac106ea0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400b0a0 tx=0x7f5aa400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 -- 172.21.15.2:0/1435997671 shutdown_connections 2023-06-23T18:20:44.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 --2- 172.21.15.2:0/1435997671 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 0x7f5aac106ea0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 --2- 172.21.15.2:0/1435997671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aac0fea60 0x7f5aac0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 --2- 172.21.15.2:0/1435997671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aac0ffc50 0x7f5aac1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 -- 172.21.15.2:0/1435997671 >> 172.21.15.2:0/1435997671 conn(0x7f5aac0fa0e0 msgr2=0x7f5aac0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:44.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 -- 172.21.15.2:0/1435997671 shutdown_connections 2023-06-23T18:20:44.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.970+0000 7f5ab235b700 1 -- 172.21.15.2:0/1435997671 wait complete. 2023-06-23T18:20:44.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab235b700 1 Processor -- start 2023-06-23T18:20:44.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab235b700 1 -- start start 2023-06-23T18:20:44.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab235b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aac0fea60 0x7f5aac193690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab235b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aac0ffc50 0x7f5aac193bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab235b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 0x7f5aac19ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab235b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac19b210 con 0x7f5aac1005e0 2023-06-23T18:20:44.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab235b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac19b350 con 0x7f5aac0fea60 2023-06-23T18:20:44.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab235b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aac19b4e0 con 0x7f5aac0ffc50 2023-06-23T18:20:44.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab08f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 0x7f5aac19ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab08f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 0x7f5aac19ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52430/0 (socket says 172.21.15.2:52430) 2023-06-23T18:20:44.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab08f8700 1 -- 172.21.15.2:0/3388804511 learned_addr learned my addr 172.21.15.2:0/3388804511 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:44.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5aab7fe700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aac0ffc50 0x7f5aac193bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5aabfff700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aac0fea60 0x7f5aac193690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab08f8700 1 -- 172.21.15.2:0/3388804511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aac0ffc50 msgr2=0x7f5aac193bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab08f8700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aac0ffc50 0x7f5aac193bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab08f8700 1 -- 172.21.15.2:0/3388804511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aac0fea60 msgr2=0x7f5aac193690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:44.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab08f8700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aac0fea60 0x7f5aac193690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:44.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5ab08f8700 1 -- 172.21.15.2:0/3388804511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa4009d00 con 0x7f5aac1005e0 2023-06-23T18:20:44.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5aab7fe700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aac0ffc50 0x7f5aac193bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:44.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.971+0000 7f5aabfff700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aac0fea60 0x7f5aac193690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:44.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.972+0000 7f5ab08f8700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 0x7f5aac19ac40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400b830 tx=0x7f5aa4004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:44.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.972+0000 7f5aa97fa700 1 -- 172.21.15.2:0/3388804511 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa400e040 con 0x7f5aac1005e0 2023-06-23T18:20:44.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.972+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac19b760 con 0x7f5aac1005e0 2023-06-23T18:20:44.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.972+0000 7f5aa97fa700 1 -- 172.21.15.2:0/3388804511 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5aa4009440 con 0x7f5aac1005e0 2023-06-23T18:20:44.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.972+0000 7f5aa97fa700 1 -- 172.21.15.2:0/3388804511 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aa401dbe0 con 0x7f5aac1005e0 2023-06-23T18:20:44.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.972+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aac19bb80 con 0x7f5aac1005e0 2023-06-23T18:20:44.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.973+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac105f40 con 0x7f5aac1005e0 2023-06-23T18:20:44.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.974+0000 7f5aa97fa700 1 -- 172.21.15.2:0/3388804511 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5aa4005450 con 0x7f5aac1005e0 2023-06-23T18:20:44.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.974+0000 7f5aa97fa700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a940a1150 0x7f5a940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:44.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.975+0000 7f5aa97fa700 1 -- 172.21.15.2:0/3388804511 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5aa40ce770 con 0x7f5aac1005e0 2023-06-23T18:20:44.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.975+0000 7f5aabfff700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a940a1150 0x7f5a940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:44.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.975+0000 7f5aabfff700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a940a1150 0x7f5a940a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c005fd0 tx=0x7f5a9c005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:44.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:44.978+0000 7f5aa97fa700 1 -- 172.21.15.2:0/3388804511 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5aa40980a0 con 0x7f5aac1005e0 2023-06-23T18:20:45.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.190+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5aac02c810 con 0x7f5aac1005e0 2023-06-23T18:20:45.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.191+0000 7f5aa97fa700 1 -- 172.21.15.2:0/3388804511 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+21953 (secure 0 0 0) 0x7f5aa409b6c0 con 0x7f5aac1005e0 2023-06-23T18:20:45.196 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:45.197 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":81,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:44.434202+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:20:45.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.193+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a940a1150 msgr2=0x7f5a940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:45.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.193+0000 7f5ab235b700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a940a1150 0x7f5a940a3600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c005fd0 tx=0x7f5a9c005f00 comp rx=0 tx=0).stop 2023-06-23T18:20:45.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.193+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 msgr2=0x7f5aac19ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:45.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.193+0000 7f5ab235b700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 0x7f5aac19ac40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400b830 tx=0x7f5aa4004d90 comp rx=0 tx=0).stop 2023-06-23T18:20:45.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.194+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 shutdown_connections 2023-06-23T18:20:45.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.194+0000 7f5ab235b700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5aac1005e0 0x7f5aac19ac40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.194+0000 7f5ab235b700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a940a1150 0x7f5a940a3600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.194+0000 7f5ab235b700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5aac0fea60 0x7f5aac193690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.194+0000 7f5ab235b700 1 --2- 172.21.15.2:0/3388804511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5aac0ffc50 0x7f5aac193bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.194+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 >> 172.21.15.2:0/3388804511 conn(0x7f5aac0fa0e0 msgr2=0x7f5aac103080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:45.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.194+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 shutdown_connections 2023-06-23T18:20:45.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.194+0000 7f5ab235b700 1 -- 172.21.15.2:0/3388804511 wait complete. 2023-06-23T18:20:45.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2395277102' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-23T18:20:45.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: osdmap e80: 12 total, 12 up, 12 in 2023-06-23T18:20:45.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/929176791' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T18:20:45.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-23T18:20:45.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: osdmap e81: 12 total, 12 up, 12 in 2023-06-23T18:20:45.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/929176791' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-23T18:20:45.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-23T18:20:45.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:45.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:45.211 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:45.211 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-23T18:20:45.211 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:45.211 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: pgmap v255: 97 pgs: 14 creating+peering, 16 active+clean, 67 unknown; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2023-06-23T18:20:45.211 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:45 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/821145901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:45.213 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-06-23T18:20:45.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2395277102' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-23T18:20:45.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: osdmap e80: 12 total, 12 up, 12 in 2023-06-23T18:20:45.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/929176791' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T18:20:45.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-23T18:20:45.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: osdmap e81: 12 total, 12 up, 12 in 2023-06-23T18:20:45.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/929176791' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-23T18:20:45.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-23T18:20:45.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:45.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:45.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:45.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-23T18:20:45.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:45.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: pgmap v255: 97 pgs: 14 creating+peering, 16 active+clean, 67 unknown; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2023-06-23T18:20:45.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:45 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/821145901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:45.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.347+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/1505336235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 msgr2=0x7fd0b80ff150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:45.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.347+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/1505336235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 0x7fd0b80ff150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac009990 tx=0x7fd0ac009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.347+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/1505336235 shutdown_connections 2023-06-23T18:20:45.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.347+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/1505336235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b80ff690 0x7fd0b8106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.347+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/1505336235 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0b8100120 0x7fd0b8100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.347+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/1505336235 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 0x7fd0b80ff150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/1505336235 >> 172.21.15.2:0/1505336235 conn(0x7fd0b80f9820 msgr2=0x7fd0b80fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:45.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/1505336235 shutdown_connections 2023-06-23T18:20:45.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/1505336235 wait complete. 2023-06-23T18:20:45.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 Processor -- start 2023-06-23T18:20:45.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 -- start start 2023-06-23T18:20:45.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 0x7fd0b81957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:45.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b80ff690 0x7fd0b8195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:45.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0b8100120 0x7fd0b819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:45.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b819d330 con 0x7fd0b80fece0 2023-06-23T18:20:45.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b819d470 con 0x7fd0b80ff690 2023-06-23T18:20:45.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.348+0000 7fd0bfaf7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b819d600 con 0x7fd0b8100120 2023-06-23T18:20:45.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 0x7fd0b81957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:45.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 0x7fd0b81957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52440/0 (socket says 172.21.15.2:52440) 2023-06-23T18:20:45.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 -- 172.21.15.2:0/650644666 learned_addr learned my addr 172.21.15.2:0/650644666 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:45.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd092700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b80ff690 0x7fd0b8195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:45.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0be094700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0b8100120 0x7fd0b819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:45.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 -- 172.21.15.2:0/650644666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0b8100120 msgr2=0x7fd0b819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:45.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0b8100120 0x7fd0b819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 -- 172.21.15.2:0/650644666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b80ff690 msgr2=0x7fd0b8195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:45.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b80ff690 0x7fd0b8195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:45.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 -- 172.21.15.2:0/650644666 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac0095f0 con 0x7fd0b80fece0 2023-06-23T18:20:45.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0be094700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0b8100120 0x7fd0b819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:45.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bd893700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 0x7fd0b81957b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0007f10 tx=0x7fd0b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0b6ffd700 1 -- 172.21.15.2:0/650644666 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b000ddd0 con 0x7fd0b80fece0 2023-06-23T18:20:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b819d8e0 con 0x7fd0b80fece0 2023-06-23T18:20:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0b6ffd700 1 -- 172.21.15.2:0/650644666 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0b000f040 con 0x7fd0b80fece0 2023-06-23T18:20:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0b6ffd700 1 -- 172.21.15.2:0/650644666 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0004400 con 0x7fd0b80fece0 2023-06-23T18:20:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.349+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b819de40 con 0x7fd0b80fece0 2023-06-23T18:20:45.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.350+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b8106b70 con 0x7fd0b80fece0 2023-06-23T18:20:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.351+0000 7fd0b6ffd700 1 -- 172.21.15.2:0/650644666 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd0b0004ad0 con 0x7fd0b80fece0 2023-06-23T18:20:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.352+0000 7fd0b6ffd700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0a00a1150 0x7fd0a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.352+0000 7fd0b6ffd700 1 -- 172.21.15.2:0/650644666 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd0b00ccc40 con 0x7fd0b80fece0 2023-06-23T18:20:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.352+0000 7fd0bd092700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0a00a1150 0x7fd0a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:45.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.352+0000 7fd0bd092700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0a00a1150 0x7fd0a00a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00b5c0 tx=0x7fd0ac005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:45.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.355+0000 7fd0b6ffd700 1 -- 172.21.15.2:0/650644666 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd0b00964e0 con 0x7fd0b80fece0 2023-06-23T18:20:45.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2395277102' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-06-23T18:20:45.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: osdmap e80: 12 total, 12 up, 12 in 2023-06-23T18:20:45.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/929176791' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-06-23T18:20:45.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-06-23T18:20:45.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: osdmap e81: 12 total, 12 up, 12 in 2023-06-23T18:20:45.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/929176791' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-06-23T18:20:45.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-06-23T18:20:45.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:45.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:45.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:45.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-06-23T18:20:45.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:20:45.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: pgmap v255: 97 pgs: 14 creating+peering, 16 active+clean, 67 unknown; 577 KiB data, 325 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 0 op/s 2023-06-23T18:20:45.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/821145901' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:45.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:45.627+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd0b802c810 con 0x7fd0b80fece0 2023-06-23T18:20:45.940 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:20:45 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-b[122882]: 2023-06-23T18:20:45.597+0000 7f728e624700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-06-23T18:20:46.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:46 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3388804511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:46.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:46 smithi002 ceph-mon[95947]: osdmap e82: 12 total, 12 up, 12 in 2023-06-23T18:20:46.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:46 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] up:active 2023-06-23T18:20:46.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:46 smithi002 ceph-mon[95947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:46.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:46 smithi002 ceph-mon[95947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:46.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:46 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/650644666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-06-23T18:20:46.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:46 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3388804511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:46.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:46 smithi082 ceph-mon[97889]: osdmap e82: 12 total, 12 up, 12 in 2023-06-23T18:20:46.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:46 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] up:active 2023-06-23T18:20:46.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:46 smithi082 ceph-mon[97889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:46.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:46 smithi082 ceph-mon[97889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:46.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:46 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/650644666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-06-23T18:20:46.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:46 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3388804511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:46.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:46 smithi017 ceph-mon[97472]: osdmap e82: 12 total, 12 up, 12 in 2023-06-23T18:20:46.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:46 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] up:active 2023-06-23T18:20:46.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:46 smithi017 ceph-mon[97472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:46.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:46 smithi017 ceph-mon[97472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:46.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:46 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/650644666' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-06-23T18:20:46.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.477+0000 7fd0b6ffd700 1 -- 172.21.15.2:0/650644666 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd0b0019020 con 0x7fd0b80fece0 2023-06-23T18:20:46.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0a00a1150 msgr2=0x7fd0a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:46.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0a00a1150 0x7fd0a00a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00b5c0 tx=0x7fd0ac005fb0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 msgr2=0x7fd0b81957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:46.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 0x7fd0b81957b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0007f10 tx=0x7fd0b000d3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 shutdown_connections 2023-06-23T18:20:46.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0b8100120 0x7fd0b819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0b80fece0 0x7fd0b81957b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0a00a1150 0x7fd0a00a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 --2- 172.21.15.2:0/650644666 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0b80ff690 0x7fd0b8195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 >> 172.21.15.2:0/650644666 conn(0x7fd0b80f9820 msgr2=0x7fd0b80fba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:46.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 shutdown_connections 2023-06-23T18:20:46.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.480+0000 7fd0bfaf7700 1 -- 172.21.15.2:0/650644666 wait complete. 2023-06-23T18:20:46.498 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-06-23T18:20:46.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-06-23T18:20:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 -- 172.21.15.2:0/2798330921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c105900 msgr2=0x7ff96c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 --2- 172.21.15.2:0/2798330921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c105900 0x7ff96c107ce0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff96400b0a0 tx=0x7ff96400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 -- 172.21.15.2:0/2798330921 shutdown_connections 2023-06-23T18:20:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 --2- 172.21.15.2:0/2798330921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c105900 0x7ff96c107ce0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 --2- 172.21.15.2:0/2798330921 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff96c1006c0 0x7ff96c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 --2- 172.21.15.2:0/2798330921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c102fe0 0x7ff96c1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 -- 172.21.15.2:0/2798330921 >> 172.21.15.2:0/2798330921 conn(0x7ff96c0fa0e0 msgr2=0x7ff96c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 -- 172.21.15.2:0/2798330921 shutdown_connections 2023-06-23T18:20:46.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.633+0000 7ff971941700 1 -- 172.21.15.2:0/2798330921 wait complete. 2023-06-23T18:20:46.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.634+0000 7ff971941700 1 Processor -- start 2023-06-23T18:20:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.634+0000 7ff971941700 1 -- start start 2023-06-23T18:20:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.634+0000 7ff971941700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c1006c0 0x7ff96c195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.634+0000 7ff971941700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c102fe0 0x7ff96c195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.634+0000 7ff971941700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff96c105900 0x7ff96c19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.634+0000 7ff971941700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c19d410 con 0x7ff96c1006c0 2023-06-23T18:20:46.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.634+0000 7ff971941700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c19d550 con 0x7ff96c102fe0 2023-06-23T18:20:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.634+0000 7ff971941700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c19d6e0 con 0x7ff96c105900 2023-06-23T18:20:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c1006c0 0x7ff96c195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c1006c0 0x7ff96c195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52452/0 (socket says 172.21.15.2:52452) 2023-06-23T18:20:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 -- 172.21.15.2:0/1415875557 learned_addr learned my addr 172.21.15.2:0/1415875557 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96a7fc700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c102fe0 0x7ff96c195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96b7fe700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff96c105900 0x7ff96c19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 -- 172.21.15.2:0/1415875557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff96c105900 msgr2=0x7ff96c19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:46.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff96c105900 0x7ff96c19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 -- 172.21.15.2:0/1415875557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c102fe0 msgr2=0x7ff96c195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c102fe0 0x7ff96c195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 -- 172.21.15.2:0/1415875557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960009520 con 0x7ff96c1006c0 2023-06-23T18:20:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff96affd700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c1006c0 0x7ff96c195890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e9a0 tx=0x7ff95c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff97093f700 1 -- 172.21.15.2:0/1415875557 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c009790 con 0x7ff96c1006c0 2023-06-23T18:20:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff97093f700 1 -- 172.21.15.2:0/1415875557 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff95c010430 con 0x7ff96c1006c0 2023-06-23T18:20:46.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff964009d00 con 0x7ff96c1006c0 2023-06-23T18:20:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff97093f700 1 -- 172.21.15.2:0/1415875557 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c00cea0 con 0x7ff96c1006c0 2023-06-23T18:20:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.635+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c19dd40 con 0x7ff96c1006c0 2023-06-23T18:20:46.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.637+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c0fcc60 con 0x7ff96c1006c0 2023-06-23T18:20:46.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.640+0000 7ff97093f700 1 -- 172.21.15.2:0/1415875557 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff95c0105a0 con 0x7ff96c1006c0 2023-06-23T18:20:46.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.641+0000 7ff97093f700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9540a1150 0x7ff9540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:46.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.641+0000 7ff96a7fc700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9540a1150 0x7ff9540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:46.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.642+0000 7ff97093f700 1 -- 172.21.15.2:0/1415875557 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff95c014070 con 0x7ff96c1006c0 2023-06-23T18:20:46.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.642+0000 7ff96a7fc700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9540a1150 0x7ff9540a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff960009df0 tx=0x7ff960009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:46.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.643+0000 7ff97093f700 1 -- 172.21.15.2:0/1415875557 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff95c096430 con 0x7ff96c1006c0 2023-06-23T18:20:46.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:46.901+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7ff96c02caa0 con 0x7ff96c1006c0 2023-06-23T18:20:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.481+0000 7ff97093f700 1 -- 172.21.15.2:0/1415875557 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7ff95c0260b0 con 0x7ff96c1006c0 2023-06-23T18:20:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9540a1150 msgr2=0x7ff9540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9540a1150 0x7ff9540a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff960009df0 tx=0x7ff960009260 comp rx=0 tx=0).stop 2023-06-23T18:20:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c1006c0 msgr2=0x7ff96c195890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c1006c0 0x7ff96c195890 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00e9a0 tx=0x7ff95c00ecb0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 shutdown_connections 2023-06-23T18:20:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff96c105900 0x7ff96c19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff96c1006c0 0x7ff96c195890 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9540a1150 0x7ff9540a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 --2- 172.21.15.2:0/1415875557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff96c102fe0 0x7ff96c195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 >> 172.21.15.2:0/1415875557 conn(0x7ff96c0fa0e0 msgr2=0x7ff96c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 shutdown_connections 2023-06-23T18:20:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.484+0000 7ff971941700 1 -- 172.21.15.2:0/1415875557 wait complete. 2023-06-23T18:20:47.505 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-06-23T18:20:47.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 -- 172.21.15.2:0/1653180301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44102f20 msgr2=0x7fea44105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:47.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 --2- 172.21.15.2:0/1653180301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44102f20 0x7fea44105300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fea38009990 tx=0x7fea38009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 -- 172.21.15.2:0/1653180301 shutdown_connections 2023-06-23T18:20:47.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 --2- 172.21.15.2:0/1653180301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea44105840 0x7fea44107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 --2- 172.21.15.2:0/1653180301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea44100600 0x7fea441029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 --2- 172.21.15.2:0/1653180301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44102f20 0x7fea44105300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 -- 172.21.15.2:0/1653180301 >> 172.21.15.2:0/1653180301 conn(0x7fea440fa0e0 msgr2=0x7fea440fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:47.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 -- 172.21.15.2:0/1653180301 shutdown_connections 2023-06-23T18:20:47.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 -- 172.21.15.2:0/1653180301 wait complete. 2023-06-23T18:20:47.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.637+0000 7fea49230700 1 Processor -- start 2023-06-23T18:20:47.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea49230700 1 -- start start 2023-06-23T18:20:47.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea49230700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44100600 0x7fea441957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:47.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea49230700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea44102f20 0x7fea44195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:47.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea49230700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea44105840 0x7fea4419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:47.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea49230700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4419d330 con 0x7fea44100600 2023-06-23T18:20:47.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea49230700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4419d470 con 0x7fea44102f20 2023-06-23T18:20:47.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea49230700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4419d600 con 0x7fea44105840 2023-06-23T18:20:47.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea42d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44100600 0x7fea441957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:47.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea42d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44100600 0x7fea441957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52466/0 (socket says 172.21.15.2:52466) 2023-06-23T18:20:47.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea42d9d700 1 -- 172.21.15.2:0/1968125424 learned_addr learned my addr 172.21.15.2:0/1968125424 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:47.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea4259c700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea44102f20 0x7fea44195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea4359e700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea44105840 0x7fea4419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea42d9d700 1 -- 172.21.15.2:0/1968125424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea44105840 msgr2=0x7fea4419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea42d9d700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea44105840 0x7fea4419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea42d9d700 1 -- 172.21.15.2:0/1968125424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea44102f20 msgr2=0x7fea44195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea42d9d700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea44102f20 0x7fea44195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea42d9d700 1 -- 172.21.15.2:0/1968125424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c009c40 con 0x7fea44100600 2023-06-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.638+0000 7fea4259c700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea44102f20 0x7fea44195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:20:47.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.639+0000 7fea42d9d700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44100600 0x7fea441957b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fea3400d980 tx=0x7fea3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:47.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.639+0000 7fea2bfff700 1 -- 172.21.15.2:0/1968125424 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34004220 con 0x7fea44100600 2023-06-23T18:20:47.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.639+0000 7fea2bfff700 1 -- 172.21.15.2:0/1968125424 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea34007640 con 0x7fea44100600 2023-06-23T18:20:47.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.639+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea380095f0 con 0x7fea44100600 2023-06-23T18:20:47.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.639+0000 7fea2bfff700 1 -- 172.21.15.2:0/1968125424 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea340054b0 con 0x7fea44100600 2023-06-23T18:20:47.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.639+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4419dcb0 con 0x7fea44100600 2023-06-23T18:20:47.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.640+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea440fcc60 con 0x7fea44100600 2023-06-23T18:20:47.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.641+0000 7fea2bfff700 1 -- 172.21.15.2:0/1968125424 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fea34005610 con 0x7fea44100600 2023-06-23T18:20:47.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.642+0000 7fea2bfff700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea2c0a11d0 0x7fea2c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:47.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.642+0000 7fea4259c700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea2c0a11d0 0x7fea2c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:47.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.642+0000 7fea2bfff700 1 -- 172.21.15.2:0/1968125424 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fea340cc9f0 con 0x7fea44100600 2023-06-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.642+0000 7fea4259c700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea2c0a11d0 0x7fea2c0a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fea380059d0 tx=0x7fea38005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:47.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.645+0000 7fea2bfff700 1 -- 172.21.15.2:0/1968125424 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fea34096290 con 0x7fea44100600 2023-06-23T18:20:47.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:47 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/650644666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-06-23T18:20:47.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:47 smithi002 ceph-mon[95947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:47.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:47 smithi002 ceph-mon[95947]: pgmap v257: 97 pgs: 14 creating+peering, 43 active+clean, 40 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2023-06-23T18:20:47.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:47 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1415875557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-06-23T18:20:47.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:47 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:47 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/650644666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-06-23T18:20:47.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:47 smithi082 ceph-mon[97889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:47.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:47 smithi082 ceph-mon[97889]: pgmap v257: 97 pgs: 14 creating+peering, 43 active+clean, 40 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2023-06-23T18:20:47.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:47 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1415875557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-06-23T18:20:47.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:47 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:47.893+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fea4402c810 con 0x7fea44100600 2023-06-23T18:20:47.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:47 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/650644666' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-06-23T18:20:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:47 smithi017 ceph-mon[97472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:47.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:47 smithi017 ceph-mon[97472]: pgmap v257: 97 pgs: 14 creating+peering, 43 active+clean, 40 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 1 op/s 2023-06-23T18:20:47.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:47 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1415875557' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-06-23T18:20:47.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:47.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:47 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.490+0000 7fea2bfff700 1 -- 172.21.15.2:0/1968125424 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fea34016070 con 0x7fea44100600 2023-06-23T18:20:48.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea2c0a11d0 msgr2=0x7fea2c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:48.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea2c0a11d0 0x7fea2c0a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fea380059d0 tx=0x7fea38005960 comp rx=0 tx=0).stop 2023-06-23T18:20:48.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44100600 msgr2=0x7fea441957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:48.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44100600 0x7fea441957b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fea3400d980 tx=0x7fea3400dc90 comp rx=0 tx=0).stop 2023-06-23T18:20:48.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 shutdown_connections 2023-06-23T18:20:48.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea44105840 0x7fea4419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea44100600 0x7fea441957b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea2c0a11d0 0x7fea2c0a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 --2- 172.21.15.2:0/1968125424 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea44102f20 0x7fea44195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 >> 172.21.15.2:0/1968125424 conn(0x7fea440fa0e0 msgr2=0x7fea44101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:48.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 shutdown_connections 2023-06-23T18:20:48.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.493+0000 7fea49230700 1 -- 172.21.15.2:0/1968125424 wait complete. 2023-06-23T18:20:48.517 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-06-23T18:20:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 -- 172.21.15.2:0/3404766084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040100600 msgr2=0x7f00401029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:48.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 --2- 172.21.15.2:0/3404766084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040100600 0x7f00401029e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0034009990 tx=0x7f0034009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 -- 172.21.15.2:0/3404766084 shutdown_connections 2023-06-23T18:20:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 --2- 172.21.15.2:0/3404766084 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0040105840 0x7f0040107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 --2- 172.21.15.2:0/3404766084 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040100600 0x7f00401029e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 --2- 172.21.15.2:0/3404766084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0040102f20 0x7f0040105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 -- 172.21.15.2:0/3404766084 >> 172.21.15.2:0/3404766084 conn(0x7f00400fa0e0 msgr2=0x7f00400fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 -- 172.21.15.2:0/3404766084 shutdown_connections 2023-06-23T18:20:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 -- 172.21.15.2:0/3404766084 wait complete. 2023-06-23T18:20:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.651+0000 7f004855f700 1 Processor -- start 2023-06-23T18:20:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f004855f700 1 -- start start 2023-06-23T18:20:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f004855f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0040100600 0x7f00400fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:48.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f004855f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0040102f20 0x7f00400fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f004855f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040105840 0x7f00400feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f004855f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00400ff290 con 0x7f0040105840 2023-06-23T18:20:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f004855f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00400ff400 con 0x7f0040100600 2023-06-23T18:20:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f004855f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004019f7c0 con 0x7f0040102f20 2023-06-23T18:20:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040105840 0x7f00400feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040105840 0x7f00400feaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52484/0 (socket says 172.21.15.2:52484) 2023-06-23T18:20:48.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 -- 172.21.15.2:0/3458102180 learned_addr learned my addr 172.21.15.2:0/3458102180 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:48.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0045afa700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0040102f20 0x7f00400fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:48.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f00462fb700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0040100600 0x7f00400fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:48.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 -- 172.21.15.2:0/3458102180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0040102f20 msgr2=0x7f00400fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:48.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0040102f20 0x7f00400fe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 -- 172.21.15.2:0/3458102180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0040100600 msgr2=0x7f00400fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:48.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0040100600 0x7f00400fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:48.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 -- 172.21.15.2:0/3458102180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00340095f0 con 0x7f0040105840 2023-06-23T18:20:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f0046afc700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040105840 0x7f00400feaf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0038007d70 tx=0x7f003800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.652+0000 7f00337fe700 1 -- 172.21.15.2:0/3458102180 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003800ad40 con 0x7f0040105840 2023-06-23T18:20:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.653+0000 7f00337fe700 1 -- 172.21.15.2:0/3458102180 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00380128b0 con 0x7f0040105840 2023-06-23T18:20:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.653+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004019faa0 con 0x7f0040105840 2023-06-23T18:20:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.653+0000 7f00337fe700 1 -- 172.21.15.2:0/3458102180 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00380115a0 con 0x7f0040105840 2023-06-23T18:20:48.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.653+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004019ffb0 con 0x7f0040105840 2023-06-23T18:20:48.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.653+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00400fcc60 con 0x7f0040105840 2023-06-23T18:20:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.655+0000 7f00337fe700 1 -- 172.21.15.2:0/3458102180 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0038012a20 con 0x7f0040105840 2023-06-23T18:20:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.655+0000 7f00337fe700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f002c0a11d0 0x7f002c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.655+0000 7f00462fb700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f002c0a11d0 0x7f002c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.655+0000 7f00337fe700 1 -- 172.21.15.2:0/3458102180 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00380ccde0 con 0x7f0040105840 2023-06-23T18:20:48.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.656+0000 7f00462fb700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f002c0a11d0 0x7f002c0a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f003400b5c0 tx=0x7f0034005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:48.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.659+0000 7f00337fe700 1 -- 172.21.15.2:0/3458102180 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0038096680 con 0x7f0040105840 2023-06-23T18:20:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-06-23T18:20:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1415875557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-06-23T18:20:48.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-06-23T18:20:48.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:48.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-06-23T18:20:48.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-06-23T18:20:48.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:48.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:48.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:48 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1968125424' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-06-23T18:20:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-06-23T18:20:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1415875557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-06-23T18:20:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-06-23T18:20:48.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-06-23T18:20:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-06-23T18:20:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:48.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:48 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1968125424' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-06-23T18:20:48.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:48.908+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f004002c810 con 0x7f0040105840 2023-06-23T18:20:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-06-23T18:20:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1415875557' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-06-23T18:20:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-06-23T18:20:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-06-23T18:20:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-06-23T18:20:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-06-23T18:20:48.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:20:48.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:20:48.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:20:48.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:48 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1968125424' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-06-23T18:20:49.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.044+0000 7f00337fe700 1 -- 172.21.15.2:0/3458102180 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f00380160b0 con 0x7f0040105840 2023-06-23T18:20:49.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f002c0a11d0 msgr2=0x7f002c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f002c0a11d0 0x7f002c0a3680 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f003400b5c0 tx=0x7f0034005fb0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040105840 msgr2=0x7f00400feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040105840 0x7f00400feaf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0038007d70 tx=0x7f003800a340 comp rx=0 tx=0).stop 2023-06-23T18:20:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 shutdown_connections 2023-06-23T18:20:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0040105840 0x7f00400feaf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f002c0a11d0 0x7f002c0a3680 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0040100600 0x7f00400fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 --2- 172.21.15.2:0/3458102180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0040102f20 0x7f00400fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 >> 172.21.15.2:0/3458102180 conn(0x7f00400fa0e0 msgr2=0x7f0040101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 shutdown_connections 2023-06-23T18:20:49.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.047+0000 7f004855f700 1 -- 172.21.15.2:0/3458102180 wait complete. 2023-06-23T18:20:49.065 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-06-23T18:20:49.065 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-06-23T18:20:49.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 -- 172.21.15.2:0/1730241010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f80fe7f0 msgr2=0x7fe5f80fec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/1730241010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f80fe7f0 0x7fe5f80fec00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009990 tx=0x7fe5e8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 -- 172.21.15.2:0/1730241010 shutdown_connections 2023-06-23T18:20:49.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/1730241010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5f8100370 0x7fe5f8106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/1730241010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f80fe7f0 0x7fe5f80fec00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/1730241010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80ff9e0 0x7fe5f80ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 -- 172.21.15.2:0/1730241010 >> 172.21.15.2:0/1730241010 conn(0x7fe5f80f9df0 msgr2=0x7fe5f80fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 -- 172.21.15.2:0/1730241010 shutdown_connections 2023-06-23T18:20:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.191+0000 7fe5ff19e700 1 -- 172.21.15.2:0/1730241010 wait complete. 2023-06-23T18:20:49.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5ff19e700 1 Processor -- start 2023-06-23T18:20:49.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5ff19e700 1 -- start start 2023-06-23T18:20:49.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5ff19e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80fe7f0 0x7fe5f8194460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:49.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5ff19e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5f80ff9e0 0x7fe5f81949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:49.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5ff19e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f8100370 0x7fe5f819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:49.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5ff19e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819bfe0 con 0x7fe5f8100370 2023-06-23T18:20:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5ff19e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819c120 con 0x7fe5f80ff9e0 2023-06-23T18:20:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5ff19e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819c2b0 con 0x7fe5f80fe7f0 2023-06-23T18:20:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.192+0000 7fe5fe99d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f8100370 0x7fe5f819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe99d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f8100370 0x7fe5f819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52498/0 (socket says 172.21.15.2:52498) 2023-06-23T18:20:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe99d700 1 -- 172.21.15.2:0/601416726 learned_addr learned my addr 172.21.15.2:0/601416726 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:49.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe19c700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80fe7f0 0x7fe5f8194460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fd99b700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5f80ff9e0 0x7fe5f81949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe99d700 1 -- 172.21.15.2:0/601416726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80fe7f0 msgr2=0x7fe5f8194460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe99d700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80fe7f0 0x7fe5f8194460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe99d700 1 -- 172.21.15.2:0/601416726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5f80ff9e0 msgr2=0x7fe5f81949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe99d700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5f80ff9e0 0x7fe5f81949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe99d700 1 -- 172.21.15.2:0/601416726 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e80095f0 con 0x7fe5f8100370 2023-06-23T18:20:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5fe99d700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f8100370 0x7fe5f819ba10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000cfb0 tx=0x7fe5f000e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:49.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5ef7fe700 1 -- 172.21.15.2:0/601416726 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0011070 con 0x7fe5f8100370 2023-06-23T18:20:49.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f8068730 con 0x7fe5f8100370 2023-06-23T18:20:49.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5ef7fe700 1 -- 172.21.15.2:0/601416726 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5f0007b30 con 0x7fe5f8100370 2023-06-23T18:20:49.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5ef7fe700 1 -- 172.21.15.2:0/601416726 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000ad90 con 0x7fe5f8100370 2023-06-23T18:20:49.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.193+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f8068bf0 con 0x7fe5f8100370 2023-06-23T18:20:49.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.194+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f804e1d0 con 0x7fe5f8100370 2023-06-23T18:20:49.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.195+0000 7fe5ef7fe700 1 -- 172.21.15.2:0/601416726 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe5f000ea40 con 0x7fe5f8100370 2023-06-23T18:20:49.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.196+0000 7fe5ef7fe700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe5e40a1150 0x7fe5e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:49.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.196+0000 7fe5fe19c700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe5e40a1150 0x7fe5e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:49.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.196+0000 7fe5ef7fe700 1 -- 172.21.15.2:0/601416726 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe5f00cea30 con 0x7fe5f8100370 2023-06-23T18:20:49.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.196+0000 7fe5fe19c700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe5e40a1150 0x7fe5e40a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c010 tx=0x7fe5e800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:49.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.199+0000 7fe5ef7fe700 1 -- 172.21.15.2:0/601416726 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe5f00982d0 con 0x7fe5f8100370 2023-06-23T18:20:49.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.434+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5f8068e10 con 0x7fe5e40a1150 2023-06-23T18:20:49.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.479+0000 7fe5ef7fe700 1 -- 172.21.15.2:0/601416726 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe5f8068e10 con 0x7fe5e40a1150 2023-06-23T18:20:49.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe5e40a1150 msgr2=0x7fe5e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe5e40a1150 0x7fe5e40a3600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800c010 tx=0x7fe5e800b540 comp rx=0 tx=0).stop 2023-06-23T18:20:49.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f8100370 msgr2=0x7fe5f819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f8100370 0x7fe5f819ba10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000cfb0 tx=0x7fe5f000e330 comp rx=0 tx=0).stop 2023-06-23T18:20:49.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 shutdown_connections 2023-06-23T18:20:49.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe5f8100370 0x7fe5f819ba10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5f80fe7f0 0x7fe5f8194460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe5e40a1150 0x7fe5e40a3600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 --2- 172.21.15.2:0/601416726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5f80ff9e0 0x7fe5f81949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 >> 172.21.15.2:0/601416726 conn(0x7fe5f80f9df0 msgr2=0x7fe5f80fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:49.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 shutdown_connections 2023-06-23T18:20:49.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.482+0000 7fe5ff19e700 1 -- 172.21.15.2:0/601416726 wait complete. 2023-06-23T18:20:49.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-06-23T18:20:49.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 -- 172.21.15.2:0/4091577153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f140fea30 msgr2=0x7f6f140fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 --2- 172.21.15.2:0/4091577153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f140fea30 0x7f6f140fee40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009990 tx=0x7f6f04009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 -- 172.21.15.2:0/4091577153 shutdown_connections 2023-06-23T18:20:49.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 --2- 172.21.15.2:0/4091577153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f14100670 0x7f6f14106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 --2- 172.21.15.2:0/4091577153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f140fea30 0x7f6f140fee40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 --2- 172.21.15.2:0/4091577153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f140ffcc0 0x7f6f14100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 -- 172.21.15.2:0/4091577153 >> 172.21.15.2:0/4091577153 conn(0x7f6f140fa030 msgr2=0x7f6f140fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:49.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 -- 172.21.15.2:0/4091577153 shutdown_connections 2023-06-23T18:20:49.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.624+0000 7f6f19499700 1 -- 172.21.15.2:0/4091577153 wait complete. 2023-06-23T18:20:49.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.625+0000 7f6f19499700 1 Processor -- start 2023-06-23T18:20:49.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.625+0000 7f6f19499700 1 -- start start 2023-06-23T18:20:49.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.625+0000 7f6f19499700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f140fea30 0x7f6f14102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:49.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.625+0000 7f6f19499700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f140ffcc0 0x7f6f141010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:49.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.625+0000 7f6f19499700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f14100670 0x7f6f141015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:49.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.625+0000 7f6f19499700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14101d80 con 0x7f6f14100670 2023-06-23T18:20:49.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.625+0000 7f6f19499700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14101ef0 con 0x7f6f140fea30 2023-06-23T18:20:49.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.625+0000 7f6f19499700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1419f7e0 con 0x7f6f140ffcc0 2023-06-23T18:20:49.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f14100670 0x7f6f141015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:49.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f14100670 0x7f6f141015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52530/0 (socket says 172.21.15.2:52530) 2023-06-23T18:20:49.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 -- 172.21.15.2:0/1528118862 learned_addr learned my addr 172.21.15.2:0/1528118862 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:49.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f12ffd700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f140fea30 0x7f6f14102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:49.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f127fc700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f140ffcc0 0x7f6f141010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:49.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 -- 172.21.15.2:0/1528118862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f140ffcc0 msgr2=0x7f6f141010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f140ffcc0 0x7f6f141010a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 -- 172.21.15.2:0/1528118862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f140fea30 msgr2=0x7f6f14102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f140fea30 0x7f6f14102a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 -- 172.21.15.2:0/1528118862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f040095f0 con 0x7f6f14100670 2023-06-23T18:20:49.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f12ffd700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f140fea30 0x7f6f14102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:49.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f137fe700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f14100670 0x7f6f141015e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c007d70 tx=0x7f6f0c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:49.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6efbfff700 1 -- 172.21.15.2:0/1528118862 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00ad40 con 0x7f6f14100670 2023-06-23T18:20:49.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1419fac0 con 0x7f6f14100670 2023-06-23T18:20:49.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6efbfff700 1 -- 172.21.15.2:0/1528118862 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6f0c0128b0 con 0x7f6f14100670 2023-06-23T18:20:49.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6efbfff700 1 -- 172.21.15.2:0/1528118862 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c0116b0 con 0x7f6f14100670 2023-06-23T18:20:49.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.626+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f141a0020 con 0x7f6f14100670 2023-06-23T18:20:49.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.627+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1410a100 con 0x7f6f14100670 2023-06-23T18:20:49.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.628+0000 7f6efbfff700 1 -- 172.21.15.2:0/1528118862 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6f0c012a20 con 0x7f6f14100670 2023-06-23T18:20:49.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.629+0000 7f6efbfff700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6efc0a1150 0x7f6efc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:49.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.629+0000 7f6f12ffd700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6efc0a1150 0x7f6efc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:49.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.629+0000 7f6efbfff700 1 -- 172.21.15.2:0/1528118862 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6f0c0ccf50 con 0x7f6f14100670 2023-06-23T18:20:49.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.629+0000 7f6f12ffd700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6efc0a1150 0x7f6efc0a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400b5c0 tx=0x7f6f04000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:49.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.632+0000 7f6efbfff700 1 -- 172.21.15.2:0/1528118862 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6f0c096750 con 0x7f6f14100670 2023-06-23T18:20:49.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:20:49.440+0000 7ff1234a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-06-23T18:20:49.801 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:20:49.440+0000 7ff1234a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-06-23T18:20:49.802 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:20:49.440+0000 7ff1234a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-06-23T18:20:49.802 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:20:49.440+0000 7ff1234a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-06-23T18:20:49.802 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:20:49.440+0000 7ff1234a6700 -1 client.0 error registering admin socket command: (17) File exists 2023-06-23T18:20:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] up:active 2023-06-23T18:20:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1968125424' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-06-23T18:20:49.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-06-23T18:20:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-06-23T18:20:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: Cluster is now healthy 2023-06-23T18:20:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-06-23T18:20:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-06-23T18:20:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-06-23T18:20:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: pgmap v258: 97 pgs: 7 creating+peering, 90 active+clean; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s wr, 10 op/s 2023-06-23T18:20:49.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3458102180' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-06-23T18:20:49.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: mds.? [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] up:active 2023-06-23T18:20:49.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3458102180' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-06-23T18:20:49.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:49 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:20:49.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] up:active 2023-06-23T18:20:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1968125424' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-06-23T18:20:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-06-23T18:20:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-06-23T18:20:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: Cluster is now healthy 2023-06-23T18:20:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-06-23T18:20:49.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-06-23T18:20:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-06-23T18:20:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: pgmap v258: 97 pgs: 7 creating+peering, 90 active+clean; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s wr, 10 op/s 2023-06-23T18:20:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3458102180' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-06-23T18:20:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: mds.? [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] up:active 2023-06-23T18:20:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3458102180' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-06-23T18:20:49.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:49 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:20:49.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.867+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f14069120 con 0x7f6efc0a1150 2023-06-23T18:20:49.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.872+0000 7f6efbfff700 1 -- 172.21.15.2:0/1528118862 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f6f14069120 con 0x7f6efc0a1150 2023-06-23T18:20:49.877 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/_nogroup/sv_0/de777b9e-6e5e-433e-950a-99a8a51ea828 2023-06-23T18:20:49.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.874+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6efc0a1150 msgr2=0x7f6efc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.874+0000 7f6f19499700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6efc0a1150 0x7f6efc0a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400b5c0 tx=0x7f6f04000bc0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.874+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f14100670 msgr2=0x7f6f141015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:49.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.874+0000 7f6f19499700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f14100670 0x7f6f141015e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c007d70 tx=0x7f6f0c00a340 comp rx=0 tx=0).stop 2023-06-23T18:20:49.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.874+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 shutdown_connections 2023-06-23T18:20:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.874+0000 7f6f19499700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f14100670 0x7f6f141015e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.875+0000 7f6f19499700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6efc0a1150 0x7f6efc0a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.875+0000 7f6f19499700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f140fea30 0x7f6f14102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.875+0000 7f6f19499700 1 --2- 172.21.15.2:0/1528118862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f140ffcc0 0x7f6f141010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.875+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 >> 172.21.15.2:0/1528118862 conn(0x7f6f140fa030 msgr2=0x7f6f14068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.875+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 shutdown_connections 2023-06-23T18:20:49.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:49.875+0000 7f6f19499700 1 -- 172.21.15.2:0/1528118862 wait complete. 2023-06-23T18:20:49.891 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-06-23T18:20:49.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] up:active 2023-06-23T18:20:49.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1968125424' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-06-23T18:20:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-06-23T18:20:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-06-23T18:20:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: Cluster is now healthy 2023-06-23T18:20:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-06-23T18:20:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-06-23T18:20:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-06-23T18:20:49.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: pgmap v258: 97 pgs: 7 creating+peering, 90 active+clean; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s wr, 10 op/s 2023-06-23T18:20:49.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3458102180' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-06-23T18:20:49.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: mds.? [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] up:active 2023-06-23T18:20:49.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3458102180' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-06-23T18:20:49.943 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:49 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:20:50.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/54711629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 msgr2=0x7f8a041011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/54711629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 0x7f8a041011e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009990 tx=0x7f89f8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/54711629 shutdown_connections 2023-06-23T18:20:50.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/54711629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a04101720 0x7f8a04107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/54711629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a040ffba0 0x7f8a040fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/54711629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 0x7f8a041011e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/54711629 >> 172.21.15.2:0/54711629 conn(0x7f8a040fb1c0 msgr2=0x7f8a040fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:50.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/54711629 shutdown_connections 2023-06-23T18:20:50.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/54711629 wait complete. 2023-06-23T18:20:50.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 Processor -- start 2023-06-23T18:20:50.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.019+0000 7f8a0aa5d700 1 -- start start 2023-06-23T18:20:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a0aa5d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a040ffba0 0x7f8a041957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a0aa5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 0x7f8a04195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a0aa5d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a04101720 0x7f8a0419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a0aa5d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0419d330 con 0x7f8a04100d90 2023-06-23T18:20:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a0aa5d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0419d470 con 0x7f8a04101720 2023-06-23T18:20:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a0aa5d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0419d600 con 0x7f8a040ffba0 2023-06-23T18:20:50.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 0x7f8a04195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 0x7f8a04195cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52540/0 (socket says 172.21.15.2:52540) 2023-06-23T18:20:50.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 -- 172.21.15.2:0/2749495296 learned_addr learned my addr 172.21.15.2:0/2749495296 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:50.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a03fff700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a040ffba0 0x7f8a041957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 -- 172.21.15.2:0/2749495296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a040ffba0 msgr2=0x7f8a041957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a08ffa700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a04101720 0x7f8a0419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a040ffba0 0x7f8a041957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 -- 172.21.15.2:0/2749495296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a04101720 msgr2=0x7f8a0419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a04101720 0x7f8a0419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 -- 172.21.15.2:0/2749495296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f80095f0 con 0x7f8a04100d90 2023-06-23T18:20:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a03fff700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a040ffba0 0x7f8a041957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a08ffa700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a04101720 0x7f8a0419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a037fe700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 0x7f8a04195cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89f8005910 tx=0x7f89f80038d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a017fa700 1 -- 172.21.15.2:0/2749495296 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f801d070 con 0x7f8a04100d90 2023-06-23T18:20:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a017fa700 1 -- 172.21.15.2:0/2749495296 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89f8022470 con 0x7f8a04100d90 2023-06-23T18:20:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0419d880 con 0x7f8a04100d90 2023-06-23T18:20:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a017fa700 1 -- 172.21.15.2:0/2749495296 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f800f6c0 con 0x7f8a04100d90 2023-06-23T18:20:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.020+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0419dd00 con 0x7f8a04100d90 2023-06-23T18:20:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.021+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0410b270 con 0x7f8a04100d90 2023-06-23T18:20:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.022+0000 7f8a017fa700 1 -- 172.21.15.2:0/2749495296 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f89f80225e0 con 0x7f8a04100d90 2023-06-23T18:20:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.023+0000 7f8a017fa700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f89ec0a11d0 0x7f89ec0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.023+0000 7f8a03fff700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f89ec0a11d0 0x7f89ec0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.023+0000 7f8a017fa700 1 -- 172.21.15.2:0/2749495296 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89f80cdd60 con 0x7f8a04100d90 2023-06-23T18:20:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.024+0000 7f8a03fff700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f89ec0a11d0 0x7f89ec0a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005fd0 tx=0x7f89f4005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.028+0000 7f8a017fa700 1 -- 172.21.15.2:0/2749495296 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f89f8097690 con 0x7f8a04100d90 2023-06-23T18:20:50.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.257+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a04060910 con 0x7f89ec0a11d0 2023-06-23T18:20:50.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.288+0000 7f8a017fa700 1 -- 172.21.15.2:0/2749495296 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8a04060910 con 0x7f89ec0a11d0 2023-06-23T18:20:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f89ec0a11d0 msgr2=0x7f89ec0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f89ec0a11d0 0x7f89ec0a3680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f89f4005fd0 tx=0x7f89f4005e40 comp rx=0 tx=0).stop 2023-06-23T18:20:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 msgr2=0x7f8a04195cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 0x7f8a04195cf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89f8005910 tx=0x7f89f80038d0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 shutdown_connections 2023-06-23T18:20:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f89ec0a11d0 0x7f89ec0a3680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a04101720 0x7f8a0419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a040ffba0 0x7f8a041957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 --2- 172.21.15.2:0/2749495296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a04100d90 0x7f8a04195cf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 >> 172.21.15.2:0/2749495296 conn(0x7f8a040fb1c0 msgr2=0x7f8a040fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:50.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 shutdown_connections 2023-06-23T18:20:50.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.290+0000 7f8a0aa5d700 1 -- 172.21.15.2:0/2749495296 wait complete. 2023-06-23T18:20:50.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-06-23T18:20:50.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 -- 172.21.15.2:0/1757615904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc105900 msgr2=0x7f26cc107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 --2- 172.21.15.2:0/1757615904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc105900 0x7f26cc107ce0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b0a0 tx=0x7f26c400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 -- 172.21.15.2:0/1757615904 shutdown_connections 2023-06-23T18:20:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 --2- 172.21.15.2:0/1757615904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc105900 0x7f26cc107ce0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 --2- 172.21.15.2:0/1757615904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26cc1006c0 0x7f26cc102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 --2- 172.21.15.2:0/1757615904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26cc102fe0 0x7f26cc1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 -- 172.21.15.2:0/1757615904 >> 172.21.15.2:0/1757615904 conn(0x7f26cc0fa0e0 msgr2=0x7f26cc0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 -- 172.21.15.2:0/1757615904 shutdown_connections 2023-06-23T18:20:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 -- 172.21.15.2:0/1757615904 wait complete. 2023-06-23T18:20:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.431+0000 7f26d2df4700 1 Processor -- start 2023-06-23T18:20:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d2df4700 1 -- start start 2023-06-23T18:20:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d2df4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc1006c0 0x7f26cc1957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d2df4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26cc102fe0 0x7f26cc195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d2df4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26cc105900 0x7f26cc19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d2df4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc19d340 con 0x7f26cc1006c0 2023-06-23T18:20:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d2df4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc19d480 con 0x7f26cc105900 2023-06-23T18:20:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d2df4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc19d610 con 0x7f26cc102fe0 2023-06-23T18:20:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc1006c0 0x7f26cc1957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc1006c0 0x7f26cc1957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52554/0 (socket says 172.21.15.2:52554) 2023-06-23T18:20:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 -- 172.21.15.2:0/3562692519 learned_addr learned my addr 172.21.15.2:0/3562692519 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26cbfff700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26cc102fe0 0x7f26cc195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d1391700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26cc105900 0x7f26cc19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 -- 172.21.15.2:0/3562692519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26cc102fe0 msgr2=0x7f26cc195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26cc102fe0 0x7f26cc195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 -- 172.21.15.2:0/3562692519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26cc105900 msgr2=0x7f26cc19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26cc105900 0x7f26cc19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 -- 172.21.15.2:0/3562692519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c4009d00 con 0x7f26cc1006c0 2023-06-23T18:20:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26d0b90700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc1006c0 0x7f26cc1957c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26c000c9b0 tx=0x7f26c000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26c9ffb700 1 -- 172.21.15.2:0/3562692519 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0007b00 con 0x7f26cc1006c0 2023-06-23T18:20:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26c9ffb700 1 -- 172.21.15.2:0/3562692519 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26c000ce50 con 0x7f26cc1006c0 2023-06-23T18:20:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.432+0000 7f26c9ffb700 1 -- 172.21.15.2:0/3562692519 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0018420 con 0x7f26cc1006c0 2023-06-23T18:20:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.433+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc19d8f0 con 0x7f26cc1006c0 2023-06-23T18:20:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.433+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc19ddb0 con 0x7f26cc1006c0 2023-06-23T18:20:50.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.434+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26cc0fcc60 con 0x7f26cc1006c0 2023-06-23T18:20:50.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.435+0000 7f26c9ffb700 1 -- 172.21.15.2:0/3562692519 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f26c000f450 con 0x7f26cc1006c0 2023-06-23T18:20:50.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.436+0000 7f26c9ffb700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26bc0a1150 0x7f26bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.436+0000 7f26cbfff700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26bc0a1150 0x7f26bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.436+0000 7f26c9ffb700 1 -- 172.21.15.2:0/3562692519 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f26c00cc9b0 con 0x7f26cc1006c0 2023-06-23T18:20:50.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.436+0000 7f26cbfff700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26bc0a1150 0x7f26bc0a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f26b8005950 tx=0x7f26b800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:50.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.439+0000 7f26c9ffb700 1 -- 172.21.15.2:0/3562692519 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f26c00962e0 con 0x7f26cc1006c0 2023-06-23T18:20:50.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.673+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f26cc060910 con 0x7f26bc0a1150 2023-06-23T18:20:50.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.677+0000 7f26c9ffb700 1 -- 172.21.15.2:0/3562692519 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f26cc060910 con 0x7f26bc0a1150 2023-06-23T18:20:50.682 INFO:teuthology.orchestra.run.smithi002.stdout:/volumes/_nogroup/sv_1/06472a1c-0cbd-4ddb-9f1c-4cb21b99a643 2023-06-23T18:20:50.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26bc0a1150 msgr2=0x7f26bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26bc0a1150 0x7f26bc0a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f26b8005950 tx=0x7f26b800b410 comp rx=0 tx=0).stop 2023-06-23T18:20:50.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc1006c0 msgr2=0x7f26cc1957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc1006c0 0x7f26cc1957c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26c000c9b0 tx=0x7f26c000ccc0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 shutdown_connections 2023-06-23T18:20:50.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26bc0a1150 0x7f26bc0a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26cc105900 0x7f26cc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26cc1006c0 0x7f26cc1957c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 --2- 172.21.15.2:0/3562692519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26cc102fe0 0x7f26cc195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 >> 172.21.15.2:0/3562692519 conn(0x7f26cc0fa0e0 msgr2=0x7f26cc101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:50.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 shutdown_connections 2023-06-23T18:20:50.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.679+0000 7f26d2df4700 1 -- 172.21.15.2:0/3562692519 wait complete. 2023-06-23T18:20:50.698 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:50 smithi002 ceph-mon[95947]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:50.699 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:20:50.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.821+0000 7ff92ac39700 1 -- 172.21.15.2:0/1158875049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 msgr2=0x7ff924100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.821+0000 7ff92ac39700 1 --2- 172.21.15.2:0/1158875049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 0x7ff924100500 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff918009990 tx=0x7ff918009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.822+0000 7ff92ac39700 1 -- 172.21.15.2:0/1158875049 shutdown_connections 2023-06-23T18:20:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.822+0000 7ff92ac39700 1 --2- 172.21.15.2:0/1158875049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240ff660 0x7ff924106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.822+0000 7ff92ac39700 1 --2- 172.21.15.2:0/1158875049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 0x7ff924100500 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.822+0000 7ff92ac39700 1 --2- 172.21.15.2:0/1158875049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9240fecb0 0x7ff9240ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.822+0000 7ff92ac39700 1 -- 172.21.15.2:0/1158875049 >> 172.21.15.2:0/1158875049 conn(0x7ff9240f9810 msgr2=0x7ff9240fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.822+0000 7ff92ac39700 1 -- 172.21.15.2:0/1158875049 shutdown_connections 2023-06-23T18:20:50.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.822+0000 7ff92ac39700 1 -- 172.21.15.2:0/1158875049 wait complete. 2023-06-23T18:20:50.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.822+0000 7ff92ac39700 1 Processor -- start 2023-06-23T18:20:50.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff92ac39700 1 -- start start 2023-06-23T18:20:50.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff92ac39700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9240fecb0 0x7ff924194710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff92ac39700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240ff660 0x7ff924194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff92ac39700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 0x7ff92419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff92ac39700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92419c290 con 0x7ff9241000f0 2023-06-23T18:20:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff92ac39700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92419c3d0 con 0x7ff9240fecb0 2023-06-23T18:20:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff92ac39700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92419c560 con 0x7ff9240ff660 2023-06-23T18:20:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 0x7ff92419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 0x7ff92419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52572/0 (socket says 172.21.15.2:52572) 2023-06-23T18:20:50.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 -- 172.21.15.2:0/59752868 learned_addr learned my addr 172.21.15.2:0/59752868 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff923fff700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240ff660 0x7ff924194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9289d5700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9240fecb0 0x7ff924194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 -- 172.21.15.2:0/59752868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240ff660 msgr2=0x7ff924194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240ff660 0x7ff924194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 -- 172.21.15.2:0/59752868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9240fecb0 msgr2=0x7ff924194710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9240fecb0 0x7ff924194710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 -- 172.21.15.2:0/59752868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180095f0 con 0x7ff9241000f0 2023-06-23T18:20:50.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.823+0000 7ff9291d6700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 0x7ff92419bcc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff91c007ee0 tx=0x7ff91c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.824+0000 7ff921ffb700 1 -- 172.21.15.2:0/59752868 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c017070 con 0x7ff9241000f0 2023-06-23T18:20:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.824+0000 7ff921ffb700 1 -- 172.21.15.2:0/59752868 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff91c00f040 con 0x7ff9241000f0 2023-06-23T18:20:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.824+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92419c840 con 0x7ff9241000f0 2023-06-23T18:20:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.824+0000 7ff921ffb700 1 -- 172.21.15.2:0/59752868 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c005570 con 0x7ff9241000f0 2023-06-23T18:20:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.824+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92419cd00 con 0x7ff9241000f0 2023-06-23T18:20:50.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.825+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92410a110 con 0x7ff9241000f0 2023-06-23T18:20:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.826+0000 7ff921ffb700 1 -- 172.21.15.2:0/59752868 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff91c00db20 con 0x7ff9241000f0 2023-06-23T18:20:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.826+0000 7ff921ffb700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9140a1150 0x7ff9140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.826+0000 7ff921ffb700 1 -- 172.21.15.2:0/59752868 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff91c0cbec0 con 0x7ff9241000f0 2023-06-23T18:20:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.826+0000 7ff9289d5700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9140a1150 0x7ff9140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:50.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.827+0000 7ff9289d5700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9140a1150 0x7ff9140a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b5c0 tx=0x7ff918005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:50.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:50.831+0000 7ff921ffb700 1 -- 172.21.15.2:0/59752868 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff91c095760 con 0x7ff9241000f0 2023-06-23T18:20:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:50 smithi082 ceph-mon[97889]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:50 smithi017 ceph-mon[97472]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.075+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff92410a2d0 con 0x7ff9241000f0 2023-06-23T18:20:51.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.076+0000 7ff921ffb700 1 -- 172.21.15.2:0/59752868 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7ff91c098d80 con 0x7ff9241000f0 2023-06-23T18:20:51.080 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:51.081 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:20:51.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9140a1150 msgr2=0x7ff9140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9140a1150 0x7ff9140a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff91800b5c0 tx=0x7ff918005fb0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 msgr2=0x7ff92419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 0x7ff92419bcc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff91c007ee0 tx=0x7ff91c00d3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 shutdown_connections 2023-06-23T18:20:51.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff9241000f0 0x7ff92419bcc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff9140a1150 0x7ff9140a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9240fecb0 0x7ff924194710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 --2- 172.21.15.2:0/59752868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240ff660 0x7ff924194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 >> 172.21.15.2:0/59752868 conn(0x7ff9240f9810 msgr2=0x7ff9240fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:51.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 shutdown_connections 2023-06-23T18:20:51.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.078+0000 7ff92ac39700 1 -- 172.21.15.2:0/59752868 wait complete. 2023-06-23T18:20:51.085 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:20:51.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:20:51.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.218+0000 7fafa6834700 1 -- 172.21.15.2:0/1179447464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00ffc50 msgr2=0x7fafa01000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.218+0000 7fafa6834700 1 --2- 172.21.15.2:0/1179447464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00ffc50 0x7fafa01000a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7faf94009990 tx=0x7faf94009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 -- 172.21.15.2:0/1179447464 shutdown_connections 2023-06-23T18:20:51.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 --2- 172.21.15.2:0/1179447464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafa01005e0 0x7fafa0106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 --2- 172.21.15.2:0/1179447464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafa00fea60 0x7fafa00fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 --2- 172.21.15.2:0/1179447464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00ffc50 0x7fafa01000a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 -- 172.21.15.2:0/1179447464 >> 172.21.15.2:0/1179447464 conn(0x7fafa00fa0e0 msgr2=0x7fafa00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:51.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 -- 172.21.15.2:0/1179447464 shutdown_connections 2023-06-23T18:20:51.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 -- 172.21.15.2:0/1179447464 wait complete. 2023-06-23T18:20:51.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 Processor -- start 2023-06-23T18:20:51.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.219+0000 7fafa6834700 1 -- start start 2023-06-23T18:20:51.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa6834700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00fea60 0x7fafa0068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa6834700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafa00ffc50 0x7fafa019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa6834700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafa01005e0 0x7fafa019cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa6834700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa019d2d0 con 0x7fafa00fea60 2023-06-23T18:20:51.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa6834700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa019d440 con 0x7fafa00ffc50 2023-06-23T18:20:51.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa6834700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa019d600 con 0x7fafa01005e0 2023-06-23T18:20:51.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00fea60 0x7fafa0068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00fea60 0x7fafa0068310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52582/0 (socket says 172.21.15.2:52582) 2023-06-23T18:20:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 -- 172.21.15.2:0/1104242371 learned_addr learned my addr 172.21.15.2:0/1104242371 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa4dd1700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafa01005e0 0x7fafa019cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9f7fe700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafa00ffc50 0x7fafa019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 -- 172.21.15.2:0/1104242371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafa01005e0 msgr2=0x7fafa019cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafa01005e0 0x7fafa019cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 -- 172.21.15.2:0/1104242371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafa00ffc50 msgr2=0x7fafa019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafa00ffc50 0x7fafa019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 -- 172.21.15.2:0/1104242371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf940095f0 con 0x7fafa00fea60 2023-06-23T18:20:51.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9f7fe700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafa00ffc50 0x7fafa019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:51.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9ffff700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00fea60 0x7fafa0068310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf9000d980 tx=0x7faf9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:51.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9d7fa700 1 -- 172.21.15.2:0/1104242371 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf90004020 con 0x7fafa00fea60 2023-06-23T18:20:51.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa019d8e0 con 0x7fafa00fea60 2023-06-23T18:20:51.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9d7fa700 1 -- 172.21.15.2:0/1104242371 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf90009730 con 0x7fafa00fea60 2023-06-23T18:20:51.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7faf9d7fa700 1 -- 172.21.15.2:0/1104242371 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf900039c0 con 0x7fafa00fea60 2023-06-23T18:20:51.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.220+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa019de40 con 0x7fafa00fea60 2023-06-23T18:20:51.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.221+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa01059e0 con 0x7fafa00fea60 2023-06-23T18:20:51.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.222+0000 7faf9d7fa700 1 -- 172.21.15.2:0/1104242371 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7faf900098a0 con 0x7fafa00fea60 2023-06-23T18:20:51.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.223+0000 7faf9d7fa700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf880a1150 0x7faf880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.223+0000 7faf9f7fe700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf880a1150 0x7faf880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.223+0000 7faf9d7fa700 1 -- 172.21.15.2:0/1104242371 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faf900cc8f0 con 0x7fafa00fea60 2023-06-23T18:20:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.224+0000 7faf9f7fe700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf880a1150 0x7faf880a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faf940059d0 tx=0x7faf94005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:51.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.227+0000 7faf9d7fa700 1 -- 172.21.15.2:0/1104242371 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7faf900960f0 con 0x7fafa00fea60 2023-06-23T18:20:51.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.428+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafa002c810 con 0x7fafa00fea60 2023-06-23T18:20:51.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.430+0000 7faf9d7fa700 1 -- 172.21.15.2:0/1104242371 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21940 (secure 0 0 0) 0x7faf90099710 con 0x7fafa00fea60 2023-06-23T18:20:51.435 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:51.435 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":82,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:45.435971+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:20:51.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf880a1150 msgr2=0x7faf880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf880a1150 0x7faf880a3600 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faf940059d0 tx=0x7faf94005960 comp rx=0 tx=0).stop 2023-06-23T18:20:51.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00fea60 msgr2=0x7fafa0068310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00fea60 0x7fafa0068310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7faf9000d980 tx=0x7faf9000dc90 comp rx=0 tx=0).stop 2023-06-23T18:20:51.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 shutdown_connections 2023-06-23T18:20:51.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fafa01005e0 0x7fafa019cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fafa00fea60 0x7fafa0068310 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faf880a1150 0x7faf880a3600 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 --2- 172.21.15.2:0/1104242371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fafa00ffc50 0x7fafa019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 >> 172.21.15.2:0/1104242371 conn(0x7fafa00fa0e0 msgr2=0x7fafa00fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:51.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 shutdown_connections 2023-06-23T18:20:51.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.432+0000 7fafa6834700 1 -- 172.21.15.2:0/1104242371 wait complete. 2023-06-23T18:20:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-06-23T18:20:51.451 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:20:51.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.577+0000 7f9aea81f700 1 -- 172.21.15.2:0/2650750517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae40fea60 msgr2=0x7f9ae40fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.577+0000 7f9aea81f700 1 --2- 172.21.15.2:0/2650750517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae40fea60 0x7f9ae40fee70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009990 tx=0x7f9ad4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.577+0000 7f9aea81f700 1 -- 172.21.15.2:0/2650750517 shutdown_connections 2023-06-23T18:20:51.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.577+0000 7f9aea81f700 1 --2- 172.21.15.2:0/2650750517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ae41005e0 0x7f9ae4106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.577+0000 7f9aea81f700 1 --2- 172.21.15.2:0/2650750517 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae40fea60 0x7f9ae40fee70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.577+0000 7f9aea81f700 1 --2- 172.21.15.2:0/2650750517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ae40ffc50 0x7f9ae41000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.577+0000 7f9aea81f700 1 -- 172.21.15.2:0/2650750517 >> 172.21.15.2:0/2650750517 conn(0x7f9ae40fa0e0 msgr2=0x7f9ae40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.578+0000 7f9aea81f700 1 -- 172.21.15.2:0/2650750517 shutdown_connections 2023-06-23T18:20:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.578+0000 7f9aea81f700 1 -- 172.21.15.2:0/2650750517 wait complete. 2023-06-23T18:20:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.578+0000 7f9aea81f700 1 Processor -- start 2023-06-23T18:20:51.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.578+0000 7f9aea81f700 1 -- start start 2023-06-23T18:20:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.578+0000 7f9aea81f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ae40fea60 0x7f9ae4068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9aea81f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ae40ffc50 0x7f9ae419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9aea81f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae41005e0 0x7f9ae419cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9aea81f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae419d2a0 con 0x7f9ae41005e0 2023-06-23T18:20:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9aea81f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae419d410 con 0x7f9ae40ffc50 2023-06-23T18:20:51.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9aea81f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae419d5d0 con 0x7f9ae40fea60 2023-06-23T18:20:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae41005e0 0x7f9ae419cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae41005e0 0x7f9ae419cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52600/0 (socket says 172.21.15.2:52600) 2023-06-23T18:20:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 -- 172.21.15.2:0/1933557087 learned_addr learned my addr 172.21.15.2:0/1933557087 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae3fff700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ae40fea60 0x7f9ae4068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae37fe700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ae40ffc50 0x7f9ae419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 -- 172.21.15.2:0/1933557087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ae40fea60 msgr2=0x7f9ae4068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ae40fea60 0x7f9ae4068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 -- 172.21.15.2:0/1933557087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ae40ffc50 msgr2=0x7f9ae419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ae40ffc50 0x7f9ae419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 -- 172.21.15.2:0/1933557087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad40095f0 con 0x7f9ae41005e0 2023-06-23T18:20:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae37fe700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ae40ffc50 0x7f9ae419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae8dbc700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae41005e0 0x7f9ae419cad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9adc007ee0 tx=0x7f9adc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae17fa700 1 -- 172.21.15.2:0/1933557087 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc017070 con 0x7f9ae41005e0 2023-06-23T18:20:51.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae17fa700 1 -- 172.21.15.2:0/1933557087 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9adc00f040 con 0x7f9ae41005e0 2023-06-23T18:20:51.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae419d8b0 con 0x7f9ae41005e0 2023-06-23T18:20:51.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9ae17fa700 1 -- 172.21.15.2:0/1933557087 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9adc005710 con 0x7f9ae41005e0 2023-06-23T18:20:51.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.579+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae419dd70 con 0x7f9ae41005e0 2023-06-23T18:20:51.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.580+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae410a0e0 con 0x7f9ae41005e0 2023-06-23T18:20:51.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.581+0000 7f9ae17fa700 1 -- 172.21.15.2:0/1933557087 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9adc005870 con 0x7f9ae41005e0 2023-06-23T18:20:51.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.582+0000 7f9ae17fa700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9acc0a11e0 0x7f9acc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.582+0000 7f9ae3fff700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9acc0a11e0 0x7f9acc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.582+0000 7f9ae17fa700 1 -- 172.21.15.2:0/1933557087 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9adc0cc850 con 0x7f9ae41005e0 2023-06-23T18:20:51.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.583+0000 7f9ae3fff700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9acc0a11e0 0x7f9acc0a3690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400b5c0 tx=0x7f9ad4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:51.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.586+0000 7f9ae17fa700 1 -- 172.21.15.2:0/1933557087 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9adc095ef0 con 0x7f9ae41005e0 2023-06-23T18:20:51.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.780+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ae4060910 con 0x7f9acc0a11e0 2023-06-23T18:20:51.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.793+0000 7f9ae17fa700 1 -- 172.21.15.2:0/1933557087 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335180 (secure 0 0 0) 0x7f9ae4060910 con 0x7f9acc0a11e0 2023-06-23T18:20:51.802 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:51 smithi002 ceph-mon[95947]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:51 smithi002 ceph-mon[95947]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:51 smithi002 ceph-mon[95947]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:51 smithi002 ceph-mon[95947]: pgmap v259: 97 pgs: 97 active+clean; 581 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2023-06-23T18:20:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:51 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/59752868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:51.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:51 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1104242371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9acc0a11e0 msgr2=0x7f9acc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9acc0a11e0 0x7f9acc0a3690 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9ad400b5c0 tx=0x7f9ad4005960 comp rx=0 tx=0).stop 2023-06-23T18:20:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae41005e0 msgr2=0x7f9ae419cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae41005e0 0x7f9ae419cad0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9adc007ee0 tx=0x7f9adc00d3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 shutdown_connections 2023-06-23T18:20:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ae41005e0 0x7f9ae419cad0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ae40fea60 0x7f9ae4068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9acc0a11e0 0x7f9acc0a3690 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 --2- 172.21.15.2:0/1933557087 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ae40ffc50 0x7f9ae419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 >> 172.21.15.2:0/1933557087 conn(0x7f9ae40fa0e0 msgr2=0x7f9ae40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:51.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 shutdown_connections 2023-06-23T18:20:51.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.797+0000 7f9aea81f700 1 -- 172.21.15.2:0/1933557087 wait complete. 2023-06-23T18:20:51.810 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-06-23T18:20:51.814 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-06-23T18:20:50.731969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594602,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":134,"num_read_kb":116,"num_write":274,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":333952,"kb_used_data":10528,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607872,"statfs":{"total":1151940427776,"available":1151598460928,"internally_reserved":0,"allocated":10780672,"data_stored":6792389,"data_compressed":107457,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4234,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":93,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.713669"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406331+0000","last_change":"2023-06-23T18:20:44.466650+0000","last_active":"2023-06-23T18:20:46.406331+0000","last_peered":"2023-06-23T18:20:46.406331+0000","last_clean":"2023-06-23T18:20:46.406331+0000","last_became_active":"2023-06-23T18:20:44.466425+0000","last_became_peered":"2023-06-23T18:20:44.466425+0000","last_unstale":"2023-06-23T18:20:46.406331+0000","last_undegraded":"2023-06-23T18:20:46.406331+0000","last_fullsized":"2023-06-23T18:20:46.406331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:46:45.756486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.466414+0000","last_change":"2023-06-23T18:20:44.465762+0000","last_active":"2023-06-23T18:20:44.466414+0000","last_peered":"2023-06-23T18:20:44.466414+0000","last_clean":"2023-06-23T18:20:44.466414+0000","last_became_active":"2023-06-23T18:20:44.463592+0000","last_became_peered":"2023-06-23T18:20:44.463592+0000","last_unstale":"2023-06-23T18:20:44.466414+0000","last_undegraded":"2023-06-23T18:20:44.466414+0000","last_fullsized":"2023-06-23T18:20:44.466414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:20:45.870897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.465699+0000","last_change":"2023-06-23T18:20:44.464162+0000","last_active":"2023-06-23T18:20:44.465699+0000","last_peered":"2023-06-23T18:20:44.465699+0000","last_clean":"2023-06-23T18:20:44.465699+0000","last_became_active":"2023-06-23T18:20:44.463854+0000","last_became_peered":"2023-06-23T18:20:44.463854+0000","last_unstale":"2023-06-23T18:20:44.465699+0000","last_undegraded":"2023-06-23T18:20:44.465699+0000","last_fullsized":"2023-06-23T18:20:44.465699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:41:24.449221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.366029+0000","last_change":"2023-06-23T18:20:44.462923+0000","last_active":"2023-06-23T18:20:46.366029+0000","last_peered":"2023-06-23T18:20:46.366029+0000","last_clean":"2023-06-23T18:20:46.366029+0000","last_became_active":"2023-06-23T18:20:44.462511+0000","last_became_peered":"2023-06-23T18:20:44.462511+0000","last_unstale":"2023-06-23T18:20:46.366029+0000","last_undegraded":"2023-06-23T18:20:46.366029+0000","last_fullsized":"2023-06-23T18:20:46.366029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:32:40.690012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530812+0000","last_change":"2023-06-23T18:20:44.454727+0000","last_active":"2023-06-23T18:20:45.530812+0000","last_peered":"2023-06-23T18:20:45.530812+0000","last_clean":"2023-06-23T18:20:45.530812+0000","last_became_active":"2023-06-23T18:20:44.454418+0000","last_became_peered":"2023-06-23T18:20:44.454418+0000","last_unstale":"2023-06-23T18:20:45.530812+0000","last_undegraded":"2023-06-23T18:20:45.530812+0000","last_fullsized":"2023-06-23T18:20:45.530812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:34:09.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905435+0000","last_change":"2023-06-23T18:20:44.460636+0000","last_active":"2023-06-23T18:20:45.905435+0000","last_peered":"2023-06-23T18:20:45.905435+0000","last_clean":"2023-06-23T18:20:45.905435+0000","last_became_active":"2023-06-23T18:20:44.460305+0000","last_became_peered":"2023-06-23T18:20:44.460305+0000","last_unstale":"2023-06-23T18:20:45.905435+0000","last_undegraded":"2023-06-23T18:20:45.905435+0000","last_fullsized":"2023-06-23T18:20:45.905435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:20:11.034702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439073+0000","last_change":"2023-06-23T18:20:44.463552+0000","last_active":"2023-06-23T18:20:45.439073+0000","last_peered":"2023-06-23T18:20:45.439073+0000","last_clean":"2023-06-23T18:20:45.439073+0000","last_became_active":"2023-06-23T18:20:44.463187+0000","last_became_peered":"2023-06-23T18:20:44.463187+0000","last_unstale":"2023-06-23T18:20:45.439073+0000","last_undegraded":"2023-06-23T18:20:45.439073+0000","last_fullsized":"2023-06-23T18:20:45.439073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:16:33.692125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439061+0000","last_change":"2023-06-23T18:20:44.457957+0000","last_active":"2023-06-23T18:20:45.439061+0000","last_peered":"2023-06-23T18:20:45.439061+0000","last_clean":"2023-06-23T18:20:45.439061+0000","last_became_active":"2023-06-23T18:20:44.457511+0000","last_became_peered":"2023-06-23T18:20:44.457511+0000","last_unstale":"2023-06-23T18:20:45.439061+0000","last_undegraded":"2023-06-23T18:20:45.439061+0000","last_fullsized":"2023-06-23T18:20:45.439061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:19:36.056785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406274+0000","last_change":"2023-06-23T18:20:44.461199+0000","last_active":"2023-06-23T18:20:46.406274+0000","last_peered":"2023-06-23T18:20:46.406274+0000","last_clean":"2023-06-23T18:20:46.406274+0000","last_became_active":"2023-06-23T18:20:44.460891+0000","last_became_peered":"2023-06-23T18:20:44.460891+0000","last_unstale":"2023-06-23T18:20:46.406274+0000","last_undegraded":"2023-06-23T18:20:46.406274+0000","last_fullsized":"2023-06-23T18:20:46.406274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:00.977889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905565+0000","last_change":"2023-06-23T18:20:44.460741+0000","last_active":"2023-06-23T18:20:45.905565+0000","last_peered":"2023-06-23T18:20:45.905565+0000","last_clean":"2023-06-23T18:20:45.905565+0000","last_became_active":"2023-06-23T18:20:44.460109+0000","last_became_peered":"2023-06-23T18:20:44.460109+0000","last_unstale":"2023-06-23T18:20:45.905565+0000","last_undegraded":"2023-06-23T18:20:45.905565+0000","last_fullsized":"2023-06-23T18:20:45.905565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:37:37.614265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439769+0000","last_change":"2023-06-23T18:20:44.453337+0000","last_active":"2023-06-23T18:20:45.439769+0000","last_peered":"2023-06-23T18:20:45.439769+0000","last_clean":"2023-06-23T18:20:45.439769+0000","last_became_active":"2023-06-23T18:20:44.452925+0000","last_became_peered":"2023-06-23T18:20:44.452925+0000","last_unstale":"2023-06-23T18:20:45.439769+0000","last_undegraded":"2023-06-23T18:20:45.439769+0000","last_fullsized":"2023-06-23T18:20:45.439769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:10.381911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.365990+0000","last_change":"2023-06-23T18:20:44.456710+0000","last_active":"2023-06-23T18:20:46.365990+0000","last_peered":"2023-06-23T18:20:46.365990+0000","last_clean":"2023-06-23T18:20:46.365990+0000","last_became_active":"2023-06-23T18:20:44.456411+0000","last_became_peered":"2023-06-23T18:20:44.456411+0000","last_unstale":"2023-06-23T18:20:46.365990+0000","last_undegraded":"2023-06-23T18:20:46.365990+0000","last_fullsized":"2023-06-23T18:20:46.365990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:01:45.669051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.455205+0000","last_change":"2023-06-23T18:20:44.453586+0000","last_active":"2023-06-23T18:20:44.455205+0000","last_peered":"2023-06-23T18:20:44.455205+0000","last_clean":"2023-06-23T18:20:44.455205+0000","last_became_active":"2023-06-23T18:20:44.453239+0000","last_became_peered":"2023-06-23T18:20:44.453239+0000","last_unstale":"2023-06-23T18:20:44.455205+0000","last_undegraded":"2023-06-23T18:20:44.455205+0000","last_fullsized":"2023-06-23T18:20:44.455205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:50:15.838769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.465959+0000","last_change":"2023-06-23T18:20:44.465627+0000","last_active":"2023-06-23T18:20:44.465959+0000","last_peered":"2023-06-23T18:20:44.465959+0000","last_clean":"2023-06-23T18:20:44.465959+0000","last_became_active":"2023-06-23T18:20:44.464568+0000","last_became_peered":"2023-06-23T18:20:44.464568+0000","last_unstale":"2023-06-23T18:20:44.465959+0000","last_undegraded":"2023-06-23T18:20:44.465959+0000","last_fullsized":"2023-06-23T18:20:44.465959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:26:31.336582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.465673+0000","last_change":"2023-06-23T18:20:44.464169+0000","last_active":"2023-06-23T18:20:44.465673+0000","last_peered":"2023-06-23T18:20:44.465673+0000","last_clean":"2023-06-23T18:20:44.465673+0000","last_became_active":"2023-06-23T18:20:44.463867+0000","last_became_peered":"2023-06-23T18:20:44.463867+0000","last_unstale":"2023-06-23T18:20:44.465673+0000","last_undegraded":"2023-06-23T18:20:44.465673+0000","last_fullsized":"2023-06-23T18:20:44.465673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:09.249987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405761+0000","last_change":"2023-06-23T18:20:44.467101+0000","last_active":"2023-06-23T18:20:46.405761+0000","last_peered":"2023-06-23T18:20:46.405761+0000","last_clean":"2023-06-23T18:20:46.405761+0000","last_became_active":"2023-06-23T18:20:44.466788+0000","last_became_peered":"2023-06-23T18:20:44.466788+0000","last_unstale":"2023-06-23T18:20:46.405761+0000","last_undegraded":"2023-06-23T18:20:46.405761+0000","last_fullsized":"2023-06-23T18:20:46.405761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:14:55.735170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406210+0000","last_change":"2023-06-23T18:20:44.466055+0000","last_active":"2023-06-23T18:20:46.406210+0000","last_peered":"2023-06-23T18:20:46.406210+0000","last_clean":"2023-06-23T18:20:46.406210+0000","last_became_active":"2023-06-23T18:20:44.465608+0000","last_became_peered":"2023-06-23T18:20:44.465608+0000","last_unstale":"2023-06-23T18:20:46.406210+0000","last_undegraded":"2023-06-23T18:20:46.406210+0000","last_fullsized":"2023-06-23T18:20:46.406210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:06:22.567595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.529187+0000","last_change":"2023-06-23T18:20:44.459414+0000","last_active":"2023-06-23T18:20:45.529187+0000","last_peered":"2023-06-23T18:20:45.529187+0000","last_clean":"2023-06-23T18:20:45.529187+0000","last_became_active":"2023-06-23T18:20:44.459014+0000","last_became_peered":"2023-06-23T18:20:44.459014+0000","last_unstale":"2023-06-23T18:20:45.529187+0000","last_undegraded":"2023-06-23T18:20:45.529187+0000","last_fullsized":"2023-06-23T18:20:45.529187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:00:16.944957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237780+0000","last_change":"2023-06-23T18:20:44.446784+0000","last_active":"2023-06-23T18:20:46.237780+0000","last_peered":"2023-06-23T18:20:46.237780+0000","last_clean":"2023-06-23T18:20:46.237780+0000","last_became_active":"2023-06-23T18:20:44.446464+0000","last_became_peered":"2023-06-23T18:20:44.446464+0000","last_unstale":"2023-06-23T18:20:46.237780+0000","last_undegraded":"2023-06-23T18:20:46.237780+0000","last_fullsized":"2023-06-23T18:20:46.237780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:03:13.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.501338+0000","last_change":"2023-06-23T18:20:44.039326+0000","last_active":"2023-06-23T18:20:48.501338+0000","last_peered":"2023-06-23T18:20:48.501338+0000","last_clean":"2023-06-23T18:20:48.501338+0000","last_became_active":"2023-06-23T18:20:44.038787+0000","last_became_peered":"2023-06-23T18:20:44.038787+0000","last_unstale":"2023-06-23T18:20:48.501338+0000","last_undegraded":"2023-06-23T18:20:48.501338+0000","last_fullsized":"2023-06-23T18:20:48.501338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:16:15.016712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439642+0000","last_change":"2023-06-23T18:20:44.465269+0000","last_active":"2023-06-23T18:20:45.439642+0000","last_peered":"2023-06-23T18:20:45.439642+0000","last_clean":"2023-06-23T18:20:45.439642+0000","last_became_active":"2023-06-23T18:20:44.464973+0000","last_became_peered":"2023-06-23T18:20:44.464973+0000","last_unstale":"2023-06-23T18:20:45.439642+0000","last_undegraded":"2023-06-23T18:20:45.439642+0000","last_fullsized":"2023-06-23T18:20:45.439642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:24:55.096958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439687+0000","last_change":"2023-06-23T18:20:44.037175+0000","last_active":"2023-06-23T18:20:45.439687+0000","last_peered":"2023-06-23T18:20:45.439687+0000","last_clean":"2023-06-23T18:20:45.439687+0000","last_became_active":"2023-06-23T18:20:44.036788+0000","last_became_peered":"2023-06-23T18:20:44.036788+0000","last_unstale":"2023-06-23T18:20:45.439687+0000","last_undegraded":"2023-06-23T18:20:45.439687+0000","last_fullsized":"2023-06-23T18:20:45.439687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:15:06.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.464521+0000","last_change":"2023-06-23T18:20:44.463344+0000","last_active":"2023-06-23T18:20:44.464521+0000","last_peered":"2023-06-23T18:20:44.464521+0000","last_clean":"2023-06-23T18:20:44.464521+0000","last_became_active":"2023-06-23T18:20:44.462649+0000","last_became_peered":"2023-06-23T18:20:44.462649+0000","last_unstale":"2023-06-23T18:20:44.464521+0000","last_undegraded":"2023-06-23T18:20:44.464521+0000","last_fullsized":"2023-06-23T18:20:44.464521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:57:45.776231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499503+0000","last_change":"2023-06-23T18:20:44.045525+0000","last_active":"2023-06-23T18:20:48.499503+0000","last_peered":"2023-06-23T18:20:48.499503+0000","last_clean":"2023-06-23T18:20:48.499503+0000","last_became_active":"2023-06-23T18:20:44.045212+0000","last_became_peered":"2023-06-23T18:20:44.045212+0000","last_unstale":"2023-06-23T18:20:48.499503+0000","last_undegraded":"2023-06-23T18:20:48.499503+0000","last_fullsized":"2023-06-23T18:20:48.499503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:44:56.572688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405870+0000","last_change":"2023-06-23T18:20:44.461071+0000","last_active":"2023-06-23T18:20:46.405870+0000","last_peered":"2023-06-23T18:20:46.405870+0000","last_clean":"2023-06-23T18:20:46.405870+0000","last_became_active":"2023-06-23T18:20:44.460726+0000","last_became_peered":"2023-06-23T18:20:44.460726+0000","last_unstale":"2023-06-23T18:20:46.405870+0000","last_undegraded":"2023-06-23T18:20:46.405870+0000","last_fullsized":"2023-06-23T18:20:46.405870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:08:47.572741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530990+0000","last_change":"2023-06-23T18:20:44.037162+0000","last_active":"2023-06-23T18:20:45.530990+0000","last_peered":"2023-06-23T18:20:45.530990+0000","last_clean":"2023-06-23T18:20:45.530990+0000","last_became_active":"2023-06-23T18:20:44.036809+0000","last_became_peered":"2023-06-23T18:20:44.036809+0000","last_unstale":"2023-06-23T18:20:45.530990+0000","last_undegraded":"2023-06-23T18:20:45.530990+0000","last_fullsized":"2023-06-23T18:20:45.530990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:42:11.269748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439457+0000","last_change":"2023-06-23T18:20:44.463525+0000","last_active":"2023-06-23T18:20:45.439457+0000","last_peered":"2023-06-23T18:20:45.439457+0000","last_clean":"2023-06-23T18:20:45.439457+0000","last_became_active":"2023-06-23T18:20:44.463245+0000","last_became_peered":"2023-06-23T18:20:44.463245+0000","last_unstale":"2023-06-23T18:20:45.439457+0000","last_undegraded":"2023-06-23T18:20:45.439457+0000","last_fullsized":"2023-06-23T18:20:45.439457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:18:47.776141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:49.050736+0000","last_change":"2023-06-23T18:20:44.044415+0000","last_active":"2023-06-23T18:20:49.050736+0000","last_peered":"2023-06-23T18:20:49.050736+0000","last_clean":"2023-06-23T18:20:49.050736+0000","last_became_active":"2023-06-23T18:20:44.044085+0000","last_became_peered":"2023-06-23T18:20:44.044085+0000","last_unstale":"2023-06-23T18:20:49.050736+0000","last_undegraded":"2023-06-23T18:20:49.050736+0000","last_fullsized":"2023-06-23T18:20:49.050736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:07:29.948276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530911+0000","last_change":"2023-06-23T18:20:44.463422+0000","last_active":"2023-06-23T18:20:45.530911+0000","last_peered":"2023-06-23T18:20:45.530911+0000","last_clean":"2023-06-23T18:20:45.530911+0000","last_became_active":"2023-06-23T18:20:44.463121+0000","last_became_peered":"2023-06-23T18:20:44.463121+0000","last_unstale":"2023-06-23T18:20:45.530911+0000","last_undegraded":"2023-06-23T18:20:45.530911+0000","last_fullsized":"2023-06-23T18:20:45.530911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:54:44.620991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905236+0000","last_change":"2023-06-23T18:20:44.039604+0000","last_active":"2023-06-23T18:20:45.905236+0000","last_peered":"2023-06-23T18:20:45.905236+0000","last_clean":"2023-06-23T18:20:45.905236+0000","last_became_active":"2023-06-23T18:20:44.039317+0000","last_became_peered":"2023-06-23T18:20:44.039317+0000","last_unstale":"2023-06-23T18:20:45.905236+0000","last_undegraded":"2023-06-23T18:20:45.905236+0000","last_fullsized":"2023-06-23T18:20:45.905236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:39:40.353762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439176+0000","last_change":"2023-06-23T18:20:44.465748+0000","last_active":"2023-06-23T18:20:45.439176+0000","last_peered":"2023-06-23T18:20:45.439176+0000","last_clean":"2023-06-23T18:20:45.439176+0000","last_became_active":"2023-06-23T18:20:44.465430+0000","last_became_peered":"2023-06-23T18:20:44.465430+0000","last_unstale":"2023-06-23T18:20:45.439176+0000","last_undegraded":"2023-06-23T18:20:45.439176+0000","last_fullsized":"2023-06-23T18:20:45.439176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:31.824195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.438495+0000","last_change":"2023-06-23T18:20:44.034388+0000","last_active":"2023-06-23T18:20:44.438495+0000","last_peered":"2023-06-23T18:20:44.438495+0000","last_clean":"2023-06-23T18:20:44.438495+0000","last_became_active":"2023-06-23T18:20:44.034067+0000","last_became_peered":"2023-06-23T18:20:44.034067+0000","last_unstale":"2023-06-23T18:20:44.438495+0000","last_undegraded":"2023-06-23T18:20:44.438495+0000","last_fullsized":"2023-06-23T18:20:44.438495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:55:24.190397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.438998+0000","last_change":"2023-06-23T18:20:44.453280+0000","last_active":"2023-06-23T18:20:45.438998+0000","last_peered":"2023-06-23T18:20:45.438998+0000","last_clean":"2023-06-23T18:20:45.438998+0000","last_became_active":"2023-06-23T18:20:44.452980+0000","last_became_peered":"2023-06-23T18:20:44.452980+0000","last_unstale":"2023-06-23T18:20:45.438998+0000","last_undegraded":"2023-06-23T18:20:45.438998+0000","last_fullsized":"2023-06-23T18:20:45.438998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:24:40.346057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439241+0000","last_change":"2023-06-23T18:20:44.036165+0000","last_active":"2023-06-23T18:20:45.439241+0000","last_peered":"2023-06-23T18:20:45.439241+0000","last_clean":"2023-06-23T18:20:45.439241+0000","last_became_active":"2023-06-23T18:20:44.035313+0000","last_became_peered":"2023-06-23T18:20:44.035313+0000","last_unstale":"2023-06-23T18:20:45.439241+0000","last_undegraded":"2023-06-23T18:20:45.439241+0000","last_fullsized":"2023-06-23T18:20:45.439241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:26:02.622257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.528993+0000","last_change":"2023-06-23T18:20:44.468358+0000","last_active":"2023-06-23T18:20:45.528993+0000","last_peered":"2023-06-23T18:20:45.528993+0000","last_clean":"2023-06-23T18:20:45.528993+0000","last_became_active":"2023-06-23T18:20:44.468032+0000","last_became_peered":"2023-06-23T18:20:44.468032+0000","last_unstale":"2023-06-23T18:20:45.528993+0000","last_undegraded":"2023-06-23T18:20:45.528993+0000","last_fullsized":"2023-06-23T18:20:45.528993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:31.361443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905449+0000","last_change":"2023-06-23T18:20:44.037921+0000","last_active":"2023-06-23T18:20:45.905449+0000","last_peered":"2023-06-23T18:20:45.905449+0000","last_clean":"2023-06-23T18:20:45.905449+0000","last_became_active":"2023-06-23T18:20:44.037571+0000","last_became_peered":"2023-06-23T18:20:44.037571+0000","last_unstale":"2023-06-23T18:20:45.905449+0000","last_undegraded":"2023-06-23T18:20:45.905449+0000","last_fullsized":"2023-06-23T18:20:45.905449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:55.796755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406218+0000","last_change":"2023-06-23T18:20:44.460897+0000","last_active":"2023-06-23T18:20:46.406218+0000","last_peered":"2023-06-23T18:20:46.406218+0000","last_clean":"2023-06-23T18:20:46.406218+0000","last_became_active":"2023-06-23T18:20:44.460476+0000","last_became_peered":"2023-06-23T18:20:44.460476+0000","last_unstale":"2023-06-23T18:20:46.406218+0000","last_undegraded":"2023-06-23T18:20:46.406218+0000","last_fullsized":"2023-06-23T18:20:46.406218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:43.488229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439412+0000","last_change":"2023-06-23T18:20:44.033013+0000","last_active":"2023-06-23T18:20:45.439412+0000","last_peered":"2023-06-23T18:20:45.439412+0000","last_clean":"2023-06-23T18:20:45.439412+0000","last_became_active":"2023-06-23T18:20:44.032580+0000","last_became_peered":"2023-06-23T18:20:44.032580+0000","last_unstale":"2023-06-23T18:20:45.439412+0000","last_undegraded":"2023-06-23T18:20:45.439412+0000","last_fullsized":"2023-06-23T18:20:45.439412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:32:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.464625+0000","last_change":"2023-06-23T18:20:44.463585+0000","last_active":"2023-06-23T18:20:44.464625+0000","last_peered":"2023-06-23T18:20:44.464625+0000","last_clean":"2023-06-23T18:20:44.464625+0000","last_became_active":"2023-06-23T18:20:44.462512+0000","last_became_peered":"2023-06-23T18:20:44.462512+0000","last_unstale":"2023-06-23T18:20:44.464625+0000","last_undegraded":"2023-06-23T18:20:44.464625+0000","last_fullsized":"2023-06-23T18:20:44.464625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:14:20.179173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497460+0000","last_change":"2023-06-23T18:20:44.037134+0000","last_active":"2023-06-23T18:20:47.497460+0000","last_peered":"2023-06-23T18:20:47.497460+0000","last_clean":"2023-06-23T18:20:47.497460+0000","last_became_active":"2023-06-23T18:20:44.036536+0000","last_became_peered":"2023-06-23T18:20:44.036536+0000","last_unstale":"2023-06-23T18:20:47.497460+0000","last_undegraded":"2023-06-23T18:20:47.497460+0000","last_fullsized":"2023-06-23T18:20:47.497460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:58:25.025236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406029+0000","last_change":"2023-06-23T18:20:44.454453+0000","last_active":"2023-06-23T18:20:46.406029+0000","last_peered":"2023-06-23T18:20:46.406029+0000","last_clean":"2023-06-23T18:20:46.406029+0000","last_became_active":"2023-06-23T18:20:44.454133+0000","last_became_peered":"2023-06-23T18:20:44.454133+0000","last_unstale":"2023-06-23T18:20:46.406029+0000","last_undegraded":"2023-06-23T18:20:46.406029+0000","last_fullsized":"2023-06-23T18:20:46.406029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:52:02.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905447+0000","last_change":"2023-06-23T18:20:44.042622+0000","last_active":"2023-06-23T18:20:45.905447+0000","last_peered":"2023-06-23T18:20:45.905447+0000","last_clean":"2023-06-23T18:20:45.905447+0000","last_became_active":"2023-06-23T18:20:44.042330+0000","last_became_peered":"2023-06-23T18:20:44.042330+0000","last_unstale":"2023-06-23T18:20:45.905447+0000","last_undegraded":"2023-06-23T18:20:45.905447+0000","last_fullsized":"2023-06-23T18:20:45.905447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:23.320076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.531053+0000","last_change":"2023-06-23T18:20:44.456558+0000","last_active":"2023-06-23T18:20:45.531053+0000","last_peered":"2023-06-23T18:20:45.531053+0000","last_clean":"2023-06-23T18:20:45.531053+0000","last_became_active":"2023-06-23T18:20:44.456199+0000","last_became_peered":"2023-06-23T18:20:44.456199+0000","last_unstale":"2023-06-23T18:20:45.531053+0000","last_undegraded":"2023-06-23T18:20:45.531053+0000","last_fullsized":"2023-06-23T18:20:45.531053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:24:13.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499315+0000","last_change":"2023-06-23T18:20:44.034101+0000","last_active":"2023-06-23T18:20:48.499315+0000","last_peered":"2023-06-23T18:20:48.499315+0000","last_clean":"2023-06-23T18:20:48.499315+0000","last_became_active":"2023-06-23T18:20:44.033287+0000","last_became_peered":"2023-06-23T18:20:44.033287+0000","last_unstale":"2023-06-23T18:20:48.499315+0000","last_undegraded":"2023-06-23T18:20:48.499315+0000","last_fullsized":"2023-06-23T18:20:48.499315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:08:52.512924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.448242+0000","last_change":"2023-06-23T18:20:44.446298+0000","last_active":"2023-06-23T18:20:44.448242+0000","last_peered":"2023-06-23T18:20:44.448242+0000","last_clean":"2023-06-23T18:20:44.448242+0000","last_became_active":"2023-06-23T18:20:44.445922+0000","last_became_peered":"2023-06-23T18:20:44.445922+0000","last_unstale":"2023-06-23T18:20:44.448242+0000","last_undegraded":"2023-06-23T18:20:44.448242+0000","last_fullsized":"2023-06-23T18:20:44.448242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:25:35.069099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498820+0000","last_change":"2023-06-23T18:20:44.035791+0000","last_active":"2023-06-23T18:20:47.498820+0000","last_peered":"2023-06-23T18:20:47.498820+0000","last_clean":"2023-06-23T18:20:47.498820+0000","last_became_active":"2023-06-23T18:20:44.035416+0000","last_became_peered":"2023-06-23T18:20:44.035416+0000","last_unstale":"2023-06-23T18:20:47.498820+0000","last_undegraded":"2023-06-23T18:20:47.498820+0000","last_fullsized":"2023-06-23T18:20:47.498820+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:02:31.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439429+0000","last_change":"2023-06-23T18:20:44.455474+0000","last_active":"2023-06-23T18:20:45.439429+0000","last_peered":"2023-06-23T18:20:45.439429+0000","last_clean":"2023-06-23T18:20:45.439429+0000","last_became_active":"2023-06-23T18:20:44.455098+0000","last_became_peered":"2023-06-23T18:20:44.455098+0000","last_unstale":"2023-06-23T18:20:45.439429+0000","last_undegraded":"2023-06-23T18:20:45.439429+0000","last_fullsized":"2023-06-23T18:20:45.439429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:53:34.377017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.365945+0000","last_change":"2023-06-23T18:20:44.030305+0000","last_active":"2023-06-23T18:20:46.365945+0000","last_peered":"2023-06-23T18:20:46.365945+0000","last_clean":"2023-06-23T18:20:46.365945+0000","last_became_active":"2023-06-23T18:20:44.029898+0000","last_became_peered":"2023-06-23T18:20:44.029898+0000","last_unstale":"2023-06-23T18:20:46.365945+0000","last_undegraded":"2023-06-23T18:20:46.365945+0000","last_fullsized":"2023-06-23T18:20:46.365945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:38:03.187411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439068+0000","last_change":"2023-06-23T18:20:44.451321+0000","last_active":"2023-06-23T18:20:45.439068+0000","last_peered":"2023-06-23T18:20:45.439068+0000","last_clean":"2023-06-23T18:20:45.439068+0000","last_became_active":"2023-06-23T18:20:44.450549+0000","last_became_peered":"2023-06-23T18:20:44.450549+0000","last_unstale":"2023-06-23T18:20:45.439068+0000","last_undegraded":"2023-06-23T18:20:45.439068+0000","last_fullsized":"2023-06-23T18:20:45.439068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:44:56.410567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.503174+0000","last_change":"2023-06-23T18:20:44.044707+0000","last_active":"2023-06-23T18:20:48.503174+0000","last_peered":"2023-06-23T18:20:48.503174+0000","last_clean":"2023-06-23T18:20:48.503174+0000","last_became_active":"2023-06-23T18:20:44.044389+0000","last_became_peered":"2023-06-23T18:20:44.044389+0000","last_unstale":"2023-06-23T18:20:48.503174+0000","last_undegraded":"2023-06-23T18:20:48.503174+0000","last_fullsized":"2023-06-23T18:20:48.503174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:45:51.078124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439091+0000","last_change":"2023-06-23T18:20:44.463637+0000","last_active":"2023-06-23T18:20:45.439091+0000","last_peered":"2023-06-23T18:20:45.439091+0000","last_clean":"2023-06-23T18:20:45.439091+0000","last_became_active":"2023-06-23T18:20:44.463289+0000","last_became_peered":"2023-06-23T18:20:44.463289+0000","last_unstale":"2023-06-23T18:20:45.439091+0000","last_undegraded":"2023-06-23T18:20:45.439091+0000","last_fullsized":"2023-06-23T18:20:45.439091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:24:04.923988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.531116+0000","last_change":"2023-06-23T18:20:44.038961+0000","last_active":"2023-06-23T18:20:45.531116+0000","last_peered":"2023-06-23T18:20:45.531116+0000","last_clean":"2023-06-23T18:20:45.531116+0000","last_became_active":"2023-06-23T18:20:44.038674+0000","last_became_peered":"2023-06-23T18:20:44.038674+0000","last_unstale":"2023-06-23T18:20:45.531116+0000","last_undegraded":"2023-06-23T18:20:45.531116+0000","last_fullsized":"2023-06-23T18:20:45.531116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:18:04.288041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406228+0000","last_change":"2023-06-23T18:20:44.461077+0000","last_active":"2023-06-23T18:20:46.406228+0000","last_peered":"2023-06-23T18:20:46.406228+0000","last_clean":"2023-06-23T18:20:46.406228+0000","last_became_active":"2023-06-23T18:20:44.460767+0000","last_became_peered":"2023-06-23T18:20:44.460767+0000","last_unstale":"2023-06-23T18:20:46.406228+0000","last_undegraded":"2023-06-23T18:20:46.406228+0000","last_fullsized":"2023-06-23T18:20:46.406228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:16.534724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.529039+0000","last_change":"2023-06-23T18:20:44.042114+0000","last_active":"2023-06-23T18:20:45.529039+0000","last_peered":"2023-06-23T18:20:45.529039+0000","last_clean":"2023-06-23T18:20:45.529039+0000","last_became_active":"2023-06-23T18:20:44.041822+0000","last_became_peered":"2023-06-23T18:20:44.041822+0000","last_unstale":"2023-06-23T18:20:45.529039+0000","last_undegraded":"2023-06-23T18:20:45.529039+0000","last_fullsized":"2023-06-23T18:20:45.529039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:36:53.587200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905482+0000","last_change":"2023-06-23T18:20:44.452903+0000","last_active":"2023-06-23T18:20:45.905482+0000","last_peered":"2023-06-23T18:20:45.905482+0000","last_clean":"2023-06-23T18:20:45.905482+0000","last_became_active":"2023-06-23T18:20:44.452539+0000","last_became_peered":"2023-06-23T18:20:44.452539+0000","last_unstale":"2023-06-23T18:20:45.905482+0000","last_undegraded":"2023-06-23T18:20:45.905482+0000","last_fullsized":"2023-06-23T18:20:45.905482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:53:53.970572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439260+0000","last_change":"2023-06-23T18:20:44.032905+0000","last_active":"2023-06-23T18:20:45.439260+0000","last_peered":"2023-06-23T18:20:45.439260+0000","last_clean":"2023-06-23T18:20:45.439260+0000","last_became_active":"2023-06-23T18:20:44.032606+0000","last_became_peered":"2023-06-23T18:20:44.032606+0000","last_unstale":"2023-06-23T18:20:45.439260+0000","last_undegraded":"2023-06-23T18:20:45.439260+0000","last_fullsized":"2023-06-23T18:20:45.439260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:05.992912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905273+0000","last_change":"2023-06-23T18:20:44.465580+0000","last_active":"2023-06-23T18:20:45.905273+0000","last_peered":"2023-06-23T18:20:45.905273+0000","last_clean":"2023-06-23T18:20:45.905273+0000","last_became_active":"2023-06-23T18:20:44.465264+0000","last_became_peered":"2023-06-23T18:20:44.465264+0000","last_unstale":"2023-06-23T18:20:45.905273+0000","last_undegraded":"2023-06-23T18:20:45.905273+0000","last_fullsized":"2023-06-23T18:20:45.905273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:20:07.333937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439575+0000","last_change":"2023-06-23T18:20:44.464324+0000","last_active":"2023-06-23T18:20:45.439575+0000","last_peered":"2023-06-23T18:20:45.439575+0000","last_clean":"2023-06-23T18:20:45.439575+0000","last_became_active":"2023-06-23T18:20:44.464021+0000","last_became_peered":"2023-06-23T18:20:44.464021+0000","last_unstale":"2023-06-23T18:20:45.439575+0000","last_undegraded":"2023-06-23T18:20:45.439575+0000","last_fullsized":"2023-06-23T18:20:45.439575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:24:15.831014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.437868+0000","last_change":"2023-06-23T18:20:44.034309+0000","last_active":"2023-06-23T18:20:44.437868+0000","last_peered":"2023-06-23T18:20:44.437868+0000","last_clean":"2023-06-23T18:20:44.437868+0000","last_became_active":"2023-06-23T18:20:44.034007+0000","last_became_peered":"2023-06-23T18:20:44.034007+0000","last_unstale":"2023-06-23T18:20:44.437868+0000","last_undegraded":"2023-06-23T18:20:44.437868+0000","last_fullsized":"2023-06-23T18:20:44.437868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:00:07.874785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439359+0000","last_change":"2023-06-23T18:20:44.468220+0000","last_active":"2023-06-23T18:20:45.439359+0000","last_peered":"2023-06-23T18:20:45.439359+0000","last_clean":"2023-06-23T18:20:45.439359+0000","last_became_active":"2023-06-23T18:20:44.467725+0000","last_became_peered":"2023-06-23T18:20:44.467725+0000","last_unstale":"2023-06-23T18:20:45.439359+0000","last_undegraded":"2023-06-23T18:20:45.439359+0000","last_fullsized":"2023-06-23T18:20:45.439359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:30.069575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498601+0000","last_change":"2023-06-23T18:20:44.030921+0000","last_active":"2023-06-23T18:20:47.498601+0000","last_peered":"2023-06-23T18:20:47.498601+0000","last_clean":"2023-06-23T18:20:47.498601+0000","last_became_active":"2023-06-23T18:20:44.030362+0000","last_became_peered":"2023-06-23T18:20:44.030362+0000","last_unstale":"2023-06-23T18:20:47.498601+0000","last_undegraded":"2023-06-23T18:20:47.498601+0000","last_fullsized":"2023-06-23T18:20:47.498601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:07:10.000171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":328,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.438999+0000","last_change":"2023-06-23T18:19:44.952262+0000","last_active":"2023-06-23T18:20:45.438999+0000","last_peered":"2023-06-23T18:20:45.438999+0000","last_clean":"2023-06-23T18:20:45.438999+0000","last_became_active":"2023-06-23T18:19:44.951803+0000","last_became_peered":"2023-06-23T18:19:44.951803+0000","last_unstale":"2023-06-23T18:20:45.438999+0000","last_undegraded":"2023-06-23T18:20:45.438999+0000","last_fullsized":"2023-06-23T18:20:45.438999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_clean_scrub_stamp":"2023-06-23T18:16:49.458922+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:49.981569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.457951+0000","last_change":"2023-06-23T18:20:44.456602+0000","last_active":"2023-06-23T18:20:44.457951+0000","last_peered":"2023-06-23T18:20:44.457951+0000","last_clean":"2023-06-23T18:20:44.457951+0000","last_became_active":"2023-06-23T18:20:44.456249+0000","last_became_peered":"2023-06-23T18:20:44.456249+0000","last_unstale":"2023-06-23T18:20:44.457951+0000","last_undegraded":"2023-06-23T18:20:44.457951+0000","last_fullsized":"2023-06-23T18:20:44.457951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:29:59.524147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439445+0000","last_change":"2023-06-23T18:20:44.039100+0000","last_active":"2023-06-23T18:20:45.439445+0000","last_peered":"2023-06-23T18:20:45.439445+0000","last_clean":"2023-06-23T18:20:45.439445+0000","last_became_active":"2023-06-23T18:20:44.038801+0000","last_became_peered":"2023-06-23T18:20:44.038801+0000","last_unstale":"2023-06-23T18:20:45.439445+0000","last_undegraded":"2023-06-23T18:20:45.439445+0000","last_fullsized":"2023-06-23T18:20:45.439445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:03.284504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.531051+0000","last_change":"2023-06-23T18:20:44.038037+0000","last_active":"2023-06-23T18:20:45.531051+0000","last_peered":"2023-06-23T18:20:45.531051+0000","last_clean":"2023-06-23T18:20:45.531051+0000","last_became_active":"2023-06-23T18:20:44.037732+0000","last_became_peered":"2023-06-23T18:20:44.037732+0000","last_unstale":"2023-06-23T18:20:45.531051+0000","last_undegraded":"2023-06-23T18:20:45.531051+0000","last_fullsized":"2023-06-23T18:20:45.531051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:30:58.279148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.460237+0000","last_change":"2023-06-23T18:20:44.459927+0000","last_active":"2023-06-23T18:20:44.460237+0000","last_peered":"2023-06-23T18:20:44.460237+0000","last_clean":"2023-06-23T18:20:44.460237+0000","last_became_active":"2023-06-23T18:20:44.459594+0000","last_became_peered":"2023-06-23T18:20:44.459594+0000","last_unstale":"2023-06-23T18:20:44.460237+0000","last_undegraded":"2023-06-23T18:20:44.460237+0000","last_fullsized":"2023-06-23T18:20:44.460237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:27:30.055435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499658+0000","last_change":"2023-06-23T18:20:44.039429+0000","last_active":"2023-06-23T18:20:48.499658+0000","last_peered":"2023-06-23T18:20:48.499658+0000","last_clean":"2023-06-23T18:20:48.499658+0000","last_became_active":"2023-06-23T18:20:44.038944+0000","last_became_peered":"2023-06-23T18:20:44.038944+0000","last_unstale":"2023-06-23T18:20:48.499658+0000","last_undegraded":"2023-06-23T18:20:48.499658+0000","last_fullsized":"2023-06-23T18:20:48.499658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:37:39.247393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439805+0000","last_change":"2023-06-23T18:20:44.458227+0000","last_active":"2023-06-23T18:20:45.439805+0000","last_peered":"2023-06-23T18:20:45.439805+0000","last_clean":"2023-06-23T18:20:45.439805+0000","last_became_active":"2023-06-23T18:20:44.457904+0000","last_became_peered":"2023-06-23T18:20:44.457904+0000","last_unstale":"2023-06-23T18:20:45.439805+0000","last_undegraded":"2023-06-23T18:20:45.439805+0000","last_fullsized":"2023-06-23T18:20:45.439805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:53:21.724545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.500253+0000","last_change":"2023-06-23T18:20:44.034089+0000","last_active":"2023-06-23T18:20:47.500253+0000","last_peered":"2023-06-23T18:20:47.500253+0000","last_clean":"2023-06-23T18:20:47.500253+0000","last_became_active":"2023-06-23T18:20:44.033745+0000","last_became_peered":"2023-06-23T18:20:44.033745+0000","last_unstale":"2023-06-23T18:20:47.500253+0000","last_undegraded":"2023-06-23T18:20:47.500253+0000","last_fullsized":"2023-06-23T18:20:47.500253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:05.244917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.469931+0000","last_change":"2023-06-23T18:20:44.468622+0000","last_active":"2023-06-23T18:20:44.469931+0000","last_peered":"2023-06-23T18:20:44.469931+0000","last_clean":"2023-06-23T18:20:44.469931+0000","last_became_active":"2023-06-23T18:20:44.466831+0000","last_became_peered":"2023-06-23T18:20:44.466831+0000","last_unstale":"2023-06-23T18:20:44.469931+0000","last_undegraded":"2023-06-23T18:20:44.469931+0000","last_fullsized":"2023-06-23T18:20:44.469931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:35:13.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497829+0000","last_change":"2023-06-23T18:20:44.030647+0000","last_active":"2023-06-23T18:20:47.497829+0000","last_peered":"2023-06-23T18:20:47.497829+0000","last_clean":"2023-06-23T18:20:47.497829+0000","last_became_active":"2023-06-23T18:20:44.030316+0000","last_became_peered":"2023-06-23T18:20:44.030316+0000","last_unstale":"2023-06-23T18:20:47.497829+0000","last_undegraded":"2023-06-23T18:20:47.497829+0000","last_fullsized":"2023-06-23T18:20:47.497829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:47.369435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530780+0000","last_change":"2023-06-23T18:20:44.458365+0000","last_active":"2023-06-23T18:20:45.530780+0000","last_peered":"2023-06-23T18:20:45.530780+0000","last_clean":"2023-06-23T18:20:45.530780+0000","last_became_active":"2023-06-23T18:20:44.458039+0000","last_became_peered":"2023-06-23T18:20:44.458039+0000","last_unstale":"2023-06-23T18:20:45.530780+0000","last_undegraded":"2023-06-23T18:20:45.530780+0000","last_fullsized":"2023-06-23T18:20:45.530780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:03:49.379689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439310+0000","last_change":"2023-06-23T18:20:44.028908+0000","last_active":"2023-06-23T18:20:45.439310+0000","last_peered":"2023-06-23T18:20:45.439310+0000","last_clean":"2023-06-23T18:20:45.439310+0000","last_became_active":"2023-06-23T18:20:44.028551+0000","last_became_peered":"2023-06-23T18:20:44.028551+0000","last_unstale":"2023-06-23T18:20:45.439310+0000","last_undegraded":"2023-06-23T18:20:45.439310+0000","last_fullsized":"2023-06-23T18:20:45.439310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:31:41.576378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406188+0000","last_change":"2023-06-23T18:20:44.465300+0000","last_active":"2023-06-23T18:20:46.406188+0000","last_peered":"2023-06-23T18:20:46.406188+0000","last_clean":"2023-06-23T18:20:46.406188+0000","last_became_active":"2023-06-23T18:20:44.465008+0000","last_became_peered":"2023-06-23T18:20:44.465008+0000","last_unstale":"2023-06-23T18:20:46.406188+0000","last_undegraded":"2023-06-23T18:20:46.406188+0000","last_fullsized":"2023-06-23T18:20:46.406188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:03:27.021563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237528+0000","last_change":"2023-06-23T18:20:44.041593+0000","last_active":"2023-06-23T18:20:46.237528+0000","last_peered":"2023-06-23T18:20:46.237528+0000","last_clean":"2023-06-23T18:20:46.237528+0000","last_became_active":"2023-06-23T18:20:44.041283+0000","last_became_peered":"2023-06-23T18:20:44.041283+0000","last_unstale":"2023-06-23T18:20:46.237528+0000","last_undegraded":"2023-06-23T18:20:46.237528+0000","last_fullsized":"2023-06-23T18:20:46.237528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:45:29.030061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439210+0000","last_change":"2023-06-23T18:20:44.465127+0000","last_active":"2023-06-23T18:20:45.439210+0000","last_peered":"2023-06-23T18:20:45.439210+0000","last_clean":"2023-06-23T18:20:45.439210+0000","last_became_active":"2023-06-23T18:20:44.464768+0000","last_became_peered":"2023-06-23T18:20:44.464768+0000","last_unstale":"2023-06-23T18:20:45.439210+0000","last_undegraded":"2023-06-23T18:20:45.439210+0000","last_fullsized":"2023-06-23T18:20:45.439210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:54:38.343097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.495850+0000","last_change":"2023-06-23T18:20:44.046245+0000","last_active":"2023-06-23T18:20:47.495850+0000","last_peered":"2023-06-23T18:20:47.495850+0000","last_clean":"2023-06-23T18:20:47.495850+0000","last_became_active":"2023-06-23T18:20:44.045945+0000","last_became_peered":"2023-06-23T18:20:44.045945+0000","last_unstale":"2023-06-23T18:20:47.495850+0000","last_undegraded":"2023-06-23T18:20:47.495850+0000","last_fullsized":"2023-06-23T18:20:47.495850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:13:06.840364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439547+0000","last_change":"2023-06-23T18:20:44.453520+0000","last_active":"2023-06-23T18:20:45.439547+0000","last_peered":"2023-06-23T18:20:45.439547+0000","last_clean":"2023-06-23T18:20:45.439547+0000","last_became_active":"2023-06-23T18:20:44.453214+0000","last_became_peered":"2023-06-23T18:20:44.453214+0000","last_unstale":"2023-06-23T18:20:45.439547+0000","last_undegraded":"2023-06-23T18:20:45.439547+0000","last_fullsized":"2023-06-23T18:20:45.439547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:47.302353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.502980+0000","last_change":"2023-06-23T18:20:44.042526+0000","last_active":"2023-06-23T18:20:48.502980+0000","last_peered":"2023-06-23T18:20:48.502980+0000","last_clean":"2023-06-23T18:20:48.502980+0000","last_became_active":"2023-06-23T18:20:44.042249+0000","last_became_peered":"2023-06-23T18:20:44.042249+0000","last_unstale":"2023-06-23T18:20:48.502980+0000","last_undegraded":"2023-06-23T18:20:48.502980+0000","last_fullsized":"2023-06-23T18:20:48.502980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:31:39.378767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439873+0000","last_change":"2023-06-23T18:20:44.454645+0000","last_active":"2023-06-23T18:20:45.439873+0000","last_peered":"2023-06-23T18:20:45.439873+0000","last_clean":"2023-06-23T18:20:45.439873+0000","last_became_active":"2023-06-23T18:20:44.453309+0000","last_became_peered":"2023-06-23T18:20:44.453309+0000","last_unstale":"2023-06-23T18:20:45.439873+0000","last_undegraded":"2023-06-23T18:20:45.439873+0000","last_fullsized":"2023-06-23T18:20:45.439873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:30:55.635983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439463+0000","last_change":"2023-06-23T18:20:44.037681+0000","last_active":"2023-06-23T18:20:45.439463+0000","last_peered":"2023-06-23T18:20:45.439463+0000","last_clean":"2023-06-23T18:20:45.439463+0000","last_became_active":"2023-06-23T18:20:44.037323+0000","last_became_peered":"2023-06-23T18:20:44.037323+0000","last_unstale":"2023-06-23T18:20:45.439463+0000","last_undegraded":"2023-06-23T18:20:45.439463+0000","last_fullsized":"2023-06-23T18:20:45.439463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:55:26.308777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439284+0000","last_change":"2023-06-23T18:20:44.456434+0000","last_active":"2023-06-23T18:20:45.439284+0000","last_peered":"2023-06-23T18:20:45.439284+0000","last_clean":"2023-06-23T18:20:45.439284+0000","last_became_active":"2023-06-23T18:20:44.456110+0000","last_became_peered":"2023-06-23T18:20:44.456110+0000","last_unstale":"2023-06-23T18:20:45.439284+0000","last_undegraded":"2023-06-23T18:20:45.439284+0000","last_fullsized":"2023-06-23T18:20:45.439284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:00:04.366221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.469319+0000","last_change":"2023-06-23T18:20:44.040722+0000","last_active":"2023-06-23T18:20:44.469319+0000","last_peered":"2023-06-23T18:20:44.469319+0000","last_clean":"2023-06-23T18:20:44.469319+0000","last_became_active":"2023-06-23T18:20:44.040390+0000","last_became_peered":"2023-06-23T18:20:44.040390+0000","last_unstale":"2023-06-23T18:20:44.469319+0000","last_undegraded":"2023-06-23T18:20:44.469319+0000","last_fullsized":"2023-06-23T18:20:44.469319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:30:30.180370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439334+0000","last_change":"2023-06-23T18:20:44.455559+0000","last_active":"2023-06-23T18:20:45.439334+0000","last_peered":"2023-06-23T18:20:45.439334+0000","last_clean":"2023-06-23T18:20:45.439334+0000","last_became_active":"2023-06-23T18:20:44.455256+0000","last_became_peered":"2023-06-23T18:20:44.455256+0000","last_unstale":"2023-06-23T18:20:45.439334+0000","last_undegraded":"2023-06-23T18:20:45.439334+0000","last_fullsized":"2023-06-23T18:20:45.439334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:40:30.169440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406131+0000","last_change":"2023-06-23T18:20:44.460403+0000","last_active":"2023-06-23T18:20:46.406131+0000","last_peered":"2023-06-23T18:20:46.406131+0000","last_clean":"2023-06-23T18:20:46.406131+0000","last_became_active":"2023-06-23T18:20:44.459928+0000","last_became_peered":"2023-06-23T18:20:44.459928+0000","last_unstale":"2023-06-23T18:20:46.406131+0000","last_undegraded":"2023-06-23T18:20:46.406131+0000","last_fullsized":"2023-06-23T18:20:46.406131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:58:06.385995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237624+0000","last_change":"2023-06-23T18:20:44.458050+0000","last_active":"2023-06-23T18:20:46.237624+0000","last_peered":"2023-06-23T18:20:46.237624+0000","last_clean":"2023-06-23T18:20:46.237624+0000","last_became_active":"2023-06-23T18:20:44.457733+0000","last_became_peered":"2023-06-23T18:20:44.457733+0000","last_unstale":"2023-06-23T18:20:46.237624+0000","last_undegraded":"2023-06-23T18:20:46.237624+0000","last_fullsized":"2023-06-23T18:20:46.237624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:23:24.448430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237525+0000","last_change":"2023-06-23T18:20:44.456233+0000","last_active":"2023-06-23T18:20:46.237525+0000","last_peered":"2023-06-23T18:20:46.237525+0000","last_clean":"2023-06-23T18:20:46.237525+0000","last_became_active":"2023-06-23T18:20:44.455677+0000","last_became_peered":"2023-06-23T18:20:44.455677+0000","last_unstale":"2023-06-23T18:20:46.237525+0000","last_undegraded":"2023-06-23T18:20:46.237525+0000","last_fullsized":"2023-06-23T18:20:46.237525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:54:14.083530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.470345+0000","last_change":"2023-06-23T18:20:44.469668+0000","last_active":"2023-06-23T18:20:44.470345+0000","last_peered":"2023-06-23T18:20:44.470345+0000","last_clean":"2023-06-23T18:20:44.470345+0000","last_became_active":"2023-06-23T18:20:44.466563+0000","last_became_peered":"2023-06-23T18:20:44.466563+0000","last_unstale":"2023-06-23T18:20:44.470345+0000","last_undegraded":"2023-06-23T18:20:44.470345+0000","last_fullsized":"2023-06-23T18:20:44.470345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:15:40.824612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.457800+0000","last_change":"2023-06-23T18:20:44.456021+0000","last_active":"2023-06-23T18:20:44.457800+0000","last_peered":"2023-06-23T18:20:44.457800+0000","last_clean":"2023-06-23T18:20:44.457800+0000","last_became_active":"2023-06-23T18:20:44.455705+0000","last_became_peered":"2023-06-23T18:20:44.455705+0000","last_unstale":"2023-06-23T18:20:44.457800+0000","last_undegraded":"2023-06-23T18:20:44.457800+0000","last_fullsized":"2023-06-23T18:20:44.457800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:13:32.159183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406226+0000","last_change":"2023-06-23T18:20:44.460359+0000","last_active":"2023-06-23T18:20:46.406226+0000","last_peered":"2023-06-23T18:20:46.406226+0000","last_clean":"2023-06-23T18:20:46.406226+0000","last_became_active":"2023-06-23T18:20:44.460034+0000","last_became_peered":"2023-06-23T18:20:44.460034+0000","last_unstale":"2023-06-23T18:20:46.406226+0000","last_undegraded":"2023-06-23T18:20:46.406226+0000","last_fullsized":"2023-06-23T18:20:46.406226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:00:06.777366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237875+0000","last_change":"2023-06-23T18:20:44.455743+0000","last_active":"2023-06-23T18:20:46.237875+0000","last_peered":"2023-06-23T18:20:46.237875+0000","last_clean":"2023-06-23T18:20:46.237875+0000","last_became_active":"2023-06-23T18:20:44.455266+0000","last_became_peered":"2023-06-23T18:20:44.455266+0000","last_unstale":"2023-06-23T18:20:46.237875+0000","last_undegraded":"2023-06-23T18:20:46.237875+0000","last_fullsized":"2023-06-23T18:20:46.237875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:33:52.673181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439651+0000","last_change":"2023-06-23T18:20:44.461434+0000","last_active":"2023-06-23T18:20:45.439651+0000","last_peered":"2023-06-23T18:20:45.439651+0000","last_clean":"2023-06-23T18:20:45.439651+0000","last_became_active":"2023-06-23T18:20:44.461137+0000","last_became_peered":"2023-06-23T18:20:44.461137+0000","last_unstale":"2023-06-23T18:20:45.439651+0000","last_undegraded":"2023-06-23T18:20:45.439651+0000","last_fullsized":"2023-06-23T18:20:45.439651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:14:08.293656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439086+0000","last_change":"2023-06-23T18:20:44.465192+0000","last_active":"2023-06-23T18:20:45.439086+0000","last_peered":"2023-06-23T18:20:45.439086+0000","last_clean":"2023-06-23T18:20:45.439086+0000","last_became_active":"2023-06-23T18:20:44.464905+0000","last_became_peered":"2023-06-23T18:20:44.464905+0000","last_unstale":"2023-06-23T18:20:45.439086+0000","last_undegraded":"2023-06-23T18:20:45.439086+0000","last_fullsized":"2023-06-23T18:20:45.439086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:01:32.627765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439130+0000","last_change":"2023-06-23T18:20:44.458725+0000","last_active":"2023-06-23T18:20:45.439130+0000","last_peered":"2023-06-23T18:20:45.439130+0000","last_clean":"2023-06-23T18:20:45.439130+0000","last_became_active":"2023-06-23T18:20:44.458414+0000","last_became_peered":"2023-06-23T18:20:44.458414+0000","last_unstale":"2023-06-23T18:20:45.439130+0000","last_undegraded":"2023-06-23T18:20:45.439130+0000","last_fullsized":"2023-06-23T18:20:45.439130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:36:29.121354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405885+0000","last_change":"2023-06-23T18:20:44.465665+0000","last_active":"2023-06-23T18:20:46.405885+0000","last_peered":"2023-06-23T18:20:46.405885+0000","last_clean":"2023-06-23T18:20:46.405885+0000","last_became_active":"2023-06-23T18:20:44.465337+0000","last_became_peered":"2023-06-23T18:20:44.465337+0000","last_unstale":"2023-06-23T18:20:46.405885+0000","last_undegraded":"2023-06-23T18:20:46.405885+0000","last_fullsized":"2023-06-23T18:20:46.405885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:58:56.987077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.457850+0000","last_change":"2023-06-23T18:20:44.455107+0000","last_active":"2023-06-23T18:20:44.457850+0000","last_peered":"2023-06-23T18:20:44.457850+0000","last_clean":"2023-06-23T18:20:44.457850+0000","last_became_active":"2023-06-23T18:20:44.454798+0000","last_became_peered":"2023-06-23T18:20:44.454798+0000","last_unstale":"2023-06-23T18:20:44.457850+0000","last_undegraded":"2023-06-23T18:20:44.457850+0000","last_fullsized":"2023-06-23T18:20:44.457850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:31:38.072092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406114+0000","last_change":"2023-06-23T18:20:44.461044+0000","last_active":"2023-06-23T18:20:46.406114+0000","last_peered":"2023-06-23T18:20:46.406114+0000","last_clean":"2023-06-23T18:20:46.406114+0000","last_became_active":"2023-06-23T18:20:44.460737+0000","last_became_peered":"2023-06-23T18:20:44.460737+0000","last_unstale":"2023-06-23T18:20:46.406114+0000","last_undegraded":"2023-06-23T18:20:46.406114+0000","last_fullsized":"2023-06-23T18:20:46.406114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:13:16.912989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281641,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4234,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":93,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":27335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481803,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":419789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":10,"up_from":70,"seq":300647710736,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":1011915,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":0.625},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.81100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.9430000000000001,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.75}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":9,"up_from":65,"seq":279172874260,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":420607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.374}]},{"osd":1,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.5409999999999999,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.432}]},{"osd":5,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.69199999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.63400000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53400000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":420084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.439}]},{"osd":1,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.308},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.42599999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38300000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.495},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.505}]},{"osd":7,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.51700000000000002}]},{"osd":9,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52400000000000002}]},{"osd":10,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332124,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":1010739,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.33300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64100000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.435,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.504},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.84199999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.623},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.54000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.53600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.56799999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70499999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.76200000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67700000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495648,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":411219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.436,"15min":0.43099999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.374}]},{"osd":1,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.43099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.02,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82699999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":1.105,"15min":1.105},"last":0.501}]},{"osd":3,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.044,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.046,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53600000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.50800000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.58599999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.63200000000000001}]},{"osd":9,"last update":"Fri Jun 23 18:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.621},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.54800000000000004}]},{"osd":10,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":421353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.376,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82599999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.41999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86499999999999999,"5min":1.218,"15min":1.218},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.317,"15min":0.317},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.37,"5min":0.312,"15min":0.312},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.45900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.53200000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45400000000000001,"15min":0.42799999999999999},"min":{"1min":0.38600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.442},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44600000000000001,"15min":0.41799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.35299999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.45100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.46800000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39400000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96099999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.376},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56200000000000006,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0629999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.68300000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.503},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.58999999999999997}]},{"osd":10,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.57899999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":868352,"data_stored":411309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.40899999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.5,"15min":0.47499999999999998},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44}]},{"osd":2,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.49199999999999999},"min":{"1min":0.41299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.624,"5min":0.628,"15min":0.628},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Jun 23 18:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56999999999999995,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.58899999999999997},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Jun 23 18:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.622}]},{"osd":8,"last update":"Fri Jun 23 18:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":422655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.34899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.438}]},{"osd":1,"last update":"Fri Jun 23 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69599999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78000000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.625}]},{"osd":3,"last update":"Fri Jun 23 18:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.311,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76000000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.626,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88700000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.623},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47699999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59799999999999998}]},{"osd":8,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45700000000000002}]},{"osd":9,"last update":"Fri Jun 23 18:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.46700000000000003}]},{"osd":10,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":3,"up_from":25,"seq":107374182444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":411323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.34799999999999998,"15min":0.33100000000000002},"min":{"1min":0.29499999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.436,"5min":0.378,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.46600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57399999999999995,"15min":0.54300000000000004},"min":{"1min":0.495,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90800000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67900000000000005}]},{"osd":4,"last update":"Fri Jun 23 18:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63}]},{"osd":5,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94099999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.496},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58699999999999997}]},{"osd":10,"last update":"Fri Jun 23 18:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.71099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920809,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.51700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.57799999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.309},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.38300000000000001},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.373}]},{"osd":2,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57799999999999996,"15min":0.54300000000000004},"min":{"1min":0.503,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78000000000000003,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.432},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.443}]},{"osd":5,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55900000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.379,"5min":0.378,"15min":0.378},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.48799999999999999}]},{"osd":8,"last update":"Fri Jun 23 18:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61099999999999999}]},{"osd":9,"last update":"Fri Jun 23 18:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.59799999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.65800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757286,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":1011585,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.49299999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.311}]},{"osd":1,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.45800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.33900000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"min":{"1min":0.32300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.248},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.26300000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.40400000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79900000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.872,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":0.89800000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.432}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.495},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41099999999999998}]},{"osd":9,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59399999999999997}]},{"osd":10,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.47699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:20:51.829 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:20:51.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:51 smithi082 ceph-mon[97889]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:51 smithi082 ceph-mon[97889]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:51 smithi082 ceph-mon[97889]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:51 smithi082 ceph-mon[97889]: pgmap v259: 97 pgs: 97 active+clean; 581 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2023-06-23T18:20:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:51 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/59752868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:51 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1104242371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:51 smithi017 ceph-mon[97472]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:51 smithi017 ceph-mon[97472]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:51 smithi017 ceph-mon[97472]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:20:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:51 smithi017 ceph-mon[97472]: pgmap v259: 97 pgs: 97 active+clean; 581 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 4.9 KiB/s wr, 15 op/s 2023-06-23T18:20:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:51 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/59752868' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:20:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:51 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1104242371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:20:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 -- 172.21.15.2:0/1384319763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f48100670 msgr2=0x7f6f48106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 --2- 172.21.15.2:0/1384319763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f48100670 0x7f6f48106f10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000b0a0 tx=0x7f6f4000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 -- 172.21.15.2:0/1384319763 shutdown_connections 2023-06-23T18:20:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 --2- 172.21.15.2:0/1384319763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f48100670 0x7f6f48106f10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 --2- 172.21.15.2:0/1384319763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f480fea30 0x7f6f480fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 --2- 172.21.15.2:0/1384319763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f480ffcc0 0x7f6f48100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 -- 172.21.15.2:0/1384319763 >> 172.21.15.2:0/1384319763 conn(0x7f6f480fa050 msgr2=0x7f6f480fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 -- 172.21.15.2:0/1384319763 shutdown_connections 2023-06-23T18:20:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.975+0000 7f6f4f633700 1 -- 172.21.15.2:0/1384319763 wait complete. 2023-06-23T18:20:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4f633700 1 Processor -- start 2023-06-23T18:20:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4f633700 1 -- start start 2023-06-23T18:20:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4f633700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f480fea30 0x7f6f48195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4f633700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f480ffcc0 0x7f6f48195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4f633700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f48100670 0x7f6f4819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4f633700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4819d410 con 0x7f6f480ffcc0 2023-06-23T18:20:51.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4f633700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4819d550 con 0x7f6f480fea30 2023-06-23T18:20:51.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4f633700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4819d6e0 con 0x7f6f48100670 2023-06-23T18:20:51.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4cbce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f480ffcc0 0x7f6f48195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4cbce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f480ffcc0 0x7f6f48195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52612/0 (socket says 172.21.15.2:52612) 2023-06-23T18:20:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4cbce700 1 -- 172.21.15.2:0/2691505217 learned_addr learned my addr 172.21.15.2:0/2691505217 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4dbd0700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f48100670 0x7f6f4819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4d3cf700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f480fea30 0x7f6f48195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4cbce700 1 -- 172.21.15.2:0/2691505217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f48100670 msgr2=0x7f6f4819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4cbce700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f48100670 0x7f6f4819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4cbce700 1 -- 172.21.15.2:0/2691505217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f480fea30 msgr2=0x7f6f48195890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4cbce700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f480fea30 0x7f6f48195890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.976+0000 7f6f4cbce700 1 -- 172.21.15.2:0/2691505217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40009d00 con 0x7f6f480ffcc0 2023-06-23T18:20:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.977+0000 7f6f4cbce700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f480ffcc0 0x7f6f48195dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9a0 tx=0x7f6f3c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.977+0000 7f6f3a7fc700 1 -- 172.21.15.2:0/2691505217 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c009790 con 0x7f6f480ffcc0 2023-06-23T18:20:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.977+0000 7f6f3a7fc700 1 -- 172.21.15.2:0/2691505217 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6f3c00cbd0 con 0x7f6f480ffcc0 2023-06-23T18:20:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.977+0000 7f6f3a7fc700 1 -- 172.21.15.2:0/2691505217 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c018680 con 0x7f6f480ffcc0 2023-06-23T18:20:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.977+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4819d9c0 con 0x7f6f480ffcc0 2023-06-23T18:20:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.977+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4819de80 con 0x7f6f480ffcc0 2023-06-23T18:20:51.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.978+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f48106bd0 con 0x7f6f480ffcc0 2023-06-23T18:20:51.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.979+0000 7f6f3a7fc700 1 -- 172.21.15.2:0/2691505217 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f6f3c00cd40 con 0x7f6f480ffcc0 2023-06-23T18:20:51.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.980+0000 7f6f3a7fc700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6f340a11e0 0x7f6f340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:51.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.980+0000 7f6f4d3cf700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6f340a11e0 0x7f6f340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:51.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.980+0000 7f6f3a7fc700 1 -- 172.21.15.2:0/2691505217 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6f3c014070 con 0x7f6f480ffcc0 2023-06-23T18:20:51.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.980+0000 7f6f4d3cf700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6f340a11e0 0x7f6f340a3690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009bf0 tx=0x7f6f44009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:51.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:51.983+0000 7f6f3a7fc700 1 -- 172.21.15.2:0/2691505217 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f6f3c0965e0 con 0x7f6f480ffcc0 2023-06-23T18:20:52.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.180+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f48060910 con 0x7f6f340a11e0 2023-06-23T18:20:52.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.192+0000 7f6f3a7fc700 1 -- 172.21.15.2:0/2691505217 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+335180 (secure 0 0 0) 0x7f6f48060910 con 0x7f6f340a11e0 2023-06-23T18:20:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:52.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.195+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6f340a11e0 msgr2=0x7f6f340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:52.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.195+0000 7f6f4f633700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6f340a11e0 0x7f6f340a3690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009bf0 tx=0x7f6f44009260 comp rx=0 tx=0).stop 2023-06-23T18:20:52.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.195+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f480ffcc0 msgr2=0x7f6f48195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:52.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.195+0000 7f6f4f633700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f480ffcc0 0x7f6f48195dd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9a0 tx=0x7f6f3c00ed60 comp rx=0 tx=0).stop 2023-06-23T18:20:52.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.196+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 shutdown_connections 2023-06-23T18:20:52.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.196+0000 7f6f4f633700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f48100670 0x7f6f4819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:52.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.196+0000 7f6f4f633700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6f340a11e0 0x7f6f340a3690 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:52.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.196+0000 7f6f4f633700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f480fea30 0x7f6f48195890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:52.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.196+0000 7f6f4f633700 1 --2- 172.21.15.2:0/2691505217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6f480ffcc0 0x7f6f48195dd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:52.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.196+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 >> 172.21.15.2:0/2691505217 conn(0x7f6f480fa050 msgr2=0x7f6f480fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:52.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.196+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 shutdown_connections 2023-06-23T18:20:52.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:52.196+0000 7f6f4f633700 1 -- 172.21.15.2:0/2691505217 wait complete. 2023-06-23T18:20:52.204 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-06-23T18:20:52.212 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2023-06-23T18:20:50.731969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594602,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":134,"num_read_kb":116,"num_write":274,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":217,"ondisk_log_size":217,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":204,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":333952,"kb_used_data":10528,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607872,"statfs":{"total":1151940427776,"available":1151598460928,"internally_reserved":0,"allocated":10780672,"data_stored":6792389,"data_compressed":107457,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4234,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":93,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.713669"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406331+0000","last_change":"2023-06-23T18:20:44.466650+0000","last_active":"2023-06-23T18:20:46.406331+0000","last_peered":"2023-06-23T18:20:46.406331+0000","last_clean":"2023-06-23T18:20:46.406331+0000","last_became_active":"2023-06-23T18:20:44.466425+0000","last_became_peered":"2023-06-23T18:20:44.466425+0000","last_unstale":"2023-06-23T18:20:46.406331+0000","last_undegraded":"2023-06-23T18:20:46.406331+0000","last_fullsized":"2023-06-23T18:20:46.406331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:46:45.756486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.466414+0000","last_change":"2023-06-23T18:20:44.465762+0000","last_active":"2023-06-23T18:20:44.466414+0000","last_peered":"2023-06-23T18:20:44.466414+0000","last_clean":"2023-06-23T18:20:44.466414+0000","last_became_active":"2023-06-23T18:20:44.463592+0000","last_became_peered":"2023-06-23T18:20:44.463592+0000","last_unstale":"2023-06-23T18:20:44.466414+0000","last_undegraded":"2023-06-23T18:20:44.466414+0000","last_fullsized":"2023-06-23T18:20:44.466414+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:20:45.870897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.465699+0000","last_change":"2023-06-23T18:20:44.464162+0000","last_active":"2023-06-23T18:20:44.465699+0000","last_peered":"2023-06-23T18:20:44.465699+0000","last_clean":"2023-06-23T18:20:44.465699+0000","last_became_active":"2023-06-23T18:20:44.463854+0000","last_became_peered":"2023-06-23T18:20:44.463854+0000","last_unstale":"2023-06-23T18:20:44.465699+0000","last_undegraded":"2023-06-23T18:20:44.465699+0000","last_fullsized":"2023-06-23T18:20:44.465699+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:41:24.449221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.366029+0000","last_change":"2023-06-23T18:20:44.462923+0000","last_active":"2023-06-23T18:20:46.366029+0000","last_peered":"2023-06-23T18:20:46.366029+0000","last_clean":"2023-06-23T18:20:46.366029+0000","last_became_active":"2023-06-23T18:20:44.462511+0000","last_became_peered":"2023-06-23T18:20:44.462511+0000","last_unstale":"2023-06-23T18:20:46.366029+0000","last_undegraded":"2023-06-23T18:20:46.366029+0000","last_fullsized":"2023-06-23T18:20:46.366029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:32:40.690012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530812+0000","last_change":"2023-06-23T18:20:44.454727+0000","last_active":"2023-06-23T18:20:45.530812+0000","last_peered":"2023-06-23T18:20:45.530812+0000","last_clean":"2023-06-23T18:20:45.530812+0000","last_became_active":"2023-06-23T18:20:44.454418+0000","last_became_peered":"2023-06-23T18:20:44.454418+0000","last_unstale":"2023-06-23T18:20:45.530812+0000","last_undegraded":"2023-06-23T18:20:45.530812+0000","last_fullsized":"2023-06-23T18:20:45.530812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:34:09.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905435+0000","last_change":"2023-06-23T18:20:44.460636+0000","last_active":"2023-06-23T18:20:45.905435+0000","last_peered":"2023-06-23T18:20:45.905435+0000","last_clean":"2023-06-23T18:20:45.905435+0000","last_became_active":"2023-06-23T18:20:44.460305+0000","last_became_peered":"2023-06-23T18:20:44.460305+0000","last_unstale":"2023-06-23T18:20:45.905435+0000","last_undegraded":"2023-06-23T18:20:45.905435+0000","last_fullsized":"2023-06-23T18:20:45.905435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:20:11.034702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439073+0000","last_change":"2023-06-23T18:20:44.463552+0000","last_active":"2023-06-23T18:20:45.439073+0000","last_peered":"2023-06-23T18:20:45.439073+0000","last_clean":"2023-06-23T18:20:45.439073+0000","last_became_active":"2023-06-23T18:20:44.463187+0000","last_became_peered":"2023-06-23T18:20:44.463187+0000","last_unstale":"2023-06-23T18:20:45.439073+0000","last_undegraded":"2023-06-23T18:20:45.439073+0000","last_fullsized":"2023-06-23T18:20:45.439073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:16:33.692125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439061+0000","last_change":"2023-06-23T18:20:44.457957+0000","last_active":"2023-06-23T18:20:45.439061+0000","last_peered":"2023-06-23T18:20:45.439061+0000","last_clean":"2023-06-23T18:20:45.439061+0000","last_became_active":"2023-06-23T18:20:44.457511+0000","last_became_peered":"2023-06-23T18:20:44.457511+0000","last_unstale":"2023-06-23T18:20:45.439061+0000","last_undegraded":"2023-06-23T18:20:45.439061+0000","last_fullsized":"2023-06-23T18:20:45.439061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:19:36.056785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406274+0000","last_change":"2023-06-23T18:20:44.461199+0000","last_active":"2023-06-23T18:20:46.406274+0000","last_peered":"2023-06-23T18:20:46.406274+0000","last_clean":"2023-06-23T18:20:46.406274+0000","last_became_active":"2023-06-23T18:20:44.460891+0000","last_became_peered":"2023-06-23T18:20:44.460891+0000","last_unstale":"2023-06-23T18:20:46.406274+0000","last_undegraded":"2023-06-23T18:20:46.406274+0000","last_fullsized":"2023-06-23T18:20:46.406274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:00.977889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905565+0000","last_change":"2023-06-23T18:20:44.460741+0000","last_active":"2023-06-23T18:20:45.905565+0000","last_peered":"2023-06-23T18:20:45.905565+0000","last_clean":"2023-06-23T18:20:45.905565+0000","last_became_active":"2023-06-23T18:20:44.460109+0000","last_became_peered":"2023-06-23T18:20:44.460109+0000","last_unstale":"2023-06-23T18:20:45.905565+0000","last_undegraded":"2023-06-23T18:20:45.905565+0000","last_fullsized":"2023-06-23T18:20:45.905565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:37:37.614265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439769+0000","last_change":"2023-06-23T18:20:44.453337+0000","last_active":"2023-06-23T18:20:45.439769+0000","last_peered":"2023-06-23T18:20:45.439769+0000","last_clean":"2023-06-23T18:20:45.439769+0000","last_became_active":"2023-06-23T18:20:44.452925+0000","last_became_peered":"2023-06-23T18:20:44.452925+0000","last_unstale":"2023-06-23T18:20:45.439769+0000","last_undegraded":"2023-06-23T18:20:45.439769+0000","last_fullsized":"2023-06-23T18:20:45.439769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:10.381911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.365990+0000","last_change":"2023-06-23T18:20:44.456710+0000","last_active":"2023-06-23T18:20:46.365990+0000","last_peered":"2023-06-23T18:20:46.365990+0000","last_clean":"2023-06-23T18:20:46.365990+0000","last_became_active":"2023-06-23T18:20:44.456411+0000","last_became_peered":"2023-06-23T18:20:44.456411+0000","last_unstale":"2023-06-23T18:20:46.365990+0000","last_undegraded":"2023-06-23T18:20:46.365990+0000","last_fullsized":"2023-06-23T18:20:46.365990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:01:45.669051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.455205+0000","last_change":"2023-06-23T18:20:44.453586+0000","last_active":"2023-06-23T18:20:44.455205+0000","last_peered":"2023-06-23T18:20:44.455205+0000","last_clean":"2023-06-23T18:20:44.455205+0000","last_became_active":"2023-06-23T18:20:44.453239+0000","last_became_peered":"2023-06-23T18:20:44.453239+0000","last_unstale":"2023-06-23T18:20:44.455205+0000","last_undegraded":"2023-06-23T18:20:44.455205+0000","last_fullsized":"2023-06-23T18:20:44.455205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:50:15.838769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.465959+0000","last_change":"2023-06-23T18:20:44.465627+0000","last_active":"2023-06-23T18:20:44.465959+0000","last_peered":"2023-06-23T18:20:44.465959+0000","last_clean":"2023-06-23T18:20:44.465959+0000","last_became_active":"2023-06-23T18:20:44.464568+0000","last_became_peered":"2023-06-23T18:20:44.464568+0000","last_unstale":"2023-06-23T18:20:44.465959+0000","last_undegraded":"2023-06-23T18:20:44.465959+0000","last_fullsized":"2023-06-23T18:20:44.465959+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:26:31.336582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.465673+0000","last_change":"2023-06-23T18:20:44.464169+0000","last_active":"2023-06-23T18:20:44.465673+0000","last_peered":"2023-06-23T18:20:44.465673+0000","last_clean":"2023-06-23T18:20:44.465673+0000","last_became_active":"2023-06-23T18:20:44.463867+0000","last_became_peered":"2023-06-23T18:20:44.463867+0000","last_unstale":"2023-06-23T18:20:44.465673+0000","last_undegraded":"2023-06-23T18:20:44.465673+0000","last_fullsized":"2023-06-23T18:20:44.465673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:21:09.249987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405761+0000","last_change":"2023-06-23T18:20:44.467101+0000","last_active":"2023-06-23T18:20:46.405761+0000","last_peered":"2023-06-23T18:20:46.405761+0000","last_clean":"2023-06-23T18:20:46.405761+0000","last_became_active":"2023-06-23T18:20:44.466788+0000","last_became_peered":"2023-06-23T18:20:44.466788+0000","last_unstale":"2023-06-23T18:20:46.405761+0000","last_undegraded":"2023-06-23T18:20:46.405761+0000","last_fullsized":"2023-06-23T18:20:46.405761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:14:55.735170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406210+0000","last_change":"2023-06-23T18:20:44.466055+0000","last_active":"2023-06-23T18:20:46.406210+0000","last_peered":"2023-06-23T18:20:46.406210+0000","last_clean":"2023-06-23T18:20:46.406210+0000","last_became_active":"2023-06-23T18:20:44.465608+0000","last_became_peered":"2023-06-23T18:20:44.465608+0000","last_unstale":"2023-06-23T18:20:46.406210+0000","last_undegraded":"2023-06-23T18:20:46.406210+0000","last_fullsized":"2023-06-23T18:20:46.406210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:06:22.567595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.529187+0000","last_change":"2023-06-23T18:20:44.459414+0000","last_active":"2023-06-23T18:20:45.529187+0000","last_peered":"2023-06-23T18:20:45.529187+0000","last_clean":"2023-06-23T18:20:45.529187+0000","last_became_active":"2023-06-23T18:20:44.459014+0000","last_became_peered":"2023-06-23T18:20:44.459014+0000","last_unstale":"2023-06-23T18:20:45.529187+0000","last_undegraded":"2023-06-23T18:20:45.529187+0000","last_fullsized":"2023-06-23T18:20:45.529187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:00:16.944957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237780+0000","last_change":"2023-06-23T18:20:44.446784+0000","last_active":"2023-06-23T18:20:46.237780+0000","last_peered":"2023-06-23T18:20:46.237780+0000","last_clean":"2023-06-23T18:20:46.237780+0000","last_became_active":"2023-06-23T18:20:44.446464+0000","last_became_peered":"2023-06-23T18:20:44.446464+0000","last_unstale":"2023-06-23T18:20:46.237780+0000","last_undegraded":"2023-06-23T18:20:46.237780+0000","last_fullsized":"2023-06-23T18:20:46.237780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:03:13.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.501338+0000","last_change":"2023-06-23T18:20:44.039326+0000","last_active":"2023-06-23T18:20:48.501338+0000","last_peered":"2023-06-23T18:20:48.501338+0000","last_clean":"2023-06-23T18:20:48.501338+0000","last_became_active":"2023-06-23T18:20:44.038787+0000","last_became_peered":"2023-06-23T18:20:44.038787+0000","last_unstale":"2023-06-23T18:20:48.501338+0000","last_undegraded":"2023-06-23T18:20:48.501338+0000","last_fullsized":"2023-06-23T18:20:48.501338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:16:15.016712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439642+0000","last_change":"2023-06-23T18:20:44.465269+0000","last_active":"2023-06-23T18:20:45.439642+0000","last_peered":"2023-06-23T18:20:45.439642+0000","last_clean":"2023-06-23T18:20:45.439642+0000","last_became_active":"2023-06-23T18:20:44.464973+0000","last_became_peered":"2023-06-23T18:20:44.464973+0000","last_unstale":"2023-06-23T18:20:45.439642+0000","last_undegraded":"2023-06-23T18:20:45.439642+0000","last_fullsized":"2023-06-23T18:20:45.439642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:24:55.096958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439687+0000","last_change":"2023-06-23T18:20:44.037175+0000","last_active":"2023-06-23T18:20:45.439687+0000","last_peered":"2023-06-23T18:20:45.439687+0000","last_clean":"2023-06-23T18:20:45.439687+0000","last_became_active":"2023-06-23T18:20:44.036788+0000","last_became_peered":"2023-06-23T18:20:44.036788+0000","last_unstale":"2023-06-23T18:20:45.439687+0000","last_undegraded":"2023-06-23T18:20:45.439687+0000","last_fullsized":"2023-06-23T18:20:45.439687+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:15:06.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.464521+0000","last_change":"2023-06-23T18:20:44.463344+0000","last_active":"2023-06-23T18:20:44.464521+0000","last_peered":"2023-06-23T18:20:44.464521+0000","last_clean":"2023-06-23T18:20:44.464521+0000","last_became_active":"2023-06-23T18:20:44.462649+0000","last_became_peered":"2023-06-23T18:20:44.462649+0000","last_unstale":"2023-06-23T18:20:44.464521+0000","last_undegraded":"2023-06-23T18:20:44.464521+0000","last_fullsized":"2023-06-23T18:20:44.464521+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:57:45.776231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499503+0000","last_change":"2023-06-23T18:20:44.045525+0000","last_active":"2023-06-23T18:20:48.499503+0000","last_peered":"2023-06-23T18:20:48.499503+0000","last_clean":"2023-06-23T18:20:48.499503+0000","last_became_active":"2023-06-23T18:20:44.045212+0000","last_became_peered":"2023-06-23T18:20:44.045212+0000","last_unstale":"2023-06-23T18:20:48.499503+0000","last_undegraded":"2023-06-23T18:20:48.499503+0000","last_fullsized":"2023-06-23T18:20:48.499503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:44:56.572688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405870+0000","last_change":"2023-06-23T18:20:44.461071+0000","last_active":"2023-06-23T18:20:46.405870+0000","last_peered":"2023-06-23T18:20:46.405870+0000","last_clean":"2023-06-23T18:20:46.405870+0000","last_became_active":"2023-06-23T18:20:44.460726+0000","last_became_peered":"2023-06-23T18:20:44.460726+0000","last_unstale":"2023-06-23T18:20:46.405870+0000","last_undegraded":"2023-06-23T18:20:46.405870+0000","last_fullsized":"2023-06-23T18:20:46.405870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:08:47.572741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530990+0000","last_change":"2023-06-23T18:20:44.037162+0000","last_active":"2023-06-23T18:20:45.530990+0000","last_peered":"2023-06-23T18:20:45.530990+0000","last_clean":"2023-06-23T18:20:45.530990+0000","last_became_active":"2023-06-23T18:20:44.036809+0000","last_became_peered":"2023-06-23T18:20:44.036809+0000","last_unstale":"2023-06-23T18:20:45.530990+0000","last_undegraded":"2023-06-23T18:20:45.530990+0000","last_fullsized":"2023-06-23T18:20:45.530990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:42:11.269748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439457+0000","last_change":"2023-06-23T18:20:44.463525+0000","last_active":"2023-06-23T18:20:45.439457+0000","last_peered":"2023-06-23T18:20:45.439457+0000","last_clean":"2023-06-23T18:20:45.439457+0000","last_became_active":"2023-06-23T18:20:44.463245+0000","last_became_peered":"2023-06-23T18:20:44.463245+0000","last_unstale":"2023-06-23T18:20:45.439457+0000","last_undegraded":"2023-06-23T18:20:45.439457+0000","last_fullsized":"2023-06-23T18:20:45.439457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:18:47.776141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:49.050736+0000","last_change":"2023-06-23T18:20:44.044415+0000","last_active":"2023-06-23T18:20:49.050736+0000","last_peered":"2023-06-23T18:20:49.050736+0000","last_clean":"2023-06-23T18:20:49.050736+0000","last_became_active":"2023-06-23T18:20:44.044085+0000","last_became_peered":"2023-06-23T18:20:44.044085+0000","last_unstale":"2023-06-23T18:20:49.050736+0000","last_undegraded":"2023-06-23T18:20:49.050736+0000","last_fullsized":"2023-06-23T18:20:49.050736+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:07:29.948276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530911+0000","last_change":"2023-06-23T18:20:44.463422+0000","last_active":"2023-06-23T18:20:45.530911+0000","last_peered":"2023-06-23T18:20:45.530911+0000","last_clean":"2023-06-23T18:20:45.530911+0000","last_became_active":"2023-06-23T18:20:44.463121+0000","last_became_peered":"2023-06-23T18:20:44.463121+0000","last_unstale":"2023-06-23T18:20:45.530911+0000","last_undegraded":"2023-06-23T18:20:45.530911+0000","last_fullsized":"2023-06-23T18:20:45.530911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:54:44.620991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905236+0000","last_change":"2023-06-23T18:20:44.039604+0000","last_active":"2023-06-23T18:20:45.905236+0000","last_peered":"2023-06-23T18:20:45.905236+0000","last_clean":"2023-06-23T18:20:45.905236+0000","last_became_active":"2023-06-23T18:20:44.039317+0000","last_became_peered":"2023-06-23T18:20:44.039317+0000","last_unstale":"2023-06-23T18:20:45.905236+0000","last_undegraded":"2023-06-23T18:20:45.905236+0000","last_fullsized":"2023-06-23T18:20:45.905236+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:39:40.353762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439176+0000","last_change":"2023-06-23T18:20:44.465748+0000","last_active":"2023-06-23T18:20:45.439176+0000","last_peered":"2023-06-23T18:20:45.439176+0000","last_clean":"2023-06-23T18:20:45.439176+0000","last_became_active":"2023-06-23T18:20:44.465430+0000","last_became_peered":"2023-06-23T18:20:44.465430+0000","last_unstale":"2023-06-23T18:20:45.439176+0000","last_undegraded":"2023-06-23T18:20:45.439176+0000","last_fullsized":"2023-06-23T18:20:45.439176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:31.824195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.438495+0000","last_change":"2023-06-23T18:20:44.034388+0000","last_active":"2023-06-23T18:20:44.438495+0000","last_peered":"2023-06-23T18:20:44.438495+0000","last_clean":"2023-06-23T18:20:44.438495+0000","last_became_active":"2023-06-23T18:20:44.034067+0000","last_became_peered":"2023-06-23T18:20:44.034067+0000","last_unstale":"2023-06-23T18:20:44.438495+0000","last_undegraded":"2023-06-23T18:20:44.438495+0000","last_fullsized":"2023-06-23T18:20:44.438495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:55:24.190397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.438998+0000","last_change":"2023-06-23T18:20:44.453280+0000","last_active":"2023-06-23T18:20:45.438998+0000","last_peered":"2023-06-23T18:20:45.438998+0000","last_clean":"2023-06-23T18:20:45.438998+0000","last_became_active":"2023-06-23T18:20:44.452980+0000","last_became_peered":"2023-06-23T18:20:44.452980+0000","last_unstale":"2023-06-23T18:20:45.438998+0000","last_undegraded":"2023-06-23T18:20:45.438998+0000","last_fullsized":"2023-06-23T18:20:45.438998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:24:40.346057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439241+0000","last_change":"2023-06-23T18:20:44.036165+0000","last_active":"2023-06-23T18:20:45.439241+0000","last_peered":"2023-06-23T18:20:45.439241+0000","last_clean":"2023-06-23T18:20:45.439241+0000","last_became_active":"2023-06-23T18:20:44.035313+0000","last_became_peered":"2023-06-23T18:20:44.035313+0000","last_unstale":"2023-06-23T18:20:45.439241+0000","last_undegraded":"2023-06-23T18:20:45.439241+0000","last_fullsized":"2023-06-23T18:20:45.439241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:26:02.622257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.528993+0000","last_change":"2023-06-23T18:20:44.468358+0000","last_active":"2023-06-23T18:20:45.528993+0000","last_peered":"2023-06-23T18:20:45.528993+0000","last_clean":"2023-06-23T18:20:45.528993+0000","last_became_active":"2023-06-23T18:20:44.468032+0000","last_became_peered":"2023-06-23T18:20:44.468032+0000","last_unstale":"2023-06-23T18:20:45.528993+0000","last_undegraded":"2023-06-23T18:20:45.528993+0000","last_fullsized":"2023-06-23T18:20:45.528993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:31.361443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905449+0000","last_change":"2023-06-23T18:20:44.037921+0000","last_active":"2023-06-23T18:20:45.905449+0000","last_peered":"2023-06-23T18:20:45.905449+0000","last_clean":"2023-06-23T18:20:45.905449+0000","last_became_active":"2023-06-23T18:20:44.037571+0000","last_became_peered":"2023-06-23T18:20:44.037571+0000","last_unstale":"2023-06-23T18:20:45.905449+0000","last_undegraded":"2023-06-23T18:20:45.905449+0000","last_fullsized":"2023-06-23T18:20:45.905449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:55.796755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406218+0000","last_change":"2023-06-23T18:20:44.460897+0000","last_active":"2023-06-23T18:20:46.406218+0000","last_peered":"2023-06-23T18:20:46.406218+0000","last_clean":"2023-06-23T18:20:46.406218+0000","last_became_active":"2023-06-23T18:20:44.460476+0000","last_became_peered":"2023-06-23T18:20:44.460476+0000","last_unstale":"2023-06-23T18:20:46.406218+0000","last_undegraded":"2023-06-23T18:20:46.406218+0000","last_fullsized":"2023-06-23T18:20:46.406218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:43.488229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439412+0000","last_change":"2023-06-23T18:20:44.033013+0000","last_active":"2023-06-23T18:20:45.439412+0000","last_peered":"2023-06-23T18:20:45.439412+0000","last_clean":"2023-06-23T18:20:45.439412+0000","last_became_active":"2023-06-23T18:20:44.032580+0000","last_became_peered":"2023-06-23T18:20:44.032580+0000","last_unstale":"2023-06-23T18:20:45.439412+0000","last_undegraded":"2023-06-23T18:20:45.439412+0000","last_fullsized":"2023-06-23T18:20:45.439412+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:32:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.464625+0000","last_change":"2023-06-23T18:20:44.463585+0000","last_active":"2023-06-23T18:20:44.464625+0000","last_peered":"2023-06-23T18:20:44.464625+0000","last_clean":"2023-06-23T18:20:44.464625+0000","last_became_active":"2023-06-23T18:20:44.462512+0000","last_became_peered":"2023-06-23T18:20:44.462512+0000","last_unstale":"2023-06-23T18:20:44.464625+0000","last_undegraded":"2023-06-23T18:20:44.464625+0000","last_fullsized":"2023-06-23T18:20:44.464625+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:14:20.179173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497460+0000","last_change":"2023-06-23T18:20:44.037134+0000","last_active":"2023-06-23T18:20:47.497460+0000","last_peered":"2023-06-23T18:20:47.497460+0000","last_clean":"2023-06-23T18:20:47.497460+0000","last_became_active":"2023-06-23T18:20:44.036536+0000","last_became_peered":"2023-06-23T18:20:44.036536+0000","last_unstale":"2023-06-23T18:20:47.497460+0000","last_undegraded":"2023-06-23T18:20:47.497460+0000","last_fullsized":"2023-06-23T18:20:47.497460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:58:25.025236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406029+0000","last_change":"2023-06-23T18:20:44.454453+0000","last_active":"2023-06-23T18:20:46.406029+0000","last_peered":"2023-06-23T18:20:46.406029+0000","last_clean":"2023-06-23T18:20:46.406029+0000","last_became_active":"2023-06-23T18:20:44.454133+0000","last_became_peered":"2023-06-23T18:20:44.454133+0000","last_unstale":"2023-06-23T18:20:46.406029+0000","last_undegraded":"2023-06-23T18:20:46.406029+0000","last_fullsized":"2023-06-23T18:20:46.406029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:52:02.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905447+0000","last_change":"2023-06-23T18:20:44.042622+0000","last_active":"2023-06-23T18:20:45.905447+0000","last_peered":"2023-06-23T18:20:45.905447+0000","last_clean":"2023-06-23T18:20:45.905447+0000","last_became_active":"2023-06-23T18:20:44.042330+0000","last_became_peered":"2023-06-23T18:20:44.042330+0000","last_unstale":"2023-06-23T18:20:45.905447+0000","last_undegraded":"2023-06-23T18:20:45.905447+0000","last_fullsized":"2023-06-23T18:20:45.905447+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:23.320076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.531053+0000","last_change":"2023-06-23T18:20:44.456558+0000","last_active":"2023-06-23T18:20:45.531053+0000","last_peered":"2023-06-23T18:20:45.531053+0000","last_clean":"2023-06-23T18:20:45.531053+0000","last_became_active":"2023-06-23T18:20:44.456199+0000","last_became_peered":"2023-06-23T18:20:44.456199+0000","last_unstale":"2023-06-23T18:20:45.531053+0000","last_undegraded":"2023-06-23T18:20:45.531053+0000","last_fullsized":"2023-06-23T18:20:45.531053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:24:13.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499315+0000","last_change":"2023-06-23T18:20:44.034101+0000","last_active":"2023-06-23T18:20:48.499315+0000","last_peered":"2023-06-23T18:20:48.499315+0000","last_clean":"2023-06-23T18:20:48.499315+0000","last_became_active":"2023-06-23T18:20:44.033287+0000","last_became_peered":"2023-06-23T18:20:44.033287+0000","last_unstale":"2023-06-23T18:20:48.499315+0000","last_undegraded":"2023-06-23T18:20:48.499315+0000","last_fullsized":"2023-06-23T18:20:48.499315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:08:52.512924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.448242+0000","last_change":"2023-06-23T18:20:44.446298+0000","last_active":"2023-06-23T18:20:44.448242+0000","last_peered":"2023-06-23T18:20:44.448242+0000","last_clean":"2023-06-23T18:20:44.448242+0000","last_became_active":"2023-06-23T18:20:44.445922+0000","last_became_peered":"2023-06-23T18:20:44.445922+0000","last_unstale":"2023-06-23T18:20:44.448242+0000","last_undegraded":"2023-06-23T18:20:44.448242+0000","last_fullsized":"2023-06-23T18:20:44.448242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:25:35.069099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498820+0000","last_change":"2023-06-23T18:20:44.035791+0000","last_active":"2023-06-23T18:20:47.498820+0000","last_peered":"2023-06-23T18:20:47.498820+0000","last_clean":"2023-06-23T18:20:47.498820+0000","last_became_active":"2023-06-23T18:20:44.035416+0000","last_became_peered":"2023-06-23T18:20:44.035416+0000","last_unstale":"2023-06-23T18:20:47.498820+0000","last_undegraded":"2023-06-23T18:20:47.498820+0000","last_fullsized":"2023-06-23T18:20:47.498820+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:02:31.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439429+0000","last_change":"2023-06-23T18:20:44.455474+0000","last_active":"2023-06-23T18:20:45.439429+0000","last_peered":"2023-06-23T18:20:45.439429+0000","last_clean":"2023-06-23T18:20:45.439429+0000","last_became_active":"2023-06-23T18:20:44.455098+0000","last_became_peered":"2023-06-23T18:20:44.455098+0000","last_unstale":"2023-06-23T18:20:45.439429+0000","last_undegraded":"2023-06-23T18:20:45.439429+0000","last_fullsized":"2023-06-23T18:20:45.439429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:53:34.377017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.365945+0000","last_change":"2023-06-23T18:20:44.030305+0000","last_active":"2023-06-23T18:20:46.365945+0000","last_peered":"2023-06-23T18:20:46.365945+0000","last_clean":"2023-06-23T18:20:46.365945+0000","last_became_active":"2023-06-23T18:20:44.029898+0000","last_became_peered":"2023-06-23T18:20:44.029898+0000","last_unstale":"2023-06-23T18:20:46.365945+0000","last_undegraded":"2023-06-23T18:20:46.365945+0000","last_fullsized":"2023-06-23T18:20:46.365945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:38:03.187411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439068+0000","last_change":"2023-06-23T18:20:44.451321+0000","last_active":"2023-06-23T18:20:45.439068+0000","last_peered":"2023-06-23T18:20:45.439068+0000","last_clean":"2023-06-23T18:20:45.439068+0000","last_became_active":"2023-06-23T18:20:44.450549+0000","last_became_peered":"2023-06-23T18:20:44.450549+0000","last_unstale":"2023-06-23T18:20:45.439068+0000","last_undegraded":"2023-06-23T18:20:45.439068+0000","last_fullsized":"2023-06-23T18:20:45.439068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:44:56.410567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.503174+0000","last_change":"2023-06-23T18:20:44.044707+0000","last_active":"2023-06-23T18:20:48.503174+0000","last_peered":"2023-06-23T18:20:48.503174+0000","last_clean":"2023-06-23T18:20:48.503174+0000","last_became_active":"2023-06-23T18:20:44.044389+0000","last_became_peered":"2023-06-23T18:20:44.044389+0000","last_unstale":"2023-06-23T18:20:48.503174+0000","last_undegraded":"2023-06-23T18:20:48.503174+0000","last_fullsized":"2023-06-23T18:20:48.503174+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:45:51.078124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439091+0000","last_change":"2023-06-23T18:20:44.463637+0000","last_active":"2023-06-23T18:20:45.439091+0000","last_peered":"2023-06-23T18:20:45.439091+0000","last_clean":"2023-06-23T18:20:45.439091+0000","last_became_active":"2023-06-23T18:20:44.463289+0000","last_became_peered":"2023-06-23T18:20:44.463289+0000","last_unstale":"2023-06-23T18:20:45.439091+0000","last_undegraded":"2023-06-23T18:20:45.439091+0000","last_fullsized":"2023-06-23T18:20:45.439091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:24:04.923988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.531116+0000","last_change":"2023-06-23T18:20:44.038961+0000","last_active":"2023-06-23T18:20:45.531116+0000","last_peered":"2023-06-23T18:20:45.531116+0000","last_clean":"2023-06-23T18:20:45.531116+0000","last_became_active":"2023-06-23T18:20:44.038674+0000","last_became_peered":"2023-06-23T18:20:44.038674+0000","last_unstale":"2023-06-23T18:20:45.531116+0000","last_undegraded":"2023-06-23T18:20:45.531116+0000","last_fullsized":"2023-06-23T18:20:45.531116+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:18:04.288041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406228+0000","last_change":"2023-06-23T18:20:44.461077+0000","last_active":"2023-06-23T18:20:46.406228+0000","last_peered":"2023-06-23T18:20:46.406228+0000","last_clean":"2023-06-23T18:20:46.406228+0000","last_became_active":"2023-06-23T18:20:44.460767+0000","last_became_peered":"2023-06-23T18:20:44.460767+0000","last_unstale":"2023-06-23T18:20:46.406228+0000","last_undegraded":"2023-06-23T18:20:46.406228+0000","last_fullsized":"2023-06-23T18:20:46.406228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:16.534724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.529039+0000","last_change":"2023-06-23T18:20:44.042114+0000","last_active":"2023-06-23T18:20:45.529039+0000","last_peered":"2023-06-23T18:20:45.529039+0000","last_clean":"2023-06-23T18:20:45.529039+0000","last_became_active":"2023-06-23T18:20:44.041822+0000","last_became_peered":"2023-06-23T18:20:44.041822+0000","last_unstale":"2023-06-23T18:20:45.529039+0000","last_undegraded":"2023-06-23T18:20:45.529039+0000","last_fullsized":"2023-06-23T18:20:45.529039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:36:53.587200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905482+0000","last_change":"2023-06-23T18:20:44.452903+0000","last_active":"2023-06-23T18:20:45.905482+0000","last_peered":"2023-06-23T18:20:45.905482+0000","last_clean":"2023-06-23T18:20:45.905482+0000","last_became_active":"2023-06-23T18:20:44.452539+0000","last_became_peered":"2023-06-23T18:20:44.452539+0000","last_unstale":"2023-06-23T18:20:45.905482+0000","last_undegraded":"2023-06-23T18:20:45.905482+0000","last_fullsized":"2023-06-23T18:20:45.905482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:53:53.970572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439260+0000","last_change":"2023-06-23T18:20:44.032905+0000","last_active":"2023-06-23T18:20:45.439260+0000","last_peered":"2023-06-23T18:20:45.439260+0000","last_clean":"2023-06-23T18:20:45.439260+0000","last_became_active":"2023-06-23T18:20:44.032606+0000","last_became_peered":"2023-06-23T18:20:44.032606+0000","last_unstale":"2023-06-23T18:20:45.439260+0000","last_undegraded":"2023-06-23T18:20:45.439260+0000","last_fullsized":"2023-06-23T18:20:45.439260+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:05.992912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905273+0000","last_change":"2023-06-23T18:20:44.465580+0000","last_active":"2023-06-23T18:20:45.905273+0000","last_peered":"2023-06-23T18:20:45.905273+0000","last_clean":"2023-06-23T18:20:45.905273+0000","last_became_active":"2023-06-23T18:20:44.465264+0000","last_became_peered":"2023-06-23T18:20:44.465264+0000","last_unstale":"2023-06-23T18:20:45.905273+0000","last_undegraded":"2023-06-23T18:20:45.905273+0000","last_fullsized":"2023-06-23T18:20:45.905273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:20:07.333937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439575+0000","last_change":"2023-06-23T18:20:44.464324+0000","last_active":"2023-06-23T18:20:45.439575+0000","last_peered":"2023-06-23T18:20:45.439575+0000","last_clean":"2023-06-23T18:20:45.439575+0000","last_became_active":"2023-06-23T18:20:44.464021+0000","last_became_peered":"2023-06-23T18:20:44.464021+0000","last_unstale":"2023-06-23T18:20:45.439575+0000","last_undegraded":"2023-06-23T18:20:45.439575+0000","last_fullsized":"2023-06-23T18:20:45.439575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:24:15.831014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.437868+0000","last_change":"2023-06-23T18:20:44.034309+0000","last_active":"2023-06-23T18:20:44.437868+0000","last_peered":"2023-06-23T18:20:44.437868+0000","last_clean":"2023-06-23T18:20:44.437868+0000","last_became_active":"2023-06-23T18:20:44.034007+0000","last_became_peered":"2023-06-23T18:20:44.034007+0000","last_unstale":"2023-06-23T18:20:44.437868+0000","last_undegraded":"2023-06-23T18:20:44.437868+0000","last_fullsized":"2023-06-23T18:20:44.437868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:00:07.874785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439359+0000","last_change":"2023-06-23T18:20:44.468220+0000","last_active":"2023-06-23T18:20:45.439359+0000","last_peered":"2023-06-23T18:20:45.439359+0000","last_clean":"2023-06-23T18:20:45.439359+0000","last_became_active":"2023-06-23T18:20:44.467725+0000","last_became_peered":"2023-06-23T18:20:44.467725+0000","last_unstale":"2023-06-23T18:20:45.439359+0000","last_undegraded":"2023-06-23T18:20:45.439359+0000","last_fullsized":"2023-06-23T18:20:45.439359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:30.069575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498601+0000","last_change":"2023-06-23T18:20:44.030921+0000","last_active":"2023-06-23T18:20:47.498601+0000","last_peered":"2023-06-23T18:20:47.498601+0000","last_clean":"2023-06-23T18:20:47.498601+0000","last_became_active":"2023-06-23T18:20:44.030362+0000","last_became_peered":"2023-06-23T18:20:44.030362+0000","last_unstale":"2023-06-23T18:20:47.498601+0000","last_undegraded":"2023-06-23T18:20:47.498601+0000","last_fullsized":"2023-06-23T18:20:47.498601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:07:10.000171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":328,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.438999+0000","last_change":"2023-06-23T18:19:44.952262+0000","last_active":"2023-06-23T18:20:45.438999+0000","last_peered":"2023-06-23T18:20:45.438999+0000","last_clean":"2023-06-23T18:20:45.438999+0000","last_became_active":"2023-06-23T18:19:44.951803+0000","last_became_peered":"2023-06-23T18:19:44.951803+0000","last_unstale":"2023-06-23T18:20:45.438999+0000","last_undegraded":"2023-06-23T18:20:45.438999+0000","last_fullsized":"2023-06-23T18:20:45.438999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_clean_scrub_stamp":"2023-06-23T18:16:49.458922+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:49.981569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.457951+0000","last_change":"2023-06-23T18:20:44.456602+0000","last_active":"2023-06-23T18:20:44.457951+0000","last_peered":"2023-06-23T18:20:44.457951+0000","last_clean":"2023-06-23T18:20:44.457951+0000","last_became_active":"2023-06-23T18:20:44.456249+0000","last_became_peered":"2023-06-23T18:20:44.456249+0000","last_unstale":"2023-06-23T18:20:44.457951+0000","last_undegraded":"2023-06-23T18:20:44.457951+0000","last_fullsized":"2023-06-23T18:20:44.457951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:29:59.524147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439445+0000","last_change":"2023-06-23T18:20:44.039100+0000","last_active":"2023-06-23T18:20:45.439445+0000","last_peered":"2023-06-23T18:20:45.439445+0000","last_clean":"2023-06-23T18:20:45.439445+0000","last_became_active":"2023-06-23T18:20:44.038801+0000","last_became_peered":"2023-06-23T18:20:44.038801+0000","last_unstale":"2023-06-23T18:20:45.439445+0000","last_undegraded":"2023-06-23T18:20:45.439445+0000","last_fullsized":"2023-06-23T18:20:45.439445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:03.284504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.531051+0000","last_change":"2023-06-23T18:20:44.038037+0000","last_active":"2023-06-23T18:20:45.531051+0000","last_peered":"2023-06-23T18:20:45.531051+0000","last_clean":"2023-06-23T18:20:45.531051+0000","last_became_active":"2023-06-23T18:20:44.037732+0000","last_became_peered":"2023-06-23T18:20:44.037732+0000","last_unstale":"2023-06-23T18:20:45.531051+0000","last_undegraded":"2023-06-23T18:20:45.531051+0000","last_fullsized":"2023-06-23T18:20:45.531051+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:30:58.279148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.460237+0000","last_change":"2023-06-23T18:20:44.459927+0000","last_active":"2023-06-23T18:20:44.460237+0000","last_peered":"2023-06-23T18:20:44.460237+0000","last_clean":"2023-06-23T18:20:44.460237+0000","last_became_active":"2023-06-23T18:20:44.459594+0000","last_became_peered":"2023-06-23T18:20:44.459594+0000","last_unstale":"2023-06-23T18:20:44.460237+0000","last_undegraded":"2023-06-23T18:20:44.460237+0000","last_fullsized":"2023-06-23T18:20:44.460237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:27:30.055435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499658+0000","last_change":"2023-06-23T18:20:44.039429+0000","last_active":"2023-06-23T18:20:48.499658+0000","last_peered":"2023-06-23T18:20:48.499658+0000","last_clean":"2023-06-23T18:20:48.499658+0000","last_became_active":"2023-06-23T18:20:44.038944+0000","last_became_peered":"2023-06-23T18:20:44.038944+0000","last_unstale":"2023-06-23T18:20:48.499658+0000","last_undegraded":"2023-06-23T18:20:48.499658+0000","last_fullsized":"2023-06-23T18:20:48.499658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:37:39.247393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439805+0000","last_change":"2023-06-23T18:20:44.458227+0000","last_active":"2023-06-23T18:20:45.439805+0000","last_peered":"2023-06-23T18:20:45.439805+0000","last_clean":"2023-06-23T18:20:45.439805+0000","last_became_active":"2023-06-23T18:20:44.457904+0000","last_became_peered":"2023-06-23T18:20:44.457904+0000","last_unstale":"2023-06-23T18:20:45.439805+0000","last_undegraded":"2023-06-23T18:20:45.439805+0000","last_fullsized":"2023-06-23T18:20:45.439805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:53:21.724545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.500253+0000","last_change":"2023-06-23T18:20:44.034089+0000","last_active":"2023-06-23T18:20:47.500253+0000","last_peered":"2023-06-23T18:20:47.500253+0000","last_clean":"2023-06-23T18:20:47.500253+0000","last_became_active":"2023-06-23T18:20:44.033745+0000","last_became_peered":"2023-06-23T18:20:44.033745+0000","last_unstale":"2023-06-23T18:20:47.500253+0000","last_undegraded":"2023-06-23T18:20:47.500253+0000","last_fullsized":"2023-06-23T18:20:47.500253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:05.244917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.469931+0000","last_change":"2023-06-23T18:20:44.468622+0000","last_active":"2023-06-23T18:20:44.469931+0000","last_peered":"2023-06-23T18:20:44.469931+0000","last_clean":"2023-06-23T18:20:44.469931+0000","last_became_active":"2023-06-23T18:20:44.466831+0000","last_became_peered":"2023-06-23T18:20:44.466831+0000","last_unstale":"2023-06-23T18:20:44.469931+0000","last_undegraded":"2023-06-23T18:20:44.469931+0000","last_fullsized":"2023-06-23T18:20:44.469931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:35:13.768286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497829+0000","last_change":"2023-06-23T18:20:44.030647+0000","last_active":"2023-06-23T18:20:47.497829+0000","last_peered":"2023-06-23T18:20:47.497829+0000","last_clean":"2023-06-23T18:20:47.497829+0000","last_became_active":"2023-06-23T18:20:44.030316+0000","last_became_peered":"2023-06-23T18:20:44.030316+0000","last_unstale":"2023-06-23T18:20:47.497829+0000","last_undegraded":"2023-06-23T18:20:47.497829+0000","last_fullsized":"2023-06-23T18:20:47.497829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:47.369435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530780+0000","last_change":"2023-06-23T18:20:44.458365+0000","last_active":"2023-06-23T18:20:45.530780+0000","last_peered":"2023-06-23T18:20:45.530780+0000","last_clean":"2023-06-23T18:20:45.530780+0000","last_became_active":"2023-06-23T18:20:44.458039+0000","last_became_peered":"2023-06-23T18:20:44.458039+0000","last_unstale":"2023-06-23T18:20:45.530780+0000","last_undegraded":"2023-06-23T18:20:45.530780+0000","last_fullsized":"2023-06-23T18:20:45.530780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:03:49.379689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439310+0000","last_change":"2023-06-23T18:20:44.028908+0000","last_active":"2023-06-23T18:20:45.439310+0000","last_peered":"2023-06-23T18:20:45.439310+0000","last_clean":"2023-06-23T18:20:45.439310+0000","last_became_active":"2023-06-23T18:20:44.028551+0000","last_became_peered":"2023-06-23T18:20:44.028551+0000","last_unstale":"2023-06-23T18:20:45.439310+0000","last_undegraded":"2023-06-23T18:20:45.439310+0000","last_fullsized":"2023-06-23T18:20:45.439310+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:31:41.576378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406188+0000","last_change":"2023-06-23T18:20:44.465300+0000","last_active":"2023-06-23T18:20:46.406188+0000","last_peered":"2023-06-23T18:20:46.406188+0000","last_clean":"2023-06-23T18:20:46.406188+0000","last_became_active":"2023-06-23T18:20:44.465008+0000","last_became_peered":"2023-06-23T18:20:44.465008+0000","last_unstale":"2023-06-23T18:20:46.406188+0000","last_undegraded":"2023-06-23T18:20:46.406188+0000","last_fullsized":"2023-06-23T18:20:46.406188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:03:27.021563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237528+0000","last_change":"2023-06-23T18:20:44.041593+0000","last_active":"2023-06-23T18:20:46.237528+0000","last_peered":"2023-06-23T18:20:46.237528+0000","last_clean":"2023-06-23T18:20:46.237528+0000","last_became_active":"2023-06-23T18:20:44.041283+0000","last_became_peered":"2023-06-23T18:20:44.041283+0000","last_unstale":"2023-06-23T18:20:46.237528+0000","last_undegraded":"2023-06-23T18:20:46.237528+0000","last_fullsized":"2023-06-23T18:20:46.237528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:45:29.030061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439210+0000","last_change":"2023-06-23T18:20:44.465127+0000","last_active":"2023-06-23T18:20:45.439210+0000","last_peered":"2023-06-23T18:20:45.439210+0000","last_clean":"2023-06-23T18:20:45.439210+0000","last_became_active":"2023-06-23T18:20:44.464768+0000","last_became_peered":"2023-06-23T18:20:44.464768+0000","last_unstale":"2023-06-23T18:20:45.439210+0000","last_undegraded":"2023-06-23T18:20:45.439210+0000","last_fullsized":"2023-06-23T18:20:45.439210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:54:38.343097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.495850+0000","last_change":"2023-06-23T18:20:44.046245+0000","last_active":"2023-06-23T18:20:47.495850+0000","last_peered":"2023-06-23T18:20:47.495850+0000","last_clean":"2023-06-23T18:20:47.495850+0000","last_became_active":"2023-06-23T18:20:44.045945+0000","last_became_peered":"2023-06-23T18:20:44.045945+0000","last_unstale":"2023-06-23T18:20:47.495850+0000","last_undegraded":"2023-06-23T18:20:47.495850+0000","last_fullsized":"2023-06-23T18:20:47.495850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:13:06.840364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439547+0000","last_change":"2023-06-23T18:20:44.453520+0000","last_active":"2023-06-23T18:20:45.439547+0000","last_peered":"2023-06-23T18:20:45.439547+0000","last_clean":"2023-06-23T18:20:45.439547+0000","last_became_active":"2023-06-23T18:20:44.453214+0000","last_became_peered":"2023-06-23T18:20:44.453214+0000","last_unstale":"2023-06-23T18:20:45.439547+0000","last_undegraded":"2023-06-23T18:20:45.439547+0000","last_fullsized":"2023-06-23T18:20:45.439547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:47.302353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.502980+0000","last_change":"2023-06-23T18:20:44.042526+0000","last_active":"2023-06-23T18:20:48.502980+0000","last_peered":"2023-06-23T18:20:48.502980+0000","last_clean":"2023-06-23T18:20:48.502980+0000","last_became_active":"2023-06-23T18:20:44.042249+0000","last_became_peered":"2023-06-23T18:20:44.042249+0000","last_unstale":"2023-06-23T18:20:48.502980+0000","last_undegraded":"2023-06-23T18:20:48.502980+0000","last_fullsized":"2023-06-23T18:20:48.502980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:31:39.378767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439873+0000","last_change":"2023-06-23T18:20:44.454645+0000","last_active":"2023-06-23T18:20:45.439873+0000","last_peered":"2023-06-23T18:20:45.439873+0000","last_clean":"2023-06-23T18:20:45.439873+0000","last_became_active":"2023-06-23T18:20:44.453309+0000","last_became_peered":"2023-06-23T18:20:44.453309+0000","last_unstale":"2023-06-23T18:20:45.439873+0000","last_undegraded":"2023-06-23T18:20:45.439873+0000","last_fullsized":"2023-06-23T18:20:45.439873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:30:55.635983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439463+0000","last_change":"2023-06-23T18:20:44.037681+0000","last_active":"2023-06-23T18:20:45.439463+0000","last_peered":"2023-06-23T18:20:45.439463+0000","last_clean":"2023-06-23T18:20:45.439463+0000","last_became_active":"2023-06-23T18:20:44.037323+0000","last_became_peered":"2023-06-23T18:20:44.037323+0000","last_unstale":"2023-06-23T18:20:45.439463+0000","last_undegraded":"2023-06-23T18:20:45.439463+0000","last_fullsized":"2023-06-23T18:20:45.439463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:55:26.308777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439284+0000","last_change":"2023-06-23T18:20:44.456434+0000","last_active":"2023-06-23T18:20:45.439284+0000","last_peered":"2023-06-23T18:20:45.439284+0000","last_clean":"2023-06-23T18:20:45.439284+0000","last_became_active":"2023-06-23T18:20:44.456110+0000","last_became_peered":"2023-06-23T18:20:44.456110+0000","last_unstale":"2023-06-23T18:20:45.439284+0000","last_undegraded":"2023-06-23T18:20:45.439284+0000","last_fullsized":"2023-06-23T18:20:45.439284+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:00:04.366221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.469319+0000","last_change":"2023-06-23T18:20:44.040722+0000","last_active":"2023-06-23T18:20:44.469319+0000","last_peered":"2023-06-23T18:20:44.469319+0000","last_clean":"2023-06-23T18:20:44.469319+0000","last_became_active":"2023-06-23T18:20:44.040390+0000","last_became_peered":"2023-06-23T18:20:44.040390+0000","last_unstale":"2023-06-23T18:20:44.469319+0000","last_undegraded":"2023-06-23T18:20:44.469319+0000","last_fullsized":"2023-06-23T18:20:44.469319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:30:30.180370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439334+0000","last_change":"2023-06-23T18:20:44.455559+0000","last_active":"2023-06-23T18:20:45.439334+0000","last_peered":"2023-06-23T18:20:45.439334+0000","last_clean":"2023-06-23T18:20:45.439334+0000","last_became_active":"2023-06-23T18:20:44.455256+0000","last_became_peered":"2023-06-23T18:20:44.455256+0000","last_unstale":"2023-06-23T18:20:45.439334+0000","last_undegraded":"2023-06-23T18:20:45.439334+0000","last_fullsized":"2023-06-23T18:20:45.439334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:40:30.169440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406131+0000","last_change":"2023-06-23T18:20:44.460403+0000","last_active":"2023-06-23T18:20:46.406131+0000","last_peered":"2023-06-23T18:20:46.406131+0000","last_clean":"2023-06-23T18:20:46.406131+0000","last_became_active":"2023-06-23T18:20:44.459928+0000","last_became_peered":"2023-06-23T18:20:44.459928+0000","last_unstale":"2023-06-23T18:20:46.406131+0000","last_undegraded":"2023-06-23T18:20:46.406131+0000","last_fullsized":"2023-06-23T18:20:46.406131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:58:06.385995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237624+0000","last_change":"2023-06-23T18:20:44.458050+0000","last_active":"2023-06-23T18:20:46.237624+0000","last_peered":"2023-06-23T18:20:46.237624+0000","last_clean":"2023-06-23T18:20:46.237624+0000","last_became_active":"2023-06-23T18:20:44.457733+0000","last_became_peered":"2023-06-23T18:20:44.457733+0000","last_unstale":"2023-06-23T18:20:46.237624+0000","last_undegraded":"2023-06-23T18:20:46.237624+0000","last_fullsized":"2023-06-23T18:20:46.237624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:23:24.448430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237525+0000","last_change":"2023-06-23T18:20:44.456233+0000","last_active":"2023-06-23T18:20:46.237525+0000","last_peered":"2023-06-23T18:20:46.237525+0000","last_clean":"2023-06-23T18:20:46.237525+0000","last_became_active":"2023-06-23T18:20:44.455677+0000","last_became_peered":"2023-06-23T18:20:44.455677+0000","last_unstale":"2023-06-23T18:20:46.237525+0000","last_undegraded":"2023-06-23T18:20:46.237525+0000","last_fullsized":"2023-06-23T18:20:46.237525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:54:14.083530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.470345+0000","last_change":"2023-06-23T18:20:44.469668+0000","last_active":"2023-06-23T18:20:44.470345+0000","last_peered":"2023-06-23T18:20:44.470345+0000","last_clean":"2023-06-23T18:20:44.470345+0000","last_became_active":"2023-06-23T18:20:44.466563+0000","last_became_peered":"2023-06-23T18:20:44.466563+0000","last_unstale":"2023-06-23T18:20:44.470345+0000","last_undegraded":"2023-06-23T18:20:44.470345+0000","last_fullsized":"2023-06-23T18:20:44.470345+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:15:40.824612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.457800+0000","last_change":"2023-06-23T18:20:44.456021+0000","last_active":"2023-06-23T18:20:44.457800+0000","last_peered":"2023-06-23T18:20:44.457800+0000","last_clean":"2023-06-23T18:20:44.457800+0000","last_became_active":"2023-06-23T18:20:44.455705+0000","last_became_peered":"2023-06-23T18:20:44.455705+0000","last_unstale":"2023-06-23T18:20:44.457800+0000","last_undegraded":"2023-06-23T18:20:44.457800+0000","last_fullsized":"2023-06-23T18:20:44.457800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:13:32.159183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406226+0000","last_change":"2023-06-23T18:20:44.460359+0000","last_active":"2023-06-23T18:20:46.406226+0000","last_peered":"2023-06-23T18:20:46.406226+0000","last_clean":"2023-06-23T18:20:46.406226+0000","last_became_active":"2023-06-23T18:20:44.460034+0000","last_became_peered":"2023-06-23T18:20:44.460034+0000","last_unstale":"2023-06-23T18:20:46.406226+0000","last_undegraded":"2023-06-23T18:20:46.406226+0000","last_fullsized":"2023-06-23T18:20:46.406226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:00:06.777366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237875+0000","last_change":"2023-06-23T18:20:44.455743+0000","last_active":"2023-06-23T18:20:46.237875+0000","last_peered":"2023-06-23T18:20:46.237875+0000","last_clean":"2023-06-23T18:20:46.237875+0000","last_became_active":"2023-06-23T18:20:44.455266+0000","last_became_peered":"2023-06-23T18:20:44.455266+0000","last_unstale":"2023-06-23T18:20:46.237875+0000","last_undegraded":"2023-06-23T18:20:46.237875+0000","last_fullsized":"2023-06-23T18:20:46.237875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:33:52.673181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439651+0000","last_change":"2023-06-23T18:20:44.461434+0000","last_active":"2023-06-23T18:20:45.439651+0000","last_peered":"2023-06-23T18:20:45.439651+0000","last_clean":"2023-06-23T18:20:45.439651+0000","last_became_active":"2023-06-23T18:20:44.461137+0000","last_became_peered":"2023-06-23T18:20:44.461137+0000","last_unstale":"2023-06-23T18:20:45.439651+0000","last_undegraded":"2023-06-23T18:20:45.439651+0000","last_fullsized":"2023-06-23T18:20:45.439651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:14:08.293656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439086+0000","last_change":"2023-06-23T18:20:44.465192+0000","last_active":"2023-06-23T18:20:45.439086+0000","last_peered":"2023-06-23T18:20:45.439086+0000","last_clean":"2023-06-23T18:20:45.439086+0000","last_became_active":"2023-06-23T18:20:44.464905+0000","last_became_peered":"2023-06-23T18:20:44.464905+0000","last_unstale":"2023-06-23T18:20:45.439086+0000","last_undegraded":"2023-06-23T18:20:45.439086+0000","last_fullsized":"2023-06-23T18:20:45.439086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:01:32.627765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439130+0000","last_change":"2023-06-23T18:20:44.458725+0000","last_active":"2023-06-23T18:20:45.439130+0000","last_peered":"2023-06-23T18:20:45.439130+0000","last_clean":"2023-06-23T18:20:45.439130+0000","last_became_active":"2023-06-23T18:20:44.458414+0000","last_became_peered":"2023-06-23T18:20:44.458414+0000","last_unstale":"2023-06-23T18:20:45.439130+0000","last_undegraded":"2023-06-23T18:20:45.439130+0000","last_fullsized":"2023-06-23T18:20:45.439130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:36:29.121354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405885+0000","last_change":"2023-06-23T18:20:44.465665+0000","last_active":"2023-06-23T18:20:46.405885+0000","last_peered":"2023-06-23T18:20:46.405885+0000","last_clean":"2023-06-23T18:20:46.405885+0000","last_became_active":"2023-06-23T18:20:44.465337+0000","last_became_peered":"2023-06-23T18:20:44.465337+0000","last_unstale":"2023-06-23T18:20:46.405885+0000","last_undegraded":"2023-06-23T18:20:46.405885+0000","last_fullsized":"2023-06-23T18:20:46.405885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:58:56.987077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-06-23T18:20:44.457850+0000","last_change":"2023-06-23T18:20:44.455107+0000","last_active":"2023-06-23T18:20:44.457850+0000","last_peered":"2023-06-23T18:20:44.457850+0000","last_clean":"2023-06-23T18:20:44.457850+0000","last_became_active":"2023-06-23T18:20:44.454798+0000","last_became_peered":"2023-06-23T18:20:44.454798+0000","last_unstale":"2023-06-23T18:20:44.457850+0000","last_undegraded":"2023-06-23T18:20:44.457850+0000","last_fullsized":"2023-06-23T18:20:44.457850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:31:38.072092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406114+0000","last_change":"2023-06-23T18:20:44.461044+0000","last_active":"2023-06-23T18:20:46.406114+0000","last_peered":"2023-06-23T18:20:46.406114+0000","last_clean":"2023-06-23T18:20:46.406114+0000","last_became_active":"2023-06-23T18:20:44.460737+0000","last_became_peered":"2023-06-23T18:20:44.460737+0000","last_unstale":"2023-06-23T18:20:46.406114+0000","last_undegraded":"2023-06-23T18:20:46.406114+0000","last_fullsized":"2023-06-23T18:20:46.406114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:13:16.912989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281641,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4234,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":8,"num_read_kb":7,"num_write":93,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":29,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":27335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481803,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":856,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":876544,"data_stored":419789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":10,"up_from":70,"seq":300647710736,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":1011915,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":0.625},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.58399999999999996}]},{"osd":5,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.81100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.9430000000000001,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.75}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":9,"up_from":65,"seq":279172874260,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":420607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.374}]},{"osd":1,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.5409999999999999,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.61699999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46999999999999997}]},{"osd":4,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.5099999999999998,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.432}]},{"osd":5,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.69199999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":5.2389999999999999,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.63400000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.53400000000000003}]}]},{"osd":8,"up_from":58,"seq":249108103192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":420084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.439}]},{"osd":1,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.308},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.39000000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.42599999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.38300000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.495},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.505}]},{"osd":7,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.51700000000000002}]},{"osd":9,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52400000000000002}]},{"osd":10,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.55300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332124,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":1010739,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.33300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64100000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.435,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.504},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.84199999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.623},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.54000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.53600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.75,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.56799999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70499999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:20:20 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.76200000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67700000000000005}]}]},{"osd":6,"up_from":46,"seq":197568495648,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":864256,"data_stored":411219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41799999999999998,"15min":0.41399999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.436,"15min":0.43099999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.374}]},{"osd":1,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.43099999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.02,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.82699999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88100000000000001,"5min":1.105,"15min":1.105},"last":0.501}]},{"osd":3,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53500000000000003,"15min":0.53100000000000003},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60199999999999998,"15min":0.59899999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.044,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.70199999999999996,"15min":0.70399999999999996},"min":{"1min":0.53700000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.046,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53600000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.50800000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.58599999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.63200000000000001}]},{"osd":9,"last update":"Fri Jun 23 18:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.621},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.54800000000000004}]},{"osd":10,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67500000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":421353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.376,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82599999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.41999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86499999999999999,"5min":1.218,"15min":1.218},"last":0.48299999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:19:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.54300000000000004,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.317,"15min":0.317},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.37,"5min":0.312,"15min":0.312},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:20:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.56100000000000005,"15min":0.54200000000000004},"min":{"1min":0.45900000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.57399999999999995,"15min":0.54400000000000004},"min":{"1min":0.53200000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45400000000000001,"15min":0.42799999999999999},"min":{"1min":0.38600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.442},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44600000000000001,"15min":0.41799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.35299999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.43099999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.45100000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:20:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.55600000000000005,"15min":0.53600000000000003},"min":{"1min":0.46800000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.62,"15min":0.58999999999999997},"min":{"1min":0.51900000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.53200000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39400000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96099999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.376},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56200000000000006,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0629999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.46899999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.68300000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.503},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.58999999999999997}]},{"osd":10,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.57899999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738426,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27800,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717352,"statfs":{"total":95995035648,"available":95966568448,"internally_reserved":0,"allocated":868352,"data_stored":411309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.40899999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.5,"15min":0.47499999999999998},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44}]},{"osd":2,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.51700000000000002,"15min":0.49199999999999999},"min":{"1min":0.41299999999999998,"5min":0.314,"15min":0.314},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.372,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.624,"5min":0.628,"15min":0.628},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40699999999999997}]},{"osd":4,"last update":"Fri Jun 23 18:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56999999999999995,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53100000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.60199999999999998,"15min":0.58899999999999997},"min":{"1min":0.495,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83999999999999997,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78900000000000003,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Jun 23 18:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.622}]},{"osd":8,"last update":"Fri Jun 23 18:20:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411379,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":422655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.48999999999999999},"min":{"1min":0.34899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.33300000000000002,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.438}]},{"osd":1,"last update":"Fri Jun 23 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52000000000000002,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69599999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.35699999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78000000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.625}]},{"osd":3,"last update":"Fri Jun 23 18:20:14 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55300000000000005,"15min":0.53800000000000003},"min":{"1min":0.311,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.76000000000000001,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.55300000000000005,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.40999999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.52500000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44800000000000001,"15min":0.443},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.626,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.47599999999999998,"15min":0.46600000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42499999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88700000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.623},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.47699999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59799999999999998}]},{"osd":8,"last update":"Fri Jun 23 18:20:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45700000000000002}]},{"osd":9,"last update":"Fri Jun 23 18:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.46700000000000003}]},{"osd":10,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":3,"up_from":25,"seq":107374182444,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27804,"kb_used_data":852,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717348,"statfs":{"total":95995035648,"available":95966564352,"internally_reserved":0,"allocated":872448,"data_stored":411323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.34799999999999998,"15min":0.33100000000000002},"min":{"1min":0.29499999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.436,"5min":0.378,"15min":0.36899999999999999},"min":{"1min":0.28499999999999998,"5min":0.245,"15min":0.245},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.55800000000000005,"15min":0.53400000000000003},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.51600000000000001,"15min":0.51000000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:20:18 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.56200000000000006,"15min":0.54900000000000004},"min":{"1min":0.46600000000000003,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.57399999999999995,"15min":0.54300000000000004},"min":{"1min":0.495,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90800000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67900000000000005}]},{"osd":4,"last update":"Fri Jun 23 18:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63}]},{"osd":5,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.434,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94099999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.46200000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55700000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.53500000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.496},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.81200000000000006}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:20:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58699999999999997}]},{"osd":10,"last update":"Fri Jun 23 18:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.71099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920809,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":419811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.51700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.57799999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.309},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.38300000000000001},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.373}]},{"osd":2,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57799999999999996,"15min":0.54300000000000004},"min":{"1min":0.503,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.67100000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78000000000000003,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.432},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.443}]},{"osd":5,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.60899999999999999,"15min":0.59699999999999998},"min":{"1min":0.47599999999999998,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.60599999999999998,"15min":0.59699999999999998},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.55900000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:20:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58799999999999997,"15min":0.57599999999999996},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.51800000000000002}]},{"osd":7,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.379,"5min":0.378,"15min":0.378},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.48799999999999999}]},{"osd":8,"last update":"Fri Jun 23 18:19:59 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61099999999999999}]},{"osd":9,"last update":"Fri Jun 23 18:20:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.59799999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.65800000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]}]},{"osd":5,"up_from":38,"seq":163208757286,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":1011585,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.46800000000000003,"15min":0.45700000000000002},"min":{"1min":0.34999999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.495,"15min":0.49299999999999999},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.311}]},{"osd":1,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.53000000000000003,"15min":0.52200000000000002},"min":{"1min":0.45800000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.33900000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.41399999999999998,"15min":0.40899999999999997},"min":{"1min":0.32300000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.248},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.436,"15min":0.42799999999999999},"min":{"1min":0.34599999999999997,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.26300000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.40400000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.38500000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.79900000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.58899999999999997,"15min":0.58099999999999996},"min":{"1min":0.40899999999999997,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.872,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.41999999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.56899999999999995}]},{"osd":7,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":0.89800000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.432}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.495},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.41099999999999998}]},{"osd":9,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.59399999999999997}]},{"osd":10,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.47699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":20231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:20:52.220 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-06-23T18:20:52.220 INFO:tasks.cephadm:Setting up client nodes... 2023-06-23T18:20:52.221 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-06-23T18:20:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:52 smithi002 ceph-mon[95947]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-06-23T18:20:52.806 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:20:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:52 smithi082 ceph-mon[97889]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-06-23T18:20:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:52 smithi017 ceph-mon[97472]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-06-23T18:20:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:53 smithi002 ceph-mon[95947]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:20:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:53 smithi002 ceph-mon[95947]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:20:53.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:53 smithi002 ceph-mon[95947]: pgmap v260: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.8 KiB/s wr, 21 op/s 2023-06-23T18:20:53.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:53 smithi082 ceph-mon[97889]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:20:53.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:53 smithi082 ceph-mon[97889]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:20:53.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:53 smithi082 ceph-mon[97889]: pgmap v260: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.8 KiB/s wr, 21 op/s 2023-06-23T18:20:53.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:53 smithi017 ceph-mon[97472]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:20:53.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:53 smithi017 ceph-mon[97472]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:20:53.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:53 smithi017 ceph-mon[97472]: pgmap v260: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 9.8 KiB/s wr, 21 op/s 2023-06-23T18:20:54.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.094+0000 7f882061a700 1 -- 172.21.15.2:0/3905644636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 msgr2=0x7f88181000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:54.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.094+0000 7f882061a700 1 --2- 172.21.15.2:0/3905644636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 0x7f88181000d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8808009990 tx=0x7f8808009ca0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.095+0000 7f882061a700 1 -- 172.21.15.2:0/3905644636 shutdown_connections 2023-06-23T18:20:54.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.095+0000 7f882061a700 1 --2- 172.21.15.2:0/3905644636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8818100610 0x7f8818106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.095+0000 7f882061a700 1 --2- 172.21.15.2:0/3905644636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88180fea80 0x7f88180fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.095+0000 7f882061a700 1 --2- 172.21.15.2:0/3905644636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 0x7f88181000d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.095+0000 7f882061a700 1 -- 172.21.15.2:0/3905644636 >> 172.21.15.2:0/3905644636 conn(0x7f88180fa0d0 msgr2=0x7f88180fc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.095+0000 7f882061a700 1 -- 172.21.15.2:0/3905644636 shutdown_connections 2023-06-23T18:20:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.095+0000 7f882061a700 1 -- 172.21.15.2:0/3905644636 wait complete. 2023-06-23T18:20:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.095+0000 7f882061a700 1 Processor -- start 2023-06-23T18:20:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f882061a700 1 -- start start 2023-06-23T18:20:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f882061a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88180fea80 0x7f88180693a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f882061a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 0x7f881819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f882061a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8818100610 0x7f881819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f882061a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881819d330 con 0x7f88180ffc80 2023-06-23T18:20:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f882061a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881819d4a0 con 0x7f8818100610 2023-06-23T18:20:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f882061a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881819d660 con 0x7f88180fea80 2023-06-23T18:20:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881dbb5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 0x7f881819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881dbb5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 0x7f881819a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52622/0 (socket says 172.21.15.2:52622) 2023-06-23T18:20:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881dbb5700 1 -- 172.21.15.2:0/1163210255 learned_addr learned my addr 172.21.15.2:0/1163210255 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881ebb7700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8818100610 0x7f881819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881e3b6700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88180fea80 0x7f88180693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881dbb5700 1 -- 172.21.15.2:0/1163210255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88180fea80 msgr2=0x7f88180693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881dbb5700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88180fea80 0x7f88180693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881dbb5700 1 -- 172.21.15.2:0/1163210255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8818100610 msgr2=0x7f881819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881dbb5700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8818100610 0x7f881819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881dbb5700 1 -- 172.21.15.2:0/1163210255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88080095f0 con 0x7f88180ffc80 2023-06-23T18:20:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.096+0000 7f881e3b6700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88180fea80 0x7f88180693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.097+0000 7f881dbb5700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 0x7f881819a540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f8808003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.097+0000 7f880f7fe700 1 -- 172.21.15.2:0/1163210255 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880801d070 con 0x7f88180ffc80 2023-06-23T18:20:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.097+0000 7f880f7fe700 1 -- 172.21.15.2:0/1163210255 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8808004640 con 0x7f88180ffc80 2023-06-23T18:20:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.097+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881819d8e0 con 0x7f88180ffc80 2023-06-23T18:20:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.097+0000 7f880f7fe700 1 -- 172.21.15.2:0/1163210255 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880800f460 con 0x7f88180ffc80 2023-06-23T18:20:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.097+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881819ddd0 con 0x7f88180ffc80 2023-06-23T18:20:54.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.098+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f881810a170 con 0x7f88180ffc80 2023-06-23T18:20:54.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.099+0000 7f880f7fe700 1 -- 172.21.15.2:0/1163210255 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8808022470 con 0x7f88180ffc80 2023-06-23T18:20:54.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.100+0000 7f880f7fe700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88040a1260 0x7f88040a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:54.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.100+0000 7f881e3b6700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88040a1260 0x7f88040a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:54.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.100+0000 7f880f7fe700 1 -- 172.21.15.2:0/1163210255 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f88080cdf50 con 0x7f88180ffc80 2023-06-23T18:20:54.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.100+0000 7f881e3b6700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88040a1260 0x7f88040a3710 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88180ffae0 tx=0x7f8814005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:54.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.104+0000 7f880f7fe700 1 -- 172.21.15.2:0/1163210255 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8808097780 con 0x7f88180ffc80 2023-06-23T18:20:54.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.384+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8818105a10 con 0x7f88180ffc80 2023-06-23T18:20:54.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.388+0000 7f880f7fe700 1 -- 172.21.15.2:0/1163210255 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f880809ada0 con 0x7f88180ffc80 2023-06-23T18:20:54.393 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2023-06-23T18:20:54.394 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQCG4pVkNCP0FhAAKQ/LPp4x+dDUKFmLl11ASg== 2023-06-23T18:20:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88040a1260 msgr2=0x7f88040a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88040a1260 0x7f88040a3710 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88180ffae0 tx=0x7f8814005f90 comp rx=0 tx=0).stop 2023-06-23T18:20:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 msgr2=0x7f881819a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 0x7f881819a540 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f8808003a50 comp rx=0 tx=0).stop 2023-06-23T18:20:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 shutdown_connections 2023-06-23T18:20:54.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88040a1260 0x7f88040a3710 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8818100610 0x7f881819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88180fea80 0x7f88180693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 --2- 172.21.15.2:0/1163210255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88180ffc80 0x7f881819a540 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 >> 172.21.15.2:0/1163210255 conn(0x7f88180fa0d0 msgr2=0x7f88180fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:54.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.390+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 shutdown_connections 2023-06-23T18:20:54.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:54.391+0000 7f882061a700 1 -- 172.21.15.2:0/1163210255 wait complete. 2023-06-23T18:20:54.768 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:20:54.769 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-06-23T18:20:54.769 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-06-23T18:20:54.811 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-06-23T18:20:54.812 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-06-23T18:20:54.813 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph mgr dump --format=json 2023-06-23T18:20:55.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:55 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1163210255' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-23T18:20:55.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:55 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1163210255' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-06-23T18:20:55.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:55 smithi002 ceph-mon[95947]: pgmap v261: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 KiB/s wr, 18 op/s 2023-06-23T18:20:55.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:55 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1163210255' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-23T18:20:55.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:55 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1163210255' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-06-23T18:20:55.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:55 smithi082 ceph-mon[97889]: pgmap v261: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 KiB/s wr, 18 op/s 2023-06-23T18:20:55.408 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:20:55.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:55 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1163210255' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-06-23T18:20:55.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:55 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1163210255' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-06-23T18:20:55.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:55 smithi017 ceph-mon[97472]: pgmap v261: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 7.5 KiB/s wr, 18 op/s 2023-06-23T18:20:56.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 -- 172.21.15.2:0/4244882146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 msgr2=0x7f059410b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 --2- 172.21.15.2:0/4244882146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 0x7f059410b8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f058c00b0a0 tx=0x7f058c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 -- 172.21.15.2:0/4244882146 shutdown_connections 2023-06-23T18:20:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 --2- 172.21.15.2:0/4244882146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 0x7f059410b8b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:56.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 --2- 172.21.15.2:0/4244882146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0594104a70 0x7f0594104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:56.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 --2- 172.21.15.2:0/4244882146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05940fea70 0x7f05940feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:56.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 -- 172.21.15.2:0/4244882146 >> 172.21.15.2:0/4244882146 conn(0x7f0594075b30 msgr2=0x7f0594075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:56.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 -- 172.21.15.2:0/4244882146 shutdown_connections 2023-06-23T18:20:56.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.729+0000 7f059a89e700 1 -- 172.21.15.2:0/4244882146 wait complete. 2023-06-23T18:20:56.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.730+0000 7f059a89e700 1 Processor -- start 2023-06-23T18:20:56.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.730+0000 7f059a89e700 1 -- start start 2023-06-23T18:20:56.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f059a89e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05940fea70 0x7f059419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:56.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f059a89e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 0x7f059419ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:56.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f059a89e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0594104a70 0x7f05941948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:56.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f059a89e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594194f50 con 0x7f05940ff420 2023-06-23T18:20:56.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f059a89e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05941950c0 con 0x7f0594104a70 2023-06-23T18:20:56.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f059a89e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0594195280 con 0x7f05940fea70 2023-06-23T18:20:56.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 0x7f059419ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:56.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 0x7f059419ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52636/0 (socket says 172.21.15.2:52636) 2023-06-23T18:20:56.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 -- 172.21.15.2:0/1258109286 learned_addr learned my addr 172.21.15.2:0/1258109286 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:56.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f0593fff700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05940fea70 0x7f059419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:56.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f0598e3b700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0594104a70 0x7f05941948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:56.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 -- 172.21.15.2:0/1258109286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05940fea70 msgr2=0x7f059419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:56.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05940fea70 0x7f059419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:56.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 -- 172.21.15.2:0/1258109286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0594104a70 msgr2=0x7f05941948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:56.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0594104a70 0x7f05941948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:56.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 -- 172.21.15.2:0/1258109286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c009d00 con 0x7f05940ff420 2023-06-23T18:20:56.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f0598e3b700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0594104a70 0x7f05941948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:20:56.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05937fe700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 0x7f059419ada0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f058400c920 tx=0x7f058400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:56.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05917fa700 1 -- 172.21.15.2:0/1258109286 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584007b00 con 0x7f05940ff420 2023-06-23T18:20:56.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0594195560 con 0x7f05940ff420 2023-06-23T18:20:56.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05917fa700 1 -- 172.21.15.2:0/1258109286 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0584004cc0 con 0x7f05940ff420 2023-06-23T18:20:56.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f05917fa700 1 -- 172.21.15.2:0/1258109286 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584005590 con 0x7f05940ff420 2023-06-23T18:20:56.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.731+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05941a1850 con 0x7f05940ff420 2023-06-23T18:20:56.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.733+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059404e1d0 con 0x7f05940ff420 2023-06-23T18:20:56.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.734+0000 7f05917fa700 1 -- 172.21.15.2:0/1258109286 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f0584004750 con 0x7f05940ff420 2023-06-23T18:20:56.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.734+0000 7f05917fa700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05800a11e0 0x7f05800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:56.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.734+0000 7f0593fff700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05800a11e0 0x7f05800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:56.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.734+0000 7f05917fa700 1 -- 172.21.15.2:0/1258109286 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f05840cc8b0 con 0x7f05940ff420 2023-06-23T18:20:56.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.735+0000 7f0593fff700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05800a11e0 0x7f05800a3690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f057c005950 tx=0x7f057c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:56.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.738+0000 7f05917fa700 1 -- 172.21.15.2:0/1258109286 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0584096050 con 0x7f05940ff420 2023-06-23T18:20:57.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:56.999+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f05941a1ba0 con 0x7f05940ff420 2023-06-23T18:20:57.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.003+0000 7f05917fa700 1 -- 172.21.15.2:0/1258109286 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260115 (secure 0 0 0) 0x7f058401d020 con 0x7f05940ff420 2023-06-23T18:20:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.006+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05800a11e0 msgr2=0x7f05800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:57.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.006+0000 7f059a89e700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05800a11e0 0x7f05800a3690 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f057c005950 tx=0x7f057c0058e0 comp rx=0 tx=0).stop 2023-06-23T18:20:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.006+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 msgr2=0x7f059419ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.006+0000 7f059a89e700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 0x7f059419ada0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f058400c920 tx=0x7f058400cce0 comp rx=0 tx=0).stop 2023-06-23T18:20:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.007+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 shutdown_connections 2023-06-23T18:20:57.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.007+0000 7f059a89e700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05800a11e0 0x7f05800a3690 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:57.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.007+0000 7f059a89e700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0594104a70 0x7f05941948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:57.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.007+0000 7f059a89e700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05940fea70 0x7f059419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:57.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.007+0000 7f059a89e700 1 --2- 172.21.15.2:0/1258109286 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05940ff420 0x7f059419ada0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:57.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.007+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 >> 172.21.15.2:0/1258109286 conn(0x7f0594075b30 msgr2=0x7f05940fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:57.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.007+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 shutdown_connections 2023-06-23T18:20:57.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:57.007+0000 7f059a89e700 1 -- 172.21.15.2:0/1258109286 wait complete. 2023-06-23T18:20:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:56 smithi082 ceph-mon[97889]: pgmap v262: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2023-06-23T18:20:57.150 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:56 smithi002 ceph-mon[95947]: pgmap v262: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2023-06-23T18:20:57.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:56 smithi017 ceph-mon[97472]: pgmap v262: 97 pgs: 97 active+clean; 620 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2023-06-23T18:20:57.368 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":3622114004},{"type":"v1","addr":"172.21.15.2:6801","nonce":3622114004}]},"active_addr":"172.21.15.2:6801/3622114004","active_change":"2023-06-23T18:14:36.653692+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.2:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":787254849}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3906344914}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":366730668}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":1228448929}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":2659730294}]}]} 2023-06-23T18:20:57.381 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-06-23T18:20:57.382 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-06-23T18:20:57.382 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd dump --format=json 2023-06-23T18:20:57.942 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:20:58.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:57 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1258109286' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-06-23T18:20:58.148 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:57 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1258109286' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-06-23T18:20:58.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:57 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1258109286' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-06-23T18:20:59.093 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:20:58 smithi002 ceph-mon[95947]: pgmap v263: 97 pgs: 97 active+clean; 628 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2023-06-23T18:20:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:20:58 smithi082 ceph-mon[97889]: pgmap v263: 97 pgs: 97 active+clean; 628 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2023-06-23T18:20:59.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:20:58 smithi017 ceph-mon[97472]: pgmap v263: 97 pgs: 97 active+clean; 628 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.2 KiB/s wr, 17 op/s 2023-06-23T18:20:59.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 -- 172.21.15.2:0/2740192663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc0ff420 msgr2=0x7fcccc10b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/2740192663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc0ff420 0x7fcccc10b8b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fccc400b0a0 tx=0x7fccc400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 -- 172.21.15.2:0/2740192663 shutdown_connections 2023-06-23T18:20:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/2740192663 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc0ff420 0x7fcccc10b8b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/2740192663 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcccc104a70 0x7fcccc104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/2740192663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc0fea70 0x7fcccc0feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 -- 172.21.15.2:0/2740192663 >> 172.21.15.2:0/2740192663 conn(0x7fcccc075b30 msgr2=0x7fcccc075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:59.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 -- 172.21.15.2:0/2740192663 shutdown_connections 2023-06-23T18:20:59.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.276+0000 7fccd0d3c700 1 -- 172.21.15.2:0/2740192663 wait complete. 2023-06-23T18:20:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccd0d3c700 1 Processor -- start 2023-06-23T18:20:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccd0d3c700 1 -- start start 2023-06-23T18:20:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccd0d3c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc0fea70 0x7fcccc196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:59.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccd0d3c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcccc0ff420 0x7fcccc196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccd0d3c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc104a70 0x7fcccc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccd0d3c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc19abf0 con 0x7fcccc104a70 2023-06-23T18:20:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccd0d3c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc19ad60 con 0x7fcccc0fea70 2023-06-23T18:20:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccd0d3c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc19af20 con 0x7fcccc0ff420 2023-06-23T18:20:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fcccad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc104a70 0x7fcccc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fcccad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc104a70 0x7fcccc19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52652/0 (socket says 172.21.15.2:52652) 2023-06-23T18:20:59.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fcccad9d700 1 -- 172.21.15.2:0/1537454049 learned_addr learned my addr 172.21.15.2:0/1537454049 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:20:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccca59c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc0fea70 0x7fcccc196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccc9d9b700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcccc0ff420 0x7fcccc196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fcccad9d700 1 -- 172.21.15.2:0/1537454049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcccc0ff420 msgr2=0x7fcccc196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fcccad9d700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcccc0ff420 0x7fcccc196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fcccad9d700 1 -- 172.21.15.2:0/1537454049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc0fea70 msgr2=0x7fcccc196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fcccad9d700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc0fea70 0x7fcccc196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fcccad9d700 1 -- 172.21.15.2:0/1537454049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4009d00 con 0x7fcccc104a70 2023-06-23T18:20:59.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.277+0000 7fccc9d9b700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcccc0ff420 0x7fcccc196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:20:59.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.278+0000 7fcccad9d700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc104a70 0x7fcccc19a5e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fccc400b830 tx=0x7fccc40094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:59.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.278+0000 7fccbb7fe700 1 -- 172.21.15.2:0/1537454049 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc400e040 con 0x7fcccc104a70 2023-06-23T18:20:59.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.278+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc19b1a0 con 0x7fcccc104a70 2023-06-23T18:20:59.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.278+0000 7fccbb7fe700 1 -- 172.21.15.2:0/1537454049 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fccc4003e90 con 0x7fcccc104a70 2023-06-23T18:20:59.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.278+0000 7fccbb7fe700 1 -- 172.21.15.2:0/1537454049 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc4004850 con 0x7fcccc104a70 2023-06-23T18:20:59.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.278+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc10f460 con 0x7fcccc104a70 2023-06-23T18:20:59.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.279+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc04e1d0 con 0x7fcccc104a70 2023-06-23T18:20:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.280+0000 7fccbb7fe700 1 -- 172.21.15.2:0/1537454049 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fccc4003a00 con 0x7fcccc104a70 2023-06-23T18:20:59.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.281+0000 7fccbb7fe700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fccb40a55f0 0x7fccb40a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:20:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.281+0000 7fccca59c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fccb40a55f0 0x7fccb40a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:20:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.281+0000 7fccbb7fe700 1 -- 172.21.15.2:0/1537454049 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fccc40ce800 con 0x7fcccc104a70 2023-06-23T18:20:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.281+0000 7fccca59c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fccb40a55f0 0x7fccb40a7aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ffbb0 tx=0x7fccbc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:20:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.284+0000 7fccbb7fe700 1 -- 172.21.15.2:0/1537454049 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fccc4097f30 con 0x7fcccc104a70 2023-06-23T18:20:59.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.487+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcccc197140 con 0x7fcccc104a70 2023-06-23T18:20:59.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.489+0000 7fccbb7fe700 1 -- 172.21.15.2:0/1537454049 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21940 (secure 0 0 0) 0x7fccc409b550 con 0x7fcccc104a70 2023-06-23T18:20:59.495 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:20:59.495 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":82,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:45.435971+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:20:59.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fccb40a55f0 msgr2=0x7fccb40a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:59.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fccb40a55f0 0x7fccb40a7aa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcccc0ffbb0 tx=0x7fccbc00b410 comp rx=0 tx=0).stop 2023-06-23T18:20:59.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc104a70 msgr2=0x7fcccc19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:20:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc104a70 0x7fcccc19a5e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fccc400b830 tx=0x7fccc40094c0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 shutdown_connections 2023-06-23T18:20:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcccc104a70 0x7fcccc19a5e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fccb40a55f0 0x7fccb40a7aa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcccc0fea70 0x7fcccc196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 --2- 172.21.15.2:0/1537454049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcccc0ff420 0x7fcccc196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:20:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 >> 172.21.15.2:0/1537454049 conn(0x7fcccc075b30 msgr2=0x7fcccc0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:20:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 shutdown_connections 2023-06-23T18:20:59.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:20:59.491+0000 7fccd0d3c700 1 -- 172.21.15.2:0/1537454049 wait complete. 2023-06-23T18:20:59.906 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-06-23T18:20:59.906 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd dump --format=json 2023-06-23T18:21:00.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:00 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1537454049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:00.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:00 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1537454049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:00.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:00 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1537454049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:00.467 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:01.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:01 smithi002 ceph-mon[95947]: pgmap v264: 97 pgs: 97 active+clean; 628 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-06-23T18:21:01.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:01 smithi082 ceph-mon[97889]: pgmap v264: 97 pgs: 97 active+clean; 628 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-06-23T18:21:01.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:01 smithi017 ceph-mon[97472]: pgmap v264: 97 pgs: 97 active+clean; 628 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.7 KiB/s wr, 14 op/s 2023-06-23T18:21:01.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2577564074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e14104e90 msgr2=0x7f2e1410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:01.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2577564074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e14104e90 0x7f2e1410ca10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00b0d0 tx=0x7f2e0c00b3e0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2577564074 shutdown_connections 2023-06-23T18:21:01.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2577564074 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e14104e90 0x7f2e1410ca10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2577564074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e140ffe40 0x7f2e14100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2577564074 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e141007e0 0x7f2e14104760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2577564074 >> 172.21.15.2:0/2577564074 conn(0x7f2e14077b60 msgr2=0x7f2e14077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:01.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2577564074 shutdown_connections 2023-06-23T18:21:01.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.658+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2577564074 wait complete. 2023-06-23T18:21:01.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1c64c700 1 Processor -- start 2023-06-23T18:21:01.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1c64c700 1 -- start start 2023-06-23T18:21:01.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1c64c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e140ffe40 0x7f2e140727d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:01.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1c64c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e141007e0 0x7f2e14072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:01.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1c64c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e14104e90 0x7f2e14073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:01.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1c64c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e141a4160 con 0x7f2e141007e0 2023-06-23T18:21:01.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1c64c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e141a42d0 con 0x7f2e140ffe40 2023-06-23T18:21:01.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1c64c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e141a4490 con 0x7f2e14104e90 2023-06-23T18:21:01.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e19be7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e141007e0 0x7f2e14072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:01.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e19be7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e141007e0 0x7f2e14072d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52664/0 (socket says 172.21.15.2:52664) 2023-06-23T18:21:01.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e19be7700 1 -- 172.21.15.2:0/2221868103 learned_addr learned my addr 172.21.15.2:0/2221868103 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:01.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1a3e8700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e140ffe40 0x7f2e140727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:01.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.659+0000 7f2e1abe9700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e14104e90 0x7f2e14073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:01.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e19be7700 1 -- 172.21.15.2:0/2221868103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e14104e90 msgr2=0x7f2e14073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:01.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e19be7700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e14104e90 0x7f2e14073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e19be7700 1 -- 172.21.15.2:0/2221868103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e140ffe40 msgr2=0x7f2e140727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:01.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e19be7700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e140ffe40 0x7f2e140727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e19be7700 1 -- 172.21.15.2:0/2221868103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c009d00 con 0x7f2e141007e0 2023-06-23T18:21:01.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e19be7700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e141007e0 0x7f2e14072d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400b500 tx=0x7f2e0400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:01.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e0b7fe700 1 -- 172.21.15.2:0/2221868103 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04011830 con 0x7f2e141007e0 2023-06-23T18:21:01.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e0b7fe700 1 -- 172.21.15.2:0/2221868103 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e04011e20 con 0x7f2e141007e0 2023-06-23T18:21:01.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e141a4770 con 0x7f2e141007e0 2023-06-23T18:21:01.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e0b7fe700 1 -- 172.21.15.2:0/2221868103 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e04010db0 con 0x7f2e141007e0 2023-06-23T18:21:01.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.660+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e141a4c70 con 0x7f2e141007e0 2023-06-23T18:21:01.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.661+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1404e1d0 con 0x7f2e141007e0 2023-06-23T18:21:01.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.662+0000 7f2e0b7fe700 1 -- 172.21.15.2:0/2221868103 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2e04011990 con 0x7f2e141007e0 2023-06-23T18:21:01.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.663+0000 7f2e0b7fe700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e000a11e0 0x7f2e000a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:01.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.663+0000 7f2e1a3e8700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e000a11e0 0x7f2e000a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:01.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.663+0000 7f2e0b7fe700 1 -- 172.21.15.2:0/2221868103 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2e040cc270 con 0x7f2e141007e0 2023-06-23T18:21:01.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.663+0000 7f2e1a3e8700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e000a11e0 0x7f2e000a3690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e10005950 tx=0x7f2e1000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:01.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.667+0000 7f2e0b7fe700 1 -- 172.21.15.2:0/2221868103 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2e04095b20 con 0x7f2e141007e0 2023-06-23T18:21:01.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.867+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e141a4ff0 con 0x7f2e141007e0 2023-06-23T18:21:01.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.868+0000 7f2e0b7fe700 1 -- 172.21.15.2:0/2221868103 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21940 (secure 0 0 0) 0x7f2e04099140 con 0x7f2e141007e0 2023-06-23T18:21:01.874 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:01.875 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":82,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:45.435971+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:21:01.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e000a11e0 msgr2=0x7f2e000a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:01.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e000a11e0 0x7f2e000a3690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2e10005950 tx=0x7f2e1000a300 comp rx=0 tx=0).stop 2023-06-23T18:21:01.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e141007e0 msgr2=0x7f2e14072d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:01.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e141007e0 0x7f2e14072d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400b500 tx=0x7f2e0400b810 comp rx=0 tx=0).stop 2023-06-23T18:21:01.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 shutdown_connections 2023-06-23T18:21:01.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e14104e90 0x7f2e14073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e000a11e0 0x7f2e000a3690 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e140ffe40 0x7f2e140727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 --2- 172.21.15.2:0/2221868103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e141007e0 0x7f2e14072d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:01.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 >> 172.21.15.2:0/2221868103 conn(0x7f2e14077b60 msgr2=0x7f2e14103740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:01.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 shutdown_connections 2023-06-23T18:21:01.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:01.870+0000 7f2e1c64c700 1 -- 172.21.15.2:0/2221868103 wait complete. 2023-06-23T18:21:02.261 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:02 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2221868103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:02.315 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-06-23T18:21:02.316 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-06-23T18:21:02.316 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-06-23T18:21:02.317 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-06-23T18:21:02.317 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-06-23T18:21:02.318 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-06-23T18:21:02.318 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-06-23T18:21:02.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-06-23T18:21:02.320 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-06-23T18:21:02.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-06-23T18:21:02.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-06-23T18:21:02.322 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-06-23T18:21:02.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:02 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2221868103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:02.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:02 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2221868103' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:03.224 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:03.245 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:03.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:03 smithi082 ceph-mon[97889]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-06-23T18:21:03.367 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:03.368 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:03.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:03 smithi017 ceph-mon[97472]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-06-23T18:21:03.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:03 smithi002 ceph-mon[95947]: pgmap v265: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.6 KiB/s wr, 12 op/s 2023-06-23T18:21:03.638 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:03.686 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:03.846 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:04.027 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:04.182 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:05.196 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:05 smithi002 ceph-mon[95947]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-06-23T18:21:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:05.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:05 smithi082 ceph-mon[97889]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-06-23T18:21:05.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:05 smithi017 ceph-mon[97472]: pgmap v266: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2023-06-23T18:21:05.796 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:06.674 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:06.994 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:06 smithi002 ceph-mon[95947]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-06-23T18:21:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:06 smithi082 ceph-mon[97889]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-06-23T18:21:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:06 smithi017 ceph-mon[97472]: pgmap v267: 97 pgs: 97 active+clean; 633 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-06-23T18:21:07.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.215+0000 7f2386ca7700 1 -- 172.21.15.2:0/3985317914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2380100db0 msgr2=0x7f2380101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:07.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.215+0000 7f2386ca7700 1 --2- 172.21.15.2:0/3985317914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2380100db0 0x7f2380101180 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2374009990 tx=0x7f2374009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:07.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.215+0000 7f2386ca7700 1 -- 172.21.15.2:0/3985317914 shutdown_connections 2023-06-23T18:21:07.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.215+0000 7f2386ca7700 1 --2- 172.21.15.2:0/3985317914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f238010a100 0x7f238010c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:07.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.215+0000 7f2386ca7700 1 --2- 172.21.15.2:0/3985317914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2380100db0 0x7f2380101180 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:07.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.215+0000 7f2386ca7700 1 --2- 172.21.15.2:0/3985317914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23801016c0 0x7f2380109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:07.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.215+0000 7f2386ca7700 1 -- 172.21.15.2:0/3985317914 >> 172.21.15.2:0/3985317914 conn(0x7f2380075b30 msgr2=0x7f2380075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:07.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 -- 172.21.15.2:0/3985317914 shutdown_connections 2023-06-23T18:21:07.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 -- 172.21.15.2:0/3985317914 wait complete. 2023-06-23T18:21:07.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 Processor -- start 2023-06-23T18:21:07.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 -- start start 2023-06-23T18:21:07.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2380100db0 0x7f2380197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:07.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23801016c0 0x7f2380198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:07.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f238010a100 0x7f238019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:07.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238019cd10 con 0x7f23801016c0 2023-06-23T18:21:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238019ce80 con 0x7f2380100db0 2023-06-23T18:21:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f2386ca7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238019d040 con 0x7f238010a100 2023-06-23T18:21:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f237ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23801016c0 0x7f2380198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f237ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23801016c0 0x7f2380198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52674/0 (socket says 172.21.15.2:52674) 2023-06-23T18:21:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.216+0000 7f237ffff700 1 -- 172.21.15.2:0/838741380 learned_addr learned my addr 172.21.15.2:0/838741380 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f2384a43700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2380100db0 0x7f2380197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f2385244700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f238010a100 0x7f238019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:07.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237ffff700 1 -- 172.21.15.2:0/838741380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f238010a100 msgr2=0x7f238019c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237ffff700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f238010a100 0x7f238019c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237ffff700 1 -- 172.21.15.2:0/838741380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2380100db0 msgr2=0x7f2380197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237ffff700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2380100db0 0x7f2380197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237ffff700 1 -- 172.21.15.2:0/838741380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23740095f0 con 0x7f23801016c0 2023-06-23T18:21:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237ffff700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23801016c0 0x7f2380198290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f236c00d950 tx=0x7f236c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:07.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00f890 con 0x7f23801016c0 2023-06-23T18:21:07.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238019d320 con 0x7f23801016c0 2023-06-23T18:21:07.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f236c00fe80 con 0x7f23801016c0 2023-06-23T18:21:07.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c00e3b0 con 0x7f23801016c0 2023-06-23T18:21:07.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.217+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23801a4b80 con 0x7f23801016c0 2023-06-23T18:21:07.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.218+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f238010fc00 con 0x7f23801016c0 2023-06-23T18:21:07.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.219+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f236c010460 con 0x7f23801016c0 2023-06-23T18:21:07.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.220+0000 7f237dffb700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f23700a11e0 0x7f23700a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:07.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.220+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f236c0cccb0 con 0x7f23801016c0 2023-06-23T18:21:07.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.220+0000 7f2384a43700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f23700a11e0 0x7f23700a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:07.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.220+0000 7f237dffb700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] conn(0x7f23700aa9b0 0x7f23700acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.220+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 --> [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23700ad470 con 0x7f23700aa9b0 2023-06-23T18:21:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.220+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f236c0cd020 con 0x7f23801016c0 2023-06-23T18:21:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.220+0000 7f2384a43700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f23700a11e0 0x7f23700a3690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f237400b5c0 tx=0x7f2374009f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.221+0000 7f2385244700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] conn(0x7f23700aa9b0 0x7f23700acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.221+0000 7f2385244700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] conn(0x7f23700aa9b0 0x7f23700acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.222+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 <== osd.7 v2:172.21.15.17:6816/3002114056 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f23700ad470 con 0x7f23700aa9b0 2023-06-23T18:21:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.238+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 --> [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f238004e1d0 con 0x7f23700aa9b0 2023-06-23T18:21:07.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f237dffb700 1 -- 172.21.15.2:0/838741380 <== osd.7 v2:172.21.15.17:6816/3002114056 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f238004e1d0 con 0x7f23700aa9b0 2023-06-23T18:21:07.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 >> [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] conn(0x7f23700aa9b0 msgr2=0x7f23700acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:07.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] conn(0x7f23700aa9b0 0x7f23700acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:07.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f23700a11e0 msgr2=0x7f23700a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:07.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f23700a11e0 0x7f23700a3690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f237400b5c0 tx=0x7f2374009f00 comp rx=0 tx=0).stop 2023-06-23T18:21:07.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23801016c0 msgr2=0x7f2380198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:07.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 --2- 172.21.15.2:0/838741380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f23801016c0 0x7f2380198290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f236c00d950 tx=0x7f236c00dc60 comp rx=0 tx=0).stop 2023-06-23T18:21:07.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2385244700 1 -- 172.21.15.2:0/838741380 reap_dead start 2023-06-23T18:21:07.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 shutdown_connections 2023-06-23T18:21:07.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 >> 172.21.15.2:0/838741380 conn(0x7f2380075b30 msgr2=0x7f23800fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:07.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 shutdown_connections 2023-06-23T18:21:07.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:07.239+0000 7f2386ca7700 1 -- 172.21.15.2:0/838741380 wait complete. 2023-06-23T18:21:08.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 -- 172.21.15.2:0/286908049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 msgr2=0x7fe1f810b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 --2- 172.21.15.2:0/286908049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 0x7fe1f810b8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000b0a0 tx=0x7fe1f000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 -- 172.21.15.2:0/286908049 shutdown_connections 2023-06-23T18:21:08.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 --2- 172.21.15.2:0/286908049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 0x7fe1f810b8b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 --2- 172.21.15.2:0/286908049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f8104a70 0x7fe1f8104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 --2- 172.21.15.2:0/286908049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1f80fea70 0x7fe1f80feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 -- 172.21.15.2:0/286908049 >> 172.21.15.2:0/286908049 conn(0x7fe1f8075b30 msgr2=0x7fe1f8075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:08.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 -- 172.21.15.2:0/286908049 shutdown_connections 2023-06-23T18:21:08.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.659+0000 7fe1ff646700 1 -- 172.21.15.2:0/286908049 wait complete. 2023-06-23T18:21:08.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1ff646700 1 Processor -- start 2023-06-23T18:21:08.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1ff646700 1 -- start start 2023-06-23T18:21:08.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1ff646700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f80fea70 0x7fe1f810cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1ff646700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 0x7fe1f810d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1ff646700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1f8104a70 0x7fe1f810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1ff646700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f81a4040 con 0x7fe1f80ff420 2023-06-23T18:21:08.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1ff646700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f81a41b0 con 0x7fe1f80fea70 2023-06-23T18:21:08.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1ff646700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f81a4370 con 0x7fe1f8104a70 2023-06-23T18:21:08.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 0x7fe1f810d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 0x7fe1f810d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52706/0 (socket says 172.21.15.2:52706) 2023-06-23T18:21:08.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 -- 172.21.15.2:0/4256579296 learned_addr learned my addr 172.21.15.2:0/4256579296 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:08.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fdbe3700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1f8104a70 0x7fe1f810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fd3e2700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f80fea70 0x7fe1f810cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 -- 172.21.15.2:0/4256579296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1f8104a70 msgr2=0x7fe1f810da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1f8104a70 0x7fe1f810da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 -- 172.21.15.2:0/4256579296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f80fea70 msgr2=0x7fe1f810cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f80fea70 0x7fe1f810cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 -- 172.21.15.2:0/4256579296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0009d00 con 0x7fe1f80ff420 2023-06-23T18:21:08.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fd3e2700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1f80fea70 0x7fe1f810cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:08.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.660+0000 7fe1fcbe1700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 0x7fe1f810d4e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800d950 tx=0x7fe1e800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:08.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.661+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8009940 con 0x7fe1f80ff420 2023-06-23T18:21:08.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.661+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1e8010460 con 0x7fe1f80ff420 2023-06-23T18:21:08.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.661+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f81a4650 con 0x7fe1f80ff420 2023-06-23T18:21:08.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.661+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800f620 con 0x7fe1f80ff420 2023-06-23T18:21:08.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.661+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f81a4ba0 con 0x7fe1f80ff420 2023-06-23T18:21:08.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.661+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1f804e1d0 con 0x7fe1f80ff420 2023-06-23T18:21:08.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.663+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe1e80105d0 con 0x7fe1f80ff420 2023-06-23T18:21:08.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.663+0000 7fe1ee7fc700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe1e40a1260 0x7fe1e40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.663+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe1e80cccb0 con 0x7fe1f80ff420 2023-06-23T18:21:08.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.664+0000 7fe1fd3e2700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe1e40a1260 0x7fe1e40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.664+0000 7fe1ee7fc700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] conn(0x7fe1e40aaa30 0x7fe1e40ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.664+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 --> [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1e40ad4f0 con 0x7fe1e40aaa30 2023-06-23T18:21:08.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.664+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1e80cd020 con 0x7fe1f80ff420 2023-06-23T18:21:08.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.664+0000 7fe1fd3e2700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe1e40a1260 0x7fe1e40a3710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4005fd0 tx=0x7fe1f4005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:08.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.664+0000 7fe1fdbe3700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] conn(0x7fe1e40aaa30 0x7fe1e40ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.664+0000 7fe1fdbe3700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] conn(0x7fe1e40aaa30 0x7fe1e40ace40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:08.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.665+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 <== osd.2 v2:172.21.15.82:6800/625655423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe1e40ad4f0 con 0x7fe1e40aaa30 2023-06-23T18:21:08.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.682+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 --> [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1f810fc40 con 0x7fe1e40aaa30 2023-06-23T18:21:08.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ee7fc700 1 -- 172.21.15.2:0/4256579296 <== osd.2 v2:172.21.15.82:6800/625655423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1f0009b80 con 0x7fe1e40aaa30 2023-06-23T18:21:08.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 >> [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] conn(0x7fe1e40aaa30 msgr2=0x7fe1e40ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ff646700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] conn(0x7fe1e40aaa30 0x7fe1e40ace40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe1e40a1260 msgr2=0x7fe1e40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ff646700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe1e40a1260 0x7fe1e40a3710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4005fd0 tx=0x7fe1f4005dc0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 msgr2=0x7fe1f810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ff646700 1 --2- 172.21.15.2:0/4256579296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1f80ff420 0x7fe1f810d4e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800d950 tx=0x7fe1e800dc60 comp rx=0 tx=0).stop 2023-06-23T18:21:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1fdbe3700 1 -- 172.21.15.2:0/4256579296 reap_dead start 2023-06-23T18:21:08.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 shutdown_connections 2023-06-23T18:21:08.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.683+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 >> 172.21.15.2:0/4256579296 conn(0x7fe1f8075b30 msgr2=0x7fe1f81026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:08.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.684+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 shutdown_connections 2023-06-23T18:21:08.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.684+0000 7fe1ff646700 1 -- 172.21.15.2:0/4256579296 wait complete. 2023-06-23T18:21:08.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 -- 172.21.15.2:0/3360194094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78640ff420 msgr2=0x7f786410b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 --2- 172.21.15.2:0/3360194094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78640ff420 0x7f786410b8b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b0a0 tx=0x7f785c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 -- 172.21.15.2:0/3360194094 shutdown_connections 2023-06-23T18:21:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 --2- 172.21.15.2:0/3360194094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78640ff420 0x7f786410b8b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 --2- 172.21.15.2:0/3360194094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7864104a70 0x7f7864104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 --2- 172.21.15.2:0/3360194094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78640fea70 0x7f78640feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 -- 172.21.15.2:0/3360194094 >> 172.21.15.2:0/3360194094 conn(0x7f7864075b30 msgr2=0x7f7864075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 -- 172.21.15.2:0/3360194094 shutdown_connections 2023-06-23T18:21:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.016+0000 7f786be77700 1 -- 172.21.15.2:0/3360194094 wait complete. 2023-06-23T18:21:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.017+0000 7f786be77700 1 Processor -- start 2023-06-23T18:21:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.017+0000 7f786be77700 1 -- start start 2023-06-23T18:21:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.017+0000 7f786be77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78640fea70 0x7f786419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786be77700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78640ff420 0x7f786419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786be77700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864104a70 0x7f78641948f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786be77700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864194f60 con 0x7f7864104a70 2023-06-23T18:21:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786be77700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78641950d0 con 0x7f78640ff420 2023-06-23T18:21:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786be77700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864195290 con 0x7f78640fea70 2023-06-23T18:21:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864104a70 0x7f78641948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864104a70 0x7f78641948f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52692/0 (socket says 172.21.15.2:52692) 2023-06-23T18:21:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 -- 172.21.15.2:0/1504608513 learned_addr learned my addr 172.21.15.2:0/1504608513 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f7869c13700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78640fea70 0x7f786419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f7869412700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78640ff420 0x7f786419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 -- 172.21.15.2:0/1504608513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78640fea70 msgr2=0x7f786419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78640fea70 0x7f786419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 -- 172.21.15.2:0/1504608513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78640ff420 msgr2=0x7f786419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78640ff420 0x7f786419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 -- 172.21.15.2:0/1504608513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c009d00 con 0x7f7864104a70 2023-06-23T18:21:08.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f7869412700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78640ff420 0x7f786419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:08.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786a414700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864104a70 0x7f78641948f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b780 tx=0x7f785c008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:08.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c00e040 con 0x7f7864104a70 2023-06-23T18:21:08.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f785c004c50 con 0x7f7864104a70 2023-06-23T18:21:08.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7864195510 con 0x7f7864104a70 2023-06-23T18:21:08.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c0052a0 con 0x7f7864104a70 2023-06-23T18:21:08.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.018+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78641a1860 con 0x7f7864104a70 2023-06-23T18:21:08.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.019+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f786404e1d0 con 0x7f7864104a70 2023-06-23T18:21:08.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.020+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f785c01d470 con 0x7f7864104a70 2023-06-23T18:21:08.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.021+0000 7f7856ffd700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78500a11e0 0x7f78500a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.021+0000 7f7869c13700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78500a11e0 0x7f78500a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.021+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f785c0ce240 con 0x7f7864104a70 2023-06-23T18:21:08.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.021+0000 7f7856ffd700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] conn(0x7f78500aa9b0 0x7f78500acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:08.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.021+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 --> [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78500ad470 con 0x7f78500aa9b0 2023-06-23T18:21:08.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.021+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f785c0ce6e0 con 0x7f7864104a70 2023-06-23T18:21:08.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.021+0000 7f7869412700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] conn(0x7f78500aa9b0 0x7f78500acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:08.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.021+0000 7f7869c13700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78500a11e0 0x7f78500a3690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78640ffbb0 tx=0x7f786000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:08.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.022+0000 7f7869412700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] conn(0x7f78500aa9b0 0x7f78500acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.022+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 <== osd.3 v2:172.21.15.2:6810/1640369824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f78500ad470 con 0x7f78500aa9b0 2023-06-23T18:21:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.039+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 --> [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78640fd710 con 0x7f78500aa9b0 2023-06-23T18:21:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f7856ffd700 1 -- 172.21.15.2:0/1504608513 <== osd.3 v2:172.21.15.2:6810/1640369824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f78640fd710 con 0x7f78500aa9b0 2023-06-23T18:21:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] conn(0x7f78500aa9b0 msgr2=0x7f78500acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] conn(0x7f78500aa9b0 0x7f78500acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78500a11e0 msgr2=0x7f78500a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78500a11e0 0x7f78500a3690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78640ffbb0 tx=0x7f786000a380 comp rx=0 tx=0).stop 2023-06-23T18:21:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864104a70 msgr2=0x7f78641948f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 --2- 172.21.15.2:0/1504608513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7864104a70 0x7f78641948f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f785c00b780 tx=0x7f785c008f90 comp rx=0 tx=0).stop 2023-06-23T18:21:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786a414700 1 -- 172.21.15.2:0/1504608513 reap_dead start 2023-06-23T18:21:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 shutdown_connections 2023-06-23T18:21:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 >> 172.21.15.2:0/1504608513 conn(0x7f7864075b30 msgr2=0x7f78640fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 shutdown_connections 2023-06-23T18:21:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.040+0000 7f786be77700 1 -- 172.21.15.2:0/1504608513 wait complete. 2023-06-23T18:21:09.064 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:08 smithi002 ceph-mon[95947]: pgmap v268: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-06-23T18:21:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:08 smithi082 ceph-mon[97889]: pgmap v268: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-06-23T18:21:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:08 smithi017 ceph-mon[97472]: pgmap v268: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2023-06-23T18:21:09.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 -- 172.21.15.2:0/1759650566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60481005d0 msgr2=0x7f6048100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 --2- 172.21.15.2:0/1759650566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60481005d0 0x7f6048100a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f603c0099c0 tx=0x7f603c009cd0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 -- 172.21.15.2:0/1759650566 shutdown_connections 2023-06-23T18:21:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 --2- 172.21.15.2:0/1759650566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6048100f80 0x7f604810b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 --2- 172.21.15.2:0/1759650566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60480fee40 0x7f60480ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 --2- 172.21.15.2:0/1759650566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60481005d0 0x7f6048100a40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 -- 172.21.15.2:0/1759650566 >> 172.21.15.2:0/1759650566 conn(0x7f6048075b20 msgr2=0x7f6048075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 -- 172.21.15.2:0/1759650566 shutdown_connections 2023-06-23T18:21:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 -- 172.21.15.2:0/1759650566 wait complete. 2023-06-23T18:21:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.318+0000 7f604e202700 1 Processor -- start 2023-06-23T18:21:09.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f604e202700 1 -- start start 2023-06-23T18:21:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f604e202700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60480fee40 0x7f6048194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f604e202700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60481005d0 0x7f6048194ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f604e202700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6048100f80 0x7f6048195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f604e202700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60481a41c0 con 0x7f60480fee40 2023-06-23T18:21:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f604e202700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60481a4330 con 0x7f60481005d0 2023-06-23T18:21:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f604e202700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60481a44f0 con 0x7f6048100f80 2023-06-23T18:21:09.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f60477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60480fee40 0x7f6048194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f60477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60480fee40 0x7f6048194980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52734/0 (socket says 172.21.15.2:52734) 2023-06-23T18:21:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f60477fe700 1 -- 172.21.15.2:0/1140115328 learned_addr learned my addr 172.21.15.2:0/1140115328 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f6046ffd700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60481005d0 0x7f6048194ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f6047fff700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6048100f80 0x7f6048195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f60477fe700 1 -- 172.21.15.2:0/1140115328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6048100f80 msgr2=0x7f6048195400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f60477fe700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6048100f80 0x7f6048195400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f60477fe700 1 -- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60481005d0 msgr2=0x7f6048194ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f60477fe700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60481005d0 0x7f6048194ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f60477fe700 1 -- 172.21.15.2:0/1140115328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c0095f0 con 0x7f60480fee40 2023-06-23T18:21:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.319+0000 7f6046ffd700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60481005d0 0x7f6048194ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.320+0000 7f60477fe700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60480fee40 0x7f6048194980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f603800e9a0 tx=0x7f603800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:09.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.320+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038009790 con 0x7f60480fee40 2023-06-23T18:21:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.320+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60481a47d0 con 0x7f60480fee40 2023-06-23T18:21:09.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.320+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f603800cbd0 con 0x7f60480fee40 2023-06-23T18:21:09.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.320+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038018680 con 0x7f60480fee40 2023-06-23T18:21:09.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.320+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60481a4ca0 con 0x7f60480fee40 2023-06-23T18:21:09.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.320+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f604810ebe0 con 0x7f60480fee40 2023-06-23T18:21:09.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.322+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f603800cd40 con 0x7f60480fee40 2023-06-23T18:21:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.323+0000 7f6044ff9700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f60300a11e0 0x7f60300a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.323+0000 7f6046ffd700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f60300a11e0 0x7f60300a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.323+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6038014070 con 0x7f60480fee40 2023-06-23T18:21:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.323+0000 7f6044ff9700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] conn(0x7f60300aa9b0 0x7f60300acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.323+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 --> [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f60300ad470 con 0x7f60300aa9b0 2023-06-23T18:21:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.323+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60380cd7e0 con 0x7f60480fee40 2023-06-23T18:21:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.323+0000 7f6046ffd700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f60300a11e0 0x7f60300a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6048069210 tx=0x7f603c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.323+0000 7f6047fff700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] conn(0x7f60300aa9b0 0x7f60300acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.324+0000 7f6047fff700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] conn(0x7f60300aa9b0 0x7f60300acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:09.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.324+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 <== osd.4 v2:172.21.15.17:6808/2671149119 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f60300ad470 con 0x7f60300aa9b0 2023-06-23T18:21:09.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.343+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 --> [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f604804e1d0 con 0x7f60300aa9b0 2023-06-23T18:21:09.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.343+0000 7f6044ff9700 1 -- 172.21.15.2:0/1140115328 <== osd.4 v2:172.21.15.17:6808/2671149119 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f604804e1d0 con 0x7f60300aa9b0 2023-06-23T18:21:09.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] conn(0x7f60300aa9b0 msgr2=0x7f60300acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] conn(0x7f60300aa9b0 0x7f60300acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f60300a11e0 msgr2=0x7f60300a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f60300a11e0 0x7f60300a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6048069210 tx=0x7f603c01a040 comp rx=0 tx=0).stop 2023-06-23T18:21:09.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60480fee40 msgr2=0x7f6048194980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 --2- 172.21.15.2:0/1140115328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60480fee40 0x7f6048194980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f603800e9a0 tx=0x7f603800ed60 comp rx=0 tx=0).stop 2023-06-23T18:21:09.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f6047fff700 1 -- 172.21.15.2:0/1140115328 reap_dead start 2023-06-23T18:21:09.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 shutdown_connections 2023-06-23T18:21:09.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 >> 172.21.15.2:0/1140115328 conn(0x7f6048075b20 msgr2=0x7f60480fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:09.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 shutdown_connections 2023-06-23T18:21:09.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:09.344+0000 7f604e202700 1 -- 172.21.15.2:0/1140115328 wait complete. 2023-06-23T18:21:09.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2449475578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240fea70 msgr2=0x7f9b240feee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 --2- 172.21.15.2:0/2449475578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240fea70 0x7f9b240feee0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9b14009990 tx=0x7f9b14009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2449475578 shutdown_connections 2023-06-23T18:21:09.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 --2- 172.21.15.2:0/2449475578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b240ff420 0x7f9b2410b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 --2- 172.21.15.2:0/2449475578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b24104a70 0x7f9b24104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 --2- 172.21.15.2:0/2449475578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240fea70 0x7f9b240feee0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2449475578 >> 172.21.15.2:0/2449475578 conn(0x7f9b24075b30 msgr2=0x7f9b24075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:09.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2449475578 shutdown_connections 2023-06-23T18:21:09.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.888+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2449475578 wait complete. 2023-06-23T18:21:09.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 Processor -- start 2023-06-23T18:21:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 -- start start 2023-06-23T18:21:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b240fea70 0x7f9b241949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240ff420 0x7f9b24194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b24104a70 0x7f9b24199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24199950 con 0x7f9b240ff420 2023-06-23T18:21:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24199ac0 con 0x7f9b24104a70 2023-06-23T18:21:09.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b24199c80 con 0x7f9b240fea70 2023-06-23T18:21:09.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240ff420 0x7f9b24194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240ff420 0x7f9b24194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52722/0 (socket says 172.21.15.2:52722) 2023-06-23T18:21:09.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 -- 172.21.15.2:0/2579846658 learned_addr learned my addr 172.21.15.2:0/2579846658 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b29257700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b240fea70 0x7f9b241949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 -- 172.21.15.2:0/2579846658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b240fea70 msgr2=0x7f9b241949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b240fea70 0x7f9b241949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 -- 172.21.15.2:0/2579846658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b24104a70 msgr2=0x7f9b24199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b24104a70 0x7f9b24199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 -- 172.21.15.2:0/2579846658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b140095f0 con 0x7f9b240ff420 2023-06-23T18:21:09.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b28a56700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240ff420 0x7f9b24194ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9b140036c0 tx=0x7f9b140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:09.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1401f070 con 0x7f9b240ff420 2023-06-23T18:21:09.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b1401aba0 con 0x7f9b240ff420 2023-06-23T18:21:09.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.889+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b24199f00 con 0x7f9b240ff420 2023-06-23T18:21:09.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.890+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b14003fb0 con 0x7f9b240ff420 2023-06-23T18:21:09.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.890+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b24068160 con 0x7f9b240ff420 2023-06-23T18:21:09.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.890+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b2419a220 con 0x7f9b240ff420 2023-06-23T18:21:09.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.892+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9b14023070 con 0x7f9b240ff420 2023-06-23T18:21:09.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.892+0000 7f9b1a7fc700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b100a1260 0x7f9b100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.892+0000 7f9b29257700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b100a1260 0x7f9b100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.892+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9b140d0420 con 0x7f9b240ff420 2023-06-23T18:21:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.892+0000 7f9b1a7fc700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] conn(0x7f9b100aaa30 0x7f9b100ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.892+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 --> [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b100ad4f0 con 0x7f9b100aaa30 2023-06-23T18:21:09.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.892+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b140d08c0 con 0x7f9b240ff420 2023-06-23T18:21:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.893+0000 7f9b29a58700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] conn(0x7f9b100aaa30 0x7f9b100ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.893+0000 7f9b29257700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b100a1260 0x7f9b100a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9b20005fd0 tx=0x7f9b20005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.893+0000 7f9b29a58700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] conn(0x7f9b100aaa30 0x7f9b100ace40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.894+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 <== osd.1 v2:172.21.15.17:6800/85914245 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9b100ad4f0 con 0x7f9b100aaa30 2023-06-23T18:21:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.910+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 --> [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9b2410eb10 con 0x7f9b100aaa30 2023-06-23T18:21:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.910+0000 7f9b1a7fc700 1 -- 172.21.15.2:0/2579846658 <== osd.1 v2:172.21.15.17:6800/85914245 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9b1c00f040 con 0x7f9b100aaa30 2023-06-23T18:21:09.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 >> [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] conn(0x7f9b100aaa30 msgr2=0x7f9b100ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] conn(0x7f9b100aaa30 0x7f9b100ace40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b100a1260 msgr2=0x7f9b100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b100a1260 0x7f9b100a3710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9b20005fd0 tx=0x7f9b20005e50 comp rx=0 tx=0).stop 2023-06-23T18:21:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240ff420 msgr2=0x7f9b24194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 --2- 172.21.15.2:0/2579846658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b240ff420 0x7f9b24194ee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9b140036c0 tx=0x7f9b140057f0 comp rx=0 tx=0).stop 2023-06-23T18:21:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b29a58700 1 -- 172.21.15.2:0/2579846658 reap_dead start 2023-06-23T18:21:09.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 shutdown_connections 2023-06-23T18:21:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 >> 172.21.15.2:0/2579846658 conn(0x7f9b24075b30 msgr2=0x7f9b240fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 shutdown_connections 2023-06-23T18:21:09.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:08.911+0000 7f9b2b4bb700 1 -- 172.21.15.2:0/2579846658 wait complete. 2023-06-23T18:21:10.237 INFO:teuthology.orchestra.run.smithi002.stdout:141733920814 2023-06-23T18:21:10.238 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-06-23T18:21:10.306 INFO:teuthology.orchestra.run.smithi002.stdout:55834574907 2023-06-23T18:21:10.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-06-23T18:21:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.779+0000 7f9cfde08700 1 -- 172.21.15.2:0/2456978210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80feac0 msgr2=0x7f9cf80fef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.779+0000 7f9cfde08700 1 --2- 172.21.15.2:0/2456978210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80feac0 0x7f9cf80fef30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009990 tx=0x7f9cec009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 -- 172.21.15.2:0/2456978210 shutdown_connections 2023-06-23T18:21:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 --2- 172.21.15.2:0/2456978210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf80ff470 0x7f9cf810b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 --2- 172.21.15.2:0/2456978210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf8104ac0 0x7f9cf8104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 --2- 172.21.15.2:0/2456978210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80feac0 0x7f9cf80fef30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 -- 172.21.15.2:0/2456978210 >> 172.21.15.2:0/2456978210 conn(0x7f9cf8075970 msgr2=0x7f9cf8075d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 -- 172.21.15.2:0/2456978210 shutdown_connections 2023-06-23T18:21:10.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 -- 172.21.15.2:0/2456978210 wait complete. 2023-06-23T18:21:10.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 Processor -- start 2023-06-23T18:21:10.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.780+0000 7f9cfde08700 1 -- start start 2023-06-23T18:21:10.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cfde08700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf80feac0 0x7f9cf8196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:10.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cfde08700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80ff470 0x7f9cf8196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:10.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cfde08700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf8104ac0 0x7f9cf81a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:10.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cfde08700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81977b0 con 0x7f9cf80ff470 2023-06-23T18:21:10.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cfde08700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81a1f20 con 0x7f9cf8104ac0 2023-06-23T18:21:10.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cfde08700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81a2090 con 0x7f9cf80feac0 2023-06-23T18:21:10.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80ff470 0x7f9cf8196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:10.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80ff470 0x7f9cf8196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52752/0 (socket says 172.21.15.2:52752) 2023-06-23T18:21:10.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf80feac0 0x7f9cf8196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:10.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 -- 172.21.15.2:0/986970778 learned_addr learned my addr 172.21.15.2:0/986970778 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:10.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf7fff700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf8104ac0 0x7f9cf81a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:10.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 -- 172.21.15.2:0/986970778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf80feac0 msgr2=0x7f9cf8196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:10.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9cf80feac0 0x7f9cf8196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:10.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 -- 172.21.15.2:0/986970778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf8104ac0 msgr2=0x7f9cf81a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:10.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf8104ac0 0x7f9cf81a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:10.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 -- 172.21.15.2:0/986970778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0095f0 con 0x7f9cf80ff470 2023-06-23T18:21:10.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf7fff700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9cf8104ac0 0x7f9cf81a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:10.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf6ffd700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80ff470 0x7f9cf8196ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006010 tx=0x7f9cec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:10.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec01d070 con 0x7f9cf80ff470 2023-06-23T18:21:10.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9cec004130 con 0x7f9cf80ff470 2023-06-23T18:21:10.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec022e30 con 0x7f9cf80ff470 2023-06-23T18:21:10.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.781+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf81a2310 con 0x7f9cf80ff470 2023-06-23T18:21:10.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.782+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf81a2720 con 0x7f9cf80ff470 2023-06-23T18:21:10.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.782+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9cf810eaf0 con 0x7f9cf80ff470 2023-06-23T18:21:10.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.784+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9cec022600 con 0x7f9cf80ff470 2023-06-23T18:21:10.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.784+0000 7f9cf4ff9700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9ce00a1260 0x7f9ce00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:10.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.784+0000 7f9cf77fe700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9ce00a1260 0x7f9ce00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:10.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.784+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9cec0ce420 con 0x7f9cf80ff470 2023-06-23T18:21:10.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.784+0000 7f9cf4ff9700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] conn(0x7f9ce00aaa30 0x7f9ce00ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:10.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.784+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 --> [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9ce00ad4f0 con 0x7f9ce00aaa30 2023-06-23T18:21:10.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.784+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cec0ce8c0 con 0x7f9cf80ff470 2023-06-23T18:21:10.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.785+0000 7f9cf7fff700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] conn(0x7f9ce00aaa30 0x7f9ce00ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:10.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.785+0000 7f9cf77fe700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9ce00a1260 0x7f9ce00a3710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80ffc00 tx=0x7f9ce800a3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:10.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.785+0000 7f9cf7fff700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] conn(0x7f9ce00aaa30 0x7f9ce00ace40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:10.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.786+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 <== osd.0 v2:172.21.15.2:6802/2079965032 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9ce00ad4f0 con 0x7f9ce00aaa30 2023-06-23T18:21:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 --> [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9cf804e1d0 con 0x7f9ce00aaa30 2023-06-23T18:21:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cf4ff9700 1 -- 172.21.15.2:0/986970778 <== osd.0 v2:172.21.15.2:6802/2079965032 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9cf804e1d0 con 0x7f9ce00aaa30 2023-06-23T18:21:10.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] conn(0x7f9ce00aaa30 msgr2=0x7f9ce00ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cfde08700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] conn(0x7f9ce00aaa30 0x7f9ce00ace40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9ce00a1260 msgr2=0x7f9ce00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cfde08700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9ce00a1260 0x7f9ce00a3710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80ffc00 tx=0x7f9ce800a3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80ff470 msgr2=0x7f9cf8196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cfde08700 1 --2- 172.21.15.2:0/986970778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9cf80ff470 0x7f9cf8196ea0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9cec006010 tx=0x7f9cec004730 comp rx=0 tx=0).stop 2023-06-23T18:21:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.802+0000 7f9cf7fff700 1 -- 172.21.15.2:0/986970778 reap_dead start 2023-06-23T18:21:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.803+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 shutdown_connections 2023-06-23T18:21:10.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.803+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 >> 172.21.15.2:0/986970778 conn(0x7f9cf8075970 msgr2=0x7f9cf80fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:10.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.803+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 shutdown_connections 2023-06-23T18:21:10.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:10.803+0000 7f9cfde08700 1 -- 172.21.15.2:0/986970778 wait complete. 2023-06-23T18:21:10.813 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:10 smithi002 ceph-mon[95947]: pgmap v269: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-06-23T18:21:11.002 INFO:teuthology.orchestra.run.smithi002.stdout:219043332129 2023-06-23T18:21:11.002 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-06-23T18:21:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:10 smithi082 ceph-mon[97889]: pgmap v269: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-06-23T18:21:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:10 smithi017 ceph-mon[97472]: pgmap v269: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 6 op/s 2023-06-23T18:21:11.573 INFO:teuthology.orchestra.run.smithi002.stdout:34359738431 2023-06-23T18:21:11.573 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-06-23T18:21:12.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.740+0000 7f396903e700 1 -- 172.21.15.2:0/1204872582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 msgr2=0x7f396410b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.740+0000 7f396903e700 1 --2- 172.21.15.2:0/1204872582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 0x7f396410b8b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f395c00b0a0 tx=0x7f395c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.740+0000 7f396903e700 1 -- 172.21.15.2:0/1204872582 shutdown_connections 2023-06-23T18:21:12.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.740+0000 7f396903e700 1 --2- 172.21.15.2:0/1204872582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 0x7f396410b8b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.740+0000 7f396903e700 1 --2- 172.21.15.2:0/1204872582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3964104a70 0x7f3964104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.740+0000 7f396903e700 1 --2- 172.21.15.2:0/1204872582 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39640fea70 0x7f39640feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.740+0000 7f396903e700 1 -- 172.21.15.2:0/1204872582 >> 172.21.15.2:0/1204872582 conn(0x7f3964075b30 msgr2=0x7f3964075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:12.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.741+0000 7f396903e700 1 -- 172.21.15.2:0/1204872582 shutdown_connections 2023-06-23T18:21:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.741+0000 7f396903e700 1 -- 172.21.15.2:0/1204872582 wait complete. 2023-06-23T18:21:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.742+0000 7f396903e700 1 Processor -- start 2023-06-23T18:21:12.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.742+0000 7f396903e700 1 -- start start 2023-06-23T18:21:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.742+0000 7f396903e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39640fea70 0x7f3964197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.742+0000 7f396903e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 0x7f3964198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.742+0000 7f396903e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3964104a70 0x7f396419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.742+0000 7f396903e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396419cd90 con 0x7f39640ff420 2023-06-23T18:21:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.742+0000 7f396903e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396419cf00 con 0x7f3964104a70 2023-06-23T18:21:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.742+0000 7f396903e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396419d0c0 con 0x7f39640fea70 2023-06-23T18:21:12.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 0x7f3964198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 0x7f3964198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52768/0 (socket says 172.21.15.2:52768) 2023-06-23T18:21:12.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f3962d9d700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39640fea70 0x7f3964197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 -- 172.21.15.2:0/556747230 learned_addr learned my addr 172.21.15.2:0/556747230 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:12.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396359e700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3964104a70 0x7f396419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 -- 172.21.15.2:0/556747230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39640fea70 msgr2=0x7f3964197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39640fea70 0x7f3964197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 -- 172.21.15.2:0/556747230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3964104a70 msgr2=0x7f396419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3964104a70 0x7f396419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 -- 172.21.15.2:0/556747230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c009d00 con 0x7f39640ff420 2023-06-23T18:21:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f3962d9d700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39640fea70 0x7f3964197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:21:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396359e700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3964104a70 0x7f396419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:21:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f396259c700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 0x7f3964198310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f395800d950 tx=0x7f395800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.743+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800f890 con 0x7f39640ff420 2023-06-23T18:21:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.744+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396419d3a0 con 0x7f39640ff420 2023-06-23T18:21:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.744+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f395800fe80 con 0x7f39640ff420 2023-06-23T18:21:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.744+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f395800e3b0 con 0x7f39640ff420 2023-06-23T18:21:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.744+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39641a4c00 con 0x7f39640ff420 2023-06-23T18:21:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.745+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f396410eac0 con 0x7f39640ff420 2023-06-23T18:21:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.747+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3958010460 con 0x7f39640ff420 2023-06-23T18:21:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.748+0000 7f394bfff700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f394c0a11e0 0x7f394c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.748+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f39580ccbb0 con 0x7f39640ff420 2023-06-23T18:21:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.748+0000 7f394bfff700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] conn(0x7f394c0aa9b0 0x7f394c0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.748+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 --> [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f394c0ad470 con 0x7f394c0aa9b0 2023-06-23T18:21:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.748+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39580cd050 con 0x7f39640ff420 2023-06-23T18:21:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.748+0000 7f396359e700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] conn(0x7f394c0aa9b0 0x7f394c0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.748+0000 7f3962d9d700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f394c0a11e0 0x7f394c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.749+0000 7f396359e700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] conn(0x7f394c0aa9b0 0x7f394c0acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.749+0000 7f3962d9d700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f394c0a11e0 0x7f394c0a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f39640ffbb0 tx=0x7f3954007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.750+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 <== osd.9 v2:172.21.15.2:6826/1361624360 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f394c0ad470 con 0x7f394c0aa9b0 2023-06-23T18:21:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.772+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 --> [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f396404e1d0 con 0x7f394c0aa9b0 2023-06-23T18:21:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f394bfff700 1 -- 172.21.15.2:0/556747230 <== osd.9 v2:172.21.15.2:6826/1361624360 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f396404e1d0 con 0x7f394c0aa9b0 2023-06-23T18:21:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] conn(0x7f394c0aa9b0 msgr2=0x7f394c0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] conn(0x7f394c0aa9b0 0x7f394c0acdc0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f394c0a11e0 msgr2=0x7f394c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f394c0a11e0 0x7f394c0a3690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f39640ffbb0 tx=0x7f3954007400 comp rx=0 tx=0).stop 2023-06-23T18:21:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 msgr2=0x7f3964198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 --2- 172.21.15.2:0/556747230 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f39640ff420 0x7f3964198310 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f395800d950 tx=0x7f395800dc60 comp rx=0 tx=0).stop 2023-06-23T18:21:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396359e700 1 -- 172.21.15.2:0/556747230 reap_dead start 2023-06-23T18:21:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 shutdown_connections 2023-06-23T18:21:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 >> 172.21.15.2:0/556747230 conn(0x7f3964075b30 msgr2=0x7f3964068010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 shutdown_connections 2023-06-23T18:21:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:11.773+0000 7f396903e700 1 -- 172.21.15.2:0/556747230 wait complete. 2023-06-23T18:21:12.364 INFO:teuthology.orchestra.run.smithi002.stdout:77309411383 2023-06-23T18:21:12.365 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-06-23T18:21:12.465 INFO:teuthology.orchestra.run.smithi002.stdout:107374182449 2023-06-23T18:21:12.466 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-06-23T18:21:12.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 -- 172.21.15.2:0/4132824609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 msgr2=0x7f705c11b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 --2- 172.21.15.2:0/4132824609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 0x7f705c11b430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f704c009990 tx=0x7f704c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 -- 172.21.15.2:0/4132824609 shutdown_connections 2023-06-23T18:21:12.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 --2- 172.21.15.2:0/4132824609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 0x7f705c11b430 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 --2- 172.21.15.2:0/4132824609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c10e300 0x7f705c06cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 --2- 172.21.15.2:0/4132824609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c10e760 0x7f705c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 -- 172.21.15.2:0/4132824609 >> 172.21.15.2:0/4132824609 conn(0x7f705c06be00 msgr2=0x7f705c070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:12.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 -- 172.21.15.2:0/4132824609 shutdown_connections 2023-06-23T18:21:12.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.789+0000 7f7062504700 1 -- 172.21.15.2:0/4132824609 wait complete. 2023-06-23T18:21:12.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.790+0000 7f7062504700 1 Processor -- start 2023-06-23T18:21:12.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.790+0000 7f7062504700 1 -- start start 2023-06-23T18:21:12.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f7062504700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 0x7f705c1a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f7062504700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c10e300 0x7f705c1a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f7062504700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c10e760 0x7f705c1a9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f7062504700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c1a9910 con 0x7f705c06d840 2023-06-23T18:21:12.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f7062504700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c1a9a80 con 0x7f705c10e760 2023-06-23T18:21:12.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f7062504700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c1a9c40 con 0x7f705c10e300 2023-06-23T18:21:12.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 0x7f705c1a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 0x7f705c1a4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52780/0 (socket says 172.21.15.2:52780) 2023-06-23T18:21:12.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705bfff700 1 -- 172.21.15.2:0/2082565145 learned_addr learned my addr 172.21.15.2:0/2082565145 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:12.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705b7fe700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c10e300 0x7f705c1a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f7060aa1700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c10e760 0x7f705c1a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705bfff700 1 -- 172.21.15.2:0/2082565145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c10e300 msgr2=0x7f705c1a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705bfff700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c10e300 0x7f705c1a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705bfff700 1 -- 172.21.15.2:0/2082565145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c10e760 msgr2=0x7f705c1a9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705bfff700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f705c10e760 0x7f705c1a9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.791+0000 7f705bfff700 1 -- 172.21.15.2:0/2082565145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c0095f0 con 0x7f705c06d840 2023-06-23T18:21:12.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.792+0000 7f705bfff700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 0x7f705c1a4960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f705400b9c0 tx=0x7f705400bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.792+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400d8e0 con 0x7f705c06d840 2023-06-23T18:21:12.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.792+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7054009900 con 0x7f705c06d840 2023-06-23T18:21:12.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.792+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400c400 con 0x7f705c06d840 2023-06-23T18:21:12.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.792+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c1a9f20 con 0x7f705c06d840 2023-06-23T18:21:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.792+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c1b17c0 con 0x7f705c06d840 2023-06-23T18:21:12.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.793+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f705c11eab0 con 0x7f705c06d840 2023-06-23T18:21:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.795+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f705400da40 con 0x7f705c06d840 2023-06-23T18:21:12.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.795+0000 7f70597fa700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70440a11e0 0x7f70440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.795+0000 7f705b7fe700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70440a11e0 0x7f70440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.795+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7054011070 con 0x7f705c06d840 2023-06-23T18:21:12.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.796+0000 7f70597fa700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] conn(0x7f70440aa9b0 0x7f70440acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.796+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 --> [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70440ad470 con 0x7f70440aa9b0 2023-06-23T18:21:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.796+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70540ce7c0 con 0x7f705c06d840 2023-06-23T18:21:12.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.796+0000 7f7060aa1700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] conn(0x7f70440aa9b0 0x7f70440acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.796+0000 7f705b7fe700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70440a11e0 0x7f70440a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f705c1a5f80 tx=0x7f7050009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.796+0000 7f7060aa1700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] conn(0x7f70440aa9b0 0x7f70440acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.797+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 <== osd.6 v2:172.21.15.2:6818/617424670 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f70440ad470 con 0x7f70440aa9b0 2023-06-23T18:21:12.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.813+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 --> [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f705c04e1d0 con 0x7f70440aa9b0 2023-06-23T18:21:12.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f70597fa700 1 -- 172.21.15.2:0/2082565145 <== osd.6 v2:172.21.15.2:6818/617424670 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f705c04e1d0 con 0x7f70440aa9b0 2023-06-23T18:21:12.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] conn(0x7f70440aa9b0 msgr2=0x7f70440acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] conn(0x7f70440aa9b0 0x7f70440acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70440a11e0 msgr2=0x7f70440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70440a11e0 0x7f70440a3690 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f705c1a5f80 tx=0x7f7050009190 comp rx=0 tx=0).stop 2023-06-23T18:21:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 msgr2=0x7f705c1a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 --2- 172.21.15.2:0/2082565145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f705c06d840 0x7f705c1a4960 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f705400b9c0 tx=0x7f705400bcd0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7060aa1700 1 -- 172.21.15.2:0/2082565145 reap_dead start 2023-06-23T18:21:12.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 shutdown_connections 2023-06-23T18:21:12.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 >> 172.21.15.2:0/2082565145 conn(0x7f705c06be00 msgr2=0x7f705c06f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:12.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.814+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 shutdown_connections 2023-06-23T18:21:12.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.815+0000 7f7062504700 1 -- 172.21.15.2:0/2082565145 wait complete. 2023-06-23T18:21:12.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/4141941726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb780fea50 msgr2=0x7fdb7810c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 --2- 172.21.15.2:0/4141941726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb780fea50 0x7fdb7810c9c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b0a0 tx=0x7fdb7000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/4141941726 shutdown_connections 2023-06-23T18:21:12.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 --2- 172.21.15.2:0/4141941726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb780fea50 0x7fdb7810c9c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 --2- 172.21.15.2:0/4141941726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb78104b00 0x7fdb78104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 --2- 172.21.15.2:0/4141941726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb780fe0a0 0x7fdb780fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/4141941726 >> 172.21.15.2:0/4141941726 conn(0x7fdb78075110 msgr2=0x7fdb78075510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:12.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/4141941726 shutdown_connections 2023-06-23T18:21:12.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.899+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/4141941726 wait complete. 2023-06-23T18:21:12.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7f4c0700 1 Processor -- start 2023-06-23T18:21:12.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7f4c0700 1 -- start start 2023-06-23T18:21:12.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7f4c0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb780fe0a0 0x7fdb7819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7f4c0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb780fea50 0x7fdb7819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7f4c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb78104b00 0x7fdb78195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7f4c0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb78196120 con 0x7fdb78104b00 2023-06-23T18:21:12.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7f4c0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb78196290 con 0x7fdb780fe0a0 2023-06-23T18:21:12.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7f4c0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb78196450 con 0x7fdb780fea50 2023-06-23T18:21:12.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7da5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb78104b00 0x7fdb78195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7da5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb78104b00 0x7fdb78195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52796/0 (socket says 172.21.15.2:52796) 2023-06-23T18:21:12.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7da5d700 1 -- 172.21.15.2:0/2309883861 learned_addr learned my addr 172.21.15.2:0/2309883861 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:12.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7d25c700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb780fe0a0 0x7fdb7819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7ca5b700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb780fea50 0x7fdb7819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7da5d700 1 -- 172.21.15.2:0/2309883861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb780fea50 msgr2=0x7fdb7819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7da5d700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb780fea50 0x7fdb7819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7da5d700 1 -- 172.21.15.2:0/2309883861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb780fe0a0 msgr2=0x7fdb7819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.900+0000 7fdb7da5d700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb780fe0a0 0x7fdb7819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.901+0000 7fdb7da5d700 1 -- 172.21.15.2:0/2309883861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70009d00 con 0x7fdb78104b00 2023-06-23T18:21:12.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.901+0000 7fdb7ca5b700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdb780fea50 0x7fdb7819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:12.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.901+0000 7fdb7da5d700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb78104b00 0x7fdb78195ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b830 tx=0x7fdb700094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.901+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000e040 con 0x7fdb78104b00 2023-06-23T18:21:12.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.901+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdb70003e90 con 0x7fdb78104b00 2023-06-23T18:21:12.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.901+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb781966d0 con 0x7fdb78104b00 2023-06-23T18:21:12.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.901+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7001d480 con 0x7fdb78104b00 2023-06-23T18:21:12.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.901+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb781a2a20 con 0x7fdb78104b00 2023-06-23T18:21:12.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.902+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb7810fd00 con 0x7fdb78104b00 2023-06-23T18:21:12.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.904+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fdb70003a00 con 0x7fdb78104b00 2023-06-23T18:21:12.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.904+0000 7fdb6e7fc700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdb640a11e0 0x7fdb640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.905+0000 7fdb7d25c700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdb640a11e0 0x7fdb640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.905+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdb700ceaf0 con 0x7fdb78104b00 2023-06-23T18:21:12.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.905+0000 7fdb6e7fc700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] conn(0x7fdb640aa9b0 0x7fdb640acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:12.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.905+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 --> [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb640ad470 con 0x7fdb640aa9b0 2023-06-23T18:21:12.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.905+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb700cef90 con 0x7fdb78104b00 2023-06-23T18:21:12.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.905+0000 7fdb7d25c700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdb640a11e0 0x7fdb640a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb74005950 tx=0x7fdb7400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.905+0000 7fdb7ca5b700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] conn(0x7fdb640aa9b0 0x7fdb640acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:12.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.906+0000 7fdb7ca5b700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] conn(0x7fdb640aa9b0 0x7fdb640acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:12.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.906+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 <== osd.10 v2:172.21.15.17:6824/3783526814 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdb640ad470 con 0x7fdb640aa9b0 2023-06-23T18:21:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.924+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 --> [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdb780fd300 con 0x7fdb640aa9b0 2023-06-23T18:21:12.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb6e7fc700 1 -- 172.21.15.2:0/2309883861 <== osd.10 v2:172.21.15.17:6824/3783526814 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdb780fd300 con 0x7fdb640aa9b0 2023-06-23T18:21:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 >> [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] conn(0x7fdb640aa9b0 msgr2=0x7fdb640acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7f4c0700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] conn(0x7fdb640aa9b0 0x7fdb640acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdb640a11e0 msgr2=0x7fdb640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7f4c0700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdb640a11e0 0x7fdb640a3690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb74005950 tx=0x7fdb7400b410 comp rx=0 tx=0).stop 2023-06-23T18:21:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb78104b00 msgr2=0x7fdb78195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7f4c0700 1 --2- 172.21.15.2:0/2309883861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdb78104b00 0x7fdb78195ab0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000b830 tx=0x7fdb700094e0 comp rx=0 tx=0).stop 2023-06-23T18:21:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7da5d700 1 -- 172.21.15.2:0/2309883861 reap_dead start 2023-06-23T18:21:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 shutdown_connections 2023-06-23T18:21:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.925+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 >> 172.21.15.2:0/2309883861 conn(0x7fdb78075110 msgr2=0x7fdb780fc930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.926+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 shutdown_connections 2023-06-23T18:21:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:12.926+0000 7fdb7f4c0700 1 -- 172.21.15.2:0/2309883861 wait complete. 2023-06-23T18:21:13.074 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:12 smithi002 ceph-mon[95947]: pgmap v270: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2023-06-23T18:21:13.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:12 smithi082 ceph-mon[97889]: pgmap v270: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2023-06-23T18:21:13.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:12 smithi017 ceph-mon[97472]: pgmap v270: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 853 B/s wr, 7 op/s 2023-06-23T18:21:13.523 INFO:teuthology.orchestra.run.smithi002.stdout:279172874265 2023-06-23T18:21:13.523 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-06-23T18:21:13.627 INFO:teuthology.orchestra.run.smithi002.stdout:300647710742 2023-06-23T18:21:13.628 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-06-23T18:21:14.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 -- 172.21.15.2:0/1723747528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fe0a0 msgr2=0x7fcd8c0fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 --2- 172.21.15.2:0/1723747528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fe0a0 0x7fcd8c0fe510 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009990 tx=0x7fcd80009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 -- 172.21.15.2:0/1723747528 shutdown_connections 2023-06-23T18:21:14.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 --2- 172.21.15.2:0/1723747528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd8c0fea50 0x7fcd8c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 --2- 172.21.15.2:0/1723747528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd8c104b00 0x7fcd8c104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 --2- 172.21.15.2:0/1723747528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fe0a0 0x7fcd8c0fe510 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 -- 172.21.15.2:0/1723747528 >> 172.21.15.2:0/1723747528 conn(0x7fcd8c075110 msgr2=0x7fcd8c075510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:14.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 -- 172.21.15.2:0/1723747528 shutdown_connections 2023-06-23T18:21:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 -- 172.21.15.2:0/1723747528 wait complete. 2023-06-23T18:21:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.074+0000 7fcd91a3a700 1 Processor -- start 2023-06-23T18:21:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd91a3a700 1 -- start start 2023-06-23T18:21:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd91a3a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd8c0fe0a0 0x7fcd8c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd91a3a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fea50 0x7fcd8c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd91a3a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd8c104b00 0x7fcd8c1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd91a3a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c196150 con 0x7fcd8c0fea50 2023-06-23T18:21:14.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd91a3a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1962c0 con 0x7fcd8c0fe0a0 2023-06-23T18:21:14.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd91a3a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c196480 con 0x7fcd8c104b00 2023-06-23T18:21:14.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fea50 0x7fcd8c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fea50 0x7fcd8c19bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52812/0 (socket says 172.21.15.2:52812) 2023-06-23T18:21:14.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8a7fc700 1 -- 172.21.15.2:0/2211638634 learned_addr learned my addr 172.21.15.2:0/2211638634 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:14.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8affd700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd8c0fe0a0 0x7fcd8c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8b7fe700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd8c104b00 0x7fcd8c1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8a7fc700 1 -- 172.21.15.2:0/2211638634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd8c104b00 msgr2=0x7fcd8c1959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8a7fc700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcd8c104b00 0x7fcd8c1959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8a7fc700 1 -- 172.21.15.2:0/2211638634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd8c0fe0a0 msgr2=0x7fcd8c19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8a7fc700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd8c0fe0a0 0x7fcd8c19b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8a7fc700 1 -- 172.21.15.2:0/2211638634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd800095f0 con 0x7fcd8c0fea50 2023-06-23T18:21:14.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.075+0000 7fcd8affd700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd8c0fe0a0 0x7fcd8c19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.076+0000 7fcd8a7fc700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fea50 0x7fcd8c19bef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006010 tx=0x7fcd80005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.076+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8001d070 con 0x7fcd8c0fea50 2023-06-23T18:21:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.076+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcd80004100 con 0x7fcd8c0fea50 2023-06-23T18:21:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.076+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c196700 con 0x7fcd8c0fea50 2023-06-23T18:21:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.076+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd80022ce0 con 0x7fcd8c0fea50 2023-06-23T18:21:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.076+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c1a2930 con 0x7fcd8c0fea50 2023-06-23T18:21:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.077+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd8c04e1d0 con 0x7fcd8c0fea50 2023-06-23T18:21:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.078+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fcd80022470 con 0x7fcd8c0fea50 2023-06-23T18:21:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.078+0000 7fcd90a38700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd740a11e0 0x7fcd740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.079+0000 7fcd8affd700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd740a11e0 0x7fcd740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.079+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcd80022e40 con 0x7fcd8c0fea50 2023-06-23T18:21:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.079+0000 7fcd90a38700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] conn(0x7fcd740aa9b0 0x7fcd740acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.079+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 --> [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd740ad470 con 0x7fcd740aa9b0 2023-06-23T18:21:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.079+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd800ce850 con 0x7fcd8c0fea50 2023-06-23T18:21:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.079+0000 7fcd8affd700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd740a11e0 0x7fcd740a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c005fd0 tx=0x7fcd7c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.079+0000 7fcd8b7fe700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] conn(0x7fcd740aa9b0 0x7fcd740acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.080+0000 7fcd8b7fe700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] conn(0x7fcd740aa9b0 0x7fcd740acdc0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.080+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 <== osd.11 v2:172.21.15.82:6824/3300059850 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fcd740ad470 con 0x7fcd740aa9b0 2023-06-23T18:21:14.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.097+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 --> [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcd8c0fd2f0 con 0x7fcd740aa9b0 2023-06-23T18:21:14.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.098+0000 7fcd90a38700 1 -- 172.21.15.2:0/2211638634 <== osd.11 v2:172.21.15.82:6824/3300059850 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fcd8c0fd2f0 con 0x7fcd740aa9b0 2023-06-23T18:21:14.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.098+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 >> [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] conn(0x7fcd740aa9b0 msgr2=0x7fcd740acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.098+0000 7fcd91a3a700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] conn(0x7fcd740aa9b0 0x7fcd740acdc0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.098+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd740a11e0 msgr2=0x7fcd740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.098+0000 7fcd91a3a700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcd740a11e0 0x7fcd740a3690 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c005fd0 tx=0x7fcd7c005ee0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.098+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fea50 msgr2=0x7fcd8c19bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.098+0000 7fcd91a3a700 1 --2- 172.21.15.2:0/2211638634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd8c0fea50 0x7fcd8c19bef0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcd80006010 tx=0x7fcd80005ef0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.098+0000 7fcd8b7fe700 1 -- 172.21.15.2:0/2211638634 reap_dead start 2023-06-23T18:21:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.099+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 shutdown_connections 2023-06-23T18:21:14.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.099+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 >> 172.21.15.2:0/2211638634 conn(0x7fcd8c075110 msgr2=0x7fcd8c0fc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:14.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.099+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 shutdown_connections 2023-06-23T18:21:14.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.099+0000 7fcd91a3a700 1 -- 172.21.15.2:0/2211638634 wait complete. 2023-06-23T18:21:14.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 -- 172.21.15.2:0/776016103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7288104a70 msgr2=0x7f7288104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 --2- 172.21.15.2:0/776016103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7288104a70 0x7f7288104e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7284009990 tx=0x7f7284009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 -- 172.21.15.2:0/776016103 shutdown_connections 2023-06-23T18:21:14.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 --2- 172.21.15.2:0/776016103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72880ff420 0x7f728810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 --2- 172.21.15.2:0/776016103 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7288104a70 0x7f7288104e40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 --2- 172.21.15.2:0/776016103 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72880fea70 0x7f72880feee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 -- 172.21.15.2:0/776016103 >> 172.21.15.2:0/776016103 conn(0x7f7288075b30 msgr2=0x7f7288075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:14.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 -- 172.21.15.2:0/776016103 shutdown_connections 2023-06-23T18:21:14.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.408+0000 7f728fd61700 1 -- 172.21.15.2:0/776016103 wait complete. 2023-06-23T18:21:14.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728fd61700 1 Processor -- start 2023-06-23T18:21:14.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728fd61700 1 -- start start 2023-06-23T18:21:14.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728fd61700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72880fea70 0x7f7288196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728fd61700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72880ff420 0x7f7288196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728fd61700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7288104a70 0x7f72881a19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728fd61700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7288197790 con 0x7f72880fea70 2023-06-23T18:21:14.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728fd61700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72881a1f00 con 0x7f72880ff420 2023-06-23T18:21:14.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728fd61700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72881a2070 con 0x7f7288104a70 2023-06-23T18:21:14.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72880fea70 0x7f7288196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72880fea70 0x7f7288196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52826/0 (socket says 172.21.15.2:52826) 2023-06-23T18:21:14.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 -- 172.21.15.2:0/2580779841 learned_addr learned my addr 172.21.15.2:0/2580779841 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:14.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728e2fe700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7288104a70 0x7f72881a19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728d2fc700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72880ff420 0x7f7288196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 -- 172.21.15.2:0/2580779841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7288104a70 msgr2=0x7f72881a19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7288104a70 0x7f72881a19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 -- 172.21.15.2:0/2580779841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72880ff420 msgr2=0x7f7288196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72880ff420 0x7f7288196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 -- 172.21.15.2:0/2580779841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72840095f0 con 0x7f72880fea70 2023-06-23T18:21:14.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728d2fc700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f72880ff420 0x7f7288196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:14.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.409+0000 7f728dafd700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72880fea70 0x7f7288196940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7284004730 tx=0x7f7284009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:14.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.410+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728401d070 con 0x7f72880fea70 2023-06-23T18:21:14.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.410+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72881a22f0 con 0x7f72880fea70 2023-06-23T18:21:14.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.410+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7284004ca0 con 0x7f72880fea70 2023-06-23T18:21:14.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.410+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284005030 con 0x7f72880fea70 2023-06-23T18:21:14.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.410+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72881a2760 con 0x7f72880fea70 2023-06-23T18:21:14.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.410+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f728804e1d0 con 0x7f72880fea70 2023-06-23T18:21:14.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.412+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7284003b70 con 0x7f72880fea70 2023-06-23T18:21:14.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.412+0000 7f727effd700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f72740a11e0 0x7f72740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.412+0000 7f728d2fc700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f72740a11e0 0x7f72740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.412+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72840ce010 con 0x7f72880fea70 2023-06-23T18:21:14.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.413+0000 7f727effd700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] conn(0x7f72740aa9b0 0x7f72740acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:14.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.413+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 --> [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72740ad470 con 0x7f72740aa9b0 2023-06-23T18:21:14.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.413+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72840ce4b0 con 0x7f72880fea70 2023-06-23T18:21:14.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.413+0000 7f728d2fc700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f72740a11e0 0x7f72740a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7278005fd0 tx=0x7f7278005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:14.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.413+0000 7f728e2fe700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] conn(0x7f72740aa9b0 0x7f72740acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:14.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.413+0000 7f728e2fe700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] conn(0x7f72740aa9b0 0x7f72740acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:14.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.414+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 <== osd.5 v2:172.21.15.82:6808/102458807 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f72740ad470 con 0x7f72740aa9b0 2023-06-23T18:21:14.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.430+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 --> [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f728810eac0 con 0x7f72740aa9b0 2023-06-23T18:21:14.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f727effd700 1 -- 172.21.15.2:0/2580779841 <== osd.5 v2:172.21.15.82:6808/102458807 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f72800098c0 con 0x7f72740aa9b0 2023-06-23T18:21:14.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 >> [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] conn(0x7f72740aa9b0 msgr2=0x7f72740acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] conn(0x7f72740aa9b0 0x7f72740acdc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:14.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f72740a11e0 msgr2=0x7f72740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f72740a11e0 0x7f72740a3690 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7278005fd0 tx=0x7f7278005f00 comp rx=0 tx=0).stop 2023-06-23T18:21:14.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72880fea70 msgr2=0x7f7288196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:14.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 --2- 172.21.15.2:0/2580779841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f72880fea70 0x7f7288196940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7284004730 tx=0x7f7284009580 comp rx=0 tx=0).stop 2023-06-23T18:21:14.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728e2fe700 1 -- 172.21.15.2:0/2580779841 reap_dead start 2023-06-23T18:21:14.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 shutdown_connections 2023-06-23T18:21:14.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 >> 172.21.15.2:0/2580779841 conn(0x7f7288075b30 msgr2=0x7f72880fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:14.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 shutdown_connections 2023-06-23T18:21:14.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:14.431+0000 7f728fd61700 1 -- 172.21.15.2:0/2580779841 wait complete. 2023-06-23T18:21:14.468 INFO:teuthology.orchestra.run.smithi002.stdout:197568495654 2023-06-23T18:21:14.469 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-06-23T18:21:14.752 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:15.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 -- 172.21.15.2:0/2574220272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc40ffdc0 msgr2=0x7efdc4100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:15.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 --2- 172.21.15.2:0/2574220272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc40ffdc0 0x7efdc4100190 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efdb80099c0 tx=0x7efdb8009cd0 comp rx=0 tx=0).stop 2023-06-23T18:21:15.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 -- 172.21.15.2:0/2574220272 shutdown_connections 2023-06-23T18:21:15.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 --2- 172.21.15.2:0/2574220272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc4104e60 0x7efdc410c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:15.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 --2- 172.21.15.2:0/2574220272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc40ffdc0 0x7efdc4100190 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:15.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 --2- 172.21.15.2:0/2574220272 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdc4100760 0x7efdc4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:15.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 -- 172.21.15.2:0/2574220272 >> 172.21.15.2:0/2574220272 conn(0x7efdc40fb620 msgr2=0x7efdc40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:15.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 -- 172.21.15.2:0/2574220272 shutdown_connections 2023-06-23T18:21:15.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.088+0000 7efdc962d700 1 -- 172.21.15.2:0/2574220272 wait complete. 2023-06-23T18:21:15.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc962d700 1 Processor -- start 2023-06-23T18:21:15.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc962d700 1 -- start start 2023-06-23T18:21:15.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc962d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdc40ffdc0 0x7efdc419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:15.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc962d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc4100760 0x7efdc419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:15.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc962d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc4104e60 0x7efdc41959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:15.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc962d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc4196010 con 0x7efdc4104e60 2023-06-23T18:21:15.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc962d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc4196180 con 0x7efdc40ffdc0 2023-06-23T18:21:15.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc962d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc4196340 con 0x7efdc4100760 2023-06-23T18:21:15.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc4104e60 0x7efdc41959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:15.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdc40ffdc0 0x7efdc419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:15.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc4104e60 0x7efdc41959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52844/0 (socket says 172.21.15.2:52844) 2023-06-23T18:21:15.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc27fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc4100760 0x7efdc419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:15.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.089+0000 7efdc37fe700 1 -- 172.21.15.2:0/3203861216 learned_addr learned my addr 172.21.15.2:0/3203861216 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:15.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdc37fe700 1 -- 172.21.15.2:0/3203861216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc4100760 msgr2=0x7efdc419be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:15.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdc37fe700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efdc4100760 0x7efdc419be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:15.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdc37fe700 1 -- 172.21.15.2:0/3203861216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdc40ffdc0 msgr2=0x7efdc419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:15.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdc37fe700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdc40ffdc0 0x7efdc419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:15.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdc37fe700 1 -- 172.21.15.2:0/3203861216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb80095f0 con 0x7efdc4104e60 2023-06-23T18:21:15.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdc37fe700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc4104e60 0x7efdc41959a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00ada0 tx=0x7efdbc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:15.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00f070 con 0x7efdc4104e60 2023-06-23T18:21:15.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc4196620 con 0x7efdc4104e60 2023-06-23T18:21:15.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdbc007b30 con 0x7efdc4104e60 2023-06-23T18:21:15.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc005280 con 0x7efdc4104e60 2023-06-23T18:21:15.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.090+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc41a2910 con 0x7efdc4104e60 2023-06-23T18:21:15.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.091+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efdc410fbd0 con 0x7efdc4104e60 2023-06-23T18:21:15.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.092+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7efdbc00a080 con 0x7efdc4104e60 2023-06-23T18:21:15.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.093+0000 7efdabfff700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdac0a11e0 0x7efdac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:15.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.093+0000 7efdc2ffd700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdac0a11e0 0x7efdac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:15.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.093+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efdbc0cc7b0 con 0x7efdc4104e60 2023-06-23T18:21:15.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.093+0000 7efdabfff700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] conn(0x7efdac0aa9b0 0x7efdac0acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:15.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.093+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 --> [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efdac0ad470 con 0x7efdac0aa9b0 2023-06-23T18:21:15.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.093+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdbc0ccc50 con 0x7efdc4104e60 2023-06-23T18:21:15.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.093+0000 7efdc2ffd700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdac0a11e0 0x7efdac0a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efdb800b580 tx=0x7efdb8009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:15.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.093+0000 7efdc27fc700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] conn(0x7efdac0aa9b0 0x7efdac0acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:15.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.094+0000 7efdc27fc700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] conn(0x7efdac0aa9b0 0x7efdac0acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:15.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.094+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 <== osd.8 v2:172.21.15.82:6816/1305476760 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efdac0ad470 con 0x7efdac0aa9b0 2023-06-23T18:21:15.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.116+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 --> [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efdc40fcbc0 con 0x7efdac0aa9b0 2023-06-23T18:21:15.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdabfff700 1 -- 172.21.15.2:0/3203861216 <== osd.8 v2:172.21.15.82:6816/1305476760 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efdc40fcbc0 con 0x7efdac0aa9b0 2023-06-23T18:21:15.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 >> [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] conn(0x7efdac0aa9b0 msgr2=0x7efdac0acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:15.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] conn(0x7efdac0aa9b0 0x7efdac0acdc0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:15.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdac0a11e0 msgr2=0x7efdac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:15.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdac0a11e0 0x7efdac0a3690 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efdb800b580 tx=0x7efdb8009f10 comp rx=0 tx=0).stop 2023-06-23T18:21:15.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc4104e60 msgr2=0x7efdc41959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:15.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 --2- 172.21.15.2:0/3203861216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efdc4104e60 0x7efdc41959a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00ada0 tx=0x7efdbc00c330 comp rx=0 tx=0).stop 2023-06-23T18:21:15.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc37fe700 1 -- 172.21.15.2:0/3203861216 reap_dead start 2023-06-23T18:21:15.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 shutdown_connections 2023-06-23T18:21:15.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 >> 172.21.15.2:0/3203861216 conn(0x7efdc40fb620 msgr2=0x7efdc40fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:15.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 shutdown_connections 2023-06-23T18:21:15.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:15.117+0000 7efdc962d700 1 -- 172.21.15.2:0/3203861216 wait complete. 2023-06-23T18:21:15.153 INFO:teuthology.orchestra.run.smithi002.stdout:330712481809 2023-06-23T18:21:15.154 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-06-23T18:21:15.345 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:15 smithi002 ceph-mon[95947]: pgmap v271: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-06-23T18:21:15.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:15 smithi082 ceph-mon[97889]: pgmap v271: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-06-23T18:21:15.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:15 smithi017 ceph-mon[97472]: pgmap v271: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2023-06-23T18:21:15.902 INFO:teuthology.orchestra.run.smithi002.stdout:249108103198 2023-06-23T18:21:15.903 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-06-23T18:21:15.971 INFO:teuthology.orchestra.run.smithi002.stdout:163208757292 2023-06-23T18:21:15.972 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-06-23T18:21:16.153 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:16.489 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:16.910 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:16.983 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:16 smithi002 ceph-mon[95947]: pgmap v272: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-06-23T18:21:17.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 -- 172.21.15.2:0/211430324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 msgr2=0x7f0320100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:17.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 --2- 172.21.15.2:0/211430324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 0x7f0320100a40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f03140099c0 tx=0x7f0314009cd0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 -- 172.21.15.2:0/211430324 shutdown_connections 2023-06-23T18:21:17.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 --2- 172.21.15.2:0/211430324 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0320100f80 0x7f032010b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 --2- 172.21.15.2:0/211430324 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03200fee40 0x7f03200ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 --2- 172.21.15.2:0/211430324 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 0x7f0320100a40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 -- 172.21.15.2:0/211430324 >> 172.21.15.2:0/211430324 conn(0x7f0320075b20 msgr2=0x7f0320075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:17.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 -- 172.21.15.2:0/211430324 shutdown_connections 2023-06-23T18:21:17.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.017+0000 7f03265eb700 1 -- 172.21.15.2:0/211430324 wait complete. 2023-06-23T18:21:17.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f03265eb700 1 Processor -- start 2023-06-23T18:21:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f03265eb700 1 -- start start 2023-06-23T18:21:17.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f03265eb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03200fee40 0x7f0320196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:17.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f03265eb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 0x7f0320196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:17.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f03265eb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100f80 0x7f03201a1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:17.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f03265eb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0320197870 con 0x7f03201005d0 2023-06-23T18:21:17.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f03265eb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201a1fe0 con 0x7f03200fee40 2023-06-23T18:21:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f03265eb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03201a2150 con 0x7f0320100f80 2023-06-23T18:21:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f031f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 0x7f0320196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f031f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 0x7f0320196f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52858/0 (socket says 172.21.15.2:52858) 2023-06-23T18:21:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.018+0000 7f031f7fe700 1 -- 172.21.15.2:0/1238259745 learned_addr learned my addr 172.21.15.2:0/1238259745 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f0324b88700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100f80 0x7f03201a1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f031ffff700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03200fee40 0x7f0320196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f0324b88700 1 -- 172.21.15.2:0/1238259745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03200fee40 msgr2=0x7f0320196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f0324b88700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03200fee40 0x7f0320196a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f0324b88700 1 -- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 msgr2=0x7f0320196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:17.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f0324b88700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 0x7f0320196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f0324b88700 1 -- 172.21.15.2:0/1238259745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140095f0 con 0x7f0320100f80 2023-06-23T18:21:17.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f031f7fe700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 0x7f0320196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:21:17.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f0324b88700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100f80 0x7f03201a1aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f031800cc50 tx=0x7f031800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:17.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f031d7fa700 1 -- 172.21.15.2:0/1238259745 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031800aa90 con 0x7f0320100f80 2023-06-23T18:21:17.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03201a2430 con 0x7f0320100f80 2023-06-23T18:21:17.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f031d7fa700 1 -- 172.21.15.2:0/1238259745 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f031800abf0 con 0x7f0320100f80 2023-06-23T18:21:17.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.019+0000 7f031d7fa700 1 -- 172.21.15.2:0/1238259745 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03180115e0 con 0x7f0320100f80 2023-06-23T18:21:17.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.020+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03201a28e0 con 0x7f0320100f80 2023-06-23T18:21:17.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.022+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032004e1d0 con 0x7f0320100f80 2023-06-23T18:21:17.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.022+0000 7f031d7fa700 1 -- 172.21.15.2:0/1238259745 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f0318012420 con 0x7f0320100f80 2023-06-23T18:21:17.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.022+0000 7f031d7fa700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f03080a11e0 0x7f03080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:17.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.022+0000 7f031d7fa700 1 -- 172.21.15.2:0/1238259745 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f03180cc990 con 0x7f0320100f80 2023-06-23T18:21:17.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.023+0000 7f031ffff700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f03080a11e0 0x7f03080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:17.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.023+0000 7f031ffff700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f03080a11e0 0x7f03080a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0310009c60 tx=0x7f0310009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:17.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.027+0000 7f031d7fa700 1 -- 172.21.15.2:0/1238259745 <== mon.1 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0318096140 con 0x7f0320100f80 2023-06-23T18:21:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:16 smithi082 ceph-mon[97889]: pgmap v272: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-06-23T18:21:17.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:16 smithi017 ceph-mon[97472]: pgmap v272: 97 pgs: 97 active+clean; 635 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 5 op/s 2023-06-23T18:21:17.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.232+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f03201a2b60 con 0x7f0320100f80 2023-06-23T18:21:17.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.232+0000 7f031d7fa700 1 -- 172.21.15.2:0/1238259745 <== mon.1 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0318016020 con 0x7f0320100f80 2023-06-23T18:21:17.238 INFO:teuthology.orchestra.run.smithi002.stdout:219043332130 2023-06-23T18:21:17.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f03080a11e0 msgr2=0x7f03080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:17.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f03080a11e0 0x7f03080a3690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0310009c60 tx=0x7f0310009260 comp rx=0 tx=0).stop 2023-06-23T18:21:17.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100f80 msgr2=0x7f03201a1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:17.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100f80 0x7f03201a1aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f031800cc50 tx=0x7f031800cf60 comp rx=0 tx=0).stop 2023-06-23T18:21:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 shutdown_connections 2023-06-23T18:21:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0320100f80 0x7f03201a1aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f03080a11e0 0x7f03080a3690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03200fee40 0x7f0320196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 --2- 172.21.15.2:0/1238259745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f03201005d0 0x7f0320196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 >> 172.21.15.2:0/1238259745 conn(0x7f0320075b20 msgr2=0x7f03200fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 shutdown_connections 2023-06-23T18:21:17.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:17.235+0000 7f03265eb700 1 -- 172.21.15.2:0/1238259745 wait complete. 2023-06-23T18:21:18.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:17 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1238259745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-06-23T18:21:18.085 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:18.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:17 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1238259745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-06-23T18:21:18.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:17 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1238259745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-06-23T18:21:18.267 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:18.268 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:18.279 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:18.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332129 got 219043332130 for osd.7 2023-06-23T18:21:18.316 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:18.722 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:18.993 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:18.994 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:18.995 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:19.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:18 smithi002 ceph-mon[95947]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-06-23T18:21:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:18 smithi082 ceph-mon[97889]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-06-23T18:21:19.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:18 smithi017 ceph-mon[97472]: pgmap v273: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 7 op/s 2023-06-23T18:21:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.223+0000 7fc82e5fe700 1 -- 172.21.15.2:0/2412071389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8280fea70 msgr2=0x7fc8280feee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.223+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/2412071389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8280fea70 0x7fc8280feee0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009990 tx=0x7fc81c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 -- 172.21.15.2:0/2412071389 shutdown_connections 2023-06-23T18:21:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/2412071389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8280ff420 0x7fc82810b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/2412071389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc828104a70 0x7fc828104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/2412071389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8280fea70 0x7fc8280feee0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 -- 172.21.15.2:0/2412071389 >> 172.21.15.2:0/2412071389 conn(0x7fc828075b30 msgr2=0x7fc828075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 -- 172.21.15.2:0/2412071389 shutdown_connections 2023-06-23T18:21:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 -- 172.21.15.2:0/2412071389 wait complete. 2023-06-23T18:21:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 Processor -- start 2023-06-23T18:21:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.224+0000 7fc82e5fe700 1 -- start start 2023-06-23T18:21:19.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82e5fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8280fea70 0x7fc828194690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:19.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82e5fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8280ff420 0x7fc828194bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:19.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82e5fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc828104a70 0x7fc828195110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82e5fe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8281a3ed0 con 0x7fc828104a70 2023-06-23T18:21:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82e5fe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8281a4040 con 0x7fc8280fea70 2023-06-23T18:21:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82e5fe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8281a4200 con 0x7fc8280ff420 2023-06-23T18:21:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc828104a70 0x7fc828195110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc828104a70 0x7fc828195110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52872/0 (socket says 172.21.15.2:52872) 2023-06-23T18:21:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 -- 172.21.15.2:0/3765675013 learned_addr learned my addr 172.21.15.2:0/3765675013 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc8277fe700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8280ff420 0x7fc828194bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc827fff700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8280fea70 0x7fc828194690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 -- 172.21.15.2:0/3765675013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8280ff420 msgr2=0x7fc828194bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8280ff420 0x7fc828194bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 -- 172.21.15.2:0/3765675013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8280fea70 msgr2=0x7fc828194690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:19.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8280fea70 0x7fc828194690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 -- 172.21.15.2:0/3765675013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0095f0 con 0x7fc828104a70 2023-06-23T18:21:19.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc827fff700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8280fea70 0x7fc828194690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:19.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.225+0000 7fc82cb9b700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc828104a70 0x7fc828195110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc820007f10 tx=0x7fc82000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.226+0000 7fc8257fa700 1 -- 172.21.15.2:0/3765675013 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820017070 con 0x7fc828104a70 2023-06-23T18:21:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.226+0000 7fc8257fa700 1 -- 172.21.15.2:0/3765675013 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc82000f040 con 0x7fc828104a70 2023-06-23T18:21:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.226+0000 7fc8257fa700 1 -- 172.21.15.2:0/3765675013 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc820005710 con 0x7fc828104a70 2023-06-23T18:21:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.226+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8281a44e0 con 0x7fc828104a70 2023-06-23T18:21:19.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.226+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8281a4960 con 0x7fc828104a70 2023-06-23T18:21:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.227+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82804e1d0 con 0x7fc828104a70 2023-06-23T18:21:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.228+0000 7fc8257fa700 1 -- 172.21.15.2:0/3765675013 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc820005870 con 0x7fc828104a70 2023-06-23T18:21:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.228+0000 7fc8257fa700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8100a11e0 0x7fc8100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:19.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.228+0000 7fc827fff700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8100a11e0 0x7fc8100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.229+0000 7fc8257fa700 1 -- 172.21.15.2:0/3765675013 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8200cc670 con 0x7fc828104a70 2023-06-23T18:21:19.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.229+0000 7fc827fff700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8100a11e0 0x7fc8100a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc818005fd0 tx=0x7fc818005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:19.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.232+0000 7fc8257fa700 1 -- 172.21.15.2:0/3765675013 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc820095e10 con 0x7fc828104a70 2023-06-23T18:21:19.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.445+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc828068580 con 0x7fc828104a70 2023-06-23T18:21:19.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.445+0000 7fc8257fa700 1 -- 172.21.15.2:0/3765675013 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc820099430 con 0x7fc828104a70 2023-06-23T18:21:19.452 INFO:teuthology.orchestra.run.smithi002.stdout:55834574909 2023-06-23T18:21:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8100a11e0 msgr2=0x7fc8100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8100a11e0 0x7fc8100a3690 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc818005fd0 tx=0x7fc818005e40 comp rx=0 tx=0).stop 2023-06-23T18:21:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc828104a70 msgr2=0x7fc828195110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:19.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc828104a70 0x7fc828195110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc820007f10 tx=0x7fc82000d3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 shutdown_connections 2023-06-23T18:21:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc828104a70 0x7fc828195110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8100a11e0 0x7fc8100a3690 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8280fea70 0x7fc828194690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 --2- 172.21.15.2:0/3765675013 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8280ff420 0x7fc828194bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:19.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 >> 172.21.15.2:0/3765675013 conn(0x7fc828075b30 msgr2=0x7fc8280fcd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:19.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 shutdown_connections 2023-06-23T18:21:19.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:19.447+0000 7fc82e5fe700 1 -- 172.21.15.2:0/3765675013 wait complete. 2023-06-23T18:21:19.862 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574909 for osd.1 2023-06-23T18:21:19.862 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:20.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:20 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3765675013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-06-23T18:21:20.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:20 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3765675013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-06-23T18:21:20.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:20 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3765675013' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-06-23T18:21:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 -- 172.21.15.2:0/3992771871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 msgr2=0x7ff1940feee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 --2- 172.21.15.2:0/3992771871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 0x7ff1940feee0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff184009990 tx=0x7ff184009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 -- 172.21.15.2:0/3992771871 shutdown_connections 2023-06-23T18:21:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 --2- 172.21.15.2:0/3992771871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1940ff420 0x7ff19410b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 --2- 172.21.15.2:0/3992771871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff194104a70 0x7ff194104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:20.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 --2- 172.21.15.2:0/3992771871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 0x7ff1940feee0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:20.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 -- 172.21.15.2:0/3992771871 >> 172.21.15.2:0/3992771871 conn(0x7ff194075b30 msgr2=0x7ff194075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:20.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 -- 172.21.15.2:0/3992771871 shutdown_connections 2023-06-23T18:21:20.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.824+0000 7ff1999c3700 1 -- 172.21.15.2:0/3992771871 wait complete. 2023-06-23T18:21:20.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.825+0000 7ff1999c3700 1 Processor -- start 2023-06-23T18:21:20.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.825+0000 7ff1999c3700 1 -- start start 2023-06-23T18:21:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.825+0000 7ff1999c3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 0x7ff194196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.825+0000 7ff1999c3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1940ff420 0x7ff194196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.825+0000 7ff1999c3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff194104a70 0x7ff1941a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.825+0000 7ff1999c3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff194197780 con 0x7ff1940fea70 2023-06-23T18:21:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.825+0000 7ff1999c3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941a1ec0 con 0x7ff1940ff420 2023-06-23T18:21:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.825+0000 7ff1999c3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1941a2080 con 0x7ff194104a70 2023-06-23T18:21:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 0x7ff194196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:20.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 0x7ff194196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52882/0 (socket says 172.21.15.2:52882) 2023-06-23T18:21:20.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 -- 172.21.15.2:0/33928862 learned_addr learned my addr 172.21.15.2:0/33928862 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:20.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff1927fc700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1940ff420 0x7ff194196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:20.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff1937fe700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff194104a70 0x7ff1941a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:20.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 -- 172.21.15.2:0/33928862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff194104a70 msgr2=0x7ff1941a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:20.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff194104a70 0x7ff1941a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:20.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 -- 172.21.15.2:0/33928862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1940ff420 msgr2=0x7ff194196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:20.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1940ff420 0x7ff194196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:20.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 -- 172.21.15.2:0/33928862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840095f0 con 0x7ff1940fea70 2023-06-23T18:21:20.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff1927fc700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1940ff420 0x7ff194196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:20.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff1937fe700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff194104a70 0x7ff1941a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:20.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff192ffd700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 0x7ff194196900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff18800d980 tx=0x7ff18800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:20.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff1989c1700 1 -- 172.21.15.2:0/33928862 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18800de60 con 0x7ff1940fea70 2023-06-23T18:21:20.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1941a2360 con 0x7ff1940fea70 2023-06-23T18:21:20.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff1989c1700 1 -- 172.21.15.2:0/33928862 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff1880044b0 con 0x7ff1940fea70 2023-06-23T18:21:20.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.826+0000 7ff1989c1700 1 -- 172.21.15.2:0/33928862 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff188003b40 con 0x7ff1940fea70 2023-06-23T18:21:20.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.827+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1941a2860 con 0x7ff1940fea70 2023-06-23T18:21:20.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.828+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19404e1d0 con 0x7ff1940fea70 2023-06-23T18:21:20.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.830+0000 7ff1989c1700 1 -- 172.21.15.2:0/33928862 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff188004870 con 0x7ff1940fea70 2023-06-23T18:21:20.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.830+0000 7ff1989c1700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff17c0a11e0 0x7ff17c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:20.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.830+0000 7ff1989c1700 1 -- 172.21.15.2:0/33928862 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff1880cc9a0 con 0x7ff1940fea70 2023-06-23T18:21:20.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.830+0000 7ff1927fc700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff17c0a11e0 0x7ff17c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:20.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.831+0000 7ff1927fc700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff17c0a11e0 0x7ff17c0a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff194068860 tx=0x7ff184005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:20.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:20.834+0000 7ff1989c1700 1 -- 172.21.15.2:0/33928862 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff188096140 con 0x7ff1940fea70 2023-06-23T18:21:21.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 -- 172.21.15.2:0/3328257962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40580fee40 msgr2=0x7f40580ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:21.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 --2- 172.21.15.2:0/3328257962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40580fee40 0x7f40580ff210 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f40540099c0 tx=0x7f4054009cd0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 -- 172.21.15.2:0/3328257962 shutdown_connections 2023-06-23T18:21:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 --2- 172.21.15.2:0/3328257962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4058100f80 0x7f405810b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 --2- 172.21.15.2:0/3328257962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40580fee40 0x7f40580ff210 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 --2- 172.21.15.2:0/3328257962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40581005d0 0x7f4058100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 -- 172.21.15.2:0/3328257962 >> 172.21.15.2:0/3328257962 conn(0x7f4058075b20 msgr2=0x7f4058075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 -- 172.21.15.2:0/3328257962 shutdown_connections 2023-06-23T18:21:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 -- 172.21.15.2:0/3328257962 wait complete. 2023-06-23T18:21:21.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 Processor -- start 2023-06-23T18:21:21.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.034+0000 7f4060225700 1 -- start start 2023-06-23T18:21:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f4060225700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40580fee40 0x7f405819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f4060225700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40581005d0 0x7f405819bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f4060225700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4058100f80 0x7f4058195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f4060225700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4058196140 con 0x7f4058100f80 2023-06-23T18:21:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f4060225700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40581962b0 con 0x7f40580fee40 2023-06-23T18:21:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f4060225700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4058196470 con 0x7f40581005d0 2023-06-23T18:21:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4058100f80 0x7f4058195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4058100f80 0x7f4058195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52900/0 (socket says 172.21.15.2:52900) 2023-06-23T18:21:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 -- 172.21.15.2:0/1461044964 learned_addr learned my addr 172.21.15.2:0/1461044964 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405dfc1700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40580fee40 0x7f405819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405d7c0700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40581005d0 0x7f405819bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 -- 172.21.15.2:0/1461044964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40581005d0 msgr2=0x7f405819bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40581005d0 0x7f405819bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 -- 172.21.15.2:0/1461044964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40580fee40 msgr2=0x7f405819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40580fee40 0x7f405819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 -- 172.21.15.2:0/1461044964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540095f0 con 0x7f4058100f80 2023-06-23T18:21:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405d7c0700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40581005d0 0x7f405819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.035+0000 7f405e7c2700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4058100f80 0x7f4058195ad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4050007ee0 tx=0x7f405000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.036+0000 7f404effd700 1 -- 172.21.15.2:0/1461044964 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050017070 con 0x7f4058100f80 2023-06-23T18:21:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.036+0000 7f404effd700 1 -- 172.21.15.2:0/1461044964 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f405000f040 con 0x7f4058100f80 2023-06-23T18:21:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.036+0000 7f404effd700 1 -- 172.21.15.2:0/1461044964 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4050005710 con 0x7f4058100f80 2023-06-23T18:21:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.036+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4058196750 con 0x7f4058100f80 2023-06-23T18:21:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.036+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40581a2a40 con 0x7f4058100f80 2023-06-23T18:21:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.037+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40581080c0 con 0x7f4058100f80 2023-06-23T18:21:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.038+0000 7f404effd700 1 -- 172.21.15.2:0/1461044964 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4050005870 con 0x7f4058100f80 2023-06-23T18:21:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.038+0000 7f404effd700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f40440a11e0 0x7f40440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.039+0000 7f405dfc1700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f40440a11e0 0x7f40440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.039+0000 7f404effd700 1 -- 172.21.15.2:0/1461044964 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f40500cc1e0 con 0x7f4058100f80 2023-06-23T18:21:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.039+0000 7f405dfc1700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f40440a11e0 0x7f40440a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f405400b580 tx=0x7f4054009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.043+0000 7f404effd700 1 -- 172.21.15.2:0/1461044964 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4050095880 con 0x7f4058100f80 2023-06-23T18:21:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.047+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff1940685a0 con 0x7ff1940fea70 2023-06-23T18:21:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.048+0000 7ff1989c1700 1 -- 172.21.15.2:0/33928862 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff188009940 con 0x7ff1940fea70 2023-06-23T18:21:21.054 INFO:teuthology.orchestra.run.smithi002.stdout:34359738433 2023-06-23T18:21:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.050+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff17c0a11e0 msgr2=0x7ff17c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.050+0000 7ff1999c3700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff17c0a11e0 0x7ff17c0a3690 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff194068860 tx=0x7ff184005fd0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.050+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 msgr2=0x7ff194196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.050+0000 7ff1999c3700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 0x7ff194196900 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff18800d980 tx=0x7ff18800dd40 comp rx=0 tx=0).stop 2023-06-23T18:21:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.053+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 shutdown_connections 2023-06-23T18:21:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.053+0000 7ff1999c3700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff194104a70 0x7ff1941a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.053+0000 7ff1999c3700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1940fea70 0x7ff194196900 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.053+0000 7ff1999c3700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff17c0a11e0 0x7ff17c0a3690 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.053+0000 7ff1999c3700 1 --2- 172.21.15.2:0/33928862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1940ff420 0x7ff194196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.053+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 >> 172.21.15.2:0/33928862 conn(0x7ff194075b30 msgr2=0x7ff1941026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.053+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 shutdown_connections 2023-06-23T18:21:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.053+0000 7ff1999c3700 1 -- 172.21.15.2:0/33928862 wait complete. 2023-06-23T18:21:21.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.252+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f405804e1d0 con 0x7f4058100f80 2023-06-23T18:21:21.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.252+0000 7f404effd700 1 -- 172.21.15.2:0/1461044964 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4050098ea0 con 0x7f4058100f80 2023-06-23T18:21:21.258 INFO:teuthology.orchestra.run.smithi002.stdout:141733920816 2023-06-23T18:21:21.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f40440a11e0 msgr2=0x7f40440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:21.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f40440a11e0 0x7f40440a3690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f405400b580 tx=0x7f4054009f10 comp rx=0 tx=0).stop 2023-06-23T18:21:21.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4058100f80 msgr2=0x7f4058195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:21.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4058100f80 0x7f4058195ad0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4050007ee0 tx=0x7f405000d3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 shutdown_connections 2023-06-23T18:21:21.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4058100f80 0x7f4058195ad0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f40440a11e0 0x7f40440a3690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40580fee40 0x7f405819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 --2- 172.21.15.2:0/1461044964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40581005d0 0x7f405819bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:21.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 >> 172.21.15.2:0/1461044964 conn(0x7f4058075b20 msgr2=0x7f40580689c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 shutdown_connections 2023-06-23T18:21:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:21.254+0000 7f4060225700 1 -- 172.21.15.2:0/1461044964 wait complete. 2023-06-23T18:21:21.288 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:21 smithi002 ceph-mon[95947]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:21.288 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:21 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/33928862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-06-23T18:21:21.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:21 smithi082 ceph-mon[97889]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:21.352 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:21 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/33928862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-06-23T18:21:21.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:21 smithi017 ceph-mon[97472]: pgmap v274: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:21.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:21 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/33928862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-06-23T18:21:21.794 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738431 got 34359738433 for osd.0 2023-06-23T18:21:21.794 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:22.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:22 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1461044964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-06-23T18:21:22.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:22 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1461044964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-06-23T18:21:22.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:22 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1461044964' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-06-23T18:21:22.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/707228794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 msgr2=0x7f9b88109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:22.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/707228794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 0x7f9b88109bc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009990 tx=0x7f9b7c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/707228794 shutdown_connections 2023-06-23T18:21:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/707228794 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b8810a100 0x7f9b8810c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/707228794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b88100db0 0x7f9b88101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/707228794 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 0x7f9b88109bc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/707228794 >> 172.21.15.2:0/707228794 conn(0x7f9b88075b30 msgr2=0x7f9b88075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/707228794 shutdown_connections 2023-06-23T18:21:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.711+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/707228794 wait complete. 2023-06-23T18:21:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.712+0000 7f9b8f4e5700 1 Processor -- start 2023-06-23T18:21:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.712+0000 7f9b8f4e5700 1 -- start start 2023-06-23T18:21:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.712+0000 7f9b8f4e5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88100db0 0x7f9b880fea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.712+0000 7f9b8f4e5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 0x7f9b880fef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8f4e5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b8810a100 0x7f9b880ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8f4e5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881a3fa0 con 0x7f9b881016c0 2023-06-23T18:21:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8f4e5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881a4110 con 0x7f9b8810a100 2023-06-23T18:21:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8f4e5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b881a42d0 con 0x7f9b88100db0 2023-06-23T18:21:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 0x7f9b880fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 0x7f9b880fef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52910/0 (socket says 172.21.15.2:52910) 2023-06-23T18:21:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 -- 172.21.15.2:0/906012513 learned_addr learned my addr 172.21.15.2:0/906012513 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8d281700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88100db0 0x7f9b880fea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 -- 172.21.15.2:0/906012513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88100db0 msgr2=0x7f9b880fea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88100db0 0x7f9b880fea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 -- 172.21.15.2:0/906012513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b8810a100 msgr2=0x7f9b880ff4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:22.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b8810a100 0x7f9b880ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 -- 172.21.15.2:0/906012513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c0095f0 con 0x7f9b881016c0 2023-06-23T18:21:22.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8d281700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88100db0 0x7f9b880fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:22.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8ca80700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 0x7f9b880fef50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c006010 tx=0x7f9b7c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:22.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b7a7fc700 1 -- 172.21.15.2:0/906012513 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c01d070 con 0x7f9b881016c0 2023-06-23T18:21:22.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b881a4550 con 0x7f9b881016c0 2023-06-23T18:21:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b7a7fc700 1 -- 172.21.15.2:0/906012513 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b7c0041a0 con 0x7f9b881016c0 2023-06-23T18:21:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b7a7fc700 1 -- 172.21.15.2:0/906012513 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00f460 con 0x7f9b881016c0 2023-06-23T18:21:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.713+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b881a49f0 con 0x7f9b881016c0 2023-06-23T18:21:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.715+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b88104990 con 0x7f9b881016c0 2023-06-23T18:21:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.716+0000 7f9b7a7fc700 1 -- 172.21.15.2:0/906012513 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9b7c022700 con 0x7f9b881016c0 2023-06-23T18:21:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.716+0000 7f9b7a7fc700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b740a11e0 0x7f9b740a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.716+0000 7f9b7a7fc700 1 -- 172.21.15.2:0/906012513 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9b7c0cdea0 con 0x7f9b881016c0 2023-06-23T18:21:22.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.716+0000 7f9b8d281700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b740a11e0 0x7f9b740a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:22.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.717+0000 7f9b8d281700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b740a11e0 0x7f9b740a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b84005950 tx=0x7f9b84005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:22.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.721+0000 7f9b7a7fc700 1 -- 172.21.15.2:0/906012513 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9b7c0976d0 con 0x7f9b881016c0 2023-06-23T18:21:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.921+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9b8804e1d0 con 0x7f9b881016c0 2023-06-23T18:21:22.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.922+0000 7f9b7a7fc700 1 -- 172.21.15.2:0/906012513 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9b7c09acf0 con 0x7f9b881016c0 2023-06-23T18:21:22.927 INFO:teuthology.orchestra.run.smithi002.stdout:107374182451 2023-06-23T18:21:22.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b740a11e0 msgr2=0x7f9b740a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:22.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b740a11e0 0x7f9b740a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b84005950 tx=0x7f9b84005f50 comp rx=0 tx=0).stop 2023-06-23T18:21:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 msgr2=0x7f9b880fef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 0x7f9b880fef50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c006010 tx=0x7f9b7c003820 comp rx=0 tx=0).stop 2023-06-23T18:21:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 shutdown_connections 2023-06-23T18:21:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9b740a11e0 0x7f9b740a3690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b8810a100 0x7f9b880ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b88100db0 0x7f9b880fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 --2- 172.21.15.2:0/906012513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b881016c0 0x7f9b880fef50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 >> 172.21.15.2:0/906012513 conn(0x7f9b88075b30 msgr2=0x7f9b88101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 shutdown_connections 2023-06-23T18:21:22.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:22.924+0000 7f9b8f4e5700 1 -- 172.21.15.2:0/906012513 wait complete. 2023-06-23T18:21:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:23 smithi002 ceph-mon[95947]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-06-23T18:21:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:23 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/906012513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-06-23T18:21:23.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:23 smithi082 ceph-mon[97889]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-06-23T18:21:23.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:23 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/906012513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-06-23T18:21:23.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:23 smithi017 ceph-mon[97472]: pgmap v275: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-06-23T18:21:23.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:23 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/906012513' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-06-23T18:21:23.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920814 got 141733920816 for osd.4 2023-06-23T18:21:23.823 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:24.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.315+0000 7f732ad9b700 1 -- 172.21.15.2:0/2567588434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7324104af0 msgr2=0x7f7324104ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.315+0000 7f732ad9b700 1 --2- 172.21.15.2:0/2567588434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7324104af0 0x7f7324104ec0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7318009990 tx=0x7f7318009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 -- 172.21.15.2:0/2567588434 shutdown_connections 2023-06-23T18:21:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 --2- 172.21.15.2:0/2567588434 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73240ff4a0 0x7f732410b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 --2- 172.21.15.2:0/2567588434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7324104af0 0x7f7324104ec0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 --2- 172.21.15.2:0/2567588434 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73240feaf0 0x7f73240fef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 -- 172.21.15.2:0/2567588434 >> 172.21.15.2:0/2567588434 conn(0x7f73240fa5b0 msgr2=0x7f73240fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:24.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 -- 172.21.15.2:0/2567588434 shutdown_connections 2023-06-23T18:21:24.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 -- 172.21.15.2:0/2567588434 wait complete. 2023-06-23T18:21:24.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 Processor -- start 2023-06-23T18:21:24.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 -- start start 2023-06-23T18:21:24.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73240feaf0 0x7f73241059d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73240ff4a0 0x7f7324105f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324104af0 0x7f7324106490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241a3f10 con 0x7f73240feaf0 2023-06-23T18:21:24.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241a4080 con 0x7f73240ff4a0 2023-06-23T18:21:24.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.316+0000 7f732ad9b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241a4240 con 0x7f7324104af0 2023-06-23T18:21:24.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73240feaf0 0x7f73241059d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73240feaf0 0x7f73241059d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52922/0 (socket says 172.21.15.2:52922) 2023-06-23T18:21:24.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 -- 172.21.15.2:0/1438802971 learned_addr learned my addr 172.21.15.2:0/1438802971 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:24.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7323fff700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73240ff4a0 0x7f7324105f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7329338700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324104af0 0x7f7324106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 -- 172.21.15.2:0/1438802971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324104af0 msgr2=0x7f7324106490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324104af0 0x7f7324106490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 -- 172.21.15.2:0/1438802971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73240ff4a0 msgr2=0x7f7324105f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73240ff4a0 0x7f7324105f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 -- 172.21.15.2:0/1438802971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73180095f0 con 0x7f73240feaf0 2023-06-23T18:21:24.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7329338700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324104af0 0x7f7324106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:24.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7328b37700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73240feaf0 0x7f73241059d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7318006010 tx=0x7f7318004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:24.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7321ffb700 1 -- 172.21.15.2:0/1438802971 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731801d070 con 0x7f73240feaf0 2023-06-23T18:21:24.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7321ffb700 1 -- 172.21.15.2:0/1438802971 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7318004000 con 0x7f73240feaf0 2023-06-23T18:21:24.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73241a44c0 con 0x7f73240feaf0 2023-06-23T18:21:24.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.317+0000 7f7321ffb700 1 -- 172.21.15.2:0/1438802971 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318022ac0 con 0x7f73240feaf0 2023-06-23T18:21:24.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.318+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73241a4930 con 0x7f73240feaf0 2023-06-23T18:21:24.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.318+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732404e1d0 con 0x7f73240feaf0 2023-06-23T18:21:24.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.320+0000 7f7321ffb700 1 -- 172.21.15.2:0/1438802971 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7318003b70 con 0x7f73240feaf0 2023-06-23T18:21:24.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.320+0000 7f7321ffb700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f73140a1260 0x7f73140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.320+0000 7f7323fff700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f73140a1260 0x7f73140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.321+0000 7f7321ffb700 1 -- 172.21.15.2:0/1438802971 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f73180cdf20 con 0x7f73240feaf0 2023-06-23T18:21:24.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.321+0000 7f7323fff700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f73140a1260 0x7f73140a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73241071c0 tx=0x7f731000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:24.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.325+0000 7f7321ffb700 1 -- 172.21.15.2:0/1438802971 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f731809b1e0 con 0x7f73240feaf0 2023-06-23T18:21:24.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 -- 172.21.15.2:0/4120755203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a34100d80 msgr2=0x7f5a34101150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/4120755203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a34100d80 0x7f5a34101150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5a24009990 tx=0x7f5a24009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 -- 172.21.15.2:0/4120755203 shutdown_connections 2023-06-23T18:21:24.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/4120755203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a3410a0d0 0x7f5a3410c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/4120755203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a34100d80 0x7f5a34101150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/4120755203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a34101690 0x7f5a34109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 -- 172.21.15.2:0/4120755203 >> 172.21.15.2:0/4120755203 conn(0x7f5a340fa540 msgr2=0x7f5a340fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:24.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 -- 172.21.15.2:0/4120755203 shutdown_connections 2023-06-23T18:21:24.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 -- 172.21.15.2:0/4120755203 wait complete. 2023-06-23T18:21:24.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 Processor -- start 2023-06-23T18:21:24.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.448+0000 7f5a3a77a700 1 -- start start 2023-06-23T18:21:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a3a77a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a34100d80 0x7f5a3419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a3a77a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a34101690 0x7f5a3419be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a3a77a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3410a0d0 0x7f5a34195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a3a77a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a34195fd0 con 0x7f5a3410a0d0 2023-06-23T18:21:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a3a77a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a34196140 con 0x7f5a34101690 2023-06-23T18:21:24.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a3a77a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a34196300 con 0x7f5a34100d80 2023-06-23T18:21:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3410a0d0 0x7f5a34195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3410a0d0 0x7f5a34195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52940/0 (socket says 172.21.15.2:52940) 2023-06-23T18:21:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 -- 172.21.15.2:0/1776512864 learned_addr learned my addr 172.21.15.2:0/1776512864 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:24.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a33fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a34100d80 0x7f5a3419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a337fe700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a34101690 0x7f5a3419be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 -- 172.21.15.2:0/1776512864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a34100d80 msgr2=0x7f5a3419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a34100d80 0x7f5a3419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 -- 172.21.15.2:0/1776512864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a34101690 msgr2=0x7f5a3419be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a34101690 0x7f5a3419be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 -- 172.21.15.2:0/1776512864 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a240095f0 con 0x7f5a3410a0d0 2023-06-23T18:21:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a38d17700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3410a0d0 0x7f5a34195960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00ada0 tx=0x7f5a2c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.449+0000 7f5a317fa700 1 -- 172.21.15.2:0/1776512864 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00f070 con 0x7f5a3410a0d0 2023-06-23T18:21:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.450+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a341965e0 con 0x7f5a3410a0d0 2023-06-23T18:21:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.450+0000 7f5a317fa700 1 -- 172.21.15.2:0/1776512864 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a2c007b30 con 0x7f5a3410a0d0 2023-06-23T18:21:24.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.450+0000 7f5a317fa700 1 -- 172.21.15.2:0/1776512864 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c005280 con 0x7f5a3410a0d0 2023-06-23T18:21:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.450+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a340feae0 con 0x7f5a3410a0d0 2023-06-23T18:21:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.451+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3404e1d0 con 0x7f5a3410a0d0 2023-06-23T18:21:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.452+0000 7f5a317fa700 1 -- 172.21.15.2:0/1776512864 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f5a2c00c8d0 con 0x7f5a3410a0d0 2023-06-23T18:21:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.452+0000 7f5a317fa700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a1c0a55f0 0x7f5a1c0a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.452+0000 7f5a33fff700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a1c0a55f0 0x7f5a1c0a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.452+0000 7f5a317fa700 1 -- 172.21.15.2:0/1776512864 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5a2c0cbde0 con 0x7f5a3410a0d0 2023-06-23T18:21:24.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.453+0000 7f5a33fff700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a1c0a55f0 0x7f5a1c0a7aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400b5c0 tx=0x7f5a24005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:24.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.458+0000 7f5a317fa700 1 -- 172.21.15.2:0/1776512864 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5a2c095610 con 0x7f5a3410a0d0 2023-06-23T18:21:24.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.539+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f73241a4c60 con 0x7f73240feaf0 2023-06-23T18:21:24.545 INFO:teuthology.orchestra.run.smithi002.stdout:77309411386 2023-06-23T18:21:24.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.539+0000 7f7321ffb700 1 -- 172.21.15.2:0/1438802971 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f731809ad70 con 0x7f73240feaf0 2023-06-23T18:21:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f73140a1260 msgr2=0x7f73140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f73140a1260 0x7f73140a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f73241071c0 tx=0x7f731000a380 comp rx=0 tx=0).stop 2023-06-23T18:21:24.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73240feaf0 msgr2=0x7f73241059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73240feaf0 0x7f73241059d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7318006010 tx=0x7f7318004730 comp rx=0 tx=0).stop 2023-06-23T18:21:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 shutdown_connections 2023-06-23T18:21:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7324104af0 0x7f7324106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f73240feaf0 0x7f73241059d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f73140a1260 0x7f73140a3710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 --2- 172.21.15.2:0/1438802971 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f73240ff4a0 0x7f7324105f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 >> 172.21.15.2:0/1438802971 conn(0x7f73240fa5b0 msgr2=0x7f73241005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 shutdown_connections 2023-06-23T18:21:24.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.542+0000 7f732ad9b700 1 -- 172.21.15.2:0/1438802971 wait complete. 2023-06-23T18:21:24.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 -- 172.21.15.2:0/2908012998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0ff2e0 msgr2=0x7fe6ac10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 --2- 172.21.15.2:0/2908012998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0ff2e0 0x7fe6ac10bfb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009990 tx=0x7fe6a0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 -- 172.21.15.2:0/2908012998 shutdown_connections 2023-06-23T18:21:24.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 --2- 172.21.15.2:0/2908012998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6ac10c4f0 0x7fe6ac10e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 --2- 172.21.15.2:0/2908012998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6ac0fe940 0x7fe6ac0fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 --2- 172.21.15.2:0/2908012998 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0ff2e0 0x7fe6ac10bfb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 -- 172.21.15.2:0/2908012998 >> 172.21.15.2:0/2908012998 conn(0x7fe6ac075b20 msgr2=0x7fe6ac075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:24.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 -- 172.21.15.2:0/2908012998 shutdown_connections 2023-06-23T18:21:24.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.659+0000 7fe6b2345700 1 -- 172.21.15.2:0/2908012998 wait complete. 2023-06-23T18:21:24.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.660+0000 7fe6b2345700 1 Processor -- start 2023-06-23T18:21:24.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.660+0000 7fe6b2345700 1 -- start start 2023-06-23T18:21:24.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6b2345700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0fe940 0x7fe6ac197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6b2345700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6ac0ff2e0 0x7fe6ac198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6b2345700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6ac10c4f0 0x7fe6ac19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6b2345700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac19cdb0 con 0x7fe6ac0fe940 2023-06-23T18:21:24.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6b2345700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac19cf20 con 0x7fe6ac10c4f0 2023-06-23T18:21:24.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6b2345700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ac19d0e0 con 0x7fe6ac0ff2e0 2023-06-23T18:21:24.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0fe940 0x7fe6ac197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0fe940 0x7fe6ac197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52950/0 (socket says 172.21.15.2:52950) 2023-06-23T18:21:24.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 -- 172.21.15.2:0/1551900549 learned_addr learned my addr 172.21.15.2:0/1551900549 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:24.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6ab7fe700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6ac0ff2e0 0x7fe6ac198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6b08e2700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6ac10c4f0 0x7fe6ac19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 -- 172.21.15.2:0/1551900549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6ac0ff2e0 msgr2=0x7fe6ac198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6ac0ff2e0 0x7fe6ac198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 -- 172.21.15.2:0/1551900549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6ac10c4f0 msgr2=0x7fe6ac19c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6ac10c4f0 0x7fe6ac19c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 -- 172.21.15.2:0/1551900549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00095f0 con 0x7fe6ac0fe940 2023-06-23T18:21:24.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6ab7fe700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6ac0ff2e0 0x7fe6ac198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:24.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6b08e2700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6ac10c4f0 0x7fe6ac19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:24.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6abfff700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0fe940 0x7fe6ac197df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00c9b0 tx=0x7fe69c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:24.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.661+0000 7fe6a97fa700 1 -- 172.21.15.2:0/1551900549 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c007b00 con 0x7fe6ac0fe940 2023-06-23T18:21:24.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.662+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6ac19d3c0 con 0x7fe6ac0fe940 2023-06-23T18:21:24.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.662+0000 7fe6a97fa700 1 -- 172.21.15.2:0/1551900549 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe69c007c60 con 0x7fe6ac0fe940 2023-06-23T18:21:24.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.662+0000 7fe6a97fa700 1 -- 172.21.15.2:0/1551900549 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c018660 con 0x7fe6ac0fe940 2023-06-23T18:21:24.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.662+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6ac1a4c20 con 0x7fe6ac0fe940 2023-06-23T18:21:24.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.663+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6ac04e1d0 con 0x7fe6ac0fe940 2023-06-23T18:21:24.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.665+0000 7fe6a97fa700 1 -- 172.21.15.2:0/1551900549 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe69c00f450 con 0x7fe6ac0fe940 2023-06-23T18:21:24.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.665+0000 7fe6a97fa700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe6940a11e0 0x7fe6940a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:24.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.665+0000 7fe6a97fa700 1 -- 172.21.15.2:0/1551900549 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe69c0ccc40 con 0x7fe6ac0fe940 2023-06-23T18:21:24.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.665+0000 7fe6ab7fe700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe6940a11e0 0x7fe6940a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:24.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.666+0000 7fe6ab7fe700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe6940a11e0 0x7fe6940a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00059d0 tx=0x7fe6a0005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:24.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.673+0000 7fe6a97fa700 1 -- 172.21.15.2:0/1551900549 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe69c096470 con 0x7fe6ac0fe940 2023-06-23T18:21:24.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.682+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f5a340fee30 con 0x7f5a3410a0d0 2023-06-23T18:21:24.688 INFO:teuthology.orchestra.run.smithi002.stdout:330712481811 2023-06-23T18:21:24.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.683+0000 7f5a317fa700 1 -- 172.21.15.2:0/1776512864 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f5a2c098c30 con 0x7f5a3410a0d0 2023-06-23T18:21:24.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a1c0a55f0 msgr2=0x7f5a1c0a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a1c0a55f0 0x7f5a1c0a7aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400b5c0 tx=0x7f5a24005fb0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3410a0d0 msgr2=0x7f5a34195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3410a0d0 0x7f5a34195960 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00ada0 tx=0x7f5a2c00c330 comp rx=0 tx=0).stop 2023-06-23T18:21:24.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 shutdown_connections 2023-06-23T18:21:24.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a3410a0d0 0x7f5a34195960 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5a34100d80 0x7f5a3419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5a1c0a55f0 0x7f5a1c0a7aa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 --2- 172.21.15.2:0/1776512864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a34101690 0x7f5a3419be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 >> 172.21.15.2:0/1776512864 conn(0x7f5a340fa540 msgr2=0x7f5a3410c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:24.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 shutdown_connections 2023-06-23T18:21:24.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.686+0000 7f5a3a77a700 1 -- 172.21.15.2:0/1776512864 wait complete. 2023-06-23T18:21:24.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.882+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fe6ac1a4e70 con 0x7fe6ac0fe940 2023-06-23T18:21:24.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.882+0000 7fe6a97fa700 1 -- 172.21.15.2:0/1551900549 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe69c00f700 con 0x7fe6ac0fe940 2023-06-23T18:21:24.887 INFO:teuthology.orchestra.run.smithi002.stdout:249108103199 2023-06-23T18:21:24.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe6940a11e0 msgr2=0x7fe6940a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe6940a11e0 0x7fe6940a3690 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00059d0 tx=0x7fe6a0005940 comp rx=0 tx=0).stop 2023-06-23T18:21:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0fe940 msgr2=0x7fe6ac197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0fe940 0x7fe6ac197df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00c9b0 tx=0x7fe69c00cd70 comp rx=0 tx=0).stop 2023-06-23T18:21:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 shutdown_connections 2023-06-23T18:21:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe6940a11e0 0x7fe6940a3690 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe6ac10c4f0 0x7fe6ac19c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6ac0fe940 0x7fe6ac197df0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 --2- 172.21.15.2:0/1551900549 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6ac0ff2e0 0x7fe6ac198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 >> 172.21.15.2:0/1551900549 conn(0x7fe6ac075b20 msgr2=0x7fe6ac0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 shutdown_connections 2023-06-23T18:21:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:24.884+0000 7fe6b2345700 1 -- 172.21.15.2:0/1551900549 wait complete. 2023-06-23T18:21:25.202 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:25 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1438802971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-06-23T18:21:25.203 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:25 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1776512864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-06-23T18:21:25.203 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:25 smithi002 ceph-mon[95947]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:25.203 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:25 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1551900549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-06-23T18:21:25.204 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411383 got 77309411386 for osd.2 2023-06-23T18:21:25.205 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:25.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:25 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1438802971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-06-23T18:21:25.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:25 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1776512864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-06-23T18:21:25.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:25 smithi082 ceph-mon[97889]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:25.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:25 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1551900549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-06-23T18:21:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:25 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1438802971' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-06-23T18:21:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:25 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1776512864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-06-23T18:21:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:25 smithi017 ceph-mon[97472]: pgmap v276: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:25.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:25 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1551900549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-06-23T18:21:25.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.939+0000 7fb95c366700 1 -- 172.21.15.2:0/1030473256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb95410c4f0 msgr2=0x7fb95410e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:25.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.939+0000 7fb95c366700 1 --2- 172.21.15.2:0/1030473256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb95410c4f0 0x7fb95410e8d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00b0a0 tx=0x7fb94c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.940+0000 7fb95c366700 1 -- 172.21.15.2:0/1030473256 shutdown_connections 2023-06-23T18:21:25.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.940+0000 7fb95c366700 1 --2- 172.21.15.2:0/1030473256 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb95410c4f0 0x7fb95410e8d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.940+0000 7fb95c366700 1 --2- 172.21.15.2:0/1030473256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9540fe940 0x7fb9540fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.940+0000 7fb95c366700 1 --2- 172.21.15.2:0/1030473256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9540ff250 0x7fb95410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.940+0000 7fb95c366700 1 -- 172.21.15.2:0/1030473256 >> 172.21.15.2:0/1030473256 conn(0x7fb954075b30 msgr2=0x7fb954075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:25.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.940+0000 7fb95c366700 1 -- 172.21.15.2:0/1030473256 shutdown_connections 2023-06-23T18:21:25.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.940+0000 7fb95c366700 1 -- 172.21.15.2:0/1030473256 wait complete. 2023-06-23T18:21:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.941+0000 7fb95c366700 1 Processor -- start 2023-06-23T18:21:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.941+0000 7fb95c366700 1 -- start start 2023-06-23T18:21:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb95c366700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9540fe940 0x7fb954197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb95c366700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9540ff250 0x7fb954198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb95c366700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95410c4f0 0x7fb95419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb95c366700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95419ceb0 con 0x7fb9540ff250 2023-06-23T18:21:25.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb95c366700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95419d020 con 0x7fb9540fe940 2023-06-23T18:21:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb95c366700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95419d1e0 con 0x7fb95410c4f0 2023-06-23T18:21:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9540ff250 0x7fb954198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9540ff250 0x7fb954198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52980/0 (socket says 172.21.15.2:52980) 2023-06-23T18:21:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 -- 172.21.15.2:0/2936687561 learned_addr learned my addr 172.21.15.2:0/2936687561 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:25.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb95a102700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9540fe940 0x7fb954197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 -- 172.21.15.2:0/2936687561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95410c4f0 msgr2=0x7fb95419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:25.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95410c4f0 0x7fb95419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 -- 172.21.15.2:0/2936687561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9540fe940 msgr2=0x7fb954197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9540fe940 0x7fb954197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 -- 172.21.15.2:0/2936687561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c009d00 con 0x7fb9540ff250 2023-06-23T18:21:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb95a102700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9540fe940 0x7fb954197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.942+0000 7fb959901700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9540ff250 0x7fb954198430 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d950 tx=0x7fb94400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.943+0000 7fb94b7fe700 1 -- 172.21.15.2:0/2936687561 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944009b40 con 0x7fb9540ff250 2023-06-23T18:21:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.943+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95419d4c0 con 0x7fb9540ff250 2023-06-23T18:21:25.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.943+0000 7fb94b7fe700 1 -- 172.21.15.2:0/2936687561 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb944009ca0 con 0x7fb9540ff250 2023-06-23T18:21:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.943+0000 7fb94b7fe700 1 -- 172.21.15.2:0/2936687561 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94400f620 con 0x7fb9540ff250 2023-06-23T18:21:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.943+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9541a4d20 con 0x7fb9540ff250 2023-06-23T18:21:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.944+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95404e1d0 con 0x7fb9540ff250 2023-06-23T18:21:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.945+0000 7fb94b7fe700 1 -- 172.21.15.2:0/2936687561 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb944010460 con 0x7fb9540ff250 2023-06-23T18:21:25.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.946+0000 7fb94b7fe700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb9400a11e0 0x7fb9400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.946+0000 7fb94b7fe700 1 -- 172.21.15.2:0/2936687561 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb9440ccc60 con 0x7fb9540ff250 2023-06-23T18:21:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.946+0000 7fb95a102700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb9400a11e0 0x7fb9400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:25.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.947+0000 7fb95a102700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb9400a11e0 0x7fb9400a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb950005950 tx=0x7fb9500058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:25.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.950+0000 7fb94b7fe700 1 -- 172.21.15.2:0/2936687561 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb944096400 con 0x7fb9540ff250 2023-06-23T18:21:25.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 -- 172.21.15.2:0/3242823870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48101480 msgr2=0x7f4b48109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:25.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/3242823870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48101480 0x7f4b48109980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4b30009990 tx=0x7f4b30009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 -- 172.21.15.2:0/3242823870 shutdown_connections 2023-06-23T18:21:25.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/3242823870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b48109f50 0x7f4b4810c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/3242823870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b48100b70 0x7f4b48100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/3242823870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48101480 0x7f4b48109980 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 -- 172.21.15.2:0/3242823870 >> 172.21.15.2:0/3242823870 conn(0x7f4b480fa430 msgr2=0x7f4b480fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:25.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 -- 172.21.15.2:0/3242823870 shutdown_connections 2023-06-23T18:21:25.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.291+0000 7f4b4cba2700 1 -- 172.21.15.2:0/3242823870 wait complete. 2023-06-23T18:21:25.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b4cba2700 1 Processor -- start 2023-06-23T18:21:25.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b4cba2700 1 -- start start 2023-06-23T18:21:25.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b4cba2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b48100b70 0x7f4b480fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:25.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b4cba2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b48101480 0x7f4b480fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:25.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b4cba2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48109f50 0x7f4b480ff290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:25.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b4cba2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481a3e40 con 0x7f4b48109f50 2023-06-23T18:21:25.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b4cba2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481a3fb0 con 0x7f4b48101480 2023-06-23T18:21:25.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b4cba2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481a4170 con 0x7f4b48100b70 2023-06-23T18:21:25.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48109f50 0x7f4b480ff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:25.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48109f50 0x7f4b480ff290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52968/0 (socket says 172.21.15.2:52968) 2023-06-23T18:21:25.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b47fff700 1 -- 172.21.15.2:0/2664882326 learned_addr learned my addr 172.21.15.2:0/2664882326 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:25.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.292+0000 7f4b46ffd700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b48101480 0x7f4b480fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:25.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b477fe700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b48100b70 0x7f4b480fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:25.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b47fff700 1 -- 172.21.15.2:0/2664882326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b48100b70 msgr2=0x7f4b480fe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:25.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b47fff700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b48100b70 0x7f4b480fe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b47fff700 1 -- 172.21.15.2:0/2664882326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b48101480 msgr2=0x7f4b480fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b47fff700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b48101480 0x7f4b480fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b47fff700 1 -- 172.21.15.2:0/2664882326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b300095f0 con 0x7f4b48109f50 2023-06-23T18:21:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b47fff700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48109f50 0x7f4b480ff290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000ef30 tx=0x7f4b4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b44ff9700 1 -- 172.21.15.2:0/2664882326 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40018070 con 0x7f4b48109f50 2023-06-23T18:21:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b44ff9700 1 -- 172.21.15.2:0/2664882326 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b4000f040 con 0x7f4b48109f50 2023-06-23T18:21:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b481a43f0 con 0x7f4b48109f50 2023-06-23T18:21:25.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b44ff9700 1 -- 172.21.15.2:0/2664882326 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4000cbc0 con 0x7f4b48109f50 2023-06-23T18:21:25.983 INFO:teuthology.orchestra.run.smithi002.stdout:163208757294 2023-06-23T18:21:25.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.293+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b481a48f0 con 0x7f4b48109f50 2023-06-23T18:21:25.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.294+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b4804ea20 con 0x7f4b48109f50 2023-06-23T18:21:25.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.295+0000 7f4b44ff9700 1 -- 172.21.15.2:0/2664882326 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4b4000cd80 con 0x7f4b48109f50 2023-06-23T18:21:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.296+0000 7f4b44ff9700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b340a1260 0x7f4b340a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.296+0000 7f4b477fe700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b340a1260 0x7f4b340a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.296+0000 7f4b44ff9700 1 -- 172.21.15.2:0/2664882326 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4b400cc3f0 con 0x7f4b48109f50 2023-06-23T18:21:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.296+0000 7f4b477fe700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b340a1260 0x7f4b340a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005950 tx=0x7f4b3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.300+0000 7f4b44ff9700 1 -- 172.21.15.2:0/2664882326 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4b40095b20 con 0x7f4b48109f50 2023-06-23T18:21:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.509+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4b481a4c50 con 0x7f4b48109f50 2023-06-23T18:21:25.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.509+0000 7f4b44ff9700 1 -- 172.21.15.2:0/2664882326 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b40099140 con 0x7f4b48109f50 2023-06-23T18:21:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b340a1260 msgr2=0x7f4b340a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b340a1260 0x7f4b340a3710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005950 tx=0x7f4b3800b410 comp rx=0 tx=0).stop 2023-06-23T18:21:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48109f50 msgr2=0x7f4b480ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48109f50 0x7f4b480ff290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000ef30 tx=0x7f4b4000c3a0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 shutdown_connections 2023-06-23T18:21:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b48109f50 0x7f4b480ff290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b48100b70 0x7f4b480fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b340a1260 0x7f4b340a3710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 --2- 172.21.15.2:0/2664882326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b48101480 0x7f4b480fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:25.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 >> 172.21.15.2:0/2664882326 conn(0x7f4b480fa430 msgr2=0x7f4b4810c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:25.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 shutdown_connections 2023-06-23T18:21:25.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:25.512+0000 7f4b4cba2700 1 -- 172.21.15.2:0/2664882326 wait complete. 2023-06-23T18:21:26.002 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182449 got 107374182451 for osd.3 2023-06-23T18:21:26.002 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:26.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103198 got 249108103199 for osd.8 2023-06-23T18:21:26.144 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:26.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.166+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb9541a4ff0 con 0x7fb9540ff250 2023-06-23T18:21:26.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.167+0000 7fb94b7fe700 1 -- 172.21.15.2:0/2936687561 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb944016020 con 0x7fb9540ff250 2023-06-23T18:21:26.174 INFO:teuthology.orchestra.run.smithi002.stdout:197568495656 2023-06-23T18:21:26.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb9400a11e0 msgr2=0x7fb9400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:26.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb9400a11e0 0x7fb9400a3690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb950005950 tx=0x7fb9500058e0 comp rx=0 tx=0).stop 2023-06-23T18:21:26.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9540ff250 msgr2=0x7fb954198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:26.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9540ff250 0x7fb954198430 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d950 tx=0x7fb94400dd10 comp rx=0 tx=0).stop 2023-06-23T18:21:26.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 shutdown_connections 2023-06-23T18:21:26.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95410c4f0 0x7fb95419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:26.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb9400a11e0 0x7fb9400a3690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:26.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9540fe940 0x7fb954197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:26.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 --2- 172.21.15.2:0/2936687561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb9540ff250 0x7fb954198430 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:26.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 >> 172.21.15.2:0/2936687561 conn(0x7fb954075b30 msgr2=0x7fb9540fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:26.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 shutdown_connections 2023-06-23T18:21:26.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:26.169+0000 7fb95c366700 1 -- 172.21.15.2:0/2936687561 wait complete. 2023-06-23T18:21:26.199 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:26 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2664882326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-06-23T18:21:26.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:26 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2664882326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-06-23T18:21:26.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:26 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2664882326' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-06-23T18:21:26.716 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481809 got 330712481811 for osd.11 2023-06-23T18:21:26.717 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:26.975 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495654 got 197568495656 for osd.6 2023-06-23T18:21:26.975 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:27.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 -- 172.21.15.2:0/4163185714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 msgr2=0x7f35100ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 --2- 172.21.15.2:0/4163185714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 0x7f35100ffe20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3500009990 tx=0x7f3500009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 -- 172.21.15.2:0/4163185714 shutdown_connections 2023-06-23T18:21:27.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 --2- 172.21.15.2:0/4163185714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3510100360 0x7f351010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 --2- 172.21.15.2:0/4163185714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35101059b0 0x7f3510105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 --2- 172.21.15.2:0/4163185714 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 0x7f35100ffe20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 -- 172.21.15.2:0/4163185714 >> 172.21.15.2:0/4163185714 conn(0x7f35100fb510 msgr2=0x7f35100fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:27.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 -- 172.21.15.2:0/4163185714 shutdown_connections 2023-06-23T18:21:27.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 -- 172.21.15.2:0/4163185714 wait complete. 2023-06-23T18:21:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.228+0000 7f351639f700 1 Processor -- start 2023-06-23T18:21:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351639f700 1 -- start start 2023-06-23T18:21:27.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351639f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 0x7f351019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:27.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351639f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3510100360 0x7f351019bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:27.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351639f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35101059b0 0x7f35101956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:27.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351639f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3510195d30 con 0x7f35100ff9b0 2023-06-23T18:21:27.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351639f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3510195ea0 con 0x7f35101059b0 2023-06-23T18:21:27.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351639f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3510196060 con 0x7f3510100360 2023-06-23T18:21:27.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 0x7f351019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:27.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 0x7f351019b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:52992/0 (socket says 172.21.15.2:52992) 2023-06-23T18:21:27.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 -- 172.21.15.2:0/3640079919 learned_addr learned my addr 172.21.15.2:0/3640079919 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:27.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f3514b9c700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3510100360 0x7f351019bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 -- 172.21.15.2:0/3640079919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3510100360 msgr2=0x7f351019bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3510100360 0x7f351019bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 -- 172.21.15.2:0/3640079919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35101059b0 msgr2=0x7f35101956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35101059b0 0x7f35101956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 -- 172.21.15.2:0/3640079919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000095f0 con 0x7f35100ff9b0 2023-06-23T18:21:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.229+0000 7f351539d700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 0x7f351019b640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c990 tx=0x7f350c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.230+0000 7f35067fc700 1 -- 172.21.15.2:0/3640079919 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c007b00 con 0x7f35100ff9b0 2023-06-23T18:21:27.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.230+0000 7f35067fc700 1 -- 172.21.15.2:0/3640079919 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f350c007c60 con 0x7f35100ff9b0 2023-06-23T18:21:27.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.230+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3510196340 con 0x7f35100ff9b0 2023-06-23T18:21:27.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.230+0000 7f35067fc700 1 -- 172.21.15.2:0/3640079919 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c018660 con 0x7f35100ff9b0 2023-06-23T18:21:27.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.230+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35101a2630 con 0x7f35100ff9b0 2023-06-23T18:21:27.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.231+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351004ea20 con 0x7f35100ff9b0 2023-06-23T18:21:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.232+0000 7f35067fc700 1 -- 172.21.15.2:0/3640079919 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f350c004020 con 0x7f35100ff9b0 2023-06-23T18:21:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.232+0000 7f35067fc700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34fc0a1260 0x7f34fc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:27.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.233+0000 7f3514b9c700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34fc0a1260 0x7f34fc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.233+0000 7f35067fc700 1 -- 172.21.15.2:0/3640079919 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f350c0cc910 con 0x7f35100ff9b0 2023-06-23T18:21:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.233+0000 7f3514b9c700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34fc0a1260 0x7f34fc0a3710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f350000b5c0 tx=0x7f3500005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:27.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.236+0000 7f35067fc700 1 -- 172.21.15.2:0/3640079919 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f350c0961c0 con 0x7f35100ff9b0 2023-06-23T18:21:27.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:27 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2936687561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-06-23T18:21:27.264 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:27 smithi002 ceph-mon[95947]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:27.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:27 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2936687561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-06-23T18:21:27.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:27 smithi082 ceph-mon[97889]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:27.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:27 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2936687561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-06-23T18:21:27.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:27 smithi017 ceph-mon[97472]: pgmap v277: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2023-06-23T18:21:27.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.449+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f3510196bd0 con 0x7f35100ff9b0 2023-06-23T18:21:27.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.449+0000 7f35067fc700 1 -- 172.21.15.2:0/3640079919 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f350c0997e0 con 0x7f35100ff9b0 2023-06-23T18:21:27.454 INFO:teuthology.orchestra.run.smithi002.stdout:279172874268 2023-06-23T18:21:27.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.451+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34fc0a1260 msgr2=0x7f34fc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.451+0000 7f351639f700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34fc0a1260 0x7f34fc0a3710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f350000b5c0 tx=0x7f3500005fb0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.451+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 msgr2=0x7f351019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.451+0000 7f351639f700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 0x7f351019b640 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f350c00c990 tx=0x7f350c00cd50 comp rx=0 tx=0).stop 2023-06-23T18:21:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.452+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 shutdown_connections 2023-06-23T18:21:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.452+0000 7f351639f700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34fc0a1260 0x7f34fc0a3710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.452+0000 7f351639f700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35101059b0 0x7f35101956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.452+0000 7f351639f700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35100ff9b0 0x7f351019b640 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.452+0000 7f351639f700 1 --2- 172.21.15.2:0/3640079919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3510100360 0x7f351019bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.452+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 >> 172.21.15.2:0/3640079919 conn(0x7f35100fb510 msgr2=0x7f3510103570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:27.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.452+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 shutdown_connections 2023-06-23T18:21:27.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.452+0000 7f351639f700 1 -- 172.21.15.2:0/3640079919 wait complete. 2023-06-23T18:21:27.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.515+0000 7f1834f2a700 1 -- 172.21.15.2:0/687262406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830104e70 msgr2=0x7f183010c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.515+0000 7f1834f2a700 1 --2- 172.21.15.2:0/687262406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830104e70 0x7f183010c9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f182800b0d0 tx=0x7f182800b3e0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.516+0000 7f1834f2a700 1 -- 172.21.15.2:0/687262406 shutdown_connections 2023-06-23T18:21:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.516+0000 7f1834f2a700 1 --2- 172.21.15.2:0/687262406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830104e70 0x7f183010c9f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.516+0000 7f1834f2a700 1 --2- 172.21.15.2:0/687262406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18300ffdd0 0x7f18301001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.516+0000 7f1834f2a700 1 --2- 172.21.15.2:0/687262406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830100770 0x7f1830104740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.516+0000 7f1834f2a700 1 -- 172.21.15.2:0/687262406 >> 172.21.15.2:0/687262406 conn(0x7f18300fb650 msgr2=0x7f18300fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.516+0000 7f1834f2a700 1 -- 172.21.15.2:0/687262406 shutdown_connections 2023-06-23T18:21:27.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.516+0000 7f1834f2a700 1 -- 172.21.15.2:0/687262406 wait complete. 2023-06-23T18:21:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f1834f2a700 1 Processor -- start 2023-06-23T18:21:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f1834f2a700 1 -- start start 2023-06-23T18:21:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f1834f2a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18300ffdd0 0x7f183019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f1834f2a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830100770 0x7f183019bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f1834f2a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830104e70 0x7f1830195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f1834f2a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830196120 con 0x7f1830100770 2023-06-23T18:21:27.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f1834f2a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830196290 con 0x7f1830104e70 2023-06-23T18:21:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f1834f2a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1830196450 con 0x7f18300ffdd0 2023-06-23T18:21:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830100770 0x7f183019bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830100770 0x7f183019bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53008/0 (socket says 172.21.15.2:53008) 2023-06-23T18:21:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182dd9b700 1 -- 172.21.15.2:0/2222270022 learned_addr learned my addr 172.21.15.2:0/2222270022 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182ed9d700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830104e70 0x7f1830195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182e59c700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18300ffdd0 0x7f183019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:27.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182dd9b700 1 -- 172.21.15.2:0/2222270022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18300ffdd0 msgr2=0x7f183019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182dd9b700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18300ffdd0 0x7f183019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182dd9b700 1 -- 172.21.15.2:0/2222270022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830104e70 msgr2=0x7f1830195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182dd9b700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830104e70 0x7f1830195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.517+0000 7f182dd9b700 1 -- 172.21.15.2:0/2222270022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1828009d00 con 0x7f1830100770 2023-06-23T18:21:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.518+0000 7f182dd9b700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830100770 0x7f183019bf70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f182400c8f0 tx=0x7f182400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.518+0000 7f181f7fe700 1 -- 172.21.15.2:0/2222270022 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824007940 con 0x7f1830100770 2023-06-23T18:21:27.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.518+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830196730 con 0x7f1830100770 2023-06-23T18:21:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.518+0000 7f181f7fe700 1 -- 172.21.15.2:0/2222270022 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f182400f450 con 0x7f1830100770 2023-06-23T18:21:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.518+0000 7f181f7fe700 1 -- 172.21.15.2:0/2222270022 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824018660 con 0x7f1830100770 2023-06-23T18:21:27.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.518+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18300728d0 con 0x7f1830100770 2023-06-23T18:21:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.519+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183004e1d0 con 0x7f1830100770 2023-06-23T18:21:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.520+0000 7f181f7fe700 1 -- 172.21.15.2:0/2222270022 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f182400f5c0 con 0x7f1830100770 2023-06-23T18:21:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.521+0000 7f181f7fe700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18180a11e0 0x7f18180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:27.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.521+0000 7f181f7fe700 1 -- 172.21.15.2:0/2222270022 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f18240cc810 con 0x7f1830100770 2023-06-23T18:21:27.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.521+0000 7f182e59c700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18180a11e0 0x7f18180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:27.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.522+0000 7f182e59c700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18180a11e0 0x7f18180a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1820005950 tx=0x7f18200058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:27.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.525+0000 7f181f7fe700 1 -- 172.21.15.2:0/2222270022 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1824096040 con 0x7f1830100770 2023-06-23T18:21:27.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757292 got 163208757294 for osd.5 2023-06-23T18:21:27.625 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:27.735 INFO:teuthology.orchestra.run.smithi002.stdout:300647710744 2023-06-23T18:21:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.730+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f1830196fc0 con 0x7f1830100770 2023-06-23T18:21:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.730+0000 7f181f7fe700 1 -- 172.21.15.2:0/2222270022 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f1824099660 con 0x7f1830100770 2023-06-23T18:21:27.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.732+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18180a11e0 msgr2=0x7f18180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.732+0000 7f1834f2a700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18180a11e0 0x7f18180a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1820005950 tx=0x7f18200058e0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.732+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830100770 msgr2=0x7f183019bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.732+0000 7f1834f2a700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830100770 0x7f183019bf70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f182400c8f0 tx=0x7f182400cc00 comp rx=0 tx=0).stop 2023-06-23T18:21:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.733+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 shutdown_connections 2023-06-23T18:21:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.733+0000 7f1834f2a700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18180a11e0 0x7f18180a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.733+0000 7f1834f2a700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1830104e70 0x7f1830195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.733+0000 7f1834f2a700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18300ffdd0 0x7f183019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.733+0000 7f1834f2a700 1 --2- 172.21.15.2:0/2222270022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1830100770 0x7f183019bf70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.733+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 >> 172.21.15.2:0/2222270022 conn(0x7f18300fb650 msgr2=0x7f18300fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.733+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 shutdown_connections 2023-06-23T18:21:27.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:27.733+0000 7f1834f2a700 1 -- 172.21.15.2:0/2222270022 wait complete. 2023-06-23T18:21:28.173 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874265 got 279172874268 for osd.9 2023-06-23T18:21:28.173 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:28.189 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:28 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3640079919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-06-23T18:21:28.190 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:28 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2222270022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-06-23T18:21:28.287 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710742 got 300647710744 for osd.10 2023-06-23T18:21:28.288 DEBUG:teuthology.parallel:result is None 2023-06-23T18:21:28.288 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-06-23T18:21:28.289 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph pg dump --format=json 2023-06-23T18:21:28.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:28 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3640079919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-06-23T18:21:28.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:28 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2222270022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-06-23T18:21:28.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:28 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3640079919' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-06-23T18:21:28.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:28 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2222270022' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-06-23T18:21:28.891 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:29.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:29 smithi082 ceph-mon[97889]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-06-23T18:21:29.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:29 smithi017 ceph-mon[97472]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-06-23T18:21:29.445 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:29 smithi002 ceph-mon[95947]: pgmap v278: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2023-06-23T18:21:30.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 -- 172.21.15.2:0/279021713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb174100f80 msgr2=0x7fb17410b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 --2- 172.21.15.2:0/279021713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb174100f80 0x7fb17410b9d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00b0d0 tx=0x7fb16c00b3e0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 -- 172.21.15.2:0/279021713 shutdown_connections 2023-06-23T18:21:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 --2- 172.21.15.2:0/279021713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb174100f80 0x7fb17410b9d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 --2- 172.21.15.2:0/279021713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1740fee40 0x7fb1740ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 --2- 172.21.15.2:0/279021713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1741005d0 0x7fb174100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 -- 172.21.15.2:0/279021713 >> 172.21.15.2:0/279021713 conn(0x7fb174075b20 msgr2=0x7fb174075f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:30.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 -- 172.21.15.2:0/279021713 shutdown_connections 2023-06-23T18:21:30.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 -- 172.21.15.2:0/279021713 wait complete. 2023-06-23T18:21:30.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.180+0000 7fb17922c700 1 Processor -- start 2023-06-23T18:21:30.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17922c700 1 -- start start 2023-06-23T18:21:30.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17922c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1740fee40 0x7fb17419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17922c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1741005d0 0x7fb17419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17922c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174100f80 0x7fb174194b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17922c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741951a0 con 0x7fb1740fee40 2023-06-23T18:21:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17922c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb174195310 con 0x7fb1741005d0 2023-06-23T18:21:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17922c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1741954d0 con 0x7fb174100f80 2023-06-23T18:21:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb172d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1740fee40 0x7fb17419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:30.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb172d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1740fee40 0x7fb17419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53018/0 (socket says 172.21.15.2:53018) 2023-06-23T18:21:30.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb172d9d700 1 -- 172.21.15.2:0/3425601000 learned_addr learned my addr 172.21.15.2:0/3425601000 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:30.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17359e700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174100f80 0x7fb174194b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:30.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb17259c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1741005d0 0x7fb17419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:30.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb172d9d700 1 -- 172.21.15.2:0/3425601000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174100f80 msgr2=0x7fb174194b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:30.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb172d9d700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174100f80 0x7fb174194b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb172d9d700 1 -- 172.21.15.2:0/3425601000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1741005d0 msgr2=0x7fb17419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:30.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb172d9d700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1741005d0 0x7fb17419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.181+0000 7fb172d9d700 1 -- 172.21.15.2:0/3425601000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16c009d00 con 0x7fb1740fee40 2023-06-23T18:21:30.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.182+0000 7fb172d9d700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1740fee40 0x7fb17419aab0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb16400d950 tx=0x7fb16400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:30.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.182+0000 7fb15bfff700 1 -- 172.21.15.2:0/3425601000 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb164009940 con 0x7fb1740fee40 2023-06-23T18:21:30.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.182+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1741957b0 con 0x7fb1740fee40 2023-06-23T18:21:30.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.182+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1741a1aa0 con 0x7fb1740fee40 2023-06-23T18:21:30.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.182+0000 7fb15bfff700 1 -- 172.21.15.2:0/3425601000 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb164010460 con 0x7fb1740fee40 2023-06-23T18:21:30.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.182+0000 7fb15bfff700 1 -- 172.21.15.2:0/3425601000 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400f620 con 0x7fb1740fee40 2023-06-23T18:21:30.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.183+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb160005320 con 0x7fb1740fee40 2023-06-23T18:21:30.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.184+0000 7fb15bfff700 1 -- 172.21.15.2:0/3425601000 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb164009aa0 con 0x7fb1740fee40 2023-06-23T18:21:30.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.185+0000 7fb15bfff700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb15c0a33f0 0x7fb15c0a58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:30.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.185+0000 7fb17259c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb15c0a33f0 0x7fb15c0a58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:30.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.185+0000 7fb15bfff700 1 -- 172.21.15.2:0/3425601000 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb1640cc990 con 0x7fb1740fee40 2023-06-23T18:21:30.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.185+0000 7fb17259c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb15c0a33f0 0x7fb15c0a58a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb168006fd0 tx=0x7fb168008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:30.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.189+0000 7fb15bfff700 1 -- 172.21.15.2:0/3425601000 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb164096130 con 0x7fb1740fee40 2023-06-23T18:21:30.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.381+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb160000bf0 con 0x7fb15c0a33f0 2023-06-23T18:21:30.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.394+0000 7fb15bfff700 1 -- 172.21.15.2:0/3425601000 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340595 (secure 0 0 0) 0x7fb160000bf0 con 0x7fb15c0a33f0 2023-06-23T18:21:30.402 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:30.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb15c0a33f0 msgr2=0x7fb15c0a58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:30.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb15c0a33f0 0x7fb15c0a58a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb168006fd0 tx=0x7fb168008040 comp rx=0 tx=0).stop 2023-06-23T18:21:30.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1740fee40 msgr2=0x7fb17419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:30.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1740fee40 0x7fb17419aab0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb16400d950 tx=0x7fb16400dc60 comp rx=0 tx=0).stop 2023-06-23T18:21:30.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 shutdown_connections 2023-06-23T18:21:30.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb174100f80 0x7fb174194b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb1740fee40 0x7fb17419aab0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb15c0a33f0 0x7fb15c0a58a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 --2- 172.21.15.2:0/3425601000 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1741005d0 0x7fb17419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 >> 172.21.15.2:0/3425601000 conn(0x7fb174075b20 msgr2=0x7fb1740fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 shutdown_connections 2023-06-23T18:21:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:30.397+0000 7fb17922c700 1 -- 172.21.15.2:0/3425601000 wait complete. 2023-06-23T18:21:30.405 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-06-23T18:21:30.838 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2023-06-23T18:21:28.742565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652016,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":383,"num_read_kb":304,"num_write":337,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334256,"kb_used_data":10832,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607568,"statfs":{"total":1151940427776,"available":1151598149632,"internally_reserved":0,"allocated":11091968,"data_stored":6993185,"data_compressed":107457,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":42,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003310"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406331+0000","last_change":"2023-06-23T18:20:44.466650+0000","last_active":"2023-06-23T18:20:46.406331+0000","last_peered":"2023-06-23T18:20:46.406331+0000","last_clean":"2023-06-23T18:20:46.406331+0000","last_became_active":"2023-06-23T18:20:44.466425+0000","last_became_peered":"2023-06-23T18:20:44.466425+0000","last_unstale":"2023-06-23T18:20:46.406331+0000","last_undegraded":"2023-06-23T18:20:46.406331+0000","last_fullsized":"2023-06-23T18:20:46.406331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:46:45.756486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109943+0000","last_change":"2023-06-23T18:20:44.465762+0000","last_active":"2023-06-23T18:20:46.109943+0000","last_peered":"2023-06-23T18:20:46.109943+0000","last_clean":"2023-06-23T18:20:46.109943+0000","last_became_active":"2023-06-23T18:20:44.463592+0000","last_became_peered":"2023-06-23T18:20:44.463592+0000","last_unstale":"2023-06-23T18:20:46.109943+0000","last_undegraded":"2023-06-23T18:20:46.109943+0000","last_fullsized":"2023-06-23T18:20:46.109943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:47:42.304223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109873+0000","last_change":"2023-06-23T18:20:44.464162+0000","last_active":"2023-06-23T18:20:46.109873+0000","last_peered":"2023-06-23T18:20:46.109873+0000","last_clean":"2023-06-23T18:20:46.109873+0000","last_became_active":"2023-06-23T18:20:44.463854+0000","last_became_peered":"2023-06-23T18:20:44.463854+0000","last_unstale":"2023-06-23T18:20:46.109873+0000","last_undegraded":"2023-06-23T18:20:46.109873+0000","last_fullsized":"2023-06-23T18:20:46.109873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:07:55.908839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.366029+0000","last_change":"2023-06-23T18:20:44.462923+0000","last_active":"2023-06-23T18:20:46.366029+0000","last_peered":"2023-06-23T18:20:46.366029+0000","last_clean":"2023-06-23T18:20:46.366029+0000","last_became_active":"2023-06-23T18:20:44.462511+0000","last_became_peered":"2023-06-23T18:20:44.462511+0000","last_unstale":"2023-06-23T18:20:46.366029+0000","last_undegraded":"2023-06-23T18:20:46.366029+0000","last_fullsized":"2023-06-23T18:20:46.366029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:32:40.690012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530812+0000","last_change":"2023-06-23T18:20:44.454727+0000","last_active":"2023-06-23T18:20:45.530812+0000","last_peered":"2023-06-23T18:20:45.530812+0000","last_clean":"2023-06-23T18:20:45.530812+0000","last_became_active":"2023-06-23T18:20:44.454418+0000","last_became_peered":"2023-06-23T18:20:44.454418+0000","last_unstale":"2023-06-23T18:20:45.530812+0000","last_undegraded":"2023-06-23T18:20:45.530812+0000","last_fullsized":"2023-06-23T18:20:45.530812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:34:09.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905435+0000","last_change":"2023-06-23T18:20:44.460636+0000","last_active":"2023-06-23T18:20:45.905435+0000","last_peered":"2023-06-23T18:20:45.905435+0000","last_clean":"2023-06-23T18:20:45.905435+0000","last_became_active":"2023-06-23T18:20:44.460305+0000","last_became_peered":"2023-06-23T18:20:44.460305+0000","last_unstale":"2023-06-23T18:20:45.905435+0000","last_undegraded":"2023-06-23T18:20:45.905435+0000","last_fullsized":"2023-06-23T18:20:45.905435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:20:11.034702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439073+0000","last_change":"2023-06-23T18:20:44.463552+0000","last_active":"2023-06-23T18:20:45.439073+0000","last_peered":"2023-06-23T18:20:45.439073+0000","last_clean":"2023-06-23T18:20:45.439073+0000","last_became_active":"2023-06-23T18:20:44.463187+0000","last_became_peered":"2023-06-23T18:20:44.463187+0000","last_unstale":"2023-06-23T18:20:45.439073+0000","last_undegraded":"2023-06-23T18:20:45.439073+0000","last_fullsized":"2023-06-23T18:20:45.439073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:16:33.692125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439061+0000","last_change":"2023-06-23T18:20:44.457957+0000","last_active":"2023-06-23T18:20:45.439061+0000","last_peered":"2023-06-23T18:20:45.439061+0000","last_clean":"2023-06-23T18:20:45.439061+0000","last_became_active":"2023-06-23T18:20:44.457511+0000","last_became_peered":"2023-06-23T18:20:44.457511+0000","last_unstale":"2023-06-23T18:20:45.439061+0000","last_undegraded":"2023-06-23T18:20:45.439061+0000","last_fullsized":"2023-06-23T18:20:45.439061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:19:36.056785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406274+0000","last_change":"2023-06-23T18:20:44.461199+0000","last_active":"2023-06-23T18:20:46.406274+0000","last_peered":"2023-06-23T18:20:46.406274+0000","last_clean":"2023-06-23T18:20:46.406274+0000","last_became_active":"2023-06-23T18:20:44.460891+0000","last_became_peered":"2023-06-23T18:20:44.460891+0000","last_unstale":"2023-06-23T18:20:46.406274+0000","last_undegraded":"2023-06-23T18:20:46.406274+0000","last_fullsized":"2023-06-23T18:20:46.406274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:00.977889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905565+0000","last_change":"2023-06-23T18:20:44.460741+0000","last_active":"2023-06-23T18:20:45.905565+0000","last_peered":"2023-06-23T18:20:45.905565+0000","last_clean":"2023-06-23T18:20:45.905565+0000","last_became_active":"2023-06-23T18:20:44.460109+0000","last_became_peered":"2023-06-23T18:20:44.460109+0000","last_unstale":"2023-06-23T18:20:45.905565+0000","last_undegraded":"2023-06-23T18:20:45.905565+0000","last_fullsized":"2023-06-23T18:20:45.905565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:37:37.614265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439769+0000","last_change":"2023-06-23T18:20:44.453337+0000","last_active":"2023-06-23T18:20:45.439769+0000","last_peered":"2023-06-23T18:20:45.439769+0000","last_clean":"2023-06-23T18:20:45.439769+0000","last_became_active":"2023-06-23T18:20:44.452925+0000","last_became_peered":"2023-06-23T18:20:44.452925+0000","last_unstale":"2023-06-23T18:20:45.439769+0000","last_undegraded":"2023-06-23T18:20:45.439769+0000","last_fullsized":"2023-06-23T18:20:45.439769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:10.381911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.365990+0000","last_change":"2023-06-23T18:20:44.456710+0000","last_active":"2023-06-23T18:20:46.365990+0000","last_peered":"2023-06-23T18:20:46.365990+0000","last_clean":"2023-06-23T18:20:46.365990+0000","last_became_active":"2023-06-23T18:20:44.456411+0000","last_became_peered":"2023-06-23T18:20:44.456411+0000","last_unstale":"2023-06-23T18:20:46.365990+0000","last_undegraded":"2023-06-23T18:20:46.365990+0000","last_fullsized":"2023-06-23T18:20:46.365990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:01:45.669051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406319+0000","last_change":"2023-06-23T18:20:44.453586+0000","last_active":"2023-06-23T18:20:46.406319+0000","last_peered":"2023-06-23T18:20:46.406319+0000","last_clean":"2023-06-23T18:20:46.406319+0000","last_became_active":"2023-06-23T18:20:44.453239+0000","last_became_peered":"2023-06-23T18:20:44.453239+0000","last_unstale":"2023-06-23T18:20:46.406319+0000","last_undegraded":"2023-06-23T18:20:46.406319+0000","last_fullsized":"2023-06-23T18:20:46.406319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:54:58.454675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109577+0000","last_change":"2023-06-23T18:20:44.465627+0000","last_active":"2023-06-23T18:20:46.109577+0000","last_peered":"2023-06-23T18:20:46.109577+0000","last_clean":"2023-06-23T18:20:46.109577+0000","last_became_active":"2023-06-23T18:20:44.464568+0000","last_became_peered":"2023-06-23T18:20:44.464568+0000","last_unstale":"2023-06-23T18:20:46.109577+0000","last_undegraded":"2023-06-23T18:20:46.109577+0000","last_fullsized":"2023-06-23T18:20:46.109577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:29:20.689694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109533+0000","last_change":"2023-06-23T18:20:44.464169+0000","last_active":"2023-06-23T18:20:46.109533+0000","last_peered":"2023-06-23T18:20:46.109533+0000","last_clean":"2023-06-23T18:20:46.109533+0000","last_became_active":"2023-06-23T18:20:44.463867+0000","last_became_peered":"2023-06-23T18:20:44.463867+0000","last_unstale":"2023-06-23T18:20:46.109533+0000","last_undegraded":"2023-06-23T18:20:46.109533+0000","last_fullsized":"2023-06-23T18:20:46.109533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:28:14.663838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405761+0000","last_change":"2023-06-23T18:20:44.467101+0000","last_active":"2023-06-23T18:20:46.405761+0000","last_peered":"2023-06-23T18:20:46.405761+0000","last_clean":"2023-06-23T18:20:46.405761+0000","last_became_active":"2023-06-23T18:20:44.466788+0000","last_became_peered":"2023-06-23T18:20:44.466788+0000","last_unstale":"2023-06-23T18:20:46.405761+0000","last_undegraded":"2023-06-23T18:20:46.405761+0000","last_fullsized":"2023-06-23T18:20:46.405761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:14:55.735170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406210+0000","last_change":"2023-06-23T18:20:44.466055+0000","last_active":"2023-06-23T18:20:46.406210+0000","last_peered":"2023-06-23T18:20:46.406210+0000","last_clean":"2023-06-23T18:20:46.406210+0000","last_became_active":"2023-06-23T18:20:44.465608+0000","last_became_peered":"2023-06-23T18:20:44.465608+0000","last_unstale":"2023-06-23T18:20:46.406210+0000","last_undegraded":"2023-06-23T18:20:46.406210+0000","last_fullsized":"2023-06-23T18:20:46.406210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:06:22.567595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.529187+0000","last_change":"2023-06-23T18:20:44.459414+0000","last_active":"2023-06-23T18:20:45.529187+0000","last_peered":"2023-06-23T18:20:45.529187+0000","last_clean":"2023-06-23T18:20:45.529187+0000","last_became_active":"2023-06-23T18:20:44.459014+0000","last_became_peered":"2023-06-23T18:20:44.459014+0000","last_unstale":"2023-06-23T18:20:45.529187+0000","last_undegraded":"2023-06-23T18:20:45.529187+0000","last_fullsized":"2023-06-23T18:20:45.529187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:00:16.944957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237780+0000","last_change":"2023-06-23T18:20:44.446784+0000","last_active":"2023-06-23T18:20:46.237780+0000","last_peered":"2023-06-23T18:20:46.237780+0000","last_clean":"2023-06-23T18:20:46.237780+0000","last_became_active":"2023-06-23T18:20:44.446464+0000","last_became_peered":"2023-06-23T18:20:44.446464+0000","last_unstale":"2023-06-23T18:20:46.237780+0000","last_undegraded":"2023-06-23T18:20:46.237780+0000","last_fullsized":"2023-06-23T18:20:46.237780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:03:13.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.501338+0000","last_change":"2023-06-23T18:20:44.039326+0000","last_active":"2023-06-23T18:20:48.501338+0000","last_peered":"2023-06-23T18:20:48.501338+0000","last_clean":"2023-06-23T18:20:48.501338+0000","last_became_active":"2023-06-23T18:20:44.038787+0000","last_became_peered":"2023-06-23T18:20:44.038787+0000","last_unstale":"2023-06-23T18:20:48.501338+0000","last_undegraded":"2023-06-23T18:20:48.501338+0000","last_fullsized":"2023-06-23T18:20:48.501338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:16:15.016712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439642+0000","last_change":"2023-06-23T18:20:44.465269+0000","last_active":"2023-06-23T18:20:45.439642+0000","last_peered":"2023-06-23T18:20:45.439642+0000","last_clean":"2023-06-23T18:20:45.439642+0000","last_became_active":"2023-06-23T18:20:44.464973+0000","last_became_peered":"2023-06-23T18:20:44.464973+0000","last_unstale":"2023-06-23T18:20:45.439642+0000","last_undegraded":"2023-06-23T18:20:45.439642+0000","last_fullsized":"2023-06-23T18:20:45.439642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:24:55.096958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.847597+0000","last_change":"2023-06-23T18:20:44.037175+0000","last_active":"2023-06-23T18:20:45.847597+0000","last_peered":"2023-06-23T18:20:45.847597+0000","last_clean":"2023-06-23T18:20:45.847597+0000","last_became_active":"2023-06-23T18:20:44.036788+0000","last_became_peered":"2023-06-23T18:20:44.036788+0000","last_unstale":"2023-06-23T18:20:45.847597+0000","last_undegraded":"2023-06-23T18:20:45.847597+0000","last_fullsized":"2023-06-23T18:20:45.847597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:15:06.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.110040+0000","last_change":"2023-06-23T18:20:44.463344+0000","last_active":"2023-06-23T18:20:46.110040+0000","last_peered":"2023-06-23T18:20:46.110040+0000","last_clean":"2023-06-23T18:20:46.110040+0000","last_became_active":"2023-06-23T18:20:44.462649+0000","last_became_peered":"2023-06-23T18:20:44.462649+0000","last_unstale":"2023-06-23T18:20:46.110040+0000","last_undegraded":"2023-06-23T18:20:46.110040+0000","last_fullsized":"2023-06-23T18:20:46.110040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:39:49.841778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499503+0000","last_change":"2023-06-23T18:20:44.045525+0000","last_active":"2023-06-23T18:20:48.499503+0000","last_peered":"2023-06-23T18:20:48.499503+0000","last_clean":"2023-06-23T18:20:48.499503+0000","last_became_active":"2023-06-23T18:20:44.045212+0000","last_became_peered":"2023-06-23T18:20:44.045212+0000","last_unstale":"2023-06-23T18:20:48.499503+0000","last_undegraded":"2023-06-23T18:20:48.499503+0000","last_fullsized":"2023-06-23T18:20:48.499503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:44:56.572688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405870+0000","last_change":"2023-06-23T18:20:44.461071+0000","last_active":"2023-06-23T18:20:46.405870+0000","last_peered":"2023-06-23T18:20:46.405870+0000","last_clean":"2023-06-23T18:20:46.405870+0000","last_became_active":"2023-06-23T18:20:44.460726+0000","last_became_peered":"2023-06-23T18:20:44.460726+0000","last_unstale":"2023-06-23T18:20:46.405870+0000","last_undegraded":"2023-06-23T18:20:46.405870+0000","last_fullsized":"2023-06-23T18:20:46.405870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:08:47.572741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530990+0000","last_change":"2023-06-23T18:20:44.037162+0000","last_active":"2023-06-23T18:20:45.530990+0000","last_peered":"2023-06-23T18:20:45.530990+0000","last_clean":"2023-06-23T18:20:45.530990+0000","last_became_active":"2023-06-23T18:20:44.036809+0000","last_became_peered":"2023-06-23T18:20:44.036809+0000","last_unstale":"2023-06-23T18:20:45.530990+0000","last_undegraded":"2023-06-23T18:20:45.530990+0000","last_fullsized":"2023-06-23T18:20:45.530990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:42:11.269748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439457+0000","last_change":"2023-06-23T18:20:44.463525+0000","last_active":"2023-06-23T18:20:45.439457+0000","last_peered":"2023-06-23T18:20:45.439457+0000","last_clean":"2023-06-23T18:20:45.439457+0000","last_became_active":"2023-06-23T18:20:44.463245+0000","last_became_peered":"2023-06-23T18:20:44.463245+0000","last_unstale":"2023-06-23T18:20:45.439457+0000","last_undegraded":"2023-06-23T18:20:45.439457+0000","last_fullsized":"2023-06-23T18:20:45.439457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:18:47.776141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:23.102989+0000","last_change":"2023-06-23T18:20:44.044415+0000","last_active":"2023-06-23T18:21:23.102989+0000","last_peered":"2023-06-23T18:21:23.102989+0000","last_clean":"2023-06-23T18:21:23.102989+0000","last_became_active":"2023-06-23T18:20:44.044085+0000","last_became_peered":"2023-06-23T18:20:44.044085+0000","last_unstale":"2023-06-23T18:21:23.102989+0000","last_undegraded":"2023-06-23T18:21:23.102989+0000","last_fullsized":"2023-06-23T18:21:23.102989+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:07:29.948276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530911+0000","last_change":"2023-06-23T18:20:44.463422+0000","last_active":"2023-06-23T18:20:45.530911+0000","last_peered":"2023-06-23T18:20:45.530911+0000","last_clean":"2023-06-23T18:20:45.530911+0000","last_became_active":"2023-06-23T18:20:44.463121+0000","last_became_peered":"2023-06-23T18:20:44.463121+0000","last_unstale":"2023-06-23T18:20:45.530911+0000","last_undegraded":"2023-06-23T18:20:45.530911+0000","last_fullsized":"2023-06-23T18:20:45.530911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:54:44.620991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.500329+0000","last_change":"2023-06-23T18:20:44.039604+0000","last_active":"2023-06-23T18:20:48.500329+0000","last_peered":"2023-06-23T18:20:48.500329+0000","last_clean":"2023-06-23T18:20:48.500329+0000","last_became_active":"2023-06-23T18:20:44.039317+0000","last_became_peered":"2023-06-23T18:20:44.039317+0000","last_unstale":"2023-06-23T18:20:48.500329+0000","last_undegraded":"2023-06-23T18:20:48.500329+0000","last_fullsized":"2023-06-23T18:20:48.500329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:39:40.353762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439176+0000","last_change":"2023-06-23T18:20:44.465748+0000","last_active":"2023-06-23T18:20:45.439176+0000","last_peered":"2023-06-23T18:20:45.439176+0000","last_clean":"2023-06-23T18:20:45.439176+0000","last_became_active":"2023-06-23T18:20:44.465430+0000","last_became_peered":"2023-06-23T18:20:44.465430+0000","last_unstale":"2023-06-23T18:20:45.439176+0000","last_undegraded":"2023-06-23T18:20:45.439176+0000","last_fullsized":"2023-06-23T18:20:45.439176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:31.824195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499906+0000","last_change":"2023-06-23T18:20:44.034388+0000","last_active":"2023-06-23T18:20:48.499906+0000","last_peered":"2023-06-23T18:20:48.499906+0000","last_clean":"2023-06-23T18:20:48.499906+0000","last_became_active":"2023-06-23T18:20:44.034067+0000","last_became_peered":"2023-06-23T18:20:44.034067+0000","last_unstale":"2023-06-23T18:20:48.499906+0000","last_undegraded":"2023-06-23T18:20:48.499906+0000","last_fullsized":"2023-06-23T18:20:48.499906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:39:02.092084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.438998+0000","last_change":"2023-06-23T18:20:44.453280+0000","last_active":"2023-06-23T18:20:45.438998+0000","last_peered":"2023-06-23T18:20:45.438998+0000","last_clean":"2023-06-23T18:20:45.438998+0000","last_became_active":"2023-06-23T18:20:44.452980+0000","last_became_peered":"2023-06-23T18:20:44.452980+0000","last_unstale":"2023-06-23T18:20:45.438998+0000","last_undegraded":"2023-06-23T18:20:45.438998+0000","last_fullsized":"2023-06-23T18:20:45.438998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:24:40.346057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439241+0000","last_change":"2023-06-23T18:20:44.036165+0000","last_active":"2023-06-23T18:20:45.439241+0000","last_peered":"2023-06-23T18:20:45.439241+0000","last_clean":"2023-06-23T18:20:45.439241+0000","last_became_active":"2023-06-23T18:20:44.035313+0000","last_became_peered":"2023-06-23T18:20:44.035313+0000","last_unstale":"2023-06-23T18:20:45.439241+0000","last_undegraded":"2023-06-23T18:20:45.439241+0000","last_fullsized":"2023-06-23T18:20:45.439241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:26:02.622257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.528993+0000","last_change":"2023-06-23T18:20:44.468358+0000","last_active":"2023-06-23T18:20:45.528993+0000","last_peered":"2023-06-23T18:20:45.528993+0000","last_clean":"2023-06-23T18:20:45.528993+0000","last_became_active":"2023-06-23T18:20:44.468032+0000","last_became_peered":"2023-06-23T18:20:44.468032+0000","last_unstale":"2023-06-23T18:20:45.528993+0000","last_undegraded":"2023-06-23T18:20:45.528993+0000","last_fullsized":"2023-06-23T18:20:45.528993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:31.361443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:26.106506+0000","last_change":"2023-06-23T18:20:44.037921+0000","last_active":"2023-06-23T18:21:26.106506+0000","last_peered":"2023-06-23T18:21:26.106506+0000","last_clean":"2023-06-23T18:21:26.106506+0000","last_became_active":"2023-06-23T18:20:44.037571+0000","last_became_peered":"2023-06-23T18:20:44.037571+0000","last_unstale":"2023-06-23T18:21:26.106506+0000","last_undegraded":"2023-06-23T18:21:26.106506+0000","last_fullsized":"2023-06-23T18:21:26.106506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:55.796755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406218+0000","last_change":"2023-06-23T18:20:44.460897+0000","last_active":"2023-06-23T18:20:46.406218+0000","last_peered":"2023-06-23T18:20:46.406218+0000","last_clean":"2023-06-23T18:20:46.406218+0000","last_became_active":"2023-06-23T18:20:44.460476+0000","last_became_peered":"2023-06-23T18:20:44.460476+0000","last_unstale":"2023-06-23T18:20:46.406218+0000","last_undegraded":"2023-06-23T18:20:46.406218+0000","last_fullsized":"2023-06-23T18:20:46.406218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:43.488229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498266+0000","last_change":"2023-06-23T18:20:44.033013+0000","last_active":"2023-06-23T18:20:47.498266+0000","last_peered":"2023-06-23T18:20:47.498266+0000","last_clean":"2023-06-23T18:20:47.498266+0000","last_became_active":"2023-06-23T18:20:44.032580+0000","last_became_peered":"2023-06-23T18:20:44.032580+0000","last_unstale":"2023-06-23T18:20:47.498266+0000","last_undegraded":"2023-06-23T18:20:47.498266+0000","last_fullsized":"2023-06-23T18:20:47.498266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:32:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.110064+0000","last_change":"2023-06-23T18:20:44.463585+0000","last_active":"2023-06-23T18:20:46.110064+0000","last_peered":"2023-06-23T18:20:46.110064+0000","last_clean":"2023-06-23T18:20:46.110064+0000","last_became_active":"2023-06-23T18:20:44.462512+0000","last_became_peered":"2023-06-23T18:20:44.462512+0000","last_unstale":"2023-06-23T18:20:46.110064+0000","last_undegraded":"2023-06-23T18:20:46.110064+0000","last_fullsized":"2023-06-23T18:20:46.110064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:53:36.151767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497460+0000","last_change":"2023-06-23T18:20:44.037134+0000","last_active":"2023-06-23T18:20:47.497460+0000","last_peered":"2023-06-23T18:20:47.497460+0000","last_clean":"2023-06-23T18:20:47.497460+0000","last_became_active":"2023-06-23T18:20:44.036536+0000","last_became_peered":"2023-06-23T18:20:44.036536+0000","last_unstale":"2023-06-23T18:20:47.497460+0000","last_undegraded":"2023-06-23T18:20:47.497460+0000","last_fullsized":"2023-06-23T18:20:47.497460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:58:25.025236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406029+0000","last_change":"2023-06-23T18:20:44.454453+0000","last_active":"2023-06-23T18:20:46.406029+0000","last_peered":"2023-06-23T18:20:46.406029+0000","last_clean":"2023-06-23T18:20:46.406029+0000","last_became_active":"2023-06-23T18:20:44.454133+0000","last_became_peered":"2023-06-23T18:20:44.454133+0000","last_unstale":"2023-06-23T18:20:46.406029+0000","last_undegraded":"2023-06-23T18:20:46.406029+0000","last_fullsized":"2023-06-23T18:20:46.406029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:52:02.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":129,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:27.107854+0000","last_change":"2023-06-23T18:20:44.042622+0000","last_active":"2023-06-23T18:21:27.107854+0000","last_peered":"2023-06-23T18:21:27.107854+0000","last_clean":"2023-06-23T18:21:27.107854+0000","last_became_active":"2023-06-23T18:20:44.042330+0000","last_became_peered":"2023-06-23T18:20:44.042330+0000","last_unstale":"2023-06-23T18:21:27.107854+0000","last_undegraded":"2023-06-23T18:21:27.107854+0000","last_fullsized":"2023-06-23T18:21:27.107854+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:23.320076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57498,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":86,"num_read_kb":99,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.531053+0000","last_change":"2023-06-23T18:20:44.456558+0000","last_active":"2023-06-23T18:20:45.531053+0000","last_peered":"2023-06-23T18:20:45.531053+0000","last_clean":"2023-06-23T18:20:45.531053+0000","last_became_active":"2023-06-23T18:20:44.456199+0000","last_became_peered":"2023-06-23T18:20:44.456199+0000","last_unstale":"2023-06-23T18:20:45.531053+0000","last_undegraded":"2023-06-23T18:20:45.531053+0000","last_fullsized":"2023-06-23T18:20:45.531053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:24:13.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499315+0000","last_change":"2023-06-23T18:20:44.034101+0000","last_active":"2023-06-23T18:20:48.499315+0000","last_peered":"2023-06-23T18:20:48.499315+0000","last_clean":"2023-06-23T18:20:48.499315+0000","last_became_active":"2023-06-23T18:20:44.033287+0000","last_became_peered":"2023-06-23T18:20:44.033287+0000","last_unstale":"2023-06-23T18:20:48.499315+0000","last_undegraded":"2023-06-23T18:20:48.499315+0000","last_fullsized":"2023-06-23T18:20:48.499315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:08:52.512924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406111+0000","last_change":"2023-06-23T18:20:44.446298+0000","last_active":"2023-06-23T18:20:46.406111+0000","last_peered":"2023-06-23T18:20:46.406111+0000","last_clean":"2023-06-23T18:20:46.406111+0000","last_became_active":"2023-06-23T18:20:44.445922+0000","last_became_peered":"2023-06-23T18:20:44.445922+0000","last_unstale":"2023-06-23T18:20:46.406111+0000","last_undegraded":"2023-06-23T18:20:46.406111+0000","last_fullsized":"2023-06-23T18:20:46.406111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:08:15.214033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498820+0000","last_change":"2023-06-23T18:20:44.035791+0000","last_active":"2023-06-23T18:20:47.498820+0000","last_peered":"2023-06-23T18:20:47.498820+0000","last_clean":"2023-06-23T18:20:47.498820+0000","last_became_active":"2023-06-23T18:20:44.035416+0000","last_became_peered":"2023-06-23T18:20:44.035416+0000","last_unstale":"2023-06-23T18:20:47.498820+0000","last_undegraded":"2023-06-23T18:20:47.498820+0000","last_fullsized":"2023-06-23T18:20:47.498820+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:02:31.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439429+0000","last_change":"2023-06-23T18:20:44.455474+0000","last_active":"2023-06-23T18:20:45.439429+0000","last_peered":"2023-06-23T18:20:45.439429+0000","last_clean":"2023-06-23T18:20:45.439429+0000","last_became_active":"2023-06-23T18:20:44.455098+0000","last_became_peered":"2023-06-23T18:20:44.455098+0000","last_unstale":"2023-06-23T18:20:45.439429+0000","last_undegraded":"2023-06-23T18:20:45.439429+0000","last_fullsized":"2023-06-23T18:20:45.439429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:53:34.377017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":60,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:26.106700+0000","last_change":"2023-06-23T18:20:44.030305+0000","last_active":"2023-06-23T18:21:26.106700+0000","last_peered":"2023-06-23T18:21:26.106700+0000","last_clean":"2023-06-23T18:21:26.106700+0000","last_became_active":"2023-06-23T18:20:44.029898+0000","last_became_peered":"2023-06-23T18:20:44.029898+0000","last_unstale":"2023-06-23T18:21:26.106700+0000","last_undegraded":"2023-06-23T18:21:26.106700+0000","last_fullsized":"2023-06-23T18:21:26.106700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:38:03.187411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439068+0000","last_change":"2023-06-23T18:20:44.451321+0000","last_active":"2023-06-23T18:20:45.439068+0000","last_peered":"2023-06-23T18:20:45.439068+0000","last_clean":"2023-06-23T18:20:45.439068+0000","last_became_active":"2023-06-23T18:20:44.450549+0000","last_became_peered":"2023-06-23T18:20:44.450549+0000","last_unstale":"2023-06-23T18:20:45.439068+0000","last_undegraded":"2023-06-23T18:20:45.439068+0000","last_fullsized":"2023-06-23T18:20:45.439068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:44:56.410567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.503294+0000","last_change":"2023-06-23T18:20:44.044707+0000","last_active":"2023-06-23T18:20:48.503294+0000","last_peered":"2023-06-23T18:20:48.503294+0000","last_clean":"2023-06-23T18:20:48.503294+0000","last_became_active":"2023-06-23T18:20:44.044389+0000","last_became_peered":"2023-06-23T18:20:44.044389+0000","last_unstale":"2023-06-23T18:20:48.503294+0000","last_undegraded":"2023-06-23T18:20:48.503294+0000","last_fullsized":"2023-06-23T18:20:48.503294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:45:51.078124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439091+0000","last_change":"2023-06-23T18:20:44.463637+0000","last_active":"2023-06-23T18:20:45.439091+0000","last_peered":"2023-06-23T18:20:45.439091+0000","last_clean":"2023-06-23T18:20:45.439091+0000","last_became_active":"2023-06-23T18:20:44.463289+0000","last_became_peered":"2023-06-23T18:20:44.463289+0000","last_unstale":"2023-06-23T18:20:45.439091+0000","last_undegraded":"2023-06-23T18:20:45.439091+0000","last_fullsized":"2023-06-23T18:20:45.439091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:24:04.923988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:49.049012+0000","last_change":"2023-06-23T18:20:44.038961+0000","last_active":"2023-06-23T18:20:49.049012+0000","last_peered":"2023-06-23T18:20:49.049012+0000","last_clean":"2023-06-23T18:20:49.049012+0000","last_became_active":"2023-06-23T18:20:44.038674+0000","last_became_peered":"2023-06-23T18:20:44.038674+0000","last_unstale":"2023-06-23T18:20:49.049012+0000","last_undegraded":"2023-06-23T18:20:49.049012+0000","last_fullsized":"2023-06-23T18:20:49.049012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:18:04.288041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406228+0000","last_change":"2023-06-23T18:20:44.461077+0000","last_active":"2023-06-23T18:20:46.406228+0000","last_peered":"2023-06-23T18:20:46.406228+0000","last_clean":"2023-06-23T18:20:46.406228+0000","last_became_active":"2023-06-23T18:20:44.460767+0000","last_became_peered":"2023-06-23T18:20:44.460767+0000","last_unstale":"2023-06-23T18:20:46.406228+0000","last_undegraded":"2023-06-23T18:20:46.406228+0000","last_fullsized":"2023-06-23T18:20:46.406228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:16.534724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.529039+0000","last_change":"2023-06-23T18:20:44.042114+0000","last_active":"2023-06-23T18:20:45.529039+0000","last_peered":"2023-06-23T18:20:45.529039+0000","last_clean":"2023-06-23T18:20:45.529039+0000","last_became_active":"2023-06-23T18:20:44.041822+0000","last_became_peered":"2023-06-23T18:20:44.041822+0000","last_unstale":"2023-06-23T18:20:45.529039+0000","last_undegraded":"2023-06-23T18:20:45.529039+0000","last_fullsized":"2023-06-23T18:20:45.529039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:36:53.587200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.456925+0000","last_change":"2023-06-23T18:20:44.452903+0000","last_active":"2023-06-23T18:20:47.456925+0000","last_peered":"2023-06-23T18:20:47.456925+0000","last_clean":"2023-06-23T18:20:47.456925+0000","last_became_active":"2023-06-23T18:20:44.452539+0000","last_became_peered":"2023-06-23T18:20:44.452539+0000","last_unstale":"2023-06-23T18:20:47.456925+0000","last_undegraded":"2023-06-23T18:20:47.456925+0000","last_fullsized":"2023-06-23T18:20:47.456925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:53:53.970572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.038724+0000","last_change":"2023-06-23T18:20:44.032905+0000","last_active":"2023-06-23T18:20:47.038724+0000","last_peered":"2023-06-23T18:20:47.038724+0000","last_clean":"2023-06-23T18:20:47.038724+0000","last_became_active":"2023-06-23T18:20:44.032606+0000","last_became_peered":"2023-06-23T18:20:44.032606+0000","last_unstale":"2023-06-23T18:20:47.038724+0000","last_undegraded":"2023-06-23T18:20:47.038724+0000","last_fullsized":"2023-06-23T18:20:47.038724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:05.992912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905273+0000","last_change":"2023-06-23T18:20:44.465580+0000","last_active":"2023-06-23T18:20:45.905273+0000","last_peered":"2023-06-23T18:20:45.905273+0000","last_clean":"2023-06-23T18:20:45.905273+0000","last_became_active":"2023-06-23T18:20:44.465264+0000","last_became_peered":"2023-06-23T18:20:44.465264+0000","last_unstale":"2023-06-23T18:20:45.905273+0000","last_undegraded":"2023-06-23T18:20:45.905273+0000","last_fullsized":"2023-06-23T18:20:45.905273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:20:07.333937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439575+0000","last_change":"2023-06-23T18:20:44.464324+0000","last_active":"2023-06-23T18:20:45.439575+0000","last_peered":"2023-06-23T18:20:45.439575+0000","last_clean":"2023-06-23T18:20:45.439575+0000","last_became_active":"2023-06-23T18:20:44.464021+0000","last_became_peered":"2023-06-23T18:20:44.464021+0000","last_unstale":"2023-06-23T18:20:45.439575+0000","last_undegraded":"2023-06-23T18:20:45.439575+0000","last_fullsized":"2023-06-23T18:20:45.439575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:24:15.831014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":56,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:25.104709+0000","last_change":"2023-06-23T18:20:44.034309+0000","last_active":"2023-06-23T18:21:25.104709+0000","last_peered":"2023-06-23T18:21:25.104709+0000","last_clean":"2023-06-23T18:21:25.104709+0000","last_became_active":"2023-06-23T18:20:44.034007+0000","last_became_peered":"2023-06-23T18:20:44.034007+0000","last_unstale":"2023-06-23T18:21:25.104709+0000","last_undegraded":"2023-06-23T18:21:25.104709+0000","last_fullsized":"2023-06-23T18:21:25.104709+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:49:31.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439359+0000","last_change":"2023-06-23T18:20:44.468220+0000","last_active":"2023-06-23T18:20:45.439359+0000","last_peered":"2023-06-23T18:20:45.439359+0000","last_clean":"2023-06-23T18:20:45.439359+0000","last_became_active":"2023-06-23T18:20:44.467725+0000","last_became_peered":"2023-06-23T18:20:44.467725+0000","last_unstale":"2023-06-23T18:20:45.439359+0000","last_undegraded":"2023-06-23T18:20:45.439359+0000","last_fullsized":"2023-06-23T18:20:45.439359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:30.069575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:49.048567+0000","last_change":"2023-06-23T18:20:44.030921+0000","last_active":"2023-06-23T18:20:49.048567+0000","last_peered":"2023-06-23T18:20:49.048567+0000","last_clean":"2023-06-23T18:20:49.048567+0000","last_became_active":"2023-06-23T18:20:44.030362+0000","last_became_peered":"2023-06-23T18:20:44.030362+0000","last_unstale":"2023-06-23T18:20:49.048567+0000","last_undegraded":"2023-06-23T18:20:49.048567+0000","last_fullsized":"2023-06-23T18:20:49.048567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:07:10.000171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":328,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.438999+0000","last_change":"2023-06-23T18:19:44.952262+0000","last_active":"2023-06-23T18:20:45.438999+0000","last_peered":"2023-06-23T18:20:45.438999+0000","last_clean":"2023-06-23T18:20:45.438999+0000","last_became_active":"2023-06-23T18:19:44.951803+0000","last_became_peered":"2023-06-23T18:19:44.951803+0000","last_unstale":"2023-06-23T18:20:45.438999+0000","last_undegraded":"2023-06-23T18:20:45.438999+0000","last_fullsized":"2023-06-23T18:20:45.438999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_clean_scrub_stamp":"2023-06-23T18:16:49.458922+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:49.981569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405900+0000","last_change":"2023-06-23T18:20:44.456602+0000","last_active":"2023-06-23T18:20:46.405900+0000","last_peered":"2023-06-23T18:20:46.405900+0000","last_clean":"2023-06-23T18:20:46.405900+0000","last_became_active":"2023-06-23T18:20:44.456249+0000","last_became_peered":"2023-06-23T18:20:44.456249+0000","last_unstale":"2023-06-23T18:20:46.405900+0000","last_undegraded":"2023-06-23T18:20:46.405900+0000","last_fullsized":"2023-06-23T18:20:46.405900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:26:13.422710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.498627+0000","last_change":"2023-06-23T18:20:44.039100+0000","last_active":"2023-06-23T18:20:48.498627+0000","last_peered":"2023-06-23T18:20:48.498627+0000","last_clean":"2023-06-23T18:20:48.498627+0000","last_became_active":"2023-06-23T18:20:44.038801+0000","last_became_peered":"2023-06-23T18:20:44.038801+0000","last_unstale":"2023-06-23T18:20:48.498627+0000","last_undegraded":"2023-06-23T18:20:48.498627+0000","last_fullsized":"2023-06-23T18:20:48.498627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:03.284504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.496038+0000","last_change":"2023-06-23T18:20:44.038037+0000","last_active":"2023-06-23T18:20:47.496038+0000","last_peered":"2023-06-23T18:20:47.496038+0000","last_clean":"2023-06-23T18:20:47.496038+0000","last_became_active":"2023-06-23T18:20:44.037732+0000","last_became_peered":"2023-06-23T18:20:44.037732+0000","last_unstale":"2023-06-23T18:20:47.496038+0000","last_undegraded":"2023-06-23T18:20:47.496038+0000","last_fullsized":"2023-06-23T18:20:47.496038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:30:58.279148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406200+0000","last_change":"2023-06-23T18:20:44.459927+0000","last_active":"2023-06-23T18:20:46.406200+0000","last_peered":"2023-06-23T18:20:46.406200+0000","last_clean":"2023-06-23T18:20:46.406200+0000","last_became_active":"2023-06-23T18:20:44.459594+0000","last_became_peered":"2023-06-23T18:20:44.459594+0000","last_unstale":"2023-06-23T18:20:46.406200+0000","last_undegraded":"2023-06-23T18:20:46.406200+0000","last_fullsized":"2023-06-23T18:20:46.406200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:35:13.951754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:55.529354+0000","last_change":"2023-06-23T18:20:44.039429+0000","last_active":"2023-06-23T18:20:55.529354+0000","last_peered":"2023-06-23T18:20:55.529354+0000","last_clean":"2023-06-23T18:20:55.529354+0000","last_became_active":"2023-06-23T18:20:44.038944+0000","last_became_peered":"2023-06-23T18:20:44.038944+0000","last_unstale":"2023-06-23T18:20:55.529354+0000","last_undegraded":"2023-06-23T18:20:55.529354+0000","last_fullsized":"2023-06-23T18:20:55.529354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:37:39.247393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439805+0000","last_change":"2023-06-23T18:20:44.458227+0000","last_active":"2023-06-23T18:20:45.439805+0000","last_peered":"2023-06-23T18:20:45.439805+0000","last_clean":"2023-06-23T18:20:45.439805+0000","last_became_active":"2023-06-23T18:20:44.457904+0000","last_became_peered":"2023-06-23T18:20:44.457904+0000","last_unstale":"2023-06-23T18:20:45.439805+0000","last_undegraded":"2023-06-23T18:20:45.439805+0000","last_fullsized":"2023-06-23T18:20:45.439805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:53:21.724545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.500253+0000","last_change":"2023-06-23T18:20:44.034089+0000","last_active":"2023-06-23T18:20:47.500253+0000","last_peered":"2023-06-23T18:20:47.500253+0000","last_clean":"2023-06-23T18:20:47.500253+0000","last_became_active":"2023-06-23T18:20:44.033745+0000","last_became_peered":"2023-06-23T18:20:44.033745+0000","last_unstale":"2023-06-23T18:20:47.500253+0000","last_undegraded":"2023-06-23T18:20:47.500253+0000","last_fullsized":"2023-06-23T18:20:47.500253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:05.244917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.110032+0000","last_change":"2023-06-23T18:20:44.468622+0000","last_active":"2023-06-23T18:20:46.110032+0000","last_peered":"2023-06-23T18:20:46.110032+0000","last_clean":"2023-06-23T18:20:46.110032+0000","last_became_active":"2023-06-23T18:20:44.466831+0000","last_became_peered":"2023-06-23T18:20:44.466831+0000","last_unstale":"2023-06-23T18:20:46.110032+0000","last_undegraded":"2023-06-23T18:20:46.110032+0000","last_fullsized":"2023-06-23T18:20:46.110032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:24:20.760069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497829+0000","last_change":"2023-06-23T18:20:44.030647+0000","last_active":"2023-06-23T18:20:47.497829+0000","last_peered":"2023-06-23T18:20:47.497829+0000","last_clean":"2023-06-23T18:20:47.497829+0000","last_became_active":"2023-06-23T18:20:44.030316+0000","last_became_peered":"2023-06-23T18:20:44.030316+0000","last_unstale":"2023-06-23T18:20:47.497829+0000","last_undegraded":"2023-06-23T18:20:47.497829+0000","last_fullsized":"2023-06-23T18:20:47.497829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:47.369435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530780+0000","last_change":"2023-06-23T18:20:44.458365+0000","last_active":"2023-06-23T18:20:45.530780+0000","last_peered":"2023-06-23T18:20:45.530780+0000","last_clean":"2023-06-23T18:20:45.530780+0000","last_became_active":"2023-06-23T18:20:44.458039+0000","last_became_peered":"2023-06-23T18:20:44.458039+0000","last_unstale":"2023-06-23T18:20:45.530780+0000","last_undegraded":"2023-06-23T18:20:45.530780+0000","last_fullsized":"2023-06-23T18:20:45.530780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:03:49.379689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498764+0000","last_change":"2023-06-23T18:20:44.028908+0000","last_active":"2023-06-23T18:20:47.498764+0000","last_peered":"2023-06-23T18:20:47.498764+0000","last_clean":"2023-06-23T18:20:47.498764+0000","last_became_active":"2023-06-23T18:20:44.028551+0000","last_became_peered":"2023-06-23T18:20:44.028551+0000","last_unstale":"2023-06-23T18:20:47.498764+0000","last_undegraded":"2023-06-23T18:20:47.498764+0000","last_fullsized":"2023-06-23T18:20:47.498764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:31:41.576378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406188+0000","last_change":"2023-06-23T18:20:44.465300+0000","last_active":"2023-06-23T18:20:46.406188+0000","last_peered":"2023-06-23T18:20:46.406188+0000","last_clean":"2023-06-23T18:20:46.406188+0000","last_became_active":"2023-06-23T18:20:44.465008+0000","last_became_peered":"2023-06-23T18:20:44.465008+0000","last_unstale":"2023-06-23T18:20:46.406188+0000","last_undegraded":"2023-06-23T18:20:46.406188+0000","last_fullsized":"2023-06-23T18:20:46.406188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:03:27.021563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237610+0000","last_change":"2023-06-23T18:20:44.041593+0000","last_active":"2023-06-23T18:20:46.237610+0000","last_peered":"2023-06-23T18:20:46.237610+0000","last_clean":"2023-06-23T18:20:46.237610+0000","last_became_active":"2023-06-23T18:20:44.041283+0000","last_became_peered":"2023-06-23T18:20:44.041283+0000","last_unstale":"2023-06-23T18:20:46.237610+0000","last_undegraded":"2023-06-23T18:20:46.237610+0000","last_fullsized":"2023-06-23T18:20:46.237610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:45:29.030061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439210+0000","last_change":"2023-06-23T18:20:44.465127+0000","last_active":"2023-06-23T18:20:45.439210+0000","last_peered":"2023-06-23T18:20:45.439210+0000","last_clean":"2023-06-23T18:20:45.439210+0000","last_became_active":"2023-06-23T18:20:44.464768+0000","last_became_peered":"2023-06-23T18:20:44.464768+0000","last_unstale":"2023-06-23T18:20:45.439210+0000","last_undegraded":"2023-06-23T18:20:45.439210+0000","last_fullsized":"2023-06-23T18:20:45.439210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:54:38.343097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497671+0000","last_change":"2023-06-23T18:20:44.046245+0000","last_active":"2023-06-23T18:20:47.497671+0000","last_peered":"2023-06-23T18:20:47.497671+0000","last_clean":"2023-06-23T18:20:47.497671+0000","last_became_active":"2023-06-23T18:20:44.045945+0000","last_became_peered":"2023-06-23T18:20:44.045945+0000","last_unstale":"2023-06-23T18:20:47.497671+0000","last_undegraded":"2023-06-23T18:20:47.497671+0000","last_fullsized":"2023-06-23T18:20:47.497671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:13:06.840364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439547+0000","last_change":"2023-06-23T18:20:44.453520+0000","last_active":"2023-06-23T18:20:45.439547+0000","last_peered":"2023-06-23T18:20:45.439547+0000","last_clean":"2023-06-23T18:20:45.439547+0000","last_became_active":"2023-06-23T18:20:44.453214+0000","last_became_peered":"2023-06-23T18:20:44.453214+0000","last_unstale":"2023-06-23T18:20:45.439547+0000","last_undegraded":"2023-06-23T18:20:45.439547+0000","last_fullsized":"2023-06-23T18:20:45.439547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:47.302353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.502980+0000","last_change":"2023-06-23T18:20:44.042526+0000","last_active":"2023-06-23T18:20:48.502980+0000","last_peered":"2023-06-23T18:20:48.502980+0000","last_clean":"2023-06-23T18:20:48.502980+0000","last_became_active":"2023-06-23T18:20:44.042249+0000","last_became_peered":"2023-06-23T18:20:44.042249+0000","last_unstale":"2023-06-23T18:20:48.502980+0000","last_undegraded":"2023-06-23T18:20:48.502980+0000","last_fullsized":"2023-06-23T18:20:48.502980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:31:39.378767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439873+0000","last_change":"2023-06-23T18:20:44.454645+0000","last_active":"2023-06-23T18:20:45.439873+0000","last_peered":"2023-06-23T18:20:45.439873+0000","last_clean":"2023-06-23T18:20:45.439873+0000","last_became_active":"2023-06-23T18:20:44.453309+0000","last_became_peered":"2023-06-23T18:20:44.453309+0000","last_unstale":"2023-06-23T18:20:45.439873+0000","last_undegraded":"2023-06-23T18:20:45.439873+0000","last_fullsized":"2023-06-23T18:20:45.439873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:30:55.635983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439463+0000","last_change":"2023-06-23T18:20:44.037681+0000","last_active":"2023-06-23T18:20:45.439463+0000","last_peered":"2023-06-23T18:20:45.439463+0000","last_clean":"2023-06-23T18:20:45.439463+0000","last_became_active":"2023-06-23T18:20:44.037323+0000","last_became_peered":"2023-06-23T18:20:44.037323+0000","last_unstale":"2023-06-23T18:20:45.439463+0000","last_undegraded":"2023-06-23T18:20:45.439463+0000","last_fullsized":"2023-06-23T18:20:45.439463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:55:26.308777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.038770+0000","last_change":"2023-06-23T18:20:44.456434+0000","last_active":"2023-06-23T18:20:47.038770+0000","last_peered":"2023-06-23T18:20:47.038770+0000","last_clean":"2023-06-23T18:20:47.038770+0000","last_became_active":"2023-06-23T18:20:44.456110+0000","last_became_peered":"2023-06-23T18:20:44.456110+0000","last_unstale":"2023-06-23T18:20:47.038770+0000","last_undegraded":"2023-06-23T18:20:47.038770+0000","last_fullsized":"2023-06-23T18:20:47.038770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:00:04.366221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406173+0000","last_change":"2023-06-23T18:20:44.040722+0000","last_active":"2023-06-23T18:20:46.406173+0000","last_peered":"2023-06-23T18:20:46.406173+0000","last_clean":"2023-06-23T18:20:46.406173+0000","last_became_active":"2023-06-23T18:20:44.040390+0000","last_became_peered":"2023-06-23T18:20:44.040390+0000","last_unstale":"2023-06-23T18:20:46.406173+0000","last_undegraded":"2023-06-23T18:20:46.406173+0000","last_fullsized":"2023-06-23T18:20:46.406173+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:56:25.584399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439334+0000","last_change":"2023-06-23T18:20:44.455559+0000","last_active":"2023-06-23T18:20:45.439334+0000","last_peered":"2023-06-23T18:20:45.439334+0000","last_clean":"2023-06-23T18:20:45.439334+0000","last_became_active":"2023-06-23T18:20:44.455256+0000","last_became_peered":"2023-06-23T18:20:44.455256+0000","last_unstale":"2023-06-23T18:20:45.439334+0000","last_undegraded":"2023-06-23T18:20:45.439334+0000","last_fullsized":"2023-06-23T18:20:45.439334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:40:30.169440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406131+0000","last_change":"2023-06-23T18:20:44.460403+0000","last_active":"2023-06-23T18:20:46.406131+0000","last_peered":"2023-06-23T18:20:46.406131+0000","last_clean":"2023-06-23T18:20:46.406131+0000","last_became_active":"2023-06-23T18:20:44.459928+0000","last_became_peered":"2023-06-23T18:20:44.459928+0000","last_unstale":"2023-06-23T18:20:46.406131+0000","last_undegraded":"2023-06-23T18:20:46.406131+0000","last_fullsized":"2023-06-23T18:20:46.406131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:58:06.385995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237624+0000","last_change":"2023-06-23T18:20:44.458050+0000","last_active":"2023-06-23T18:20:46.237624+0000","last_peered":"2023-06-23T18:20:46.237624+0000","last_clean":"2023-06-23T18:20:46.237624+0000","last_became_active":"2023-06-23T18:20:44.457733+0000","last_became_peered":"2023-06-23T18:20:44.457733+0000","last_unstale":"2023-06-23T18:20:46.237624+0000","last_undegraded":"2023-06-23T18:20:46.237624+0000","last_fullsized":"2023-06-23T18:20:46.237624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:23:24.448430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237525+0000","last_change":"2023-06-23T18:20:44.456233+0000","last_active":"2023-06-23T18:20:46.237525+0000","last_peered":"2023-06-23T18:20:46.237525+0000","last_clean":"2023-06-23T18:20:46.237525+0000","last_became_active":"2023-06-23T18:20:44.455677+0000","last_became_peered":"2023-06-23T18:20:44.455677+0000","last_unstale":"2023-06-23T18:20:46.237525+0000","last_undegraded":"2023-06-23T18:20:46.237525+0000","last_fullsized":"2023-06-23T18:20:46.237525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:54:14.083530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109745+0000","last_change":"2023-06-23T18:20:44.469668+0000","last_active":"2023-06-23T18:20:46.109745+0000","last_peered":"2023-06-23T18:20:46.109745+0000","last_clean":"2023-06-23T18:20:46.109745+0000","last_became_active":"2023-06-23T18:20:44.466563+0000","last_became_peered":"2023-06-23T18:20:44.466563+0000","last_unstale":"2023-06-23T18:20:46.109745+0000","last_undegraded":"2023-06-23T18:20:46.109745+0000","last_fullsized":"2023-06-23T18:20:46.109745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:24:09.129588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406322+0000","last_change":"2023-06-23T18:20:44.456021+0000","last_active":"2023-06-23T18:20:46.406322+0000","last_peered":"2023-06-23T18:20:46.406322+0000","last_clean":"2023-06-23T18:20:46.406322+0000","last_became_active":"2023-06-23T18:20:44.455705+0000","last_became_peered":"2023-06-23T18:20:44.455705+0000","last_unstale":"2023-06-23T18:20:46.406322+0000","last_undegraded":"2023-06-23T18:20:46.406322+0000","last_fullsized":"2023-06-23T18:20:46.406322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:16:56.043843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406226+0000","last_change":"2023-06-23T18:20:44.460359+0000","last_active":"2023-06-23T18:20:46.406226+0000","last_peered":"2023-06-23T18:20:46.406226+0000","last_clean":"2023-06-23T18:20:46.406226+0000","last_became_active":"2023-06-23T18:20:44.460034+0000","last_became_peered":"2023-06-23T18:20:44.460034+0000","last_unstale":"2023-06-23T18:20:46.406226+0000","last_undegraded":"2023-06-23T18:20:46.406226+0000","last_fullsized":"2023-06-23T18:20:46.406226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:00:06.777366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237875+0000","last_change":"2023-06-23T18:20:44.455743+0000","last_active":"2023-06-23T18:20:46.237875+0000","last_peered":"2023-06-23T18:20:46.237875+0000","last_clean":"2023-06-23T18:20:46.237875+0000","last_became_active":"2023-06-23T18:20:44.455266+0000","last_became_peered":"2023-06-23T18:20:44.455266+0000","last_unstale":"2023-06-23T18:20:46.237875+0000","last_undegraded":"2023-06-23T18:20:46.237875+0000","last_fullsized":"2023-06-23T18:20:46.237875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:33:52.673181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439651+0000","last_change":"2023-06-23T18:20:44.461434+0000","last_active":"2023-06-23T18:20:45.439651+0000","last_peered":"2023-06-23T18:20:45.439651+0000","last_clean":"2023-06-23T18:20:45.439651+0000","last_became_active":"2023-06-23T18:20:44.461137+0000","last_became_peered":"2023-06-23T18:20:44.461137+0000","last_unstale":"2023-06-23T18:20:45.439651+0000","last_undegraded":"2023-06-23T18:20:45.439651+0000","last_fullsized":"2023-06-23T18:20:45.439651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:14:08.293656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439086+0000","last_change":"2023-06-23T18:20:44.465192+0000","last_active":"2023-06-23T18:20:45.439086+0000","last_peered":"2023-06-23T18:20:45.439086+0000","last_clean":"2023-06-23T18:20:45.439086+0000","last_became_active":"2023-06-23T18:20:44.464905+0000","last_became_peered":"2023-06-23T18:20:44.464905+0000","last_unstale":"2023-06-23T18:20:45.439086+0000","last_undegraded":"2023-06-23T18:20:45.439086+0000","last_fullsized":"2023-06-23T18:20:45.439086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:01:32.627765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439130+0000","last_change":"2023-06-23T18:20:44.458725+0000","last_active":"2023-06-23T18:20:45.439130+0000","last_peered":"2023-06-23T18:20:45.439130+0000","last_clean":"2023-06-23T18:20:45.439130+0000","last_became_active":"2023-06-23T18:20:44.458414+0000","last_became_peered":"2023-06-23T18:20:44.458414+0000","last_unstale":"2023-06-23T18:20:45.439130+0000","last_undegraded":"2023-06-23T18:20:45.439130+0000","last_fullsized":"2023-06-23T18:20:45.439130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:36:29.121354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405885+0000","last_change":"2023-06-23T18:20:44.465665+0000","last_active":"2023-06-23T18:20:46.405885+0000","last_peered":"2023-06-23T18:20:46.405885+0000","last_clean":"2023-06-23T18:20:46.405885+0000","last_became_active":"2023-06-23T18:20:44.465337+0000","last_became_peered":"2023-06-23T18:20:44.465337+0000","last_unstale":"2023-06-23T18:20:46.405885+0000","last_undegraded":"2023-06-23T18:20:46.405885+0000","last_fullsized":"2023-06-23T18:20:46.405885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:58:56.987077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406238+0000","last_change":"2023-06-23T18:20:44.455107+0000","last_active":"2023-06-23T18:20:46.406238+0000","last_peered":"2023-06-23T18:20:46.406238+0000","last_clean":"2023-06-23T18:20:46.406238+0000","last_became_active":"2023-06-23T18:20:44.454798+0000","last_became_peered":"2023-06-23T18:20:44.454798+0000","last_unstale":"2023-06-23T18:20:46.406238+0000","last_undegraded":"2023-06-23T18:20:46.406238+0000","last_fullsized":"2023-06-23T18:20:46.406238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:10:04.410538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406114+0000","last_change":"2023-06-23T18:20:44.461044+0000","last_active":"2023-06-23T18:20:46.406114+0000","last_peered":"2023-06-23T18:20:46.406114+0000","last_clean":"2023-06-23T18:20:46.406114+0000","last_became_active":"2023-06-23T18:20:44.460737+0000","last_became_peered":"2023-06-23T18:20:44.460737+0000","last_unstale":"2023-06-23T18:20:46.406114+0000","last_undegraded":"2023-06-23T18:20:46.406114+0000","last_fullsized":"2023-06-23T18:20:46.406114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:13:16.912989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281641,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61402,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":257,"num_read_kb":195,"num_write":154,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":184206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481812,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":420371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.7590000000000003,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.40899999999999997}]},{"osd":1,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.47999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.64400000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6580000000000004,"5min":5.6580000000000004,"15min":5.6580000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.627}]},{"osd":7,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.754}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.72699999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.74099999999999999}]}]},{"osd":10,"up_from":70,"seq":300647710745,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":1028096,"data_stored":1069698,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.373}]},{"osd":2,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.48799999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.69999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.75900000000000001}]},{"osd":9,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.9430000000000001,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.84099999999999997}]},{"osd":11,"last update":"Fri Jun 23 18:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.82499999999999996}]}]},{"osd":9,"up_from":65,"seq":279172874268,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":420607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.311},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44600000000000001,"15min":0.45200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.28199999999999997}]},{"osd":1,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.67100000000000004,"15min":0.69499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.72999999999999998,"15min":0.753},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.378}]},{"osd":4,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.75,"15min":0.77500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83899999999999997,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.437}]},{"osd":5,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.70599999999999996,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.034,"5min":1.04,"15min":1.04},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0669999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":1.397,"15min":1.397},"last":0.45200000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.81599999999999995,"15min":0.83399999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0960000000000001,"5min":1.661,"15min":1.661},"last":0.52600000000000002}]},{"osd":8,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0349999999999999,"5min":1.079,"15min":1.079},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.93000000000000005,"15min":0.97299999999999998},"min":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.91900000000000004,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.622}]},{"osd":10,"last update":"Fri Jun 23 18:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.5}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":420196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.27600000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.621},{"interface":"front","average":{"1min":0.443,"5min":0.45700000000000002,"15min":0.45900000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.64400000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56000000000000005,"15min":0.56499999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88600000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.78200000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.872,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88600000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90200000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.82999999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.75}]},{"osd":10,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332133,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":1012366,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.33300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64100000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.498},{"interface":"front","average":{"1min":0.503,"5min":0.435,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.749},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.80300000000000005}]},{"osd":3,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.84199999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.52400000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.54000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.75900000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.53600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.56799999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.627}]},{"osd":8,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.97799999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72399999999999998,"15min":0.71599999999999997},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81899999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.93999999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jun 23 18:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.90200000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":6,"up_from":46,"seq":197568495657,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":420002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.58899999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.28699999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.30099999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.89400000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70799999999999996,"5min":1.105,"15min":1.105},"last":0.503}]},{"osd":3,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65400000000000003,"5min":1.141,"15min":1.141},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72199999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.872,"15min":0.872},"last":0.442},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77200000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68300000000000005,"15min":0.69699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63700000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53600000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.50800000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.54100000000000004}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97199999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69199999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96799999999999997,"5min":1.008,"15min":1.008},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71699999999999997,"15min":0.73599999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.40600000000000003}]},{"osd":10,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.52800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":477287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.376,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82599999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.41999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86499999999999999,"5min":1.218,"15min":1.218},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.53900000000000003},"min":{"1min":0.52600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.071,"5min":1.25,"15min":1.25},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56399999999999995,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0960000000000001,"5min":1.343,"15min":1.343},"last":0.745}]},{"osd":3,"last update":"Fri Jun 23 18:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.42799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.93200000000000005,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.439}]},{"osd":4,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45400000000000001,"15min":0.42799999999999999},"min":{"1min":0.38600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44600000000000001,"15min":0.41799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.63100000000000001,"15min":0.58399999999999996},"min":{"1min":0.54800000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.032,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.623,"15min":0.59699999999999998},"min":{"1min":0.54300000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.052,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84599999999999997,"5min":1.169,"15min":1.169},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.60099999999999998},"min":{"1min":0.53200000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.093,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.47799999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39400000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96099999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56200000000000006,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0629999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.94299999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.66600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.216,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77200000000000002,"15min":0.76600000000000001},"min":{"1min":0.61699999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.149,"5min":1.319,"15min":1.319},"last":0.73199999999999998}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.876,"15min":0.88900000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1180000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96399999999999997,"5min":1.288,"15min":1.288},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":421267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.40899999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.5,"15min":0.47499999999999998},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.55900000000000005,"15min":0.51900000000000002},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Jun 23 18:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.624,"5min":0.628,"15min":0.628},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.64000000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56999999999999995,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51300000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79000000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77000000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82499999999999996,"5min":1.073,"15min":1.073},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.622}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.876,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.624}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.76400000000000001,"15min":0.79000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0449999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.72099999999999997}]},{"osd":10,"last update":"Fri Jun 23 18:20:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.83899999999999997}]},{"osd":11,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.745},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.76200000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":478589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.34899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.503,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68999999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.59399999999999997}]},{"osd":1,"last update":"Fri Jun 23 18:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79000000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.84699999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Jun 23 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.627},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.47099999999999997},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Jun 23 18:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.88700000000000001,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.623,"15min":0.623},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.81899999999999995}]},{"osd":7,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59799999999999998}]},{"osd":8,"last update":"Fri Jun 23 18:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.45400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:20:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.871},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.92600000000000005}]},{"osd":10,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.88400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":420319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.502,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":12.246,"5min":12.246,"15min":12.246},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.32700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58099999999999996,"15min":0.54200000000000004},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":8.3569999999999993,"5min":8.3569999999999993,"15min":8.3569999999999993},"last":0.57099999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.57899999999999996}]},{"osd":4,"last update":"Fri Jun 23 18:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.45300000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51300000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:20:25 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.498,"15min":0.48899999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41399999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.69799999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.504}]},{"osd":10,"last update":"Fri Jun 23 18:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.71099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":420775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.51700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.57799999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.38300000000000001},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57799999999999996,"15min":0.54300000000000004},"min":{"1min":0.503,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78000000000000003,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.88},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.74299999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":1.129,"15min":1.129},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.379,"5min":0.378,"15min":0.378},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76000000000000001}]},{"osd":8,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94199999999999995,"5min":1.202,"15min":1.202},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.747,"15min":0.75600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91400000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.85899999999999999}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":0.79400000000000004}]},{"osd":10,"last update":"Fri Jun 23 18:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.91400000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":1011708,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47699999999999998,"15min":0.46000000000000002},"min":{"1min":0.39200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70599999999999996,"5min":0.875,"15min":0.875},"last":0.37},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65400000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56100000000000005}]},{"osd":2,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41099999999999998},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.439}]},{"osd":3,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.67800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72999999999999998,"5min":0.751,"15min":0.751},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80700000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.59399999999999997,"15min":0.58199999999999996},"min":{"1min":0.45800000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.64300000000000002,"15min":0.63300000000000001},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":0.89800000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70499999999999996}]},{"osd":8,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.998,"15min":0.998},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.57599999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:20:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.64300000000000002}]},{"osd":10,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:21:30.850 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph pg dump --format=json 2023-06-23T18:21:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:30 smithi002 ceph-mon[95947]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:21:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:30 smithi002 ceph-mon[95947]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:30 smithi082 ceph-mon[97889]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:21:31.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:30 smithi082 ceph-mon[97889]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:30 smithi017 ceph-mon[97472]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:21:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:30 smithi017 ceph-mon[97472]: pgmap v279: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:31.418 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:32.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.716+0000 7f8a89823700 1 -- 172.21.15.2:0/1926297534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840ff470 msgr2=0x7f8a8410b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:32.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.716+0000 7f8a89823700 1 --2- 172.21.15.2:0/1926297534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840ff470 0x7f8a8410b900 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00b0a0 tx=0x7f8a7c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 -- 172.21.15.2:0/1926297534 shutdown_connections 2023-06-23T18:21:32.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 --2- 172.21.15.2:0/1926297534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840ff470 0x7f8a8410b900 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 --2- 172.21.15.2:0/1926297534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a84104ac0 0x7f8a84104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 --2- 172.21.15.2:0/1926297534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a840feac0 0x7f8a840fef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 -- 172.21.15.2:0/1926297534 >> 172.21.15.2:0/1926297534 conn(0x7f8a84075970 msgr2=0x7f8a84075d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:32.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 -- 172.21.15.2:0/1926297534 shutdown_connections 2023-06-23T18:21:32.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 -- 172.21.15.2:0/1926297534 wait complete. 2023-06-23T18:21:32.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 Processor -- start 2023-06-23T18:21:32.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.717+0000 7f8a89823700 1 -- start start 2023-06-23T18:21:32.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a89823700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840feac0 0x7f8a8419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:32.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a89823700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a840ff470 0x7f8a8419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:32.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a89823700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a84104ac0 0x7f8a84195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:32.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a89823700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a841960e0 con 0x7f8a840feac0 2023-06-23T18:21:32.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a89823700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84196250 con 0x7f8a840ff470 2023-06-23T18:21:32.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a89823700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a84196410 con 0x7f8a84104ac0 2023-06-23T18:21:32.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840feac0 0x7f8a8419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:32.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840feac0 0x7f8a8419b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53030/0 (socket says 172.21.15.2:53030) 2023-06-23T18:21:32.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 -- 172.21.15.2:0/2813970945 learned_addr learned my addr 172.21.15.2:0/2813970945 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:32.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a827fc700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a840ff470 0x7f8a8419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:32.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 -- 172.21.15.2:0/2813970945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a84104ac0 msgr2=0x7f8a84195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:32.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a84104ac0 0x7f8a84195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 -- 172.21.15.2:0/2813970945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a840ff470 msgr2=0x7f8a8419bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:32.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a840ff470 0x7f8a8419bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 -- 172.21.15.2:0/2813970945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c009d00 con 0x7f8a840feac0 2023-06-23T18:21:32.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a827fc700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a840ff470 0x7f8a8419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:32.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a82ffd700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840feac0 0x7f8a8419b9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400c990 tx=0x7f8a7400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:32.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a88821700 1 -- 172.21.15.2:0/2813970945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74007b00 con 0x7f8a840feac0 2023-06-23T18:21:32.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a88821700 1 -- 172.21.15.2:0/2813970945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a74007c60 con 0x7f8a840feac0 2023-06-23T18:21:32.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a88821700 1 -- 172.21.15.2:0/2813970945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74018660 con 0x7f8a840feac0 2023-06-23T18:21:32.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.718+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a841966f0 con 0x7f8a840feac0 2023-06-23T18:21:32.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.719+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a841a29e0 con 0x7f8a840feac0 2023-06-23T18:21:32.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.719+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8404e1d0 con 0x7f8a840feac0 2023-06-23T18:21:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.721+0000 7f8a88821700 1 -- 172.21.15.2:0/2813970945 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8a7400f450 con 0x7f8a840feac0 2023-06-23T18:21:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.721+0000 7f8a88821700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a6c0a1260 0x7f8a6c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:32.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.721+0000 7f8a827fc700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a6c0a1260 0x7f8a6c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:32.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.721+0000 7f8a88821700 1 -- 172.21.15.2:0/2813970945 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8a740cc8d0 con 0x7f8a840feac0 2023-06-23T18:21:32.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.722+0000 7f8a827fc700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a6c0a1260 0x7f8a6c0a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a78005950 tx=0x7f8a780058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:32.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.724+0000 7f8a88821700 1 -- 172.21.15.2:0/2813970945 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8a74096180 con 0x7f8a840feac0 2023-06-23T18:21:32.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.921+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a840690e0 con 0x7f8a6c0a1260 2023-06-23T18:21:32.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.933+0000 7f8a88821700 1 -- 172.21.15.2:0/2813970945 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+340568 (secure 0 0 0) 0x7f8a840690e0 con 0x7f8a6c0a1260 2023-06-23T18:21:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:32.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a6c0a1260 msgr2=0x7f8a6c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:32.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a6c0a1260 0x7f8a6c0a3710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a78005950 tx=0x7f8a780058e0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840feac0 msgr2=0x7f8a8419b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:32.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840feac0 0x7f8a8419b9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8a7400c990 tx=0x7f8a7400cd50 comp rx=0 tx=0).stop 2023-06-23T18:21:32.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 shutdown_connections 2023-06-23T18:21:32.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a84104ac0 0x7f8a84195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a840feac0 0x7f8a8419b9f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a6c0a1260 0x7f8a6c0a3710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 --2- 172.21.15.2:0/2813970945 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a840ff470 0x7f8a8419bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:32.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.936+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 >> 172.21.15.2:0/2813970945 conn(0x7f8a84075970 msgr2=0x7f8a840689d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:32.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.937+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 shutdown_connections 2023-06-23T18:21:32.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:32.937+0000 7f8a89823700 1 -- 172.21.15.2:0/2813970945 wait complete. 2023-06-23T18:21:32.946 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-06-23T18:21:33.086 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:32 smithi002 ceph-mon[95947]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-06-23T18:21:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:32 smithi082 ceph-mon[97889]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-06-23T18:21:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:32 smithi017 ceph-mon[97472]: pgmap v280: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-06-23T18:21:33.378 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2023-06-23T18:21:32.743845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652016,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":413,"num_read_kb":319,"num_write":337,"num_write_kb":3045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334256,"kb_used_data":10832,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607568,"statfs":{"total":1151940427776,"available":1151598149632,"internally_reserved":0,"allocated":11091968,"data_stored":6993185,"data_compressed":107457,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003529"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406331+0000","last_change":"2023-06-23T18:20:44.466650+0000","last_active":"2023-06-23T18:20:46.406331+0000","last_peered":"2023-06-23T18:20:46.406331+0000","last_clean":"2023-06-23T18:20:46.406331+0000","last_became_active":"2023-06-23T18:20:44.466425+0000","last_became_peered":"2023-06-23T18:20:44.466425+0000","last_unstale":"2023-06-23T18:20:46.406331+0000","last_undegraded":"2023-06-23T18:20:46.406331+0000","last_fullsized":"2023-06-23T18:20:46.406331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:46:45.756486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109943+0000","last_change":"2023-06-23T18:20:44.465762+0000","last_active":"2023-06-23T18:20:46.109943+0000","last_peered":"2023-06-23T18:20:46.109943+0000","last_clean":"2023-06-23T18:20:46.109943+0000","last_became_active":"2023-06-23T18:20:44.463592+0000","last_became_peered":"2023-06-23T18:20:44.463592+0000","last_unstale":"2023-06-23T18:20:46.109943+0000","last_undegraded":"2023-06-23T18:20:46.109943+0000","last_fullsized":"2023-06-23T18:20:46.109943+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:47:42.304223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109873+0000","last_change":"2023-06-23T18:20:44.464162+0000","last_active":"2023-06-23T18:20:46.109873+0000","last_peered":"2023-06-23T18:20:46.109873+0000","last_clean":"2023-06-23T18:20:46.109873+0000","last_became_active":"2023-06-23T18:20:44.463854+0000","last_became_peered":"2023-06-23T18:20:44.463854+0000","last_unstale":"2023-06-23T18:20:46.109873+0000","last_undegraded":"2023-06-23T18:20:46.109873+0000","last_fullsized":"2023-06-23T18:20:46.109873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:07:55.908839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.366029+0000","last_change":"2023-06-23T18:20:44.462923+0000","last_active":"2023-06-23T18:20:46.366029+0000","last_peered":"2023-06-23T18:20:46.366029+0000","last_clean":"2023-06-23T18:20:46.366029+0000","last_became_active":"2023-06-23T18:20:44.462511+0000","last_became_peered":"2023-06-23T18:20:44.462511+0000","last_unstale":"2023-06-23T18:20:46.366029+0000","last_undegraded":"2023-06-23T18:20:46.366029+0000","last_fullsized":"2023-06-23T18:20:46.366029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:32:40.690012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530812+0000","last_change":"2023-06-23T18:20:44.454727+0000","last_active":"2023-06-23T18:20:45.530812+0000","last_peered":"2023-06-23T18:20:45.530812+0000","last_clean":"2023-06-23T18:20:45.530812+0000","last_became_active":"2023-06-23T18:20:44.454418+0000","last_became_peered":"2023-06-23T18:20:44.454418+0000","last_unstale":"2023-06-23T18:20:45.530812+0000","last_undegraded":"2023-06-23T18:20:45.530812+0000","last_fullsized":"2023-06-23T18:20:45.530812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:34:09.454407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905435+0000","last_change":"2023-06-23T18:20:44.460636+0000","last_active":"2023-06-23T18:20:45.905435+0000","last_peered":"2023-06-23T18:20:45.905435+0000","last_clean":"2023-06-23T18:20:45.905435+0000","last_became_active":"2023-06-23T18:20:44.460305+0000","last_became_peered":"2023-06-23T18:20:44.460305+0000","last_unstale":"2023-06-23T18:20:45.905435+0000","last_undegraded":"2023-06-23T18:20:45.905435+0000","last_fullsized":"2023-06-23T18:20:45.905435+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:20:11.034702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439073+0000","last_change":"2023-06-23T18:20:44.463552+0000","last_active":"2023-06-23T18:20:45.439073+0000","last_peered":"2023-06-23T18:20:45.439073+0000","last_clean":"2023-06-23T18:20:45.439073+0000","last_became_active":"2023-06-23T18:20:44.463187+0000","last_became_peered":"2023-06-23T18:20:44.463187+0000","last_unstale":"2023-06-23T18:20:45.439073+0000","last_undegraded":"2023-06-23T18:20:45.439073+0000","last_fullsized":"2023-06-23T18:20:45.439073+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:16:33.692125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439061+0000","last_change":"2023-06-23T18:20:44.457957+0000","last_active":"2023-06-23T18:20:45.439061+0000","last_peered":"2023-06-23T18:20:45.439061+0000","last_clean":"2023-06-23T18:20:45.439061+0000","last_became_active":"2023-06-23T18:20:44.457511+0000","last_became_peered":"2023-06-23T18:20:44.457511+0000","last_unstale":"2023-06-23T18:20:45.439061+0000","last_undegraded":"2023-06-23T18:20:45.439061+0000","last_fullsized":"2023-06-23T18:20:45.439061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:19:36.056785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406274+0000","last_change":"2023-06-23T18:20:44.461199+0000","last_active":"2023-06-23T18:20:46.406274+0000","last_peered":"2023-06-23T18:20:46.406274+0000","last_clean":"2023-06-23T18:20:46.406274+0000","last_became_active":"2023-06-23T18:20:44.460891+0000","last_became_peered":"2023-06-23T18:20:44.460891+0000","last_unstale":"2023-06-23T18:20:46.406274+0000","last_undegraded":"2023-06-23T18:20:46.406274+0000","last_fullsized":"2023-06-23T18:20:46.406274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:51:00.977889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905565+0000","last_change":"2023-06-23T18:20:44.460741+0000","last_active":"2023-06-23T18:20:45.905565+0000","last_peered":"2023-06-23T18:20:45.905565+0000","last_clean":"2023-06-23T18:20:45.905565+0000","last_became_active":"2023-06-23T18:20:44.460109+0000","last_became_peered":"2023-06-23T18:20:44.460109+0000","last_unstale":"2023-06-23T18:20:45.905565+0000","last_undegraded":"2023-06-23T18:20:45.905565+0000","last_fullsized":"2023-06-23T18:20:45.905565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:37:37.614265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439769+0000","last_change":"2023-06-23T18:20:44.453337+0000","last_active":"2023-06-23T18:20:45.439769+0000","last_peered":"2023-06-23T18:20:45.439769+0000","last_clean":"2023-06-23T18:20:45.439769+0000","last_became_active":"2023-06-23T18:20:44.452925+0000","last_became_peered":"2023-06-23T18:20:44.452925+0000","last_unstale":"2023-06-23T18:20:45.439769+0000","last_undegraded":"2023-06-23T18:20:45.439769+0000","last_fullsized":"2023-06-23T18:20:45.439769+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:18:10.381911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.365990+0000","last_change":"2023-06-23T18:20:44.456710+0000","last_active":"2023-06-23T18:20:46.365990+0000","last_peered":"2023-06-23T18:20:46.365990+0000","last_clean":"2023-06-23T18:20:46.365990+0000","last_became_active":"2023-06-23T18:20:44.456411+0000","last_became_peered":"2023-06-23T18:20:44.456411+0000","last_unstale":"2023-06-23T18:20:46.365990+0000","last_undegraded":"2023-06-23T18:20:46.365990+0000","last_fullsized":"2023-06-23T18:20:46.365990+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:01:45.669051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406319+0000","last_change":"2023-06-23T18:20:44.453586+0000","last_active":"2023-06-23T18:20:46.406319+0000","last_peered":"2023-06-23T18:20:46.406319+0000","last_clean":"2023-06-23T18:20:46.406319+0000","last_became_active":"2023-06-23T18:20:44.453239+0000","last_became_peered":"2023-06-23T18:20:44.453239+0000","last_unstale":"2023-06-23T18:20:46.406319+0000","last_undegraded":"2023-06-23T18:20:46.406319+0000","last_fullsized":"2023-06-23T18:20:46.406319+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:54:58.454675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109577+0000","last_change":"2023-06-23T18:20:44.465627+0000","last_active":"2023-06-23T18:20:46.109577+0000","last_peered":"2023-06-23T18:20:46.109577+0000","last_clean":"2023-06-23T18:20:46.109577+0000","last_became_active":"2023-06-23T18:20:44.464568+0000","last_became_peered":"2023-06-23T18:20:44.464568+0000","last_unstale":"2023-06-23T18:20:46.109577+0000","last_undegraded":"2023-06-23T18:20:46.109577+0000","last_fullsized":"2023-06-23T18:20:46.109577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:29:20.689694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109533+0000","last_change":"2023-06-23T18:20:44.464169+0000","last_active":"2023-06-23T18:20:46.109533+0000","last_peered":"2023-06-23T18:20:46.109533+0000","last_clean":"2023-06-23T18:20:46.109533+0000","last_became_active":"2023-06-23T18:20:44.463867+0000","last_became_peered":"2023-06-23T18:20:44.463867+0000","last_unstale":"2023-06-23T18:20:46.109533+0000","last_undegraded":"2023-06-23T18:20:46.109533+0000","last_fullsized":"2023-06-23T18:20:46.109533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:28:14.663838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405761+0000","last_change":"2023-06-23T18:20:44.467101+0000","last_active":"2023-06-23T18:20:46.405761+0000","last_peered":"2023-06-23T18:20:46.405761+0000","last_clean":"2023-06-23T18:20:46.405761+0000","last_became_active":"2023-06-23T18:20:44.466788+0000","last_became_peered":"2023-06-23T18:20:44.466788+0000","last_unstale":"2023-06-23T18:20:46.405761+0000","last_undegraded":"2023-06-23T18:20:46.405761+0000","last_fullsized":"2023-06-23T18:20:46.405761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:14:55.735170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406210+0000","last_change":"2023-06-23T18:20:44.466055+0000","last_active":"2023-06-23T18:20:46.406210+0000","last_peered":"2023-06-23T18:20:46.406210+0000","last_clean":"2023-06-23T18:20:46.406210+0000","last_became_active":"2023-06-23T18:20:44.465608+0000","last_became_peered":"2023-06-23T18:20:44.465608+0000","last_unstale":"2023-06-23T18:20:46.406210+0000","last_undegraded":"2023-06-23T18:20:46.406210+0000","last_fullsized":"2023-06-23T18:20:46.406210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:06:22.567595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.529187+0000","last_change":"2023-06-23T18:20:44.459414+0000","last_active":"2023-06-23T18:20:45.529187+0000","last_peered":"2023-06-23T18:20:45.529187+0000","last_clean":"2023-06-23T18:20:45.529187+0000","last_became_active":"2023-06-23T18:20:44.459014+0000","last_became_peered":"2023-06-23T18:20:44.459014+0000","last_unstale":"2023-06-23T18:20:45.529187+0000","last_undegraded":"2023-06-23T18:20:45.529187+0000","last_fullsized":"2023-06-23T18:20:45.529187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:00:16.944957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237780+0000","last_change":"2023-06-23T18:20:44.446784+0000","last_active":"2023-06-23T18:20:46.237780+0000","last_peered":"2023-06-23T18:20:46.237780+0000","last_clean":"2023-06-23T18:20:46.237780+0000","last_became_active":"2023-06-23T18:20:44.446464+0000","last_became_peered":"2023-06-23T18:20:44.446464+0000","last_unstale":"2023-06-23T18:20:46.237780+0000","last_undegraded":"2023-06-23T18:20:46.237780+0000","last_fullsized":"2023-06-23T18:20:46.237780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:03:13.595346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.501338+0000","last_change":"2023-06-23T18:20:44.039326+0000","last_active":"2023-06-23T18:20:48.501338+0000","last_peered":"2023-06-23T18:20:48.501338+0000","last_clean":"2023-06-23T18:20:48.501338+0000","last_became_active":"2023-06-23T18:20:44.038787+0000","last_became_peered":"2023-06-23T18:20:44.038787+0000","last_unstale":"2023-06-23T18:20:48.501338+0000","last_undegraded":"2023-06-23T18:20:48.501338+0000","last_fullsized":"2023-06-23T18:20:48.501338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:16:15.016712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439642+0000","last_change":"2023-06-23T18:20:44.465269+0000","last_active":"2023-06-23T18:20:45.439642+0000","last_peered":"2023-06-23T18:20:45.439642+0000","last_clean":"2023-06-23T18:20:45.439642+0000","last_became_active":"2023-06-23T18:20:44.464973+0000","last_became_peered":"2023-06-23T18:20:44.464973+0000","last_unstale":"2023-06-23T18:20:45.439642+0000","last_undegraded":"2023-06-23T18:20:45.439642+0000","last_fullsized":"2023-06-23T18:20:45.439642+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:24:55.096958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.847597+0000","last_change":"2023-06-23T18:20:44.037175+0000","last_active":"2023-06-23T18:20:45.847597+0000","last_peered":"2023-06-23T18:20:45.847597+0000","last_clean":"2023-06-23T18:20:45.847597+0000","last_became_active":"2023-06-23T18:20:44.036788+0000","last_became_peered":"2023-06-23T18:20:44.036788+0000","last_unstale":"2023-06-23T18:20:45.847597+0000","last_undegraded":"2023-06-23T18:20:45.847597+0000","last_fullsized":"2023-06-23T18:20:45.847597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:15:06.903564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.110040+0000","last_change":"2023-06-23T18:20:44.463344+0000","last_active":"2023-06-23T18:20:46.110040+0000","last_peered":"2023-06-23T18:20:46.110040+0000","last_clean":"2023-06-23T18:20:46.110040+0000","last_became_active":"2023-06-23T18:20:44.462649+0000","last_became_peered":"2023-06-23T18:20:44.462649+0000","last_unstale":"2023-06-23T18:20:46.110040+0000","last_undegraded":"2023-06-23T18:20:46.110040+0000","last_fullsized":"2023-06-23T18:20:46.110040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:39:49.841778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499503+0000","last_change":"2023-06-23T18:20:44.045525+0000","last_active":"2023-06-23T18:20:48.499503+0000","last_peered":"2023-06-23T18:20:48.499503+0000","last_clean":"2023-06-23T18:20:48.499503+0000","last_became_active":"2023-06-23T18:20:44.045212+0000","last_became_peered":"2023-06-23T18:20:44.045212+0000","last_unstale":"2023-06-23T18:20:48.499503+0000","last_undegraded":"2023-06-23T18:20:48.499503+0000","last_fullsized":"2023-06-23T18:20:48.499503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:44:56.572688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405870+0000","last_change":"2023-06-23T18:20:44.461071+0000","last_active":"2023-06-23T18:20:46.405870+0000","last_peered":"2023-06-23T18:20:46.405870+0000","last_clean":"2023-06-23T18:20:46.405870+0000","last_became_active":"2023-06-23T18:20:44.460726+0000","last_became_peered":"2023-06-23T18:20:44.460726+0000","last_unstale":"2023-06-23T18:20:46.405870+0000","last_undegraded":"2023-06-23T18:20:46.405870+0000","last_fullsized":"2023-06-23T18:20:46.405870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:08:47.572741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530990+0000","last_change":"2023-06-23T18:20:44.037162+0000","last_active":"2023-06-23T18:20:45.530990+0000","last_peered":"2023-06-23T18:20:45.530990+0000","last_clean":"2023-06-23T18:20:45.530990+0000","last_became_active":"2023-06-23T18:20:44.036809+0000","last_became_peered":"2023-06-23T18:20:44.036809+0000","last_unstale":"2023-06-23T18:20:45.530990+0000","last_undegraded":"2023-06-23T18:20:45.530990+0000","last_fullsized":"2023-06-23T18:20:45.530990+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:42:11.269748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439457+0000","last_change":"2023-06-23T18:20:44.463525+0000","last_active":"2023-06-23T18:20:45.439457+0000","last_peered":"2023-06-23T18:20:45.439457+0000","last_clean":"2023-06-23T18:20:45.439457+0000","last_became_active":"2023-06-23T18:20:44.463245+0000","last_became_peered":"2023-06-23T18:20:44.463245+0000","last_unstale":"2023-06-23T18:20:45.439457+0000","last_undegraded":"2023-06-23T18:20:45.439457+0000","last_fullsized":"2023-06-23T18:20:45.439457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:18:47.776141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"82'5","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:28.110146+0000","last_change":"2023-06-23T18:20:44.044415+0000","last_active":"2023-06-23T18:21:28.110146+0000","last_peered":"2023-06-23T18:21:28.110146+0000","last_clean":"2023-06-23T18:21:28.110146+0000","last_became_active":"2023-06-23T18:20:44.044085+0000","last_became_peered":"2023-06-23T18:20:44.044085+0000","last_unstale":"2023-06-23T18:21:28.110146+0000","last_undegraded":"2023-06-23T18:21:28.110146+0000","last_fullsized":"2023-06-23T18:21:28.110146+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:07:29.948276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530911+0000","last_change":"2023-06-23T18:20:44.463422+0000","last_active":"2023-06-23T18:20:45.530911+0000","last_peered":"2023-06-23T18:20:45.530911+0000","last_clean":"2023-06-23T18:20:45.530911+0000","last_became_active":"2023-06-23T18:20:44.463121+0000","last_became_peered":"2023-06-23T18:20:44.463121+0000","last_unstale":"2023-06-23T18:20:45.530911+0000","last_undegraded":"2023-06-23T18:20:45.530911+0000","last_fullsized":"2023-06-23T18:20:45.530911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:54:44.620991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.500329+0000","last_change":"2023-06-23T18:20:44.039604+0000","last_active":"2023-06-23T18:20:48.500329+0000","last_peered":"2023-06-23T18:20:48.500329+0000","last_clean":"2023-06-23T18:20:48.500329+0000","last_became_active":"2023-06-23T18:20:44.039317+0000","last_became_peered":"2023-06-23T18:20:44.039317+0000","last_unstale":"2023-06-23T18:20:48.500329+0000","last_undegraded":"2023-06-23T18:20:48.500329+0000","last_fullsized":"2023-06-23T18:20:48.500329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:39:40.353762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439176+0000","last_change":"2023-06-23T18:20:44.465748+0000","last_active":"2023-06-23T18:20:45.439176+0000","last_peered":"2023-06-23T18:20:45.439176+0000","last_clean":"2023-06-23T18:20:45.439176+0000","last_became_active":"2023-06-23T18:20:44.465430+0000","last_became_peered":"2023-06-23T18:20:44.465430+0000","last_unstale":"2023-06-23T18:20:45.439176+0000","last_undegraded":"2023-06-23T18:20:45.439176+0000","last_fullsized":"2023-06-23T18:20:45.439176+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:31:31.824195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499906+0000","last_change":"2023-06-23T18:20:44.034388+0000","last_active":"2023-06-23T18:20:48.499906+0000","last_peered":"2023-06-23T18:20:48.499906+0000","last_clean":"2023-06-23T18:20:48.499906+0000","last_became_active":"2023-06-23T18:20:44.034067+0000","last_became_peered":"2023-06-23T18:20:44.034067+0000","last_unstale":"2023-06-23T18:20:48.499906+0000","last_undegraded":"2023-06-23T18:20:48.499906+0000","last_fullsized":"2023-06-23T18:20:48.499906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:39:02.092084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.438998+0000","last_change":"2023-06-23T18:20:44.453280+0000","last_active":"2023-06-23T18:20:45.438998+0000","last_peered":"2023-06-23T18:20:45.438998+0000","last_clean":"2023-06-23T18:20:45.438998+0000","last_became_active":"2023-06-23T18:20:44.452980+0000","last_became_peered":"2023-06-23T18:20:44.452980+0000","last_unstale":"2023-06-23T18:20:45.438998+0000","last_undegraded":"2023-06-23T18:20:45.438998+0000","last_fullsized":"2023-06-23T18:20:45.438998+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:24:40.346057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439241+0000","last_change":"2023-06-23T18:20:44.036165+0000","last_active":"2023-06-23T18:20:45.439241+0000","last_peered":"2023-06-23T18:20:45.439241+0000","last_clean":"2023-06-23T18:20:45.439241+0000","last_became_active":"2023-06-23T18:20:44.035313+0000","last_became_peered":"2023-06-23T18:20:44.035313+0000","last_unstale":"2023-06-23T18:20:45.439241+0000","last_undegraded":"2023-06-23T18:20:45.439241+0000","last_fullsized":"2023-06-23T18:20:45.439241+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:26:02.622257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.528993+0000","last_change":"2023-06-23T18:20:44.468358+0000","last_active":"2023-06-23T18:20:45.528993+0000","last_peered":"2023-06-23T18:20:45.528993+0000","last_clean":"2023-06-23T18:20:45.528993+0000","last_became_active":"2023-06-23T18:20:44.468032+0000","last_became_peered":"2023-06-23T18:20:44.468032+0000","last_unstale":"2023-06-23T18:20:45.528993+0000","last_undegraded":"2023-06-23T18:20:45.528993+0000","last_fullsized":"2023-06-23T18:20:45.528993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:31.361443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'3","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:31.113861+0000","last_change":"2023-06-23T18:20:44.037921+0000","last_active":"2023-06-23T18:21:31.113861+0000","last_peered":"2023-06-23T18:21:31.113861+0000","last_clean":"2023-06-23T18:21:31.113861+0000","last_became_active":"2023-06-23T18:20:44.037571+0000","last_became_peered":"2023-06-23T18:20:44.037571+0000","last_unstale":"2023-06-23T18:21:31.113861+0000","last_undegraded":"2023-06-23T18:21:31.113861+0000","last_fullsized":"2023-06-23T18:21:31.113861+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:54:55.796755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":44,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406218+0000","last_change":"2023-06-23T18:20:44.460897+0000","last_active":"2023-06-23T18:20:46.406218+0000","last_peered":"2023-06-23T18:20:46.406218+0000","last_clean":"2023-06-23T18:20:46.406218+0000","last_became_active":"2023-06-23T18:20:44.460476+0000","last_became_peered":"2023-06-23T18:20:44.460476+0000","last_unstale":"2023-06-23T18:20:46.406218+0000","last_undegraded":"2023-06-23T18:20:46.406218+0000","last_fullsized":"2023-06-23T18:20:46.406218+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:53:43.488229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498266+0000","last_change":"2023-06-23T18:20:44.033013+0000","last_active":"2023-06-23T18:20:47.498266+0000","last_peered":"2023-06-23T18:20:47.498266+0000","last_clean":"2023-06-23T18:20:47.498266+0000","last_became_active":"2023-06-23T18:20:44.032580+0000","last_became_peered":"2023-06-23T18:20:44.032580+0000","last_unstale":"2023-06-23T18:20:47.498266+0000","last_undegraded":"2023-06-23T18:20:47.498266+0000","last_fullsized":"2023-06-23T18:20:47.498266+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:32:32.936824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.110064+0000","last_change":"2023-06-23T18:20:44.463585+0000","last_active":"2023-06-23T18:20:46.110064+0000","last_peered":"2023-06-23T18:20:46.110064+0000","last_clean":"2023-06-23T18:20:46.110064+0000","last_became_active":"2023-06-23T18:20:44.462512+0000","last_became_peered":"2023-06-23T18:20:44.462512+0000","last_unstale":"2023-06-23T18:20:46.110064+0000","last_undegraded":"2023-06-23T18:20:46.110064+0000","last_fullsized":"2023-06-23T18:20:46.110064+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:53:36.151767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497460+0000","last_change":"2023-06-23T18:20:44.037134+0000","last_active":"2023-06-23T18:20:47.497460+0000","last_peered":"2023-06-23T18:20:47.497460+0000","last_clean":"2023-06-23T18:20:47.497460+0000","last_became_active":"2023-06-23T18:20:44.036536+0000","last_became_peered":"2023-06-23T18:20:44.036536+0000","last_unstale":"2023-06-23T18:20:47.497460+0000","last_undegraded":"2023-06-23T18:20:47.497460+0000","last_fullsized":"2023-06-23T18:20:47.497460+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:58:25.025236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406029+0000","last_change":"2023-06-23T18:20:44.454453+0000","last_active":"2023-06-23T18:20:46.406029+0000","last_peered":"2023-06-23T18:20:46.406029+0000","last_clean":"2023-06-23T18:20:46.406029+0000","last_became_active":"2023-06-23T18:20:44.454133+0000","last_became_peered":"2023-06-23T18:20:44.454133+0000","last_unstale":"2023-06-23T18:20:46.406029+0000","last_undegraded":"2023-06-23T18:20:46.406029+0000","last_fullsized":"2023-06-23T18:20:46.406029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:52:02.235360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":139,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:32.115262+0000","last_change":"2023-06-23T18:20:44.042622+0000","last_active":"2023-06-23T18:21:32.115262+0000","last_peered":"2023-06-23T18:21:32.115262+0000","last_clean":"2023-06-23T18:21:32.115262+0000","last_became_active":"2023-06-23T18:20:44.042330+0000","last_became_peered":"2023-06-23T18:20:44.042330+0000","last_unstale":"2023-06-23T18:21:32.115262+0000","last_undegraded":"2023-06-23T18:21:32.115262+0000","last_fullsized":"2023-06-23T18:21:32.115262+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:23.320076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57498,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":96,"num_read_kb":104,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.531053+0000","last_change":"2023-06-23T18:20:44.456558+0000","last_active":"2023-06-23T18:20:45.531053+0000","last_peered":"2023-06-23T18:20:45.531053+0000","last_clean":"2023-06-23T18:20:45.531053+0000","last_became_active":"2023-06-23T18:20:44.456199+0000","last_became_peered":"2023-06-23T18:20:44.456199+0000","last_unstale":"2023-06-23T18:20:45.531053+0000","last_undegraded":"2023-06-23T18:20:45.531053+0000","last_fullsized":"2023-06-23T18:20:45.531053+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:24:13.285535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.499315+0000","last_change":"2023-06-23T18:20:44.034101+0000","last_active":"2023-06-23T18:20:48.499315+0000","last_peered":"2023-06-23T18:20:48.499315+0000","last_clean":"2023-06-23T18:20:48.499315+0000","last_became_active":"2023-06-23T18:20:44.033287+0000","last_became_peered":"2023-06-23T18:20:44.033287+0000","last_unstale":"2023-06-23T18:20:48.499315+0000","last_undegraded":"2023-06-23T18:20:48.499315+0000","last_fullsized":"2023-06-23T18:20:48.499315+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:08:52.512924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406111+0000","last_change":"2023-06-23T18:20:44.446298+0000","last_active":"2023-06-23T18:20:46.406111+0000","last_peered":"2023-06-23T18:20:46.406111+0000","last_clean":"2023-06-23T18:20:46.406111+0000","last_became_active":"2023-06-23T18:20:44.445922+0000","last_became_peered":"2023-06-23T18:20:44.445922+0000","last_unstale":"2023-06-23T18:20:46.406111+0000","last_undegraded":"2023-06-23T18:20:46.406111+0000","last_fullsized":"2023-06-23T18:20:46.406111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:08:15.214033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498820+0000","last_change":"2023-06-23T18:20:44.035791+0000","last_active":"2023-06-23T18:20:47.498820+0000","last_peered":"2023-06-23T18:20:47.498820+0000","last_clean":"2023-06-23T18:20:47.498820+0000","last_became_active":"2023-06-23T18:20:44.035416+0000","last_became_peered":"2023-06-23T18:20:44.035416+0000","last_unstale":"2023-06-23T18:20:47.498820+0000","last_undegraded":"2023-06-23T18:20:47.498820+0000","last_fullsized":"2023-06-23T18:20:47.498820+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:02:31.790408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439429+0000","last_change":"2023-06-23T18:20:44.455474+0000","last_active":"2023-06-23T18:20:45.439429+0000","last_peered":"2023-06-23T18:20:45.439429+0000","last_clean":"2023-06-23T18:20:45.439429+0000","last_became_active":"2023-06-23T18:20:44.455098+0000","last_became_peered":"2023-06-23T18:20:44.455098+0000","last_unstale":"2023-06-23T18:20:45.439429+0000","last_undegraded":"2023-06-23T18:20:45.439429+0000","last_fullsized":"2023-06-23T18:20:45.439429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:53:34.377017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'4","reported_seq":65,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:31.113800+0000","last_change":"2023-06-23T18:20:44.030305+0000","last_active":"2023-06-23T18:21:31.113800+0000","last_peered":"2023-06-23T18:21:31.113800+0000","last_clean":"2023-06-23T18:21:31.113800+0000","last_became_active":"2023-06-23T18:20:44.029898+0000","last_became_peered":"2023-06-23T18:20:44.029898+0000","last_unstale":"2023-06-23T18:21:31.113800+0000","last_undegraded":"2023-06-23T18:21:31.113800+0000","last_fullsized":"2023-06-23T18:21:31.113800+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:38:03.187411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":930,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439068+0000","last_change":"2023-06-23T18:20:44.451321+0000","last_active":"2023-06-23T18:20:45.439068+0000","last_peered":"2023-06-23T18:20:45.439068+0000","last_clean":"2023-06-23T18:20:45.439068+0000","last_became_active":"2023-06-23T18:20:44.450549+0000","last_became_peered":"2023-06-23T18:20:44.450549+0000","last_unstale":"2023-06-23T18:20:45.439068+0000","last_undegraded":"2023-06-23T18:20:45.439068+0000","last_fullsized":"2023-06-23T18:20:45.439068+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:44:56.410567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.503294+0000","last_change":"2023-06-23T18:20:44.044707+0000","last_active":"2023-06-23T18:20:48.503294+0000","last_peered":"2023-06-23T18:20:48.503294+0000","last_clean":"2023-06-23T18:20:48.503294+0000","last_became_active":"2023-06-23T18:20:44.044389+0000","last_became_peered":"2023-06-23T18:20:44.044389+0000","last_unstale":"2023-06-23T18:20:48.503294+0000","last_undegraded":"2023-06-23T18:20:48.503294+0000","last_fullsized":"2023-06-23T18:20:48.503294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:45:51.078124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439091+0000","last_change":"2023-06-23T18:20:44.463637+0000","last_active":"2023-06-23T18:20:45.439091+0000","last_peered":"2023-06-23T18:20:45.439091+0000","last_clean":"2023-06-23T18:20:45.439091+0000","last_became_active":"2023-06-23T18:20:44.463289+0000","last_became_peered":"2023-06-23T18:20:44.463289+0000","last_unstale":"2023-06-23T18:20:45.439091+0000","last_undegraded":"2023-06-23T18:20:45.439091+0000","last_fullsized":"2023-06-23T18:20:45.439091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:24:04.923988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:49.049012+0000","last_change":"2023-06-23T18:20:44.038961+0000","last_active":"2023-06-23T18:20:49.049012+0000","last_peered":"2023-06-23T18:20:49.049012+0000","last_clean":"2023-06-23T18:20:49.049012+0000","last_became_active":"2023-06-23T18:20:44.038674+0000","last_became_peered":"2023-06-23T18:20:44.038674+0000","last_unstale":"2023-06-23T18:20:49.049012+0000","last_undegraded":"2023-06-23T18:20:49.049012+0000","last_fullsized":"2023-06-23T18:20:49.049012+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:18:04.288041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406228+0000","last_change":"2023-06-23T18:20:44.461077+0000","last_active":"2023-06-23T18:20:46.406228+0000","last_peered":"2023-06-23T18:20:46.406228+0000","last_clean":"2023-06-23T18:20:46.406228+0000","last_became_active":"2023-06-23T18:20:44.460767+0000","last_became_peered":"2023-06-23T18:20:44.460767+0000","last_unstale":"2023-06-23T18:20:46.406228+0000","last_undegraded":"2023-06-23T18:20:46.406228+0000","last_fullsized":"2023-06-23T18:20:46.406228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:47:16.534724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.529039+0000","last_change":"2023-06-23T18:20:44.042114+0000","last_active":"2023-06-23T18:20:45.529039+0000","last_peered":"2023-06-23T18:20:45.529039+0000","last_clean":"2023-06-23T18:20:45.529039+0000","last_became_active":"2023-06-23T18:20:44.041822+0000","last_became_peered":"2023-06-23T18:20:44.041822+0000","last_unstale":"2023-06-23T18:20:45.529039+0000","last_undegraded":"2023-06-23T18:20:45.529039+0000","last_fullsized":"2023-06-23T18:20:45.529039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:36:53.587200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.456925+0000","last_change":"2023-06-23T18:20:44.452903+0000","last_active":"2023-06-23T18:20:47.456925+0000","last_peered":"2023-06-23T18:20:47.456925+0000","last_clean":"2023-06-23T18:20:47.456925+0000","last_became_active":"2023-06-23T18:20:44.452539+0000","last_became_peered":"2023-06-23T18:20:44.452539+0000","last_unstale":"2023-06-23T18:20:47.456925+0000","last_undegraded":"2023-06-23T18:20:47.456925+0000","last_fullsized":"2023-06-23T18:20:47.456925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:53:53.970572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.038724+0000","last_change":"2023-06-23T18:20:44.032905+0000","last_active":"2023-06-23T18:20:47.038724+0000","last_peered":"2023-06-23T18:20:47.038724+0000","last_clean":"2023-06-23T18:20:47.038724+0000","last_became_active":"2023-06-23T18:20:44.032606+0000","last_became_peered":"2023-06-23T18:20:44.032606+0000","last_unstale":"2023-06-23T18:20:47.038724+0000","last_undegraded":"2023-06-23T18:20:47.038724+0000","last_fullsized":"2023-06-23T18:20:47.038724+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:08:05.992912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.905273+0000","last_change":"2023-06-23T18:20:44.465580+0000","last_active":"2023-06-23T18:20:45.905273+0000","last_peered":"2023-06-23T18:20:45.905273+0000","last_clean":"2023-06-23T18:20:45.905273+0000","last_became_active":"2023-06-23T18:20:44.465264+0000","last_became_peered":"2023-06-23T18:20:44.465264+0000","last_unstale":"2023-06-23T18:20:45.905273+0000","last_undegraded":"2023-06-23T18:20:45.905273+0000","last_fullsized":"2023-06-23T18:20:45.905273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:20:07.333937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439575+0000","last_change":"2023-06-23T18:20:44.464324+0000","last_active":"2023-06-23T18:20:45.439575+0000","last_peered":"2023-06-23T18:20:45.439575+0000","last_clean":"2023-06-23T18:20:45.439575+0000","last_became_active":"2023-06-23T18:20:44.464021+0000","last_became_peered":"2023-06-23T18:20:44.464021+0000","last_unstale":"2023-06-23T18:20:45.439575+0000","last_undegraded":"2023-06-23T18:20:45.439575+0000","last_fullsized":"2023-06-23T18:20:45.439575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:24:15.831014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":61,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:21:30.111508+0000","last_change":"2023-06-23T18:20:44.034309+0000","last_active":"2023-06-23T18:21:30.111508+0000","last_peered":"2023-06-23T18:21:30.111508+0000","last_clean":"2023-06-23T18:21:30.111508+0000","last_became_active":"2023-06-23T18:20:44.034007+0000","last_became_peered":"2023-06-23T18:20:44.034007+0000","last_unstale":"2023-06-23T18:21:30.111508+0000","last_undegraded":"2023-06-23T18:21:30.111508+0000","last_fullsized":"2023-06-23T18:21:30.111508+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:49:31.359893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439359+0000","last_change":"2023-06-23T18:20:44.468220+0000","last_active":"2023-06-23T18:20:45.439359+0000","last_peered":"2023-06-23T18:20:45.439359+0000","last_clean":"2023-06-23T18:20:45.439359+0000","last_became_active":"2023-06-23T18:20:44.467725+0000","last_became_peered":"2023-06-23T18:20:44.467725+0000","last_unstale":"2023-06-23T18:20:45.439359+0000","last_undegraded":"2023-06-23T18:20:45.439359+0000","last_fullsized":"2023-06-23T18:20:45.439359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:30.069575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:49.048567+0000","last_change":"2023-06-23T18:20:44.030921+0000","last_active":"2023-06-23T18:20:49.048567+0000","last_peered":"2023-06-23T18:20:49.048567+0000","last_clean":"2023-06-23T18:20:49.048567+0000","last_became_active":"2023-06-23T18:20:44.030362+0000","last_became_peered":"2023-06-23T18:20:44.030362+0000","last_unstale":"2023-06-23T18:20:49.048567+0000","last_undegraded":"2023-06-23T18:20:49.048567+0000","last_fullsized":"2023-06-23T18:20:49.048567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:07:10.000171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":328,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.438999+0000","last_change":"2023-06-23T18:19:44.952262+0000","last_active":"2023-06-23T18:20:45.438999+0000","last_peered":"2023-06-23T18:20:45.438999+0000","last_clean":"2023-06-23T18:20:45.438999+0000","last_became_active":"2023-06-23T18:19:44.951803+0000","last_became_peered":"2023-06-23T18:19:44.951803+0000","last_unstale":"2023-06-23T18:20:45.438999+0000","last_undegraded":"2023-06-23T18:20:45.438999+0000","last_fullsized":"2023-06-23T18:20:45.438999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_clean_scrub_stamp":"2023-06-23T18:16:49.458922+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:08:49.981569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405900+0000","last_change":"2023-06-23T18:20:44.456602+0000","last_active":"2023-06-23T18:20:46.405900+0000","last_peered":"2023-06-23T18:20:46.405900+0000","last_clean":"2023-06-23T18:20:46.405900+0000","last_became_active":"2023-06-23T18:20:44.456249+0000","last_became_peered":"2023-06-23T18:20:44.456249+0000","last_unstale":"2023-06-23T18:20:46.405900+0000","last_undegraded":"2023-06-23T18:20:46.405900+0000","last_fullsized":"2023-06-23T18:20:46.405900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:26:13.422710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.498627+0000","last_change":"2023-06-23T18:20:44.039100+0000","last_active":"2023-06-23T18:20:48.498627+0000","last_peered":"2023-06-23T18:20:48.498627+0000","last_clean":"2023-06-23T18:20:48.498627+0000","last_became_active":"2023-06-23T18:20:44.038801+0000","last_became_peered":"2023-06-23T18:20:44.038801+0000","last_unstale":"2023-06-23T18:20:48.498627+0000","last_undegraded":"2023-06-23T18:20:48.498627+0000","last_fullsized":"2023-06-23T18:20:48.498627+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:52:03.284504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.496038+0000","last_change":"2023-06-23T18:20:44.038037+0000","last_active":"2023-06-23T18:20:47.496038+0000","last_peered":"2023-06-23T18:20:47.496038+0000","last_clean":"2023-06-23T18:20:47.496038+0000","last_became_active":"2023-06-23T18:20:44.037732+0000","last_became_peered":"2023-06-23T18:20:44.037732+0000","last_unstale":"2023-06-23T18:20:47.496038+0000","last_undegraded":"2023-06-23T18:20:47.496038+0000","last_fullsized":"2023-06-23T18:20:47.496038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:30:58.279148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406200+0000","last_change":"2023-06-23T18:20:44.459927+0000","last_active":"2023-06-23T18:20:46.406200+0000","last_peered":"2023-06-23T18:20:46.406200+0000","last_clean":"2023-06-23T18:20:46.406200+0000","last_became_active":"2023-06-23T18:20:44.459594+0000","last_became_peered":"2023-06-23T18:20:44.459594+0000","last_unstale":"2023-06-23T18:20:46.406200+0000","last_undegraded":"2023-06-23T18:20:46.406200+0000","last_fullsized":"2023-06-23T18:20:46.406200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:35:13.951754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:55.529354+0000","last_change":"2023-06-23T18:20:44.039429+0000","last_active":"2023-06-23T18:20:55.529354+0000","last_peered":"2023-06-23T18:20:55.529354+0000","last_clean":"2023-06-23T18:20:55.529354+0000","last_became_active":"2023-06-23T18:20:44.038944+0000","last_became_peered":"2023-06-23T18:20:44.038944+0000","last_unstale":"2023-06-23T18:20:55.529354+0000","last_undegraded":"2023-06-23T18:20:55.529354+0000","last_fullsized":"2023-06-23T18:20:55.529354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:37:39.247393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439805+0000","last_change":"2023-06-23T18:20:44.458227+0000","last_active":"2023-06-23T18:20:45.439805+0000","last_peered":"2023-06-23T18:20:45.439805+0000","last_clean":"2023-06-23T18:20:45.439805+0000","last_became_active":"2023-06-23T18:20:44.457904+0000","last_became_peered":"2023-06-23T18:20:44.457904+0000","last_unstale":"2023-06-23T18:20:45.439805+0000","last_undegraded":"2023-06-23T18:20:45.439805+0000","last_fullsized":"2023-06-23T18:20:45.439805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:53:21.724545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.500253+0000","last_change":"2023-06-23T18:20:44.034089+0000","last_active":"2023-06-23T18:20:47.500253+0000","last_peered":"2023-06-23T18:20:47.500253+0000","last_clean":"2023-06-23T18:20:47.500253+0000","last_became_active":"2023-06-23T18:20:44.033745+0000","last_became_peered":"2023-06-23T18:20:44.033745+0000","last_unstale":"2023-06-23T18:20:47.500253+0000","last_undegraded":"2023-06-23T18:20:47.500253+0000","last_fullsized":"2023-06-23T18:20:47.500253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:03:05.244917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.110032+0000","last_change":"2023-06-23T18:20:44.468622+0000","last_active":"2023-06-23T18:20:46.110032+0000","last_peered":"2023-06-23T18:20:46.110032+0000","last_clean":"2023-06-23T18:20:46.110032+0000","last_became_active":"2023-06-23T18:20:44.466831+0000","last_became_peered":"2023-06-23T18:20:44.466831+0000","last_unstale":"2023-06-23T18:20:46.110032+0000","last_undegraded":"2023-06-23T18:20:46.110032+0000","last_fullsized":"2023-06-23T18:20:46.110032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:24:20.760069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497829+0000","last_change":"2023-06-23T18:20:44.030647+0000","last_active":"2023-06-23T18:20:47.497829+0000","last_peered":"2023-06-23T18:20:47.497829+0000","last_clean":"2023-06-23T18:20:47.497829+0000","last_became_active":"2023-06-23T18:20:44.030316+0000","last_became_peered":"2023-06-23T18:20:44.030316+0000","last_unstale":"2023-06-23T18:20:47.497829+0000","last_undegraded":"2023-06-23T18:20:47.497829+0000","last_fullsized":"2023-06-23T18:20:47.497829+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:59:47.369435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.530780+0000","last_change":"2023-06-23T18:20:44.458365+0000","last_active":"2023-06-23T18:20:45.530780+0000","last_peered":"2023-06-23T18:20:45.530780+0000","last_clean":"2023-06-23T18:20:45.530780+0000","last_became_active":"2023-06-23T18:20:44.458039+0000","last_became_peered":"2023-06-23T18:20:44.458039+0000","last_unstale":"2023-06-23T18:20:45.530780+0000","last_undegraded":"2023-06-23T18:20:45.530780+0000","last_fullsized":"2023-06-23T18:20:45.530780+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:03:49.379689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.498764+0000","last_change":"2023-06-23T18:20:44.028908+0000","last_active":"2023-06-23T18:20:47.498764+0000","last_peered":"2023-06-23T18:20:47.498764+0000","last_clean":"2023-06-23T18:20:47.498764+0000","last_became_active":"2023-06-23T18:20:44.028551+0000","last_became_peered":"2023-06-23T18:20:44.028551+0000","last_unstale":"2023-06-23T18:20:47.498764+0000","last_undegraded":"2023-06-23T18:20:47.498764+0000","last_fullsized":"2023-06-23T18:20:47.498764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:31:41.576378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406188+0000","last_change":"2023-06-23T18:20:44.465300+0000","last_active":"2023-06-23T18:20:46.406188+0000","last_peered":"2023-06-23T18:20:46.406188+0000","last_clean":"2023-06-23T18:20:46.406188+0000","last_became_active":"2023-06-23T18:20:44.465008+0000","last_became_peered":"2023-06-23T18:20:44.465008+0000","last_unstale":"2023-06-23T18:20:46.406188+0000","last_undegraded":"2023-06-23T18:20:46.406188+0000","last_fullsized":"2023-06-23T18:20:46.406188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:03:27.021563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237610+0000","last_change":"2023-06-23T18:20:44.041593+0000","last_active":"2023-06-23T18:20:46.237610+0000","last_peered":"2023-06-23T18:20:46.237610+0000","last_clean":"2023-06-23T18:20:46.237610+0000","last_became_active":"2023-06-23T18:20:44.041283+0000","last_became_peered":"2023-06-23T18:20:44.041283+0000","last_unstale":"2023-06-23T18:20:46.237610+0000","last_undegraded":"2023-06-23T18:20:46.237610+0000","last_fullsized":"2023-06-23T18:20:46.237610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:45:29.030061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439210+0000","last_change":"2023-06-23T18:20:44.465127+0000","last_active":"2023-06-23T18:20:45.439210+0000","last_peered":"2023-06-23T18:20:45.439210+0000","last_clean":"2023-06-23T18:20:45.439210+0000","last_became_active":"2023-06-23T18:20:44.464768+0000","last_became_peered":"2023-06-23T18:20:44.464768+0000","last_unstale":"2023-06-23T18:20:45.439210+0000","last_undegraded":"2023-06-23T18:20:45.439210+0000","last_fullsized":"2023-06-23T18:20:45.439210+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:54:38.343097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.497671+0000","last_change":"2023-06-23T18:20:44.046245+0000","last_active":"2023-06-23T18:20:47.497671+0000","last_peered":"2023-06-23T18:20:47.497671+0000","last_clean":"2023-06-23T18:20:47.497671+0000","last_became_active":"2023-06-23T18:20:44.045945+0000","last_became_peered":"2023-06-23T18:20:44.045945+0000","last_unstale":"2023-06-23T18:20:47.497671+0000","last_undegraded":"2023-06-23T18:20:47.497671+0000","last_fullsized":"2023-06-23T18:20:47.497671+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:13:06.840364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439547+0000","last_change":"2023-06-23T18:20:44.453520+0000","last_active":"2023-06-23T18:20:45.439547+0000","last_peered":"2023-06-23T18:20:45.439547+0000","last_clean":"2023-06-23T18:20:45.439547+0000","last_became_active":"2023-06-23T18:20:44.453214+0000","last_became_peered":"2023-06-23T18:20:44.453214+0000","last_unstale":"2023-06-23T18:20:45.439547+0000","last_undegraded":"2023-06-23T18:20:45.439547+0000","last_fullsized":"2023-06-23T18:20:45.439547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:47.302353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:48.502980+0000","last_change":"2023-06-23T18:20:44.042526+0000","last_active":"2023-06-23T18:20:48.502980+0000","last_peered":"2023-06-23T18:20:48.502980+0000","last_clean":"2023-06-23T18:20:48.502980+0000","last_became_active":"2023-06-23T18:20:44.042249+0000","last_became_peered":"2023-06-23T18:20:44.042249+0000","last_unstale":"2023-06-23T18:20:48.502980+0000","last_undegraded":"2023-06-23T18:20:48.502980+0000","last_fullsized":"2023-06-23T18:20:48.502980+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:31:39.378767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439873+0000","last_change":"2023-06-23T18:20:44.454645+0000","last_active":"2023-06-23T18:20:45.439873+0000","last_peered":"2023-06-23T18:20:45.439873+0000","last_clean":"2023-06-23T18:20:45.439873+0000","last_became_active":"2023-06-23T18:20:44.453309+0000","last_became_peered":"2023-06-23T18:20:44.453309+0000","last_unstale":"2023-06-23T18:20:45.439873+0000","last_undegraded":"2023-06-23T18:20:45.439873+0000","last_fullsized":"2023-06-23T18:20:45.439873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:30:55.635983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439463+0000","last_change":"2023-06-23T18:20:44.037681+0000","last_active":"2023-06-23T18:20:45.439463+0000","last_peered":"2023-06-23T18:20:45.439463+0000","last_clean":"2023-06-23T18:20:45.439463+0000","last_became_active":"2023-06-23T18:20:44.037323+0000","last_became_peered":"2023-06-23T18:20:44.037323+0000","last_unstale":"2023-06-23T18:20:45.439463+0000","last_undegraded":"2023-06-23T18:20:45.439463+0000","last_fullsized":"2023-06-23T18:20:45.439463+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:55:26.308777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'1","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:47.038770+0000","last_change":"2023-06-23T18:20:44.456434+0000","last_active":"2023-06-23T18:20:47.038770+0000","last_peered":"2023-06-23T18:20:47.038770+0000","last_clean":"2023-06-23T18:20:47.038770+0000","last_became_active":"2023-06-23T18:20:44.456110+0000","last_became_peered":"2023-06-23T18:20:44.456110+0000","last_unstale":"2023-06-23T18:20:47.038770+0000","last_undegraded":"2023-06-23T18:20:47.038770+0000","last_fullsized":"2023-06-23T18:20:47.038770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:00:04.366221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406173+0000","last_change":"2023-06-23T18:20:44.040722+0000","last_active":"2023-06-23T18:20:46.406173+0000","last_peered":"2023-06-23T18:20:46.406173+0000","last_clean":"2023-06-23T18:20:46.406173+0000","last_became_active":"2023-06-23T18:20:44.040390+0000","last_became_peered":"2023-06-23T18:20:44.040390+0000","last_unstale":"2023-06-23T18:20:46.406173+0000","last_undegraded":"2023-06-23T18:20:46.406173+0000","last_fullsized":"2023-06-23T18:20:46.406173+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:56:25.584399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439334+0000","last_change":"2023-06-23T18:20:44.455559+0000","last_active":"2023-06-23T18:20:45.439334+0000","last_peered":"2023-06-23T18:20:45.439334+0000","last_clean":"2023-06-23T18:20:45.439334+0000","last_became_active":"2023-06-23T18:20:44.455256+0000","last_became_peered":"2023-06-23T18:20:44.455256+0000","last_unstale":"2023-06-23T18:20:45.439334+0000","last_undegraded":"2023-06-23T18:20:45.439334+0000","last_fullsized":"2023-06-23T18:20:45.439334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:40:30.169440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406131+0000","last_change":"2023-06-23T18:20:44.460403+0000","last_active":"2023-06-23T18:20:46.406131+0000","last_peered":"2023-06-23T18:20:46.406131+0000","last_clean":"2023-06-23T18:20:46.406131+0000","last_became_active":"2023-06-23T18:20:44.459928+0000","last_became_peered":"2023-06-23T18:20:44.459928+0000","last_unstale":"2023-06-23T18:20:46.406131+0000","last_undegraded":"2023-06-23T18:20:46.406131+0000","last_fullsized":"2023-06-23T18:20:46.406131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:58:06.385995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237624+0000","last_change":"2023-06-23T18:20:44.458050+0000","last_active":"2023-06-23T18:20:46.237624+0000","last_peered":"2023-06-23T18:20:46.237624+0000","last_clean":"2023-06-23T18:20:46.237624+0000","last_became_active":"2023-06-23T18:20:44.457733+0000","last_became_peered":"2023-06-23T18:20:44.457733+0000","last_unstale":"2023-06-23T18:20:46.237624+0000","last_undegraded":"2023-06-23T18:20:46.237624+0000","last_fullsized":"2023-06-23T18:20:46.237624+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:23:24.448430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237525+0000","last_change":"2023-06-23T18:20:44.456233+0000","last_active":"2023-06-23T18:20:46.237525+0000","last_peered":"2023-06-23T18:20:46.237525+0000","last_clean":"2023-06-23T18:20:46.237525+0000","last_became_active":"2023-06-23T18:20:44.455677+0000","last_became_peered":"2023-06-23T18:20:44.455677+0000","last_unstale":"2023-06-23T18:20:46.237525+0000","last_undegraded":"2023-06-23T18:20:46.237525+0000","last_fullsized":"2023-06-23T18:20:46.237525+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:54:14.083530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.109745+0000","last_change":"2023-06-23T18:20:44.469668+0000","last_active":"2023-06-23T18:20:46.109745+0000","last_peered":"2023-06-23T18:20:46.109745+0000","last_clean":"2023-06-23T18:20:46.109745+0000","last_became_active":"2023-06-23T18:20:44.466563+0000","last_became_peered":"2023-06-23T18:20:44.466563+0000","last_unstale":"2023-06-23T18:20:46.109745+0000","last_undegraded":"2023-06-23T18:20:46.109745+0000","last_fullsized":"2023-06-23T18:20:46.109745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:24:09.129588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406322+0000","last_change":"2023-06-23T18:20:44.456021+0000","last_active":"2023-06-23T18:20:46.406322+0000","last_peered":"2023-06-23T18:20:46.406322+0000","last_clean":"2023-06-23T18:20:46.406322+0000","last_became_active":"2023-06-23T18:20:44.455705+0000","last_became_peered":"2023-06-23T18:20:44.455705+0000","last_unstale":"2023-06-23T18:20:46.406322+0000","last_undegraded":"2023-06-23T18:20:46.406322+0000","last_fullsized":"2023-06-23T18:20:46.406322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:16:56.043843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406226+0000","last_change":"2023-06-23T18:20:44.460359+0000","last_active":"2023-06-23T18:20:46.406226+0000","last_peered":"2023-06-23T18:20:46.406226+0000","last_clean":"2023-06-23T18:20:46.406226+0000","last_became_active":"2023-06-23T18:20:44.460034+0000","last_became_peered":"2023-06-23T18:20:44.460034+0000","last_unstale":"2023-06-23T18:20:46.406226+0000","last_undegraded":"2023-06-23T18:20:46.406226+0000","last_fullsized":"2023-06-23T18:20:46.406226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:00:06.777366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.237875+0000","last_change":"2023-06-23T18:20:44.455743+0000","last_active":"2023-06-23T18:20:46.237875+0000","last_peered":"2023-06-23T18:20:46.237875+0000","last_clean":"2023-06-23T18:20:46.237875+0000","last_became_active":"2023-06-23T18:20:44.455266+0000","last_became_peered":"2023-06-23T18:20:44.455266+0000","last_unstale":"2023-06-23T18:20:46.237875+0000","last_undegraded":"2023-06-23T18:20:46.237875+0000","last_fullsized":"2023-06-23T18:20:46.237875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:33:52.673181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439651+0000","last_change":"2023-06-23T18:20:44.461434+0000","last_active":"2023-06-23T18:20:45.439651+0000","last_peered":"2023-06-23T18:20:45.439651+0000","last_clean":"2023-06-23T18:20:45.439651+0000","last_became_active":"2023-06-23T18:20:44.461137+0000","last_became_peered":"2023-06-23T18:20:44.461137+0000","last_unstale":"2023-06-23T18:20:45.439651+0000","last_undegraded":"2023-06-23T18:20:45.439651+0000","last_fullsized":"2023-06-23T18:20:45.439651+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:14:08.293656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439086+0000","last_change":"2023-06-23T18:20:44.465192+0000","last_active":"2023-06-23T18:20:45.439086+0000","last_peered":"2023-06-23T18:20:45.439086+0000","last_clean":"2023-06-23T18:20:45.439086+0000","last_became_active":"2023-06-23T18:20:44.464905+0000","last_became_peered":"2023-06-23T18:20:44.464905+0000","last_unstale":"2023-06-23T18:20:45.439086+0000","last_undegraded":"2023-06-23T18:20:45.439086+0000","last_fullsized":"2023-06-23T18:20:45.439086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:01:32.627765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:45.439130+0000","last_change":"2023-06-23T18:20:44.458725+0000","last_active":"2023-06-23T18:20:45.439130+0000","last_peered":"2023-06-23T18:20:45.439130+0000","last_clean":"2023-06-23T18:20:45.439130+0000","last_became_active":"2023-06-23T18:20:44.458414+0000","last_became_peered":"2023-06-23T18:20:44.458414+0000","last_unstale":"2023-06-23T18:20:45.439130+0000","last_undegraded":"2023-06-23T18:20:45.439130+0000","last_fullsized":"2023-06-23T18:20:45.439130+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:36:29.121354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.405885+0000","last_change":"2023-06-23T18:20:44.465665+0000","last_active":"2023-06-23T18:20:46.405885+0000","last_peered":"2023-06-23T18:20:46.405885+0000","last_clean":"2023-06-23T18:20:46.405885+0000","last_became_active":"2023-06-23T18:20:44.465337+0000","last_became_peered":"2023-06-23T18:20:44.465337+0000","last_unstale":"2023-06-23T18:20:46.405885+0000","last_undegraded":"2023-06-23T18:20:46.405885+0000","last_fullsized":"2023-06-23T18:20:46.405885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:58:56.987077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406238+0000","last_change":"2023-06-23T18:20:44.455107+0000","last_active":"2023-06-23T18:20:46.406238+0000","last_peered":"2023-06-23T18:20:46.406238+0000","last_clean":"2023-06-23T18:20:46.406238+0000","last_became_active":"2023-06-23T18:20:44.454798+0000","last_became_peered":"2023-06-23T18:20:44.454798+0000","last_unstale":"2023-06-23T18:20:46.406238+0000","last_undegraded":"2023-06-23T18:20:46.406238+0000","last_fullsized":"2023-06-23T18:20:46.406238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:10:04.410538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-06-23T18:20:46.406114+0000","last_change":"2023-06-23T18:20:44.461044+0000","last_active":"2023-06-23T18:20:46.406114+0000","last_peered":"2023-06-23T18:20:46.406114+0000","last_clean":"2023-06-23T18:20:46.406114+0000","last_became_active":"2023-06-23T18:20:44.460737+0000","last_became_peered":"2023-06-23T18:20:44.460737+0000","last_unstale":"2023-06-23T18:20:46.406114+0000","last_undegraded":"2023-06-23T18:20:46.406114+0000","last_fullsized":"2023-06-23T18:20:46.406114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:13:16.912989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281641,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61402,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":287,"num_read_kb":210,"num_write":154,"num_write_kb":105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":184206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481813,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":860,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":880640,"data_stored":420371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":5.7590000000000003,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.439},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.628},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.58099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69899999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6580000000000004,"5min":5.6580000000000004,"15min":5.6580000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.75600000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":9,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.76900000000000002}]},{"osd":10,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":0.80800000000000005}]}]},{"osd":10,"up_from":70,"seq":300647710746,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":1028096,"data_stored":1069698,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":3.6970000000000001,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":2.9279999999999999,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":2.6669999999999998,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.73199999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":4.5499999999999998,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":2.9780000000000002,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":2.8559999999999999,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.7210000000000001,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":2.8100000000000001,"5min":2.8100000000000001,"15min":2.8100000000000001},"last":0.753},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.7210000000000001,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":0.81599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.782,"5min":1.782,"15min":1.782},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.9089999999999998,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.85999999999999999}]},{"osd":9,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.871},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.9430000000000001,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.77100000000000002}]},{"osd":11,"last update":"Fri Jun 23 18:21:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.93100000000000005}]}]},{"osd":9,"up_from":65,"seq":279172874269,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":897024,"data_stored":420607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44600000000000001,"15min":0.45200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.67100000000000004,"15min":0.69499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.72999999999999998,"15min":0.753},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.47299999999999998}]},{"osd":2,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.62}]},{"osd":4,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.442},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.75,"15min":0.77500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83899999999999997,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.70599999999999996,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.034,"5min":1.04,"15min":1.04},"last":0.70299999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0669999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":1.397,"15min":1.397},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.81599999999999995,"15min":0.83399999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0960000000000001,"5min":1.661,"15min":1.661},"last":0.73899999999999999}]},{"osd":8,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0349999999999999,"5min":1.079,"15min":1.079},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.93000000000000005,"15min":0.97299999999999998},"min":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.91900000000000004,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.60599999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:20:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.59099999999999997}]}]},{"osd":8,"up_from":58,"seq":249108103201,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":892928,"data_stored":420196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.51900000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.60399999999999998,"15min":0.60699999999999998},"min":{"1min":0.38800000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.76800000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.44600000000000001,"15min":0.44800000000000001},"min":{"1min":0.27600000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.67200000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.621},{"interface":"front","average":{"1min":0.443,"5min":0.45700000000000002,"15min":0.45900000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.64400000000000002,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.56000000000000005,"15min":0.56499999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72699999999999998,"5min":0.752,"15min":0.752},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.39000000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.40300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82699999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.38200000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88600000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.78200000000000003}]},{"osd":5,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54100000000000004,"15min":0.54200000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.872,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.88600000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:21:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.432,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90200000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.82999999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.75}]},{"osd":10,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.88300000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":1012366,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.33300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.64100000000000001,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.435,"15min":0.42299999999999999},"min":{"1min":0.29899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.54600000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.56499999999999995,"15min":0.55700000000000005},"min":{"1min":0.36899999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.753},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.499,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64700000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.49199999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.80000000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.52000000000000002,"15min":0.50900000000000001},"min":{"1min":0.34000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.84199999999999997,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.54100000000000004,"15min":0.53100000000000003},"min":{"1min":0.371,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.621}]},{"osd":5,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64100000000000001,"15min":0.63},"min":{"1min":0.54000000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.65800000000000003,"15min":0.64800000000000002},"min":{"1min":0.55600000000000005,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.877}]},{"osd":6,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.53600000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.66800000000000004,"15min":0.65400000000000003},"min":{"1min":0.56799999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.84699999999999998}]},{"osd":8,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71499999999999997,"15min":0.71099999999999997},"min":{"1min":0.55800000000000005,"5min":0.443,"15min":0.443},"max":{"1min":0.97799999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.72399999999999998,"15min":0.71599999999999997},"min":{"1min":0.51500000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.81399999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.93999999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.92400000000000004}]},{"osd":10,"last update":"Fri Jun 23 18:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.93600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96999999999999997}]}]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":864,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":884736,"data_stored":420002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.58899999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.28699999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.433}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.89400000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70799999999999996,"5min":1.105,"15min":1.105},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65400000000000003,"5min":1.141,"15min":1.141},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72199999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.872,"15min":0.872},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.5}]},{"osd":5,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77200000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68300000000000005,"15min":0.69699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53600000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.69499999999999995,"15min":0.69299999999999995},"min":{"1min":0.50800000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.64800000000000002}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97199999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.623}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69199999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96799999999999997,"5min":1.008,"15min":1.008},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71699999999999997,"15min":0.73599999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.63700000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:20:43 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67000000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":477287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.51100000000000001,"15min":0.47899999999999998},"min":{"1min":0.376,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82599999999999996,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.41999999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86499999999999999,"5min":1.218,"15min":1.218},"last":0.56799999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.59199999999999997,"15min":0.53900000000000003},"min":{"1min":0.52600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":1.071,"5min":1.25,"15min":1.25},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.56399999999999995,"15min":0.53800000000000003},"min":{"1min":0.44900000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0960000000000001,"5min":1.343,"15min":1.343},"last":0.745}]},{"osd":3,"last update":"Fri Jun 23 18:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.42799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.93200000000000005,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.439}]},{"osd":4,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.45400000000000001,"15min":0.42799999999999999},"min":{"1min":0.38600000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85499999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.44600000000000001,"15min":0.41799999999999998},"min":{"1min":0.35699999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82899999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.40100000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.63100000000000001,"15min":0.58399999999999996},"min":{"1min":0.54800000000000004,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.032,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.623,"15min":0.59699999999999998},"min":{"1min":0.54300000000000004,"5min":0.374,"15min":0.374},"max":{"1min":1.052,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.52400000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84599999999999997,"5min":1.169,"15min":1.169},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.60099999999999998},"min":{"1min":0.53200000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.093,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.47799999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.61699999999999999,"15min":0.59699999999999998},"min":{"1min":0.39400000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96099999999999997,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.69399999999999995,"15min":0.67000000000000004},"min":{"1min":0.56200000000000006,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.0629999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.94299999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.66600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.216,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77200000000000002,"15min":0.76600000000000001},"min":{"1min":0.61699999999999999,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.149,"5min":1.319,"15min":1.319},"last":0.73199999999999998}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.876,"15min":0.88900000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1180000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96399999999999997,"5min":1.288,"15min":1.288},"last":0.78800000000000003}]},{"osd":10,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.89600000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27836,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717316,"statfs":{"total":95995035648,"available":95966531584,"internally_reserved":0,"allocated":905216,"data_stored":421267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:20:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.50900000000000001,"15min":0.47499999999999998},"min":{"1min":0.40899999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.5,"15min":0.47499999999999998},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.499},"min":{"1min":0.38200000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.77900000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.55900000000000005,"15min":0.51900000000000002},"min":{"1min":0.432,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79300000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Fri Jun 23 18:19:17 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.38500000000000001,"15min":0.377},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.624,"5min":0.628,"15min":0.628},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.45600000000000002,"15min":0.45200000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.27100000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.57099999999999995,"15min":0.54900000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.56999999999999995,"15min":0.55400000000000005},"min":{"1min":0.41999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41399999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.60799999999999998,"15min":0.59099999999999997},"min":{"1min":0.439,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.79000000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.77000000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82499999999999996,"5min":1.073,"15min":1.073},"last":0.503}]},{"osd":7,"last update":"Fri Jun 23 18:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.622}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.876,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.56499999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.76400000000000001,"15min":0.79000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0449999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.48299999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:20:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53300000000000003}]},{"osd":11,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59699999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":478589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.34899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.70499999999999996,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68999999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.54700000000000004,"15min":0.53200000000000003},"min":{"1min":0.38500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79000000000000004,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.81499999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.56899999999999995,"15min":0.56299999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.56599999999999995,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.45500000000000002,"15min":0.44500000000000001},"min":{"1min":0.30399999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.49099999999999999,"15min":0.47099999999999997},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0069999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59799999999999998}]},{"osd":8,"last update":"Fri Jun 23 18:21:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.35899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.64100000000000001,"15min":0.63600000000000001},"min":{"1min":0.45400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78800000000000003}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61899999999999999}]},{"osd":10,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.82199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27832,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717320,"statfs":{"total":95995035648,"available":95966535680,"internally_reserved":0,"allocated":901120,"data_stored":420319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.502,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":12.246,"5min":12.246,"15min":12.246},"last":0.309},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40400000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58099999999999996,"15min":0.54200000000000004},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":8.3569999999999993,"5min":8.3569999999999993,"15min":8.3569999999999993},"last":0.74199999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:20:39 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60999999999999999,"15min":0.58599999999999997},"min":{"1min":0.44,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.63900000000000001,"15min":0.625},"min":{"1min":0.47399999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47699999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.65100000000000002},"min":{"1min":0.45300000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59399999999999997,"15min":0.58699999999999997},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.80500000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.505}]},{"osd":6,"last update":"Fri Jun 23 18:21:26 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.70699999999999996,"15min":0.59799999999999998},"min":{"1min":0.377,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":12.183999999999999,"5min":12.183999999999999,"15min":12.183999999999999},"last":0.379},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.59099999999999997,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.34699999999999998}]},{"osd":7,"last update":"Fri Jun 23 18:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.61299999999999999,"15min":0.60699999999999998},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.54200000000000004,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.51900000000000002}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.43099999999999999}]},{"osd":10,"last update":"Fri Jun 23 18:20:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.73599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":888832,"data_stored":420775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.51700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29799999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.57799999999999996,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.39900000000000002,"15min":0.38300000000000001},"min":{"1min":0.29999999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.50900000000000001},"min":{"1min":0.38100000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.73899999999999999,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.57799999999999996,"15min":0.54300000000000004},"min":{"1min":0.503,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.71999999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55500000000000005,"15min":0.52300000000000002},"min":{"1min":0.41999999999999998,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78000000000000003,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.371,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Jun 23 18:20:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61799999999999999,"15min":0.59999999999999998},"min":{"1min":0.437,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82799999999999996,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60899999999999999,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.74299999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.54600000000000004}]},{"osd":6,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":1.129,"15min":1.129},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61199999999999999,"15min":0.60999999999999999},"min":{"1min":0.379,"5min":0.378,"15min":0.378},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.59799999999999998,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.42299999999999999}]},{"osd":8,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67100000000000004,"15min":0.67700000000000005},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94199999999999995,"5min":1.202,"15min":1.202},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.747,"15min":0.75600000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.91400000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":0.76100000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:20:42 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.70699999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":1011708,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.47699999999999998,"15min":0.46000000000000002},"min":{"1min":0.39200000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.70599999999999996,"5min":0.875,"15min":0.875},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49399999999999999,"15min":0.49299999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.65400000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53200000000000003,"15min":0.52200000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41099999999999998},"min":{"1min":0.248,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.63600000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.435,"15min":0.42799999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.58199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.45500000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.40400000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.67800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.433},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53700000000000003,"15min":0.53100000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.72999999999999998,"5min":0.751,"15min":0.751},"last":0.60999999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58599999999999997,"15min":0.57599999999999996},"min":{"1min":0.40200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80700000000000005,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.59899999999999998,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.745}]},{"osd":6,"last update":"Fri Jun 23 18:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.503,"5min":0.439,"15min":0.439},"max":{"1min":0.89800000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67500000000000004,"15min":0.68000000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70099999999999996}]},{"osd":8,"last update":"Fri Jun 23 18:20:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.624,"15min":0.63400000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.78200000000000003,"5min":0.998,"15min":0.998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.61699999999999999,"15min":0.627},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73599999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.76200000000000001}]},{"osd":9,"last update":"Fri Jun 23 18:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Jun 23 18:20:45 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:21:33.390 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-06-23T18:21:33.390 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-06-23T18:21:33.390 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-06-23T18:21:33.391 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph health --format=json 2023-06-23T18:21:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:34.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:33 smithi082 ceph-mon[97889]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:21:34.159 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:33 smithi002 ceph-mon[95947]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:21:34.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:33 smithi017 ceph-mon[97472]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:21:35.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 -- 172.21.15.2:0/1810678039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 msgr2=0x7fcc540feee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:35.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/1810678039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 0x7fcc540feee0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009990 tx=0x7fcc44009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 -- 172.21.15.2:0/1810678039 shutdown_connections 2023-06-23T18:21:35.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/1810678039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc540ff420 0x7fcc5410b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/1810678039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54104a70 0x7fcc54104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/1810678039 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 0x7fcc540feee0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 -- 172.21.15.2:0/1810678039 >> 172.21.15.2:0/1810678039 conn(0x7fcc54075b30 msgr2=0x7fcc54075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:35.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 -- 172.21.15.2:0/1810678039 shutdown_connections 2023-06-23T18:21:35.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.146+0000 7fcc58c0f700 1 -- 172.21.15.2:0/1810678039 wait complete. 2023-06-23T18:21:35.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc58c0f700 1 Processor -- start 2023-06-23T18:21:35.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc58c0f700 1 -- start start 2023-06-23T18:21:35.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc58c0f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 0x7fcc5419a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:35.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc58c0f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc540ff420 0x7fcc5419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:35.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc58c0f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54104a70 0x7fcc541948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:35.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc58c0f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54194f10 con 0x7fcc540fea70 2023-06-23T18:21:35.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc58c0f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54195080 con 0x7fcc540ff420 2023-06-23T18:21:35.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc58c0f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc54195240 con 0x7fcc54104a70 2023-06-23T18:21:35.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc5259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 0x7fcc5419a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:35.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc5259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 0x7fcc5419a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53042/0 (socket says 172.21.15.2:53042) 2023-06-23T18:21:35.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc5259c700 1 -- 172.21.15.2:0/3304666591 learned_addr learned my addr 172.21.15.2:0/3304666591 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:35.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.147+0000 7fcc51d9b700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc540ff420 0x7fcc5419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:35.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc52d9d700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54104a70 0x7fcc541948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:35.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc5259c700 1 -- 172.21.15.2:0/3304666591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54104a70 msgr2=0x7fcc541948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:35.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc5259c700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54104a70 0x7fcc541948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc5259c700 1 -- 172.21.15.2:0/3304666591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc540ff420 msgr2=0x7fcc5419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:35.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc5259c700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc540ff420 0x7fcc5419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc5259c700 1 -- 172.21.15.2:0/3304666591 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc440095f0 con 0x7fcc540fea70 2023-06-23T18:21:35.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc5259c700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 0x7fcc5419a820 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00d980 tx=0x7fcc3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:35.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3304666591 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c004020 con 0x7fcc540fea70 2023-06-23T18:21:35.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc54195520 con 0x7fcc540fea70 2023-06-23T18:21:35.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3304666591 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc3c009730 con 0x7fcc540fea70 2023-06-23T18:21:35.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3304666591 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3c0039c0 con 0x7fcc540fea70 2023-06-23T18:21:35.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.148+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc541a1810 con 0x7fcc540fea70 2023-06-23T18:21:35.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.149+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5404e1d0 con 0x7fcc540fea70 2023-06-23T18:21:35.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.150+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3304666591 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fcc3c0098a0 con 0x7fcc540fea70 2023-06-23T18:21:35.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.151+0000 7fcc4b7fe700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc400a11e0 0x7fcc400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:35.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.151+0000 7fcc51d9b700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc400a11e0 0x7fcc400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:35.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.151+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3304666591 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcc3c0cc5c0 con 0x7fcc540fea70 2023-06-23T18:21:35.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.151+0000 7fcc51d9b700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc400a11e0 0x7fcc400a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc540fc750 tx=0x7fcc44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:35.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.155+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3304666591 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcc3c095d60 con 0x7fcc540fea70 2023-06-23T18:21:35.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:35 smithi002 ceph-mon[95947]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:35.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:35 smithi082 ceph-mon[97889]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:35.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.400+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcc54195db0 con 0x7fcc540fea70 2023-06-23T18:21:35.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.400+0000 7fcc4b7fe700 1 -- 172.21.15.2:0/3304666591 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fcc3c099380 con 0x7fcc540fea70 2023-06-23T18:21:35.406 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:35.407 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-06-23T18:21:35.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.402+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc400a11e0 msgr2=0x7fcc400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:35.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.402+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc400a11e0 0x7fcc400a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc540fc750 tx=0x7fcc44005fb0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.402+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 msgr2=0x7fcc5419a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:35.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.402+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 0x7fcc5419a820 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c00d980 tx=0x7fcc3c00dc90 comp rx=0 tx=0).stop 2023-06-23T18:21:35.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.402+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 shutdown_connections 2023-06-23T18:21:35.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.402+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcc54104a70 0x7fcc541948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.403+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcc540fea70 0x7fcc5419a820 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.403+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc400a11e0 0x7fcc400a3690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.403+0000 7fcc58c0f700 1 --2- 172.21.15.2:0/3304666591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc540ff420 0x7fcc5419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.403+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 >> 172.21.15.2:0/3304666591 conn(0x7fcc54075b30 msgr2=0x7fcc541026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:35.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.403+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 shutdown_connections 2023-06-23T18:21:35.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.403+0000 7fcc58c0f700 1 -- 172.21.15.2:0/3304666591 wait complete. 2023-06-23T18:21:35.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:35 smithi017 ceph-mon[97472]: pgmap v281: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:35.815 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-06-23T18:21:35.816 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:21:35.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.942+0000 7f50d7e38700 1 -- 172.21.15.2:0/3310918618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d01005e0 msgr2=0x7f50d0106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:35.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.942+0000 7f50d7e38700 1 --2- 172.21.15.2:0/3310918618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d01005e0 0x7f50d0106ea0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f50c800b0a0 tx=0x7f50c800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.942+0000 7f50d7e38700 1 -- 172.21.15.2:0/3310918618 shutdown_connections 2023-06-23T18:21:35.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.942+0000 7f50d7e38700 1 --2- 172.21.15.2:0/3310918618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d01005e0 0x7f50d0106ea0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.942+0000 7f50d7e38700 1 --2- 172.21.15.2:0/3310918618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50d00fea60 0x7f50d00fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.942+0000 7f50d7e38700 1 --2- 172.21.15.2:0/3310918618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50d00ffc50 0x7f50d01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.942+0000 7f50d7e38700 1 -- 172.21.15.2:0/3310918618 >> 172.21.15.2:0/3310918618 conn(0x7f50d00fa0e0 msgr2=0x7f50d00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:35.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 -- 172.21.15.2:0/3310918618 shutdown_connections 2023-06-23T18:21:35.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 -- 172.21.15.2:0/3310918618 wait complete. 2023-06-23T18:21:35.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 Processor -- start 2023-06-23T18:21:35.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 -- start start 2023-06-23T18:21:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50d00fea60 0x7f50d0068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d00ffc50 0x7f50d019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50d01005e0 0x7f50d019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019d440 con 0x7f50d00ffc50 2023-06-23T18:21:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019d5b0 con 0x7f50d01005e0 2023-06-23T18:21:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.943+0000 7f50d7e38700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d019d770 con 0x7f50d00fea60 2023-06-23T18:21:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d00ffc50 0x7f50d019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d00ffc50 0x7f50d019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53056/0 (socket says 172.21.15.2:53056) 2023-06-23T18:21:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 -- 172.21.15.2:0/1726073485 learned_addr learned my addr 172.21.15.2:0/1726073485 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d5bd4700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50d00fea60 0x7f50d0068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d63d5700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50d01005e0 0x7f50d019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 -- 172.21.15.2:0/1726073485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50d00fea60 msgr2=0x7f50d0068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50d00fea60 0x7f50d0068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 -- 172.21.15.2:0/1726073485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50d01005e0 msgr2=0x7f50d019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50d01005e0 0x7f50d019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 -- 172.21.15.2:0/1726073485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c8009d00 con 0x7f50d00ffc50 2023-06-23T18:21:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d53d3700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d00ffc50 0x7f50d019a620 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50c000d950 tx=0x7f50c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50c6ffd700 1 -- 172.21.15.2:0/1726073485 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000f890 con 0x7f50d00ffc50 2023-06-23T18:21:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50c6ffd700 1 -- 172.21.15.2:0/1726073485 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50c000fe80 con 0x7f50d00ffc50 2023-06-23T18:21:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d019da50 con 0x7f50d00ffc50 2023-06-23T18:21:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50c6ffd700 1 -- 172.21.15.2:0/1726073485 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c000e3b0 con 0x7f50d00ffc50 2023-06-23T18:21:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.944+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d019df60 con 0x7f50d00ffc50 2023-06-23T18:21:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.945+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d0106050 con 0x7f50d00ffc50 2023-06-23T18:21:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.946+0000 7f50c6ffd700 1 -- 172.21.15.2:0/1726073485 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f50c0010460 con 0x7f50d00ffc50 2023-06-23T18:21:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.947+0000 7f50c6ffd700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f50bc0a11e0 0x7f50bc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.947+0000 7f50d5bd4700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f50bc0a11e0 0x7f50bc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.947+0000 7f50c6ffd700 1 -- 172.21.15.2:0/1726073485 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f50c00cc670 con 0x7f50d00ffc50 2023-06-23T18:21:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.947+0000 7f50d5bd4700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f50bc0a11e0 0x7f50bc0a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f50d00ffab0 tx=0x7f50cc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:35.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:35.950+0000 7f50c6ffd700 1 -- 172.21.15.2:0/1726073485 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f50c0095e10 con 0x7f50d00ffc50 2023-06-23T18:21:36.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.151+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50d002c9e0 con 0x7f50d00ffc50 2023-06-23T18:21:36.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.152+0000 7f50c6ffd700 1 -- 172.21.15.2:0/1726073485 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21940 (secure 0 0 0) 0x7f50c0020090 con 0x7f50d00ffc50 2023-06-23T18:21:36.157 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:36.157 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":82,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:20:45.435971+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:21:36.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f50bc0a11e0 msgr2=0x7f50bc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f50bc0a11e0 0x7f50bc0a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f50d00ffab0 tx=0x7f50cc00b410 comp rx=0 tx=0).stop 2023-06-23T18:21:36.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d00ffc50 msgr2=0x7f50d019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d00ffc50 0x7f50d019a620 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50c000d950 tx=0x7f50c000dc60 comp rx=0 tx=0).stop 2023-06-23T18:21:36.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 shutdown_connections 2023-06-23T18:21:36.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f50bc0a11e0 0x7f50bc0a3690 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50d01005e0 0x7f50d019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f50d00fea60 0x7f50d0068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 --2- 172.21.15.2:0/1726073485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50d00ffc50 0x7f50d019a620 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 >> 172.21.15.2:0/1726073485 conn(0x7f50d00fa0e0 msgr2=0x7f50d0103190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:36.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 shutdown_connections 2023-06-23T18:21:36.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.154+0000 7f50d7e38700 1 -- 172.21.15.2:0/1726073485 wait complete. 2023-06-23T18:21:36.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:21:36.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:21:36.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.297+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/389019796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d40fea80 msgr2=0x7fe7d40fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.297+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/389019796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d40fea80 0x7fe7d40fee90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009990 tx=0x7fe7c4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.297+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/389019796 shutdown_connections 2023-06-23T18:21:36.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.297+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/389019796 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d4100600 0x7fe7d4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.297+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/389019796 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d40fea80 0x7fe7d40fee90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.297+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/389019796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d40ffc70 0x7fe7d41000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.297+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/389019796 >> 172.21.15.2:0/389019796 conn(0x7fe7d40fa0e0 msgr2=0x7fe7d40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:36.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/389019796 shutdown_connections 2023-06-23T18:21:36.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/389019796 wait complete. 2023-06-23T18:21:36.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 Processor -- start 2023-06-23T18:21:36.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 -- start start 2023-06-23T18:21:36.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d40fea80 0x7fe7d4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:36.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d40ffc70 0x7fe7d4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:36.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d4100600 0x7fe7d419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:36.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d419f5b0 con 0x7fe7d4100600 2023-06-23T18:21:36.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d419f6f0 con 0x7fe7d40ffc70 2023-06-23T18:21:36.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.298+0000 7fe7d9d1e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d419f880 con 0x7fe7d40fea80 2023-06-23T18:21:36.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d4100600 0x7fe7d419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:36.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d4100600 0x7fe7d419efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53072/0 (socket says 172.21.15.2:53072) 2023-06-23T18:21:36.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 -- 172.21.15.2:0/1542648290 learned_addr learned my addr 172.21.15.2:0/1542648290 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:36.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d37fe700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d40fea80 0x7fe7d4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:36.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d2ffd700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d40ffc70 0x7fe7d4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:36.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 -- 172.21.15.2:0/1542648290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d40fea80 msgr2=0x7fe7d4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d40fea80 0x7fe7d4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 -- 172.21.15.2:0/1542648290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d40ffc70 msgr2=0x7fe7d4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d40ffc70 0x7fe7d4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 -- 172.21.15.2:0/1542648290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c40095f0 con 0x7fe7d4100600 2023-06-23T18:21:36.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d2ffd700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d40ffc70 0x7fe7d4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:36.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d3fff700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d4100600 0x7fe7d419efe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00b970 tx=0x7fe7cc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:36.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d0ff9700 1 -- 172.21.15.2:0/1542648290 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00c930 con 0x7fe7d4100600 2023-06-23T18:21:36.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d0ff9700 1 -- 172.21.15.2:0/1542648290 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe7cc014440 con 0x7fe7d4100600 2023-06-23T18:21:36.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d419fb60 con 0x7fe7d4100600 2023-06-23T18:21:36.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d0ff9700 1 -- 172.21.15.2:0/1542648290 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0075d0 con 0x7fe7d4100600 2023-06-23T18:21:36.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.299+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d41a0070 con 0x7fe7d4100600 2023-06-23T18:21:36.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.300+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d410a180 con 0x7fe7d4100600 2023-06-23T18:21:36.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.301+0000 7fe7d0ff9700 1 -- 172.21.15.2:0/1542648290 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe7cc0145b0 con 0x7fe7d4100600 2023-06-23T18:21:36.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.302+0000 7fe7d0ff9700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe7bc0a1260 0x7fe7bc0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:36.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.302+0000 7fe7d0ff9700 1 -- 172.21.15.2:0/1542648290 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe7cc006e40 con 0x7fe7d4100600 2023-06-23T18:21:36.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.302+0000 7fe7d37fe700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe7bc0a1260 0x7fe7bc0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:36.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.302+0000 7fe7d37fe700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe7bc0a1260 0x7fe7bc0a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400b5c0 tx=0x7fe7c4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:36.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.305+0000 7fe7d0ff9700 1 -- 172.21.15.2:0/1542648290 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe7cc096040 con 0x7fe7d4100600 2023-06-23T18:21:36.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:36 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3304666591' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-06-23T18:21:36.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:36 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3304666591' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-06-23T18:21:36.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.503+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7d41a06e0 con 0x7fe7d4100600 2023-06-23T18:21:36.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.503+0000 7fe7d0ff9700 1 -- 172.21.15.2:0/1542648290 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7cc019020 con 0x7fe7d4100600 2023-06-23T18:21:36.508 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:21:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe7bc0a1260 msgr2=0x7fe7bc0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe7bc0a1260 0x7fe7bc0a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400b5c0 tx=0x7fe7c4005960 comp rx=0 tx=0).stop 2023-06-23T18:21:36.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d4100600 msgr2=0x7fe7d419efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d4100600 0x7fe7d419efe0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00b970 tx=0x7fe7cc00bc80 comp rx=0 tx=0).stop 2023-06-23T18:21:36.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 shutdown_connections 2023-06-23T18:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d4100600 0x7fe7d419efe0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7d40fea80 0x7fe7d4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe7bc0a1260 0x7fe7bc0a3710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 --2- 172.21.15.2:0/1542648290 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d40ffc70 0x7fe7d4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 >> 172.21.15.2:0/1542648290 conn(0x7fe7d40fa0e0 msgr2=0x7fe7d40677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 shutdown_connections 2023-06-23T18:21:36.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.506+0000 7fe7d9d1e700 1 -- 172.21.15.2:0/1542648290 wait complete. 2023-06-23T18:21:36.522 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:36 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3304666591' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-06-23T18:21:36.523 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:21:36.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3290460403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 msgr2=0x7f1c240feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3290460403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 0x7f1c240feeb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009990 tx=0x7f1c18009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3290460403 shutdown_connections 2023-06-23T18:21:36.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3290460403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c24100620 0x7f1c24106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3290460403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 0x7f1c240feeb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3290460403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c240ffc90 0x7f1c241000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3290460403 >> 172.21.15.2:0/3290460403 conn(0x7f1c240fa0c0 msgr2=0x7f1c240fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:36.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3290460403 shutdown_connections 2023-06-23T18:21:36.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3290460403 wait complete. 2023-06-23T18:21:36.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 Processor -- start 2023-06-23T18:21:36.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.648+0000 7f1c2bf37700 1 -- start start 2023-06-23T18:21:36.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2bf37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 0x7f1c241957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:36.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2bf37700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c240ffc90 0x7f1c24195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:36.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2bf37700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c24100620 0x7f1c2419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:36.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2bf37700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2419d350 con 0x7f1c240feaa0 2023-06-23T18:21:36.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2bf37700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2419d490 con 0x7f1c240ffc90 2023-06-23T18:21:36.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2bf37700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2419d620 con 0x7f1c24100620 2023-06-23T18:21:36.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 0x7f1c241957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:36.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 0x7f1c241957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53082/0 (socket says 172.21.15.2:53082) 2023-06-23T18:21:36.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 -- 172.21.15.2:0/3103504598 learned_addr learned my addr 172.21.15.2:0/3103504598 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:36.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c294d2700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c240ffc90 0x7f1c24195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:36.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2a4d4700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c24100620 0x7f1c2419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:36.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 -- 172.21.15.2:0/3103504598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c24100620 msgr2=0x7f1c2419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c24100620 0x7f1c2419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 -- 172.21.15.2:0/3103504598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c240ffc90 msgr2=0x7f1c24195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c240ffc90 0x7f1c24195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 -- 172.21.15.2:0/3103504598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180095f0 con 0x7f1c240feaa0 2023-06-23T18:21:36.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c29cd3700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 0x7f1c241957d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c18000c00 tx=0x7f1c180047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:36.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c16ffd700 1 -- 172.21.15.2:0/3103504598 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1801d070 con 0x7f1c240feaa0 2023-06-23T18:21:36.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c16ffd700 1 -- 172.21.15.2:0/3103504598 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c1800be50 con 0x7f1c240feaa0 2023-06-23T18:21:36.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c16ffd700 1 -- 172.21.15.2:0/3103504598 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1800f460 con 0x7f1c240feaa0 2023-06-23T18:21:36.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2419d8a0 con 0x7f1c240feaa0 2023-06-23T18:21:36.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.649+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2419dc70 con 0x7f1c240feaa0 2023-06-23T18:21:36.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.650+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c24106c20 con 0x7f1c240feaa0 2023-06-23T18:21:36.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.651+0000 7f1c16ffd700 1 -- 172.21.15.2:0/3103504598 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1c18022670 con 0x7f1c240feaa0 2023-06-23T18:21:36.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.652+0000 7f1c16ffd700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c100a11e0 0x7f1c100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:36.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.652+0000 7f1c294d2700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c100a11e0 0x7f1c100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:36.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.652+0000 7f1c16ffd700 1 -- 172.21.15.2:0/3103504598 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1c180ce2e0 con 0x7f1c240feaa0 2023-06-23T18:21:36.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.653+0000 7f1c294d2700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c100a11e0 0x7f1c100a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009c60 tx=0x7f1c20009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:36.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.656+0000 7f1c16ffd700 1 -- 172.21.15.2:0/3103504598 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1c18097b10 con 0x7f1c240feaa0 2023-06-23T18:21:36.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.857+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c2402c9a0 con 0x7f1c240feaa0 2023-06-23T18:21:36.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.858+0000 7f1c16ffd700 1 -- 172.21.15.2:0/3103504598 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c1809b130 con 0x7f1c240feaa0 2023-06-23T18:21:36.863 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:21:36.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c100a11e0 msgr2=0x7f1c100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c100a11e0 0x7f1c100a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009c60 tx=0x7f1c20009260 comp rx=0 tx=0).stop 2023-06-23T18:21:36.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 msgr2=0x7f1c241957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:36.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 0x7f1c241957d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1c18000c00 tx=0x7f1c180047a0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 shutdown_connections 2023-06-23T18:21:36.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c24100620 0x7f1c2419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c240feaa0 0x7f1c241957d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c100a11e0 0x7f1c100a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 --2- 172.21.15.2:0/3103504598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c240ffc90 0x7f1c24195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:36.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 >> 172.21.15.2:0/3103504598 conn(0x7f1c240fa0c0 msgr2=0x7f1c240fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:36.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 shutdown_connections 2023-06-23T18:21:36.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:36.860+0000 7f1c2bf37700 1 -- 172.21.15.2:0/3103504598 wait complete. 2023-06-23T18:21:36.877 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:21:37.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 -- 172.21.15.2:0/130284511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 msgr2=0x7f71fc0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 --2- 172.21.15.2:0/130284511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 0x7f71fc0fffa0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009990 tx=0x7f71f8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 -- 172.21.15.2:0/130284511 shutdown_connections 2023-06-23T18:21:37.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 --2- 172.21.15.2:0/130284511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71fc101710 0x7f71fc107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 --2- 172.21.15.2:0/130284511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 0x7f71fc0fffa0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 --2- 172.21.15.2:0/130284511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71fc100d80 0x7f71fc1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 -- 172.21.15.2:0/130284511 >> 172.21.15.2:0/130284511 conn(0x7f71fc0fb1b0 msgr2=0x7f71fc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:37.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 -- 172.21.15.2:0/130284511 shutdown_connections 2023-06-23T18:21:37.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.001+0000 7f7204419700 1 -- 172.21.15.2:0/130284511 wait complete. 2023-06-23T18:21:37.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f7204419700 1 Processor -- start 2023-06-23T18:21:37.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f7204419700 1 -- start start 2023-06-23T18:21:37.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f7204419700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 0x7f71fc074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f7204419700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71fc100d80 0x7f71fc0728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f7204419700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71fc101710 0x7f71fc072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f7204419700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0735a0 con 0x7f71fc0ffb90 2023-06-23T18:21:37.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f7204419700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc073710 con 0x7f71fc101710 2023-06-23T18:21:37.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f7204419700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc19f830 con 0x7f71fc100d80 2023-06-23T18:21:37.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 0x7f71fc074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 0x7f71fc074280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53096/0 (socket says 172.21.15.2:53096) 2023-06-23T18:21:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 -- 172.21.15.2:0/237789699 learned_addr learned my addr 172.21.15.2:0/237789699 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72029b6700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71fc101710 0x7f71fc072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72019b4700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71fc100d80 0x7f71fc0728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 -- 172.21.15.2:0/237789699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71fc100d80 msgr2=0x7f71fc0728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71fc100d80 0x7f71fc0728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 -- 172.21.15.2:0/237789699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71fc101710 msgr2=0x7f71fc072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71fc101710 0x7f71fc072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 -- 172.21.15.2:0/237789699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f80095f0 con 0x7f71fc0ffb90 2023-06-23T18:21:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.002+0000 7f72021b5700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 0x7f71fc074280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f71f800b5c0 tx=0x7f71f8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.003+0000 7f71f37fe700 1 -- 172.21.15.2:0/237789699 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f801d070 con 0x7f71fc0ffb90 2023-06-23T18:21:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.003+0000 7f71f37fe700 1 -- 172.21.15.2:0/237789699 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71f8003ec0 con 0x7f71fc0ffb90 2023-06-23T18:21:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.003+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc19fab0 con 0x7f71fc0ffb90 2023-06-23T18:21:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.003+0000 7f71f37fe700 1 -- 172.21.15.2:0/237789699 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8022be0 con 0x7f71fc0ffb90 2023-06-23T18:21:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.003+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc19fed0 con 0x7f71fc0ffb90 2023-06-23T18:21:37.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.004+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71fc106b10 con 0x7f71fc0ffb90 2023-06-23T18:21:37.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.005+0000 7f71f37fe700 1 -- 172.21.15.2:0/237789699 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f71f8004030 con 0x7f71fc0ffb90 2023-06-23T18:21:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.006+0000 7f71f37fe700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71e80a11e0 0x7f71e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.006+0000 7f72019b4700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71e80a11e0 0x7f71e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.006+0000 7f71f37fe700 1 -- 172.21.15.2:0/237789699 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f71f80ce620 con 0x7f71fc0ffb90 2023-06-23T18:21:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.006+0000 7f72019b4700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71e80a11e0 0x7f71e80a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005950 tx=0x7f71ec0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.009+0000 7f71f37fe700 1 -- 172.21.15.2:0/237789699 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f71f8097e50 con 0x7f71fc0ffb90 2023-06-23T18:21:37.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.214+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71fc02ca60 con 0x7f71fc0ffb90 2023-06-23T18:21:37.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.214+0000 7f71f37fe700 1 -- 172.21.15.2:0/237789699 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f71f8027020 con 0x7f71fc0ffb90 2023-06-23T18:21:37.219 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:21:37.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71e80a11e0 msgr2=0x7f71e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71e80a11e0 0x7f71e80a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f71ec005950 tx=0x7f71ec0058e0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 msgr2=0x7f71fc074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 0x7f71fc074280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f71f800b5c0 tx=0x7f71f8005740 comp rx=0 tx=0).stop 2023-06-23T18:21:37.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 shutdown_connections 2023-06-23T18:21:37.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71e80a11e0 0x7f71e80a3690 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71fc101710 0x7f71fc072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71fc0ffb90 0x7f71fc074280 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 --2- 172.21.15.2:0/237789699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71fc100d80 0x7f71fc0728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 >> 172.21.15.2:0/237789699 conn(0x7f71fc0fb1b0 msgr2=0x7f71fc0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:37.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 shutdown_connections 2023-06-23T18:21:37.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.217+0000 7f7204419700 1 -- 172.21.15.2:0/237789699 wait complete. 2023-06-23T18:21:37.233 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1726073485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:37.233 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1542648290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:37.233 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-06-23T18:21:37.233 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-06-23T18:21:37.233 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-06-23T18:21:37.233 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-06-23T18:21:37.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-06-23T18:21:37.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-06-23T18:21:37.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2023-06-23T18:21:37.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-06-23T18:21:37.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-06-23T18:21:37.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:37.234 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:37 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3103504598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:37.235 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:21:37.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1726073485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1542648290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-06-23T18:21:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-06-23T18:21:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-06-23T18:21:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-06-23T18:21:37.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-06-23T18:21:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-06-23T18:21:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2023-06-23T18:21:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-06-23T18:21:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-06-23T18:21:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:37.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:37 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3103504598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:37.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 -- 172.21.15.2:0/4228592620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f78100e00 msgr2=0x7f2f78101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4228592620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f78100e00 0x7f2f78101250 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009990 tx=0x7f2f60009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 -- 172.21.15.2:0/4228592620 shutdown_connections 2023-06-23T18:21:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4228592620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f78101790 0x7f2f78108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4228592620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f780fde30 0x7f2f780fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4228592620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f78100e00 0x7f2f78101250 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 -- 172.21.15.2:0/4228592620 >> 172.21.15.2:0/4228592620 conn(0x7f2f780f9810 msgr2=0x7f2f780fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 -- 172.21.15.2:0/4228592620 shutdown_connections 2023-06-23T18:21:37.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.356+0000 7f2f7c950700 1 -- 172.21.15.2:0/4228592620 wait complete. 2023-06-23T18:21:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7c950700 1 Processor -- start 2023-06-23T18:21:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7c950700 1 -- start start 2023-06-23T18:21:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7c950700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f780fde30 0x7f2f781957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7c950700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f78100e00 0x7f2f78195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7c950700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f78101790 0x7f2f7819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7c950700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7819d350 con 0x7f2f780fde30 2023-06-23T18:21:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7c950700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7819d490 con 0x7f2f78101790 2023-06-23T18:21:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7c950700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7819d620 con 0x7f2f78100e00 2023-06-23T18:21:37.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f780fde30 0x7f2f781957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f780fde30 0x7f2f781957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53114/0 (socket says 172.21.15.2:53114) 2023-06-23T18:21:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7659c700 1 -- 172.21.15.2:0/4293626925 learned_addr learned my addr 172.21.15.2:0/4293626925 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f76d9d700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f78101790 0x7f2f7819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f75d9b700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f78100e00 0x7f2f78195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7659c700 1 -- 172.21.15.2:0/4293626925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f78100e00 msgr2=0x7f2f78195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7659c700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f78100e00 0x7f2f78195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7659c700 1 -- 172.21.15.2:0/4293626925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f78101790 msgr2=0x7f2f7819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7659c700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f78101790 0x7f2f7819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f7659c700 1 -- 172.21.15.2:0/4293626925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f600095f0 con 0x7f2f780fde30 2023-06-23T18:21:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.357+0000 7f2f75d9b700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f78100e00 0x7f2f78195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.358+0000 7f2f7659c700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f780fde30 0x7f2f781957d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800d980 tx=0x7f2f6800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.358+0000 7f2f6f7fe700 1 -- 172.21.15.2:0/4293626925 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6800de60 con 0x7f2f780fde30 2023-06-23T18:21:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.358+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7819d900 con 0x7f2f780fde30 2023-06-23T18:21:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.358+0000 7f2f6f7fe700 1 -- 172.21.15.2:0/4293626925 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f680044b0 con 0x7f2f780fde30 2023-06-23T18:21:37.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.358+0000 7f2f6f7fe700 1 -- 172.21.15.2:0/4293626925 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68003b40 con 0x7f2f780fde30 2023-06-23T18:21:37.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.358+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7819dee0 con 0x7f2f780fde30 2023-06-23T18:21:37.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.359+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f78106b40 con 0x7f2f780fde30 2023-06-23T18:21:37.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.360+0000 7f2f6f7fe700 1 -- 172.21.15.2:0/4293626925 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2f68003ca0 con 0x7f2f780fde30 2023-06-23T18:21:37.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.361+0000 7f2f6f7fe700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2f640a11e0 0x7f2f640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.361+0000 7f2f75d9b700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2f640a11e0 0x7f2f640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.361+0000 7f2f6f7fe700 1 -- 172.21.15.2:0/4293626925 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2f680ccb10 con 0x7f2f780fde30 2023-06-23T18:21:37.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.361+0000 7f2f75d9b700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2f640a11e0 0x7f2f640a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f600095c0 tx=0x7f2f60009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:37.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.365+0000 7f2f6f7fe700 1 -- 172.21.15.2:0/4293626925 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2f68096100 con 0x7f2f780fde30 2023-06-23T18:21:37.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1726073485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1542648290' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-06-23T18:21:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-06-23T18:21:37.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2023-06-23T18:21:37.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2023-06-23T18:21:37.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-06-23T18:21:37.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]: dispatch 2023-06-23T18:21:37.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2023-06-23T18:21:37.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2023-06-23T18:21:37.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2023-06-23T18:21:37.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: pgmap v282: 97 pgs: 97 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:37.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:37 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3103504598' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:37.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.610+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f7802c810 con 0x7f2f780fde30 2023-06-23T18:21:37.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.610+0000 7f2f6f7fe700 1 -- 172.21.15.2:0/4293626925 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7f2f68009940 con 0x7f2f780fde30 2023-06-23T18:21:37.615 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:37.615 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:21:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2f640a11e0 msgr2=0x7f2f640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2f640a11e0 0x7f2f640a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2f600095c0 tx=0x7f2f60009550 comp rx=0 tx=0).stop 2023-06-23T18:21:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f780fde30 msgr2=0x7f2f781957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f780fde30 0x7f2f781957d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800d980 tx=0x7f2f6800dd40 comp rx=0 tx=0).stop 2023-06-23T18:21:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 shutdown_connections 2023-06-23T18:21:37.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2f640a11e0 0x7f2f640a3690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f78101790 0x7f2f7819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f780fde30 0x7f2f781957d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 --2- 172.21.15.2:0/4293626925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2f78100e00 0x7f2f78195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.612+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 >> 172.21.15.2:0/4293626925 conn(0x7f2f780f9810 msgr2=0x7f2f780fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.613+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 shutdown_connections 2023-06-23T18:21:37.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.613+0000 7f2f7c950700 1 -- 172.21.15.2:0/4293626925 wait complete. 2023-06-23T18:21:37.619 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:21:37.630 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:21:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 -- 172.21.15.2:0/4024944988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40fea60 msgr2=0x7fd8d40fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/4024944988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40fea60 0x7fd8d40fee70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009990 tx=0x7fd8d0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 -- 172.21.15.2:0/4024944988 shutdown_connections 2023-06-23T18:21:37.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/4024944988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d41005e0 0x7fd8d4106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/4024944988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40fea60 0x7fd8d40fee70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/4024944988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8d40ffc50 0x7fd8d41000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 -- 172.21.15.2:0/4024944988 >> 172.21.15.2:0/4024944988 conn(0x7fd8d40fa0e0 msgr2=0x7fd8d40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 -- 172.21.15.2:0/4024944988 shutdown_connections 2023-06-23T18:21:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 -- 172.21.15.2:0/4024944988 wait complete. 2023-06-23T18:21:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 Processor -- start 2023-06-23T18:21:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 -- start start 2023-06-23T18:21:37.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d40fea60 0x7fd8d4068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40ffc50 0x7fd8d419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8d41005e0 0x7fd8d419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d419d290 con 0x7fd8d40ffc50 2023-06-23T18:21:37.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.752+0000 7fd8dc15c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d419d400 con 0x7fd8d41005e0 2023-06-23T18:21:37.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8dc15c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d419d5c0 con 0x7fd8d40fea60 2023-06-23T18:21:37.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40ffc50 0x7fd8d419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40ffc50 0x7fd8d419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53130/0 (socket says 172.21.15.2:53130) 2023-06-23T18:21:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 -- 172.21.15.2:0/2498308621 learned_addr learned my addr 172.21.15.2:0/2498308621 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d9ef8700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d40fea60 0x7fd8d4068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8da6f9700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8d41005e0 0x7fd8d419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 -- 172.21.15.2:0/2498308621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d40fea60 msgr2=0x7fd8d4068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d40fea60 0x7fd8d4068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 -- 172.21.15.2:0/2498308621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8d41005e0 msgr2=0x7fd8d419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8d41005e0 0x7fd8d419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 -- 172.21.15.2:0/2498308621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d00095f0 con 0x7fd8d40ffc50 2023-06-23T18:21:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8d96f7700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40ffc50 0x7fd8d419a4b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400c9b0 tx=0x7fd8c400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:37.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8caffd700 1 -- 172.21.15.2:0/2498308621 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4007b00 con 0x7fd8d40ffc50 2023-06-23T18:21:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8caffd700 1 -- 172.21.15.2:0/2498308621 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd8c4007c60 con 0x7fd8d40ffc50 2023-06-23T18:21:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d419d8a0 con 0x7fd8d40ffc50 2023-06-23T18:21:37.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8caffd700 1 -- 172.21.15.2:0/2498308621 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4018660 con 0x7fd8d40ffc50 2023-06-23T18:21:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.753+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d419ddb0 con 0x7fd8d40ffc50 2023-06-23T18:21:37.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.754+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d4105940 con 0x7fd8d40ffc50 2023-06-23T18:21:37.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.755+0000 7fd8caffd700 1 -- 172.21.15.2:0/2498308621 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd8c400f450 con 0x7fd8d40ffc50 2023-06-23T18:21:37.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.756+0000 7fd8caffd700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8c00a11e0 0x7fd8c00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:37.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.756+0000 7fd8d9ef8700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8c00a11e0 0x7fd8c00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:37.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.756+0000 7fd8caffd700 1 -- 172.21.15.2:0/2498308621 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd8c40ccb60 con 0x7fd8d40ffc50 2023-06-23T18:21:37.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.757+0000 7fd8d9ef8700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8c00a11e0 0x7fd8c00a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00098e0 tx=0x7fd8d000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:37.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.761+0000 7fd8caffd700 1 -- 172.21.15.2:0/2498308621 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd8c4096150 con 0x7fd8d40ffc50 2023-06-23T18:21:37.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.959+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8d402c810 con 0x7fd8d40ffc50 2023-06-23T18:21:37.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.960+0000 7fd8caffd700 1 -- 172.21.15.2:0/2498308621 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+22421 (secure 0 0 0) 0x7fd8c4099770 con 0x7fd8d40ffc50 2023-06-23T18:21:37.965 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:37.966 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":83,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:37.060840+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:21:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.962+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8c00a11e0 msgr2=0x7fd8c00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.962+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8c00a11e0 0x7fd8c00a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd8d00098e0 tx=0x7fd8d000b260 comp rx=0 tx=0).stop 2023-06-23T18:21:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.962+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40ffc50 msgr2=0x7fd8d419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:37.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40ffc50 0x7fd8d419a4b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd8c400c9b0 tx=0x7fd8c400cd70 comp rx=0 tx=0).stop 2023-06-23T18:21:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 shutdown_connections 2023-06-23T18:21:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd8c00a11e0 0x7fd8c00a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8d41005e0 0x7fd8d419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8d40fea60 0x7fd8d4068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 --2- 172.21.15.2:0/2498308621 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd8d40ffc50 0x7fd8d419a4b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 >> 172.21.15.2:0/2498308621 conn(0x7fd8d40fa0e0 msgr2=0x7fd8d40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 shutdown_connections 2023-06-23T18:21:37.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:37.963+0000 7fd8dc15c700 1 -- 172.21.15.2:0/2498308621 wait complete. 2023-06-23T18:21:37.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:21:37.981 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:21:38.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 -- 172.21.15.2:0/4132735649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0ffd10 msgr2=0x7fae9c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 --2- 172.21.15.2:0/4132735649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0ffd10 0x7fae9c100180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fae90009990 tx=0x7fae90009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 -- 172.21.15.2:0/4132735649 shutdown_connections 2023-06-23T18:21:38.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 --2- 172.21.15.2:0/4132735649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9c1006c0 0x7fae9c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 --2- 172.21.15.2:0/4132735649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9c0fea80 0x7fae9c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 --2- 172.21.15.2:0/4132735649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0ffd10 0x7fae9c100180 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 -- 172.21.15.2:0/4132735649 >> 172.21.15.2:0/4132735649 conn(0x7fae9c0fa0c0 msgr2=0x7fae9c0fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:38.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 -- 172.21.15.2:0/4132735649 shutdown_connections 2023-06-23T18:21:38.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.110+0000 7faea4515700 1 -- 172.21.15.2:0/4132735649 wait complete. 2023-06-23T18:21:38.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea4515700 1 Processor -- start 2023-06-23T18:21:38.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea4515700 1 -- start start 2023-06-23T18:21:38.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea4515700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0fea80 0x7fae9c102e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea4515700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9c0ffd10 0x7fae9c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea4515700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9c1006c0 0x7fae9c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea4515700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c102130 con 0x7fae9c0fea80 2023-06-23T18:21:38.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea4515700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c1022a0 con 0x7fae9c0ffd10 2023-06-23T18:21:38.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea4515700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c19f850 con 0x7fae9c1006c0 2023-06-23T18:21:38.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0fea80 0x7fae9c102e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0fea80 0x7fae9c102e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53142/0 (socket says 172.21.15.2:53142) 2023-06-23T18:21:38.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 -- 172.21.15.2:0/639705758 learned_addr learned my addr 172.21.15.2:0/639705758 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:38.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea2ab2700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9c1006c0 0x7fae9c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea1ab0700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9c0ffd10 0x7fae9c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 -- 172.21.15.2:0/639705758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9c1006c0 msgr2=0x7fae9c101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9c1006c0 0x7fae9c101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 -- 172.21.15.2:0/639705758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9c0ffd10 msgr2=0x7fae9c101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9c0ffd10 0x7fae9c101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 -- 172.21.15.2:0/639705758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae900095f0 con 0x7fae9c0fea80 2023-06-23T18:21:38.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.111+0000 7faea22b1700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0fea80 0x7fae9c102e10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9a0 tx=0x7fae9800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:38.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.112+0000 7fae8f7fe700 1 -- 172.21.15.2:0/639705758 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98009790 con 0x7fae9c0fea80 2023-06-23T18:21:38.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.112+0000 7fae8f7fe700 1 -- 172.21.15.2:0/639705758 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae98010430 con 0x7fae9c0fea80 2023-06-23T18:21:38.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.112+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c19fb30 con 0x7fae9c0fea80 2023-06-23T18:21:38.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.112+0000 7fae8f7fe700 1 -- 172.21.15.2:0/639705758 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9800cea0 con 0x7fae9c0fea80 2023-06-23T18:21:38.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.112+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c1a0090 con 0x7fae9c0fea80 2023-06-23T18:21:38.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.113+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c106c50 con 0x7fae9c0fea80 2023-06-23T18:21:38.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.114+0000 7fae8f7fe700 1 -- 172.21.15.2:0/639705758 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fae980105a0 con 0x7fae9c0fea80 2023-06-23T18:21:38.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.114+0000 7fae8f7fe700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae880a1260 0x7fae880a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.114+0000 7fae8f7fe700 1 -- 172.21.15.2:0/639705758 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae98014070 con 0x7fae9c0fea80 2023-06-23T18:21:38.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.114+0000 7faea1ab0700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae880a1260 0x7fae880a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.115+0000 7faea1ab0700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae880a1260 0x7fae880a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae90006010 tx=0x7fae9000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:38.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.119+0000 7fae8f7fe700 1 -- 172.21.15.2:0/639705758 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fae98051770 con 0x7fae9c0fea80 2023-06-23T18:21:38.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-06-23T18:21:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-06-23T18:21:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-06-23T18:21:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-06-23T18:21:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-06-23T18:21:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-06-23T18:21:38.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2023-06-23T18:21:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-06-23T18:21:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-06-23T18:21:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: osdmap e83: 12 total, 12 up, 12 in 2023-06-23T18:21:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/237789699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:21:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/4293626925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:38.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:38 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2498308621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:38.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.316+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae9c02ca70 con 0x7fae9c0fea80 2023-06-23T18:21:38.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.316+0000 7fae8f7fe700 1 -- 172.21.15.2:0/639705758 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae980961f0 con 0x7fae9c0fea80 2023-06-23T18:21:38.321 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:21:38.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae880a1260 msgr2=0x7fae880a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae880a1260 0x7fae880a3710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fae90006010 tx=0x7fae9000b260 comp rx=0 tx=0).stop 2023-06-23T18:21:38.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0fea80 msgr2=0x7fae9c102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0fea80 0x7fae9c102e10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9a0 tx=0x7fae9800ecb0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 shutdown_connections 2023-06-23T18:21:38.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae9c1006c0 0x7fae9c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae9c0fea80 0x7fae9c102e10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae880a1260 0x7fae880a3710 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 --2- 172.21.15.2:0/639705758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae9c0ffd10 0x7fae9c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 >> 172.21.15.2:0/639705758 conn(0x7fae9c0fa0c0 msgr2=0x7fae9c0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:38.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 shutdown_connections 2023-06-23T18:21:38.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.318+0000 7faea4515700 1 -- 172.21.15.2:0/639705758 wait complete. 2023-06-23T18:21:38.335 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:21:38.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-06-23T18:21:38.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-06-23T18:21:38.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-06-23T18:21:38.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-06-23T18:21:38.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-06-23T18:21:38.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-06-23T18:21:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2023-06-23T18:21:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-06-23T18:21:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-06-23T18:21:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: osdmap e83: 12 total, 12 up, 12 in 2023-06-23T18:21:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/237789699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:21:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/4293626925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:38.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:38 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2498308621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:38.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-06-23T18:21:38.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-06-23T18:21:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2023-06-23T18:21:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2023-06-23T18:21:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-06-23T18:21:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2023-06-23T18:21:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2023-06-23T18:21:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2023-06-23T18:21:38.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2023-06-23T18:21:38.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: osdmap e83: 12 total, 12 up, 12 in 2023-06-23T18:21:38.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/237789699' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:21:38.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/4293626925' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:38.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:38 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2498308621' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:38.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 -- 172.21.15.2:0/347638355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 msgr2=0x7f592c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 --2- 172.21.15.2:0/347638355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 0x7f592c101260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f591c009990 tx=0x7f591c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 -- 172.21.15.2:0/347638355 shutdown_connections 2023-06-23T18:21:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 --2- 172.21.15.2:0/347638355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f592c1017a0 0x7f592c108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 --2- 172.21.15.2:0/347638355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f592c0ffb60 0x7f592c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 --2- 172.21.15.2:0/347638355 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 0x7f592c101260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 -- 172.21.15.2:0/347638355 >> 172.21.15.2:0/347638355 conn(0x7f592c0fb180 msgr2=0x7f592c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 -- 172.21.15.2:0/347638355 shutdown_connections 2023-06-23T18:21:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.459+0000 7f593470d700 1 -- 172.21.15.2:0/347638355 wait complete. 2023-06-23T18:21:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f593470d700 1 Processor -- start 2023-06-23T18:21:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f593470d700 1 -- start start 2023-06-23T18:21:38.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f593470d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f592c0ffb60 0x7f592c074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f593470d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 0x7f592c0728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f593470d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f592c1017a0 0x7f592c072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f593470d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c0735a0 con 0x7f592c100df0 2023-06-23T18:21:38.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f593470d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c073710 con 0x7f592c1017a0 2023-06-23T18:21:38.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f593470d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c19f840 con 0x7f592c0ffb60 2023-06-23T18:21:38.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5931ca8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 0x7f592c0728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5931ca8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 0x7f592c0728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53158/0 (socket says 172.21.15.2:53158) 2023-06-23T18:21:38.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5931ca8700 1 -- 172.21.15.2:0/1558892100 learned_addr learned my addr 172.21.15.2:0/1558892100 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:38.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f59324a9700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f592c0ffb60 0x7f592c074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5932caa700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f592c1017a0 0x7f592c072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5931ca8700 1 -- 172.21.15.2:0/1558892100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f592c0ffb60 msgr2=0x7f592c074280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5931ca8700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f592c0ffb60 0x7f592c074280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5931ca8700 1 -- 172.21.15.2:0/1558892100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f592c1017a0 msgr2=0x7f592c072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5931ca8700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f592c1017a0 0x7f592c072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.460+0000 7f5931ca8700 1 -- 172.21.15.2:0/1558892100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0095f0 con 0x7f592c100df0 2023-06-23T18:21:38.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.461+0000 7f5931ca8700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 0x7f592c0728c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f591c0059d0 tx=0x7f591c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:38.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.461+0000 7f59237fe700 1 -- 172.21.15.2:0/1558892100 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c01d070 con 0x7f592c100df0 2023-06-23T18:21:38.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.461+0000 7f59237fe700 1 -- 172.21.15.2:0/1558892100 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f591c00f460 con 0x7f592c100df0 2023-06-23T18:21:38.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.461+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c19fac0 con 0x7f592c100df0 2023-06-23T18:21:38.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.461+0000 7f59237fe700 1 -- 172.21.15.2:0/1558892100 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c021670 con 0x7f592c100df0 2023-06-23T18:21:38.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.461+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c19ffc0 con 0x7f592c100df0 2023-06-23T18:21:38.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.462+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c10b1e0 con 0x7f592c100df0 2023-06-23T18:21:38.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.463+0000 7f59237fe700 1 -- 172.21.15.2:0/1558892100 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f591c00f5d0 con 0x7f592c100df0 2023-06-23T18:21:38.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.464+0000 7f59237fe700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f59180a1260 0x7f59180a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.464+0000 7f59324a9700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f59180a1260 0x7f59180a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.464+0000 7f59237fe700 1 -- 172.21.15.2:0/1558892100 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f591c0cdd80 con 0x7f592c100df0 2023-06-23T18:21:38.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.464+0000 7f59324a9700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f59180a1260 0x7f59180a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5928009c60 tx=0x7f5928009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:38.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.467+0000 7f59237fe700 1 -- 172.21.15.2:0/1558892100 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f591c0973c0 con 0x7f592c100df0 2023-06-23T18:21:38.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.668+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f592c106bd0 con 0x7f592c100df0 2023-06-23T18:21:38.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.668+0000 7f59237fe700 1 -- 172.21.15.2:0/1558892100 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f591c09a9e0 con 0x7f592c100df0 2023-06-23T18:21:38.673 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:21:38.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.670+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f59180a1260 msgr2=0x7f59180a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.670+0000 7f593470d700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f59180a1260 0x7f59180a3710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5928009c60 tx=0x7f5928009260 comp rx=0 tx=0).stop 2023-06-23T18:21:38.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.670+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 msgr2=0x7f592c0728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.670+0000 7f593470d700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 0x7f592c0728c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f591c0059d0 tx=0x7f591c004790 comp rx=0 tx=0).stop 2023-06-23T18:21:38.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.671+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 shutdown_connections 2023-06-23T18:21:38.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.671+0000 7f593470d700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f59180a1260 0x7f59180a3710 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.671+0000 7f593470d700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f592c1017a0 0x7f592c072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.671+0000 7f593470d700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f592c0ffb60 0x7f592c074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.671+0000 7f593470d700 1 --2- 172.21.15.2:0/1558892100 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f592c100df0 0x7f592c0728c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.671+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 >> 172.21.15.2:0/1558892100 conn(0x7f592c0fb180 msgr2=0x7f592c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:38.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.671+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 shutdown_connections 2023-06-23T18:21:38.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.671+0000 7f593470d700 1 -- 172.21.15.2:0/1558892100 wait complete. 2023-06-23T18:21:38.687 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:21:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 -- 172.21.15.2:0/1655182867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef40fea80 msgr2=0x7f8ef40fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/1655182867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef40fea80 0x7f8ef40fee90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009990 tx=0x7f8ee8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 -- 172.21.15.2:0/1655182867 shutdown_connections 2023-06-23T18:21:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/1655182867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef4100600 0x7f8ef4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/1655182867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef40fea80 0x7f8ef40fee90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/1655182867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40ffc70 0x7f8ef41000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 -- 172.21.15.2:0/1655182867 >> 172.21.15.2:0/1655182867 conn(0x7f8ef40fa0e0 msgr2=0x7f8ef40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 -- 172.21.15.2:0/1655182867 shutdown_connections 2023-06-23T18:21:38.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 -- 172.21.15.2:0/1655182867 wait complete. 2023-06-23T18:21:38.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 Processor -- start 2023-06-23T18:21:38.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.812+0000 7f8ef8f81700 1 -- start start 2023-06-23T18:21:38.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef8f81700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40fea80 0x7f8ef41958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef8f81700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40ffc70 0x7f8ef4195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef8f81700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef4100600 0x7f8ef419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef8f81700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef419d430 con 0x7f8ef4100600 2023-06-23T18:21:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef8f81700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef419d570 con 0x7f8ef40ffc70 2023-06-23T18:21:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef8f81700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef419d700 con 0x7f8ef40fea80 2023-06-23T18:21:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef4100600 0x7f8ef419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef4100600 0x7f8ef419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53174/0 (socket says 172.21.15.2:53174) 2023-06-23T18:21:38.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40fea80 0x7f8ef41958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef2d9d700 1 -- 172.21.15.2:0/476734020 learned_addr learned my addr 172.21.15.2:0/476734020 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef1d9b700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40ffc70 0x7f8ef4195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef2d9d700 1 -- 172.21.15.2:0/476734020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40fea80 msgr2=0x7f8ef41958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef2d9d700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40fea80 0x7f8ef41958b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef2d9d700 1 -- 172.21.15.2:0/476734020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40ffc70 msgr2=0x7f8ef4195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef2d9d700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40ffc70 0x7f8ef4195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef2d9d700 1 -- 172.21.15.2:0/476734020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80095f0 con 0x7f8ef4100600 2023-06-23T18:21:38.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.813+0000 7f8ef1d9b700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40ffc70 0x7f8ef4195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:38.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.814+0000 7f8ef2d9d700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef4100600 0x7f8ef419ce60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00df60 tx=0x7f8eec00df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:38.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.814+0000 7f8ee37fe700 1 -- 172.21.15.2:0/476734020 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec00b2e0 con 0x7f8ef4100600 2023-06-23T18:21:38.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.814+0000 7f8ee37fe700 1 -- 172.21.15.2:0/476734020 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8eec00dce0 con 0x7f8ef4100600 2023-06-23T18:21:38.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.814+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef4101270 con 0x7f8ef4100600 2023-06-23T18:21:38.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.814+0000 7f8ee37fe700 1 -- 172.21.15.2:0/476734020 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec00be20 con 0x7f8ef4100600 2023-06-23T18:21:38.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.814+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef4101730 con 0x7f8ef4100600 2023-06-23T18:21:38.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.815+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef4106bd0 con 0x7f8ef4100600 2023-06-23T18:21:38.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.816+0000 7f8ee37fe700 1 -- 172.21.15.2:0/476734020 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8eec00d0d0 con 0x7f8ef4100600 2023-06-23T18:21:38.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.816+0000 7f8ee37fe700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8edc0a11e0 0x7f8edc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:38.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.817+0000 7f8ef259c700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8edc0a11e0 0x7f8edc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:38.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.817+0000 7f8ee37fe700 1 -- 172.21.15.2:0/476734020 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8eec006e40 con 0x7f8ef4100600 2023-06-23T18:21:38.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.817+0000 7f8ef259c700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8edc0a11e0 0x7f8edc0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c010 tx=0x7f8ee8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:38.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:38.820+0000 7f8ee37fe700 1 -- 172.21.15.2:0/476734020 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8eec0a6370 con 0x7f8ef4100600 2023-06-23T18:21:39.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.018+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ef402c9a0 con 0x7f8ef4100600 2023-06-23T18:21:39.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.019+0000 7f8ee37fe700 1 -- 172.21.15.2:0/476734020 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8eec0a9990 con 0x7f8ef4100600 2023-06-23T18:21:39.024 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:21:39.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8edc0a11e0 msgr2=0x7f8edc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8edc0a11e0 0x7f8edc0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c010 tx=0x7f8ee8005fb0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef4100600 msgr2=0x7f8ef419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef4100600 0x7f8ef419ce60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8eec00df60 tx=0x7f8eec00df90 comp rx=0 tx=0).stop 2023-06-23T18:21:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 shutdown_connections 2023-06-23T18:21:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8ef4100600 0x7f8ef419ce60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40fea80 0x7f8ef41958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8edc0a11e0 0x7f8edc0a3690 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 --2- 172.21.15.2:0/476734020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40ffc70 0x7f8ef4195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 >> 172.21.15.2:0/476734020 conn(0x7f8ef40fa0e0 msgr2=0x7f8ef40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 shutdown_connections 2023-06-23T18:21:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.021+0000 7f8ef8f81700 1 -- 172.21.15.2:0/476734020 wait complete. 2023-06-23T18:21:39.038 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:21:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 -- 172.21.15.2:0/2155029946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 msgr2=0x7f71c00feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 --2- 172.21.15.2:0/2155029946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 0x7f71c00feeb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009990 tx=0x7f71b4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 -- 172.21.15.2:0/2155029946 shutdown_connections 2023-06-23T18:21:39.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 --2- 172.21.15.2:0/2155029946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0100620 0x7f71c0106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 --2- 172.21.15.2:0/2155029946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 0x7f71c00feeb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 --2- 172.21.15.2:0/2155029946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c00ffc90 0x7f71c01000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 -- 172.21.15.2:0/2155029946 >> 172.21.15.2:0/2155029946 conn(0x7f71c00fa0c0 msgr2=0x7f71c00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:39.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 -- 172.21.15.2:0/2155029946 shutdown_connections 2023-06-23T18:21:39.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 -- 172.21.15.2:0/2155029946 wait complete. 2023-06-23T18:21:39.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 Processor -- start 2023-06-23T18:21:39.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.161+0000 7f71c6e16700 1 -- start start 2023-06-23T18:21:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c6e16700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 0x7f71c0102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c6e16700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c00ffc90 0x7f71c01013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c6e16700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0100620 0x7f71c0101910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c6e16700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c01020b0 con 0x7f71c00feaa0 2023-06-23T18:21:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c6e16700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c0102220 con 0x7f71c00ffc90 2023-06-23T18:21:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c6e16700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c019f840 con 0x7f71c0100620 2023-06-23T18:21:39.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 0x7f71c0102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 0x7f71c0102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53184/0 (socket says 172.21.15.2:53184) 2023-06-23T18:21:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 -- 172.21.15.2:0/885138841 learned_addr learned my addr 172.21.15.2:0/885138841 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c53b3700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0100620 0x7f71c0101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 -- 172.21.15.2:0/885138841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0100620 msgr2=0x7f71c0101910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0100620 0x7f71c0101910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 -- 172.21.15.2:0/885138841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c00ffc90 msgr2=0x7f71c01013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c00ffc90 0x7f71c01013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 -- 172.21.15.2:0/885138841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b40095f0 con 0x7f71c00feaa0 2023-06-23T18:21:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c53b3700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0100620 0x7f71c0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c4bb2700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 0x7f71c0102d90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f71b4000c00 tx=0x7f71b4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71bdffb700 1 -- 172.21.15.2:0/885138841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b401d070 con 0x7f71c00feaa0 2023-06-23T18:21:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71bdffb700 1 -- 172.21.15.2:0/885138841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71b40040b0 con 0x7f71c00feaa0 2023-06-23T18:21:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c019fac0 con 0x7f71c00feaa0 2023-06-23T18:21:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.162+0000 7f71bdffb700 1 -- 172.21.15.2:0/885138841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4022cb0 con 0x7f71c00feaa0 2023-06-23T18:21:39.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.163+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c019ff70 con 0x7f71c00feaa0 2023-06-23T18:21:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.163+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c0106bd0 con 0x7f71c00feaa0 2023-06-23T18:21:39.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.165+0000 7f71bdffb700 1 -- 172.21.15.2:0/885138841 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f71b4022470 con 0x7f71c00feaa0 2023-06-23T18:21:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.165+0000 7f71bdffb700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71b00a1260 0x7f71b00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.165+0000 7f71bffff700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71b00a1260 0x7f71b00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.165+0000 7f71bdffb700 1 -- 172.21.15.2:0/885138841 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f71b40ce6b0 con 0x7f71c00feaa0 2023-06-23T18:21:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.166+0000 7f71bffff700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71b00a1260 0x7f71b00a3710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005950 tx=0x7f71ac0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:39.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.169+0000 7f71bdffb700 1 -- 172.21.15.2:0/885138841 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f71b4097cf0 con 0x7f71c00feaa0 2023-06-23T18:21:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:39 smithi002 ceph-mon[95947]: osdmap e84: 12 total, 12 up, 12 in 2023-06-23T18:21:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:39 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/639705758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:39 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1558892100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:39 smithi002 ceph-mon[95947]: pgmap v285: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:39.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:39 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/476734020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:39 smithi082 ceph-mon[97889]: osdmap e84: 12 total, 12 up, 12 in 2023-06-23T18:21:39.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:39 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/639705758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:39 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1558892100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:39 smithi082 ceph-mon[97889]: pgmap v285: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:39.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:39 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/476734020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.419+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71c002c810 con 0x7f71c00feaa0 2023-06-23T18:21:39.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.420+0000 7f71bdffb700 1 -- 172.21.15.2:0/885138841 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7f71b409b310 con 0x7f71c00feaa0 2023-06-23T18:21:39.424 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:39.425 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:21:39.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71b00a1260 msgr2=0x7f71b00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71b00a1260 0x7f71b00a3710 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71ac005950 tx=0x7f71ac0058e0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 msgr2=0x7f71c0102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 0x7f71c0102d90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f71b4000c00 tx=0x7f71b4009f90 comp rx=0 tx=0).stop 2023-06-23T18:21:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 shutdown_connections 2023-06-23T18:21:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71c0100620 0x7f71c0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71c00feaa0 0x7f71c0102d90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f71b00a1260 0x7f71b00a3710 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 --2- 172.21.15.2:0/885138841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c00ffc90 0x7f71c01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 >> 172.21.15.2:0/885138841 conn(0x7f71c00fa0c0 msgr2=0x7f71c00fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:39.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 shutdown_connections 2023-06-23T18:21:39.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.422+0000 7f71c6e16700 1 -- 172.21.15.2:0/885138841 wait complete. 2023-06-23T18:21:39.428 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:21:39.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:21:39.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:39 smithi017 ceph-mon[97472]: osdmap e84: 12 total, 12 up, 12 in 2023-06-23T18:21:39.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:39 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/639705758' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:39 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1558892100' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:39 smithi017 ceph-mon[97472]: pgmap v285: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:39.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:39 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/476734020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:21:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.561+0000 7f9f4a587700 1 -- 172.21.15.2:0/278059740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440fe1f0 msgr2=0x7f9f440fe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.561+0000 7f9f4a587700 1 --2- 172.21.15.2:0/278059740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440fe1f0 0x7f9f440fe600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009990 tx=0x7f9f34009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.561+0000 7f9f4a587700 1 -- 172.21.15.2:0/278059740 shutdown_connections 2023-06-23T18:21:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.561+0000 7f9f4a587700 1 --2- 172.21.15.2:0/278059740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f440ffd70 0x7f9f44106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.561+0000 7f9f4a587700 1 --2- 172.21.15.2:0/278059740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440fe1f0 0x7f9f440fe600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.561+0000 7f9f4a587700 1 --2- 172.21.15.2:0/278059740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f440ff3e0 0x7f9f440ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.561+0000 7f9f4a587700 1 -- 172.21.15.2:0/278059740 >> 172.21.15.2:0/278059740 conn(0x7f9f440f9810 msgr2=0x7f9f440fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 -- 172.21.15.2:0/278059740 shutdown_connections 2023-06-23T18:21:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 -- 172.21.15.2:0/278059740 wait complete. 2023-06-23T18:21:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 Processor -- start 2023-06-23T18:21:39.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 -- start start 2023-06-23T18:21:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f440fe1f0 0x7f9f44194710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f440ff3e0 0x7f9f44194c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440ffd70 0x7f9f4419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4419c290 con 0x7f9f440ffd70 2023-06-23T18:21:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4419c3d0 con 0x7f9f440fe1f0 2023-06-23T18:21:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.562+0000 7f9f4a587700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4419c560 con 0x7f9f440ff3e0 2023-06-23T18:21:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440ffd70 0x7f9f4419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440ffd70 0x7f9f4419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53202/0 (socket says 172.21.15.2:53202) 2023-06-23T18:21:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 -- 172.21.15.2:0/1977739089 learned_addr learned my addr 172.21.15.2:0/1977739089 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f43fff700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f440fe1f0 0x7f9f44194710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f437fe700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f440ff3e0 0x7f9f44194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 -- 172.21.15.2:0/1977739089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f440ff3e0 msgr2=0x7f9f44194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f440ff3e0 0x7f9f44194c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 -- 172.21.15.2:0/1977739089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f440fe1f0 msgr2=0x7f9f44194710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f440fe1f0 0x7f9f44194710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 -- 172.21.15.2:0/1977739089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f340095f0 con 0x7f9f440ffd70 2023-06-23T18:21:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f48b24700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440ffd70 0x7f9f4419bcc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007ee0 tx=0x7f9f3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f417fa700 1 -- 172.21.15.2:0/1977739089 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c017070 con 0x7f9f440ffd70 2023-06-23T18:21:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f417fa700 1 -- 172.21.15.2:0/1977739089 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f3c00f040 con 0x7f9f440ffd70 2023-06-23T18:21:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f417fa700 1 -- 172.21.15.2:0/1977739089 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c005710 con 0x7f9f440ffd70 2023-06-23T18:21:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4419c840 con 0x7f9f440ffd70 2023-06-23T18:21:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.563+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4419cd00 con 0x7f9f440ffd70 2023-06-23T18:21:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.564+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44104dc0 con 0x7f9f440ffd70 2023-06-23T18:21:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.565+0000 7f9f417fa700 1 -- 172.21.15.2:0/1977739089 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9f3c005870 con 0x7f9f440ffd70 2023-06-23T18:21:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.566+0000 7f9f417fa700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f2c0a11e0 0x7f9f2c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.566+0000 7f9f417fa700 1 -- 172.21.15.2:0/1977739089 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9f3c0cc900 con 0x7f9f440ffd70 2023-06-23T18:21:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.566+0000 7f9f43fff700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f2c0a11e0 0x7f9f2c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.566+0000 7f9f43fff700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f2c0a11e0 0x7f9f2c0a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400b5c0 tx=0x7f9f34005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.569+0000 7f9f417fa700 1 -- 172.21.15.2:0/1977739089 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9f3c095ef0 con 0x7f9f440ffd70 2023-06-23T18:21:39.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.772+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f4402c810 con 0x7f9f440ffd70 2023-06-23T18:21:39.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.774+0000 7f9f417fa700 1 -- 172.21.15.2:0/1977739089 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f9f3c099510 con 0x7f9f440ffd70 2023-06-23T18:21:39.779 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:39.779 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:21:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f2c0a11e0 msgr2=0x7f9f2c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f2c0a11e0 0x7f9f2c0a3690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400b5c0 tx=0x7f9f34005960 comp rx=0 tx=0).stop 2023-06-23T18:21:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440ffd70 msgr2=0x7f9f4419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440ffd70 0x7f9f4419bcc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007ee0 tx=0x7f9f3c00d3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 shutdown_connections 2023-06-23T18:21:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f440ffd70 0x7f9f4419bcc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f2c0a11e0 0x7f9f2c0a3690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f440fe1f0 0x7f9f44194710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 --2- 172.21.15.2:0/1977739089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f440ff3e0 0x7f9f44194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 >> 172.21.15.2:0/1977739089 conn(0x7f9f440f9810 msgr2=0x7f9f440fb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 shutdown_connections 2023-06-23T18:21:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.776+0000 7f9f4a587700 1 -- 172.21.15.2:0/1977739089 wait complete. 2023-06-23T18:21:39.794 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:21:39.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 -- 172.21.15.2:0/1410611546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 msgr2=0x7f7b10106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 --2- 172.21.15.2:0/1410611546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 0x7f7b10106ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800b0a0 tx=0x7f7b0800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 -- 172.21.15.2:0/1410611546 shutdown_connections 2023-06-23T18:21:39.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 --2- 172.21.15.2:0/1410611546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 0x7f7b10106ea0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 --2- 172.21.15.2:0/1410611546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b100fea60 0x7f7b100fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 --2- 172.21.15.2:0/1410611546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b100ffc50 0x7f7b101000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 -- 172.21.15.2:0/1410611546 >> 172.21.15.2:0/1410611546 conn(0x7f7b100fa0e0 msgr2=0x7f7b100fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:39.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 -- 172.21.15.2:0/1410611546 shutdown_connections 2023-06-23T18:21:39.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.915+0000 7f7b17186700 1 -- 172.21.15.2:0/1410611546 wait complete. 2023-06-23T18:21:39.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b17186700 1 Processor -- start 2023-06-23T18:21:39.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b17186700 1 -- start start 2023-06-23T18:21:39.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b17186700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b100fea60 0x7f7b10068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b17186700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b100ffc50 0x7f7b1019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b17186700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 0x7f7b1019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b17186700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1019d380 con 0x7f7b101005e0 2023-06-23T18:21:39.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b17186700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1019d4f0 con 0x7f7b100fea60 2023-06-23T18:21:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b17186700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1019d6b0 con 0x7f7b100ffc50 2023-06-23T18:21:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 0x7f7b1019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 0x7f7b1019cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53216/0 (socket says 172.21.15.2:53216) 2023-06-23T18:21:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 -- 172.21.15.2:0/373086637 learned_addr learned my addr 172.21.15.2:0/373086637 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b14f22700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b100fea60 0x7f7b10068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b0ffff700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b100ffc50 0x7f7b1019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 -- 172.21.15.2:0/373086637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b100ffc50 msgr2=0x7f7b1019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b100ffc50 0x7f7b1019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 -- 172.21.15.2:0/373086637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b100fea60 msgr2=0x7f7b10068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b100fea60 0x7f7b10068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 -- 172.21.15.2:0/373086637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08009d00 con 0x7f7b101005e0 2023-06-23T18:21:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b14f22700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b100fea60 0x7f7b10068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.916+0000 7f7b15723700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 0x7f7b1019cbb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800b830 tx=0x7f7b08004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.917+0000 7f7b0dffb700 1 -- 172.21.15.2:0/373086637 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800e040 con 0x7f7b101005e0 2023-06-23T18:21:39.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.917+0000 7f7b0dffb700 1 -- 172.21.15.2:0/373086637 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7b08009440 con 0x7f7b101005e0 2023-06-23T18:21:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.917+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1019d930 con 0x7f7b101005e0 2023-06-23T18:21:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.917+0000 7f7b0dffb700 1 -- 172.21.15.2:0/373086637 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0801dbe0 con 0x7f7b101005e0 2023-06-23T18:21:39.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.917+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1019ddb0 con 0x7f7b101005e0 2023-06-23T18:21:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.918+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b101058c0 con 0x7f7b101005e0 2023-06-23T18:21:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.919+0000 7f7b0dffb700 1 -- 172.21.15.2:0/373086637 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7b08005450 con 0x7f7b101005e0 2023-06-23T18:21:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.919+0000 7f7b0dffb700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7b000a1260 0x7f7b000a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.920+0000 7f7b0dffb700 1 -- 172.21.15.2:0/373086637 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7b080ce470 con 0x7f7b101005e0 2023-06-23T18:21:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.920+0000 7f7b14f22700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7b000a1260 0x7f7b000a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:39.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.920+0000 7f7b14f22700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7b000a1260 0x7f7b000a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7b04005950 tx=0x7f7b040058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:39.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:39.924+0000 7f7b0dffb700 1 -- 172.21.15.2:0/373086637 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7b08097af0 con 0x7f7b101005e0 2023-06-23T18:21:40.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.168+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b1002c810 con 0x7f7b101005e0 2023-06-23T18:21:40.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.168+0000 7f7b0dffb700 1 -- 172.21.15.2:0/373086637 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7f7b0809b110 con 0x7f7b101005e0 2023-06-23T18:21:40.173 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:40.173 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:21:40.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7b000a1260 msgr2=0x7f7b000a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7b000a1260 0x7f7b000a3710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7b04005950 tx=0x7f7b040058e0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 msgr2=0x7f7b1019cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 0x7f7b1019cbb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800b830 tx=0x7f7b08004d90 comp rx=0 tx=0).stop 2023-06-23T18:21:40.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 shutdown_connections 2023-06-23T18:21:40.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7b101005e0 0x7f7b1019cbb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7b000a1260 0x7f7b000a3710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b100fea60 0x7f7b10068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 --2- 172.21.15.2:0/373086637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b100ffc50 0x7f7b1019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.170+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 >> 172.21.15.2:0/373086637 conn(0x7f7b100fa0e0 msgr2=0x7f7b100fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.171+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 shutdown_connections 2023-06-23T18:21:40.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.171+0000 7f7b17186700 1 -- 172.21.15.2:0/373086637 wait complete. 2023-06-23T18:21:40.177 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:21:40.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-06-23T18:20:44.434596+0000', 'modified': '2023-06-23T18:20:49.041282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24533, 'mds_1': 24526, 'mds_2': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6835/2425660103', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2425660103}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2425660103}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/2287539900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 2287539900}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 2287539900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6837/2388271800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6836', 'nonce': 2388271800}, {'type': 'v1', 'addr': '172.21.15.17:6837', 'nonce': 2388271800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.82:6837/2740251056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 2740251056}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2740251056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.82:6839/2781114494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2781114494}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2781114494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.17:6839/4283937693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6838', 'nonce': 4283937693}, {'type': 'v1', 'addr': '172.21.15.17:6839', 'nonce': 4283937693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-23T18:21:40.188 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-06-23T18:21:40.188 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-06-23T18:21:40.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.310+0000 7febe2edf700 1 -- 172.21.15.2:0/3493485354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc0ffc70 msgr2=0x7febdc1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.310+0000 7febe2edf700 1 --2- 172.21.15.2:0/3493485354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc0ffc70 0x7febdc1000c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7febc8009990 tx=0x7febc8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.310+0000 7febe2edf700 1 -- 172.21.15.2:0/3493485354 shutdown_connections 2023-06-23T18:21:40.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.310+0000 7febe2edf700 1 --2- 172.21.15.2:0/3493485354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc100600 0x7febdc106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 --2- 172.21.15.2:0/3493485354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febdc0fea80 0x7febdc0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 --2- 172.21.15.2:0/3493485354 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc0ffc70 0x7febdc1000c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 -- 172.21.15.2:0/3493485354 >> 172.21.15.2:0/3493485354 conn(0x7febdc0fa0e0 msgr2=0x7febdc0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 -- 172.21.15.2:0/3493485354 shutdown_connections 2023-06-23T18:21:40.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 -- 172.21.15.2:0/3493485354 wait complete. 2023-06-23T18:21:40.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 Processor -- start 2023-06-23T18:21:40.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 -- start start 2023-06-23T18:21:40.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febdc0fea80 0x7febdc195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc0ffc70 0x7febdc195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc100600 0x7febdc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc19d380 con 0x7febdc100600 2023-06-23T18:21:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc19d4c0 con 0x7febdc0fea80 2023-06-23T18:21:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe2edf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc19d650 con 0x7febdc0ffc70 2023-06-23T18:21:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe147c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc100600 0x7febdc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe147c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc100600 0x7febdc19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53230/0 (socket says 172.21.15.2:53230) 2023-06-23T18:21:40.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe147c700 1 -- 172.21.15.2:0/340848356 learned_addr learned my addr 172.21.15.2:0/340848356 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.311+0000 7febe0c7b700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febdc0fea80 0x7febdc195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febdbfff700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc0ffc70 0x7febdc195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febe147c700 1 -- 172.21.15.2:0/340848356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc0ffc70 msgr2=0x7febdc195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febe147c700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc0ffc70 0x7febdc195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febe147c700 1 -- 172.21.15.2:0/340848356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febdc0fea80 msgr2=0x7febdc195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febe147c700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febdc0fea80 0x7febdc195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febe147c700 1 -- 172.21.15.2:0/340848356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80095f0 con 0x7febdc100600 2023-06-23T18:21:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febe147c700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc100600 0x7febdc19cdb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febd4007ee0 tx=0x7febd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febd9ffb700 1 -- 172.21.15.2:0/340848356 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd400ddd0 con 0x7febdc100600 2023-06-23T18:21:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febd9ffb700 1 -- 172.21.15.2:0/340848356 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febd400f040 con 0x7febdc100600 2023-06-23T18:21:40.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febe2edf700 1 -- 172.21.15.2:0/340848356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc19d930 con 0x7febdc100600 2023-06-23T18:21:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febd9ffb700 1 -- 172.21.15.2:0/340848356 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd40047d0 con 0x7febdc100600 2023-06-23T18:21:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.312+0000 7febe2edf700 1 -- 172.21.15.2:0/340848356 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc19ddf0 con 0x7febdc100600 2023-06-23T18:21:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.314+0000 7febd9ffb700 1 -- 172.21.15.2:0/340848356 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7febd4003a00 con 0x7febdc100600 2023-06-23T18:21:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.315+0000 7febd9ffb700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7febcc0a11e0 0x7febcc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.315+0000 7febd9ffb700 1 -- 172.21.15.2:0/340848356 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7febd40cc440 con 0x7febdc100600 2023-06-23T18:21:40.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.315+0000 7febe0c7b700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7febcc0a11e0 0x7febcc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.315+0000 7febe0c7b700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7febcc0a11e0 0x7febcc0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7febdc0ffad0 tx=0x7febd000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febef064b80 1 Processor -- start 2023-06-23T18:21:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febef064b80 1 -- start start 2023-06-23T18:21:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febef064b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9f7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febef064b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6ea9f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febef064b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6ea9f8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febef064b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea8fbc60 con 0x55d6ea9f7580 2023-06-23T18:21:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febef064b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea94e9f0 con 0x55d6ea9f7fb0 2023-06-23T18:21:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febef064b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea947e80 con 0x55d6ea9f6c30 2023-06-23T18:21:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febdbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6ea9f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febdbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6ea9f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53236/0 (socket says 172.21.15.2:53236) 2023-06-23T18:21:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febdbfff700 1 -- 172.21.15.2:0/430267484 learned_addr learned my addr 172.21.15.2:0/430267484 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febe0c7b700 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.316+0000 7febe147c700 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6ea9f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febdbfff700 1 -- 172.21.15.2:0/430267484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f6c30 msgr2=0x55d6ea9f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febdbfff700 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febdbfff700 1 -- 172.21.15.2:0/430267484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 msgr2=0x55d6ea9f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febdbfff700 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6ea9f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febdbfff700 1 -- 172.21.15.2:0/430267484 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ea7c95e0 con 0x55d6ea9f7580 2023-06-23T18:21:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febe147c700 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6ea9f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febdbfff700 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6ea9f7a20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7febc8005c30 tx=0x7febc8005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcbc766e3f4ccc5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febc6ffd700 1 -- 172.21.15.2:0/430267484 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc800b9f0 con 0x55d6ea9f7580 2023-06-23T18:21:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febc6ffd700 1 -- 172.21.15.2:0/430267484 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febc801e040 con 0x55d6ea9f7580 2023-06-23T18:21:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.317+0000 7febc6ffd700 1 -- 172.21.15.2:0/430267484 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8004830 con 0x55d6ea9f7580 2023-06-23T18:21:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 -- 172.21.15.2:0/430267484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 msgr2=0x55d6ea9f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6ea9f7a20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7febc8005c30 tx=0x7febc8005c60 comp rx=0 tx=0).stop 2023-06-23T18:21:40.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 -- 172.21.15.2:0/430267484 shutdown_connections 2023-06-23T18:21:40.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6ea9f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 --2- 172.21.15.2:0/430267484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6ea9f7a20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 -- 172.21.15.2:0/430267484 >> 172.21.15.2:0/430267484 conn(0x55d6ea9ec3a0 msgr2=0x55d6ea9f14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 -- 172.21.15.2:0/430267484 shutdown_connections 2023-06-23T18:21:40.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 -- 172.21.15.2:0/430267484 wait complete. 2023-06-23T18:21:40.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 -- start start 2023-06-23T18:21:40.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 Processor -- start 2023-06-23T18:21:40.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.318+0000 7febef064b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa00480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6eaa009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa07a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea8e7a70 con 0x55d6ea9f7580 2023-06-23T18:21:40.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea8e7770 con 0x55d6ea9f6c30 2023-06-23T18:21:40.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea7cb8d0 con 0x55d6ea9f7fb0 2023-06-23T18:21:40.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 10 client.0 _ll_register_callbacks cb 0x55d6ea8ed210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:21:40.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6eaa009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6eaa009c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53242/0 (socket says 172.21.15.2:53242) 2023-06-23T18:21:40.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 -- 172.21.15.2:0/166254758 learned_addr learned my addr 172.21.15.2:0/166254758 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febe0c7b700 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa00480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febe147c700 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa07a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 -- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 msgr2=0x55d6eaa07a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa07a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 -- 172.21.15.2:0/166254758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f6c30 msgr2=0x55d6eaa00480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa00480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 -- 172.21.15.2:0/166254758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc1077f0 con 0x55d6ea9f7580 2023-06-23T18:21:40.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febe147c700 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa07a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:40.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febdbfff700 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6eaa009c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7febc8009990 tx=0x7febc80092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1e38a157b9d156 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:21:40.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febbbfff700 1 -- 172.21.15.2:0/166254758 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8003e60 con 0x55d6ea9f7580 2023-06-23T18:21:40.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febbbfff700 1 -- 172.21.15.2:0/166254758 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7febc8003fc0 con 0x55d6ea9f7580 2023-06-23T18:21:40.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 1 -- 172.21.15.2:0/166254758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6ea9e9b60 con 0x55d6ea9f7580 2023-06-23T18:21:40.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febbbfff700 1 -- 172.21.15.2:0/166254758 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7febc8004180 con 0x55d6ea9f7580 2023-06-23T18:21:40.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 10 client.15042 fetch_fsmap learned FSMap version 22 2023-06-23T18:21:40.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.319+0000 7febef064b80 1 -- 172.21.15.2:0/166254758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d6ea775d90 con 0x55d6ea9f7580 2023-06-23T18:21:40.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febbbfff700 1 -- 172.21.15.2:0/166254758 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febc803d020 con 0x55d6ea9f7580 2023-06-23T18:21:40.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febbbfff700 1 -- 172.21.15.2:0/166254758 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7febc8037520 con 0x55d6ea9f7580 2023-06-23T18:21:40.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febef064b80 10 client.15042 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:21:40.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febef064b80 10 client.15042 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-23T18:21:40.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febef064b80 20 client.15042 populate_metadata read hostname 'smithi002' 2023-06-23T18:21:40.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febef064b80 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55d6eaa0b1f0 0x55d6eaa0a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febef064b80 4 client.15042 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:21:40.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febef064b80 1 -- 172.21.15.2:0/166254758 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6ea940e60 con 0x55d6eaa0b1f0 2023-06-23T18:21:40.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.320+0000 7febe0c7b700 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55d6eaa0b1f0 0x55d6eaa0a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.321+0000 7febbbfff700 10 client.15042 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:21:40.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.321+0000 7febe0c7b700 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55d6eaa0b1f0 0x55d6eaa0a930 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eccfbcb1419ee335 server_cookie=46525c4879091530 in_seq=0 out_seq=0 2023-06-23T18:21:40.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febbbfff700 1 -- 172.21.15.2:0/166254758 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d6ea940e60 con 0x55d6eaa0b1f0 2023-06-23T18:21:40.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febbbfff700 10 client.15042 handle_command_reply: tid=0 2023-06-23T18:21:40.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 client.15042 shutdown 2023-06-23T18:21:40.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 -- 172.21.15.2:0/166254758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 msgr2=0x55d6eaa009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6eaa009c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7febc8009990 tx=0x7febc80092d0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 -- 172.21.15.2:0/166254758 shutdown_connections 2023-06-23T18:21:40.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55d6eaa0b1f0 0x55d6eaa0a930 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa07a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa00480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 --2- 172.21.15.2:0/166254758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f7580 0x55d6eaa009c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febbbfff700 0 client.15042 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:21:40.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 -- 172.21.15.2:0/166254758 >> 172.21.15.2:0/166254758 conn(0x55d6ea9ec3a0 msgr2=0x55d6ea9f0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 -- 172.21.15.2:0/166254758 shutdown_connections 2023-06-23T18:21:40.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 1 -- 172.21.15.2:0/166254758 wait complete. 2023-06-23T18:21:40.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.322+0000 7febef064b80 20 client.15042 trim_cache size 0 max 16384 2023-06-23T18:21:40.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febef064b80 1 Processor -- start 2023-06-23T18:21:40.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febef064b80 1 -- start start 2023-06-23T18:21:40.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febef064b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febef064b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7580 0x55d6eaa079c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febef064b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa13f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febef064b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea9f2fc0 con 0x55d6ea9f6c30 2023-06-23T18:21:40.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febef064b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea940e90 con 0x55d6ea9f7fb0 2023-06-23T18:21:40.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febef064b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6ea916790 con 0x55d6ea9f7580 2023-06-23T18:21:40.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe0c7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe0c7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9ed870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53248/0 (socket says 172.21.15.2:53248) 2023-06-23T18:21:40.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe0c7b700 1 -- 172.21.15.2:0/492722857 learned_addr learned my addr 172.21.15.2:0/492722857 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febdbfff700 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7580 0x55d6eaa079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe147c700 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa13f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe0c7b700 1 -- 172.21.15.2:0/492722857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7580 msgr2=0x55d6eaa079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe0c7b700 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7580 0x55d6eaa079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe0c7b700 1 -- 172.21.15.2:0/492722857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 msgr2=0x55d6eaa13f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe0c7b700 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa13f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe0c7b700 1 -- 172.21.15.2:0/492722857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6ea9f88a0 con 0x55d6ea9f6c30 2023-06-23T18:21:40.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.340+0000 7febe147c700 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa13f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:40.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febe0c7b700 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9ed870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febd0005e40 tx=0x7febd0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1ff7591ad21704 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febc77fe700 1 -- 172.21.15.2:0/492722857 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0022040 con 0x55d6ea9f6c30 2023-06-23T18:21:40.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febc77fe700 1 -- 172.21.15.2:0/492722857 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febd0014070 con 0x55d6ea9f6c30 2023-06-23T18:21:40.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febc77fe700 1 -- 172.21.15.2:0/492722857 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd001c420 con 0x55d6ea9f6c30 2023-06-23T18:21:40.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 -- 172.21.15.2:0/492722857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 msgr2=0x55d6ea9ed870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9ed870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7febd0005e40 tx=0x7febd0012000 comp rx=0 tx=0).stop 2023-06-23T18:21:40.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 -- 172.21.15.2:0/492722857 shutdown_connections 2023-06-23T18:21:40.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa13f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6ea9ed870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 --2- 172.21.15.2:0/492722857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7580 0x55d6eaa079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 -- 172.21.15.2:0/492722857 >> 172.21.15.2:0/492722857 conn(0x55d6ea9ec3a0 msgr2=0x55d6ea9f14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 -- 172.21.15.2:0/492722857 shutdown_connections 2023-06-23T18:21:40.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.341+0000 7febef064b80 1 -- 172.21.15.2:0/492722857 wait complete. 2023-06-23T18:21:40.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 1 -- start start 2023-06-23T18:21:40.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 1 Processor -- start 2023-06-23T18:21:40.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa02b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7580 0x55d6eaa03050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa06070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6eaa065b0 con 0x55d6ea9f6c30 2023-06-23T18:21:40.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6eaa06720 con 0x55d6ea9f7580 2023-06-23T18:21:40.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6eaa068e0 con 0x55d6ea9f7fb0 2023-06-23T18:21:40.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febef064b80 10 client.0 _ll_register_callbacks cb 0x55d6ea8ed210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:21:40.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe0c7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa02b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe0c7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa02b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53254/0 (socket says 172.21.15.2:53254) 2023-06-23T18:21:40.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe0c7b700 1 -- 172.21.15.2:0/678378025 learned_addr learned my addr 172.21.15.2:0/678378025 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe147c700 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa06070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febdbfff700 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7580 0x55d6eaa03050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe0c7b700 1 -- 172.21.15.2:0/678378025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 msgr2=0x55d6eaa06070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe0c7b700 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa06070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe0c7b700 1 -- 172.21.15.2:0/678378025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7580 msgr2=0x55d6eaa03050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe0c7b700 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7580 0x55d6eaa03050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febe0c7b700 1 -- 172.21.15.2:0/678378025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd0010430 con 0x55d6ea9f6c30 2023-06-23T18:21:40.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.342+0000 7febdbfff700 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7580 0x55d6eaa03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:40.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febe0c7b700 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa02b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7febd0027e20 tx=0x7febd0027e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5029c9748e2cf70 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:21:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febbbfff700 1 -- 172.21.15.2:0/678378025 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0022070 con 0x55d6ea9f6c30 2023-06-23T18:21:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febbbfff700 1 -- 172.21.15.2:0/678378025 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7febd001c420 con 0x55d6ea9f6c30 2023-06-23T18:21:40.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 1 -- 172.21.15.2:0/678378025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febc801e040 con 0x55d6ea9f6c30 2023-06-23T18:21:40.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febbbfff700 1 -- 172.21.15.2:0/678378025 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd001c5e0 con 0x55d6ea9f6c30 2023-06-23T18:21:40.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 10 client.15048 fetch_fsmap learned FSMap version 22 2023-06-23T18:21:40.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 1 -- 172.21.15.2:0/678378025 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55d6ea75ec10 con 0x55d6ea9f6c30 2023-06-23T18:21:40.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febbbfff700 1 -- 172.21.15.2:0/678378025 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febc801e040 con 0x55d6ea9f6c30 2023-06-23T18:21:40.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febbbfff700 1 -- 172.21.15.2:0/678378025 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7febd002f5b0 con 0x55d6ea9f6c30 2023-06-23T18:21:40.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:21:40.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 10 client.15048 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-23T18:21:40.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 20 client.15048 populate_metadata read hostname 'smithi002' 2023-06-23T18:21:40.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55d6eaa64650 0x55d6eaa610a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 4 client.15048 mds_command: new command op to 24505 tid=0[{"prefix": "status"}] 2023-06-23T18:21:40.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.343+0000 7febef064b80 1 -- 172.21.15.2:0/678378025 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d6ea903000 con 0x55d6eaa64650 2023-06-23T18:21:40.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febdbfff700 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55d6eaa64650 0x55d6eaa610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febbbfff700 10 client.15048 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:21:40.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febdbfff700 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55d6eaa64650 0x55d6eaa610a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3bdb1e0c6def7b9 server_cookie=ef85b0ae2611e41f in_seq=0 out_seq=0 2023-06-23T18:21:40.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febbbfff700 1 -- 172.21.15.2:0/678378025 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d6ea903000 con 0x55d6eaa64650 2023-06-23T18:21:40.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febbbfff700 10 client.15048 handle_command_reply: tid=0 2023-06-23T18:21:40.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 client.15048 shutdown 2023-06-23T18:21:40.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 -- 172.21.15.2:0/678378025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 msgr2=0x55d6eaa02b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa02b10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7febd0027e20 tx=0x7febd0027e50 comp rx=0 tx=0).stop 2023-06-23T18:21:40.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 -- 172.21.15.2:0/678378025 shutdown_connections 2023-06-23T18:21:40.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55d6eaa64650 0x55d6eaa610a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d6ea9f7fb0 0x55d6eaa06070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d6ea9f6c30 0x55d6eaa02b10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 --2- 172.21.15.2:0/678378025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d6ea9f7580 0x55d6eaa03050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febbbfff700 0 client.15048 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:21:40.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.344+0000 7febef064b80 1 -- 172.21.15.2:0/678378025 >> 172.21.15.2:0/678378025 conn(0x55d6ea9ec3a0 msgr2=0x55d6ea9f1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febef064b80 1 -- 172.21.15.2:0/678378025 shutdown_connections 2023-06-23T18:21:40.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febef064b80 1 -- 172.21.15.2:0/678378025 wait complete. 2023-06-23T18:21:40.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febef064b80 20 client.15048 trim_cache size 0 max 16384 2023-06-23T18:21:40.368 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:21:40.368 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:21:40.368 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2023-06-23T18:21:40.369 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24505, 2023-06-23T18:21:40.369 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-06-23T18:21:40.369 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-06-23T18:21:40.369 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-06-23T18:21:40.369 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 51.852756880999998, 2023-06-23T18:21:40.369 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 22, 2023-06-23T18:21:40.369 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 84, 2023-06-23T18:21:40.370 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 82, 2023-06-23T18:21:40.370 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 71.824480170000001 2023-06-23T18:21:40.370 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:21:40.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 -- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7febcc0a11e0 msgr2=0x7febcc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7febcc0a11e0 0x7febcc0a3690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7febdc0ffad0 tx=0x7febd000a380 comp rx=0 tx=0).stop 2023-06-23T18:21:40.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 -- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc100600 msgr2=0x7febdc19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc100600 0x7febdc19cdb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febd4007ee0 tx=0x7febd400d3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 -- 172.21.15.2:0/340848356 shutdown_connections 2023-06-23T18:21:40.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febdc100600 0x7febdc19cdb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7febcc0a11e0 0x7febcc0a3690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7febdc0fea80 0x7febdc195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 --2- 172.21.15.2:0/340848356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febdc0ffc70 0x7febdc195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 -- 172.21.15.2:0/340848356 >> 172.21.15.2:0/340848356 conn(0x7febdc0fa0e0 msgr2=0x7febdc0677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 -- 172.21.15.2:0/340848356 shutdown_connections 2023-06-23T18:21:40.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.345+0000 7febe2edf700 1 -- 172.21.15.2:0/340848356 wait complete. 2023-06-23T18:21:40.372 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:40 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/885138841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:40.373 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:40 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1977739089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:40.373 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-06-23T18:21:40.374 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:40 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/885138841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:40.374 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:40 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1977739089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:40.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:40 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/885138841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:40.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:40 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1977739089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:40.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 -- 172.21.15.2:0/3719285276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 msgr2=0x7f15641000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 --2- 172.21.15.2:0/3719285276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 0x7f15641000c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1554009990 tx=0x7f1554009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 -- 172.21.15.2:0/3719285276 shutdown_connections 2023-06-23T18:21:40.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 --2- 172.21.15.2:0/3719285276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1564100600 0x7f1564106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 --2- 172.21.15.2:0/3719285276 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15640fea80 0x7f15640fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 --2- 172.21.15.2:0/3719285276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 0x7f15641000c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 -- 172.21.15.2:0/3719285276 >> 172.21.15.2:0/3719285276 conn(0x7f15640fa0e0 msgr2=0x7f15640fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 -- 172.21.15.2:0/3719285276 shutdown_connections 2023-06-23T18:21:40.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 -- 172.21.15.2:0/3719285276 wait complete. 2023-06-23T18:21:40.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.499+0000 7f156c16d700 1 Processor -- start 2023-06-23T18:21:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156c16d700 1 -- start start 2023-06-23T18:21:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156c16d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15640fea80 0x7f1564109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156c16d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 0x7f15641079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156c16d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1564100600 0x7f1564107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156c16d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1564108690 con 0x7f15640ffc70 2023-06-23T18:21:40.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156c16d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1564108800 con 0x7f1564100600 2023-06-23T18:21:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156c16d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156419f850 con 0x7f15640fea80 2023-06-23T18:21:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 0x7f15641079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 0x7f15641079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53270/0 (socket says 172.21.15.2:53270) 2023-06-23T18:21:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 -- 172.21.15.2:0/563457853 learned_addr learned my addr 172.21.15.2:0/563457853 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569f09700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15640fea80 0x7f1564109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156a70a700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1564100600 0x7f1564107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 -- 172.21.15.2:0/563457853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15640fea80 msgr2=0x7f1564109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15640fea80 0x7f1564109370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 -- 172.21.15.2:0/563457853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1564100600 msgr2=0x7f1564107ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1564100600 0x7f1564107ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 -- 172.21.15.2:0/563457853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540095f0 con 0x7f15640ffc70 2023-06-23T18:21:40.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f156a70a700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1564100600 0x7f1564107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f1569708700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 0x7f15641079b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f15540098e0 tx=0x7f1554004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.500+0000 7f155affd700 1 -- 172.21.15.2:0/563457853 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155401d070 con 0x7f15640ffc70 2023-06-23T18:21:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.501+0000 7f155affd700 1 -- 172.21.15.2:0/563457853 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f155400bdd0 con 0x7f15640ffc70 2023-06-23T18:21:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.501+0000 7f155affd700 1 -- 172.21.15.2:0/563457853 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554022db0 con 0x7f15640ffc70 2023-06-23T18:21:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.501+0000 7f156c16d700 1 -- 172.21.15.2:0/563457853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156419fad0 con 0x7f15640ffc70 2023-06-23T18:21:40.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.501+0000 7f156c16d700 1 -- 172.21.15.2:0/563457853 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156419fef0 con 0x7f15640ffc70 2023-06-23T18:21:40.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.503+0000 7f155affd700 1 -- 172.21.15.2:0/563457853 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1554022470 con 0x7f15640ffc70 2023-06-23T18:21:40.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.503+0000 7f155affd700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15500a1260 0x7f15500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f1569f09700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15500a1260 0x7f15500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f155affd700 1 -- 172.21.15.2:0/563457853 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f15540ce080 con 0x7f15640ffc70 2023-06-23T18:21:40.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f1569f09700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15500a1260 0x7f15500a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15640ffad0 tx=0x7f1560009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f15782f2b80 1 Processor -- start 2023-06-23T18:21:40.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f15782f2b80 1 -- start start 2023-06-23T18:21:40.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37348c30 0x55fc37349040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349580 0x55fc37349a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc3734a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f15782f2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3724dc60 con 0x55fc37349580 2023-06-23T18:21:40.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f15782f2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc372a09f0 con 0x55fc37348c30 2023-06-23T18:21:40.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.504+0000 7f15782f2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc37299e80 con 0x55fc37349fb0 2023-06-23T18:21:40.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349580 0x55fc37349a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349580 0x55fc37349a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53278/0 (socket says 172.21.15.2:53278) 2023-06-23T18:21:40.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 -- 172.21.15.2:0/602248741 learned_addr learned my addr 172.21.15.2:0/602248741 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569f09700 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37348c30 0x55fc37349040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f156a70a700 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc3734a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 -- 172.21.15.2:0/602248741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 msgr2=0x55fc3734a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc3734a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 -- 172.21.15.2:0/602248741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37348c30 msgr2=0x55fc37349040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37348c30 0x55fc37349040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 -- 172.21.15.2:0/602248741 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc3711b5e0 con 0x55fc37349580 2023-06-23T18:21:40.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f156a70a700 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc3734a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:40.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f1569708700 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349580 0x55fc37349a20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1554097df0 tx=0x7f1554009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c44d19d4329c69 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f154ffff700 1 -- 172.21.15.2:0/602248741 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155400b940 con 0x55fc37349580 2023-06-23T18:21:40.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f154ffff700 1 -- 172.21.15.2:0/602248741 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f155400bc10 con 0x55fc37349580 2023-06-23T18:21:40.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.505+0000 7f154ffff700 1 -- 172.21.15.2:0/602248741 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15540a3740 con 0x55fc37349580 2023-06-23T18:21:40.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 -- 172.21.15.2:0/602248741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349580 msgr2=0x55fc37349a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349580 0x55fc37349a20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1554097df0 tx=0x7f1554009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 -- 172.21.15.2:0/602248741 shutdown_connections 2023-06-23T18:21:40.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc3734a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37348c30 0x55fc37349040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 --2- 172.21.15.2:0/602248741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349580 0x55fc37349a20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 -- 172.21.15.2:0/602248741 >> 172.21.15.2:0/602248741 conn(0x55fc3733e3a0 msgr2=0x55fc373434d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 -- 172.21.15.2:0/602248741 shutdown_connections 2023-06-23T18:21:40.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 -- 172.21.15.2:0/602248741 wait complete. 2023-06-23T18:21:40.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.506+0000 7f15782f2b80 1 -- start start 2023-06-23T18:21:40.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f15782f2b80 1 Processor -- start 2023-06-23T18:21:40.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37348c30 0x55fc373521a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc373526e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349fb0 0x55fc37359720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f15782f2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc37239a70 con 0x55fc37349fb0 2023-06-23T18:21:40.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f15782f2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc37239770 con 0x55fc37349580 2023-06-23T18:21:40.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f15782f2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3711d8d0 con 0x55fc37348c30 2023-06-23T18:21:40.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f15782f2b80 10 client.0 _ll_register_callbacks cb 0x55fc3723f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:21:40.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f156a70a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349fb0 0x55fc37359720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f156a70a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349fb0 0x55fc37359720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53286/0 (socket says 172.21.15.2:53286) 2023-06-23T18:21:40.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f156a70a700 1 -- 172.21.15.2:0/3116484515 learned_addr learned my addr 172.21.15.2:0/3116484515 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f1569708700 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc373526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f1569f09700 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37348c30 0x55fc373521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f156a70a700 1 -- 172.21.15.2:0/3116484515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37348c30 msgr2=0x55fc373521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f156a70a700 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37348c30 0x55fc373521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f156a70a700 1 -- 172.21.15.2:0/3116484515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 msgr2=0x55fc373526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f156a70a700 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc373526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.507+0000 7f156a70a700 1 -- 172.21.15.2:0/3116484515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156419fef0 con 0x55fc37349fb0 2023-06-23T18:21:40.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f156a70a700 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349fb0 0x55fc37359720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f155c007be0 tx=0x7f155c0094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c4bca295697fc8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f154d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:21:40.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f154d7fa700 1 -- 172.21.15.2:0/3116484515 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00e070 con 0x55fc37349fb0 2023-06-23T18:21:40.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f154d7fa700 1 -- 172.21.15.2:0/3116484515 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f155c012cb0 con 0x55fc37349fb0 2023-06-23T18:21:40.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 1 -- 172.21.15.2:0/3116484515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc3733bb60 con 0x55fc37349fb0 2023-06-23T18:21:40.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f154d7fa700 1 -- 172.21.15.2:0/3116484515 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f155c012eb0 con 0x55fc37349fb0 2023-06-23T18:21:40.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 10 client.15060 fetch_fsmap learned FSMap version 22 2023-06-23T18:21:40.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 1 -- 172.21.15.2:0/3116484515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55fc370c7d90 con 0x55fc37349fb0 2023-06-23T18:21:40.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f154d7fa700 1 -- 172.21.15.2:0/3116484515 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fc3733bb60 con 0x55fc37349fb0 2023-06-23T18:21:40.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f154d7fa700 1 -- 172.21.15.2:0/3116484515 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f155c014040 con 0x55fc37349fb0 2023-06-23T18:21:40.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:21:40.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 10 client.15060 resolve_mds: resolved name 'i' to daemon mds.i 2023-06-23T18:21:40.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 20 client.15060 populate_metadata read hostname 'smithi002' 2023-06-23T18:21:40.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55fc3735d4d0 0x55fc3735c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 4 client.15060 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:21:40.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.508+0000 7f15782f2b80 1 -- 172.21.15.2:0/3116484515 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc37292e60 con 0x55fc3735d4d0 2023-06-23T18:21:40.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.509+0000 7f1569f09700 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55fc3735d4d0 0x55fc3735c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.509+0000 7f1569f09700 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55fc3735d4d0 0x55fc3735c880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2895a6eb4fad2ddb server_cookie=761d180c87b91a83 in_seq=0 out_seq=0 2023-06-23T18:21:40.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.509+0000 7f154d7fa700 10 client.15060 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:21:40.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f154d7fa700 1 -- 172.21.15.2:0/3116484515 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fc37292e60 con 0x55fc3735d4d0 2023-06-23T18:21:40.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f154d7fa700 10 client.15060 handle_command_reply: tid=0 2023-06-23T18:21:40.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 client.15060 shutdown 2023-06-23T18:21:40.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 -- 172.21.15.2:0/3116484515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349fb0 msgr2=0x55fc37359720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349fb0 0x55fc37359720 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f155c007be0 tx=0x7f155c0094a0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 -- 172.21.15.2:0/3116484515 shutdown_connections 2023-06-23T18:21:40.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55fc3735d4d0 0x55fc3735c880 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37349fb0 0x55fc37359720 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37348c30 0x55fc373521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 --2- 172.21.15.2:0/3116484515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc373526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f154d7fa700 0 client.15060 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:21:40.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 -- 172.21.15.2:0/3116484515 >> 172.21.15.2:0/3116484515 conn(0x55fc3733e3a0 msgr2=0x55fc37342c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 -- 172.21.15.2:0/3116484515 shutdown_connections 2023-06-23T18:21:40.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 1 -- 172.21.15.2:0/3116484515 wait complete. 2023-06-23T18:21:40.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.510+0000 7f15782f2b80 20 client.15060 trim_cache size 0 max 16384 2023-06-23T18:21:40.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f15782f2b80 1 Processor -- start 2023-06-23T18:21:40.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f15782f2b80 1 -- start start 2023-06-23T18:21:40.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc3735c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc3735cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc37359640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f15782f2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc37351d70 con 0x55fc37348c30 2023-06-23T18:21:40.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f15782f2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3733df20 con 0x55fc37349580 2023-06-23T18:21:40.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f15782f2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc3724dc60 con 0x55fc37349fb0 2023-06-23T18:21:40.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f1569f09700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc3735c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f1569f09700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc3735c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53290/0 (socket says 172.21.15.2:53290) 2023-06-23T18:21:40.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f1569f09700 1 -- 172.21.15.2:0/2083227192 learned_addr learned my addr 172.21.15.2:0/2083227192 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.528+0000 7f1569708700 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc3735cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f156a70a700 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc37359640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1569f09700 1 -- 172.21.15.2:0/2083227192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 msgr2=0x55fc37359640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1569f09700 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc37359640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1569f09700 1 -- 172.21.15.2:0/2083227192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 msgr2=0x55fc3735cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1569f09700 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc3735cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1569f09700 1 -- 172.21.15.2:0/2083227192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc3734a650 con 0x55fc37348c30 2023-06-23T18:21:40.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1569708700 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc3735cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:40.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1569f09700 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc3735c880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55fc3735e6d0 tx=0x7f1560014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677a49427cd7e71c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1558ff9700 1 -- 172.21.15.2:0/2083227192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156002a600 con 0x55fc37348c30 2023-06-23T18:21:40.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1558ff9700 1 -- 172.21.15.2:0/2083227192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1560023040 con 0x55fc37348c30 2023-06-23T18:21:40.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.529+0000 7f1558ff9700 1 -- 172.21.15.2:0/2083227192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15600044f0 con 0x55fc37348c30 2023-06-23T18:21:40.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- 172.21.15.2:0/2083227192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 msgr2=0x55fc3735c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc3735c880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55fc3735e6d0 tx=0x7f1560014000 comp rx=0 tx=0).stop 2023-06-23T18:21:40.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- 172.21.15.2:0/2083227192 shutdown_connections 2023-06-23T18:21:40.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc37359640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc3735c880 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 --2- 172.21.15.2:0/2083227192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc3735cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- 172.21.15.2:0/2083227192 >> 172.21.15.2:0/2083227192 conn(0x55fc3733e3a0 msgr2=0x55fc37366220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- 172.21.15.2:0/2083227192 shutdown_connections 2023-06-23T18:21:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- 172.21.15.2:0/2083227192 wait complete. 2023-06-23T18:21:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- start start 2023-06-23T18:21:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 Processor -- start 2023-06-23T18:21:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc373548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc37354de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc37357e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc373449b0 con 0x55fc37348c30 2023-06-23T18:21:40.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc37299e80 con 0x55fc37349580 2023-06-23T18:21:40.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc37358390 con 0x55fc37349fb0 2023-06-23T18:21:40.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f15782f2b80 10 client.0 _ll_register_callbacks cb 0x55fc3723f210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:21:40.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.530+0000 7f1569f09700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc373548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569f09700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc373548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53296/0 (socket says 172.21.15.2:53296) 2023-06-23T18:21:40.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569f09700 1 -- 172.21.15.2:0/1387998672 learned_addr learned my addr 172.21.15.2:0/1387998672 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f156a70a700 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc37357e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569708700 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc37354de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569f09700 1 -- 172.21.15.2:0/1387998672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 msgr2=0x55fc37357e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569f09700 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc37357e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569f09700 1 -- 172.21.15.2:0/1387998672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 msgr2=0x55fc37354de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569f09700 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc37354de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569f09700 1 -- 172.21.15.2:0/1387998672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1560017070 con 0x55fc37348c30 2023-06-23T18:21:40.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569708700 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc37354de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f1569f09700 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc373548a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f15600097c0 tx=0x7f156002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd96606b60d82ae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f154d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:21:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f154d7fa700 1 -- 172.21.15.2:0/1387998672 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560004520 con 0x55fc37348c30 2023-06-23T18:21:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f154d7fa700 1 -- 172.21.15.2:0/1387998672 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f156002aa40 con 0x55fc37348c30 2023-06-23T18:21:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f15782f2b80 1 -- 172.21.15.2:0/1387998672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f155400bc10 con 0x55fc37348c30 2023-06-23T18:21:40.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f154d7fa700 1 -- 172.21.15.2:0/1387998672 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f156002ac00 con 0x55fc37348c30 2023-06-23T18:21:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f15782f2b80 10 client.15066 fetch_fsmap learned FSMap version 22 2023-06-23T18:21:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f15782f2b80 1 -- 172.21.15.2:0/1387998672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55fc370b0c10 con 0x55fc37348c30 2023-06-23T18:21:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.531+0000 7f154d7fa700 1 -- 172.21.15.2:0/1387998672 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f155400bc10 con 0x55fc37348c30 2023-06-23T18:21:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f154d7fa700 1 -- 172.21.15.2:0/1387998672 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1560030030 con 0x55fc37348c30 2023-06-23T18:21:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f15782f2b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:21:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f15782f2b80 10 client.15066 resolve_mds: resolved name 'i' to daemon mds.i 2023-06-23T18:21:40.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f15782f2b80 20 client.15066 populate_metadata read hostname 'smithi002' 2023-06-23T18:21:40.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f15782f2b80 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55fc373b66c0 0x55fc373b3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f15782f2b80 4 client.15066 mds_command: new command op to 24526 tid=0[{"prefix": "status"}] 2023-06-23T18:21:40.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f15782f2b80 1 -- 172.21.15.2:0/1387998672 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fc37255000 con 0x55fc373b66c0 2023-06-23T18:21:40.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f1569708700 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55fc373b66c0 0x55fc373b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f154d7fa700 10 client.15066 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:21:40.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.532+0000 7f1569708700 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55fc373b66c0 0x55fc373b3360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f62e3f30d3b8ead server_cookie=96cbc7d5de2cac15 in_seq=0 out_seq=0 2023-06-23T18:21:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f154d7fa700 1 -- 172.21.15.2:0/1387998672 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55fc37255000 con 0x55fc373b66c0 2023-06-23T18:21:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f154d7fa700 10 client.15066 handle_command_reply: tid=0 2023-06-23T18:21:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 client.15066 shutdown 2023-06-23T18:21:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 -- 172.21.15.2:0/1387998672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 msgr2=0x55fc373548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc373548a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f15600097c0 tx=0x7f156002a6a0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 -- 172.21.15.2:0/1387998672 shutdown_connections 2023-06-23T18:21:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55fc373b66c0 0x55fc373b3360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fc37349fb0 0x55fc37357e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55fc37348c30 0x55fc373548a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 --2- 172.21.15.2:0/1387998672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55fc37349580 0x55fc37354de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f154d7fa700 0 client.15066 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:21:40.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 -- 172.21.15.2:0/1387998672 >> 172.21.15.2:0/1387998672 conn(0x55fc3733e3a0 msgr2=0x55fc3735c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 -- 172.21.15.2:0/1387998672 shutdown_connections 2023-06-23T18:21:40.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 1 -- 172.21.15.2:0/1387998672 wait complete. 2023-06-23T18:21:40.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.533+0000 7f15782f2b80 20 client.15066 trim_cache size 0 max 16384 2023-06-23T18:21:40.549 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:21:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:21:40.549 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2023-06-23T18:21:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24526, 2023-06-23T18:21:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-06-23T18:21:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-06-23T18:21:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-06-23T18:21:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 53.043595515, 2023-06-23T18:21:40.550 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 22, 2023-06-23T18:21:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 84, 2023-06-23T18:21:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 82, 2023-06-23T18:21:40.551 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 67.122122880000006 2023-06-23T18:21:40.551 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:21:40.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 -- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15500a1260 msgr2=0x7f15500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15500a1260 0x7f15500a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f15640ffad0 tx=0x7f1560009260 comp rx=0 tx=0).stop 2023-06-23T18:21:40.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 -- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 msgr2=0x7f15641079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 0x7f15641079b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f15540098e0 tx=0x7f1554004730 comp rx=0 tx=0).stop 2023-06-23T18:21:40.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 -- 172.21.15.2:0/563457853 shutdown_connections 2023-06-23T18:21:40.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15500a1260 0x7f15500a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1564100600 0x7f1564107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15640fea80 0x7f1564109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 --2- 172.21.15.2:0/563457853 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15640ffc70 0x7f15641079b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 -- 172.21.15.2:0/563457853 >> 172.21.15.2:0/563457853 conn(0x7f15640fa0e0 msgr2=0x7f15640fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 -- 172.21.15.2:0/563457853 shutdown_connections 2023-06-23T18:21:40.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.534+0000 7f156c16d700 1 -- 172.21.15.2:0/563457853 wait complete. 2023-06-23T18:21:40.555 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-06-23T18:21:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 -- 172.21.15.2:0/2597519882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d80feaa0 msgr2=0x7fd1d80feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 --2- 172.21.15.2:0/2597519882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d80feaa0 0x7fd1d80feeb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009990 tx=0x7fd1c8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 -- 172.21.15.2:0/2597519882 shutdown_connections 2023-06-23T18:21:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 --2- 172.21.15.2:0/2597519882 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1d8100620 0x7fd1d8106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 --2- 172.21.15.2:0/2597519882 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d80feaa0 0x7fd1d80feeb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 --2- 172.21.15.2:0/2597519882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d80ffc90 0x7fd1d81000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 -- 172.21.15.2:0/2597519882 >> 172.21.15.2:0/2597519882 conn(0x7fd1d80fa0c0 msgr2=0x7fd1d80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 -- 172.21.15.2:0/2597519882 shutdown_connections 2023-06-23T18:21:40.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.677+0000 7fd1dce65700 1 -- 172.21.15.2:0/2597519882 wait complete. 2023-06-23T18:21:40.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.678+0000 7fd1dce65700 1 Processor -- start 2023-06-23T18:21:40.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.678+0000 7fd1dce65700 1 -- start start 2023-06-23T18:21:40.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.678+0000 7fd1dce65700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1d80feaa0 0x7fd1d8195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.678+0000 7fd1dce65700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d80ffc90 0x7fd1d8195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.678+0000 7fd1dce65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d8100620 0x7fd1d819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.678+0000 7fd1dce65700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d819d3b0 con 0x7fd1d8100620 2023-06-23T18:21:40.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.678+0000 7fd1dce65700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d819d4f0 con 0x7fd1d80feaa0 2023-06-23T18:21:40.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.678+0000 7fd1dce65700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d819d680 con 0x7fd1d80ffc90 2023-06-23T18:21:40.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d8100620 0x7fd1d819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d8100620 0x7fd1d819cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53314/0 (socket says 172.21.15.2:53314) 2023-06-23T18:21:40.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/3845201474 learned_addr learned my addr 172.21.15.2:0/3845201474 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d80ffc90 0x7fd1d8195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d659c700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1d80feaa0 0x7fd1d8195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/3845201474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d80ffc90 msgr2=0x7fd1d8195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d80ffc90 0x7fd1d8195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/3845201474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1d80feaa0 msgr2=0x7fd1d8195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1d80feaa0 0x7fd1d8195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/3845201474 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c80095f0 con 0x7fd1d8100620 2023-06-23T18:21:40.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d8100620 0x7fd1d819cde0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0007ee0 tx=0x7fd1d000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1c77fe700 1 -- 172.21.15.2:0/3845201474 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d000dd20 con 0x7fd1d8100620 2023-06-23T18:21:40.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1dce65700 1 -- 172.21.15.2:0/3845201474 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d819d960 con 0x7fd1d8100620 2023-06-23T18:21:40.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1c77fe700 1 -- 172.21.15.2:0/3845201474 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1d000f040 con 0x7fd1d8100620 2023-06-23T18:21:40.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1c77fe700 1 -- 172.21.15.2:0/3845201474 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0004750 con 0x7fd1d8100620 2023-06-23T18:21:40.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.679+0000 7fd1dce65700 1 -- 172.21.15.2:0/3845201474 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d819dec0 con 0x7fd1d8100620 2023-06-23T18:21:40.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.682+0000 7fd1c77fe700 1 -- 172.21.15.2:0/3845201474 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd1d0003a00 con 0x7fd1d8100620 2023-06-23T18:21:40.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.682+0000 7fd1c77fe700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd1c00a11e0 0x7fd1c00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.683+0000 7fd1c77fe700 1 -- 172.21.15.2:0/3845201474 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd1d00cc320 con 0x7fd1d8100620 2023-06-23T18:21:40.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.683+0000 7fd1d659c700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd1c00a11e0 0x7fd1c00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.683+0000 7fd1d659c700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd1c00a11e0 0x7fd1c00a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800b5c0 tx=0x7fd1c8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.683+0000 7fd1e8feab80 1 Processor -- start 2023-06-23T18:21:40.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.683+0000 7fd1e8feab80 1 -- start start 2023-06-23T18:21:40.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 0x55af7474c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7474ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7474d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af74650c60 con 0x55af7474cfb0 2023-06-23T18:21:40.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af746a39f0 con 0x55af7474c580 2023-06-23T18:21:40.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7469ce80 con 0x55af7474bc30 2023-06-23T18:21:40.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7474d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7474d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53322/0 (socket says 172.21.15.2:53322) 2023-06-23T18:21:40.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/2200836692 learned_addr learned my addr 172.21.15.2:0/2200836692 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d659c700 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 0x55af7474c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7474ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/2200836692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 msgr2=0x55af7474c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 0x55af7474c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/2200836692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 msgr2=0x55af7474ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7474ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/2200836692 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af7451e5e0 con 0x55af7474cfb0 2023-06-23T18:21:40.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7474ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:40.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7474d420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0099520 tx=0x7fd1d0099200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5eb710ef0ec12b8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1c4ff9700 1 -- 172.21.15.2:0/2200836692 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d00b90b0 con 0x55af7474cfb0 2023-06-23T18:21:40.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1c4ff9700 1 -- 172.21.15.2:0/2200836692 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1d00048b0 con 0x55af7474cfb0 2023-06-23T18:21:40.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.684+0000 7fd1c4ff9700 1 -- 172.21.15.2:0/2200836692 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0009450 con 0x55af7474cfb0 2023-06-23T18:21:40.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2200836692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 msgr2=0x55af7474d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7474d420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0099520 tx=0x7fd1d0099200 comp rx=0 tx=0).stop 2023-06-23T18:21:40.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2200836692 shutdown_connections 2023-06-23T18:21:40.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7474d420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 0x55af7474c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2200836692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7474ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2200836692 >> 172.21.15.2:0/2200836692 conn(0x55af747413a0 msgr2=0x55af747464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2200836692 shutdown_connections 2023-06-23T18:21:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2200836692 wait complete. 2023-06-23T18:21:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.685+0000 7fd1e8feab80 1 -- start start 2023-06-23T18:21:40.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1e8feab80 1 Processor -- start 2023-06-23T18:21:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 0x55af74755500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af74755a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7475ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7463ca70 con 0x55af7474cfb0 2023-06-23T18:21:40.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7463c770 con 0x55af7474c580 2023-06-23T18:21:40.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af745208d0 con 0x55af7474bc30 2023-06-23T18:21:40.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1e8feab80 10 client.0 _ll_register_callbacks cb 0x55af74642210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:21:40.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7475ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7475ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53328/0 (socket says 172.21.15.2:53328) 2023-06-23T18:21:40.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/3152664602 learned_addr learned my addr 172.21.15.2:0/3152664602 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d659c700 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 0x55af74755500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af74755a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/3152664602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 msgr2=0x55af74755500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 0x55af74755500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/3152664602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 msgr2=0x55af74755a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af74755a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 -- 172.21.15.2:0/3152664602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d819dec0 con 0x55af7474cfb0 2023-06-23T18:21:40.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7475ca80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00a14b0 tx=0x7fd1d00b9130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d00e8321916be91 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.686+0000 7fd1bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:21:40.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1bdffb700 1 -- 172.21.15.2:0/3152664602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0099590 con 0x55af7474cfb0 2023-06-23T18:21:40.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1bdffb700 1 -- 172.21.15.2:0/3152664602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd1d009d380 con 0x55af7474cfb0 2023-06-23T18:21:40.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 1 -- 172.21.15.2:0/3152664602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af7473eb60 con 0x55af7474cfb0 2023-06-23T18:21:40.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1bdffb700 1 -- 172.21.15.2:0/3152664602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1d0009410 con 0x55af7474cfb0 2023-06-23T18:21:40.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 10 client.15078 fetch_fsmap learned FSMap version 22 2023-06-23T18:21:40.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 1 -- 172.21.15.2:0/3152664602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55af744cad90 con 0x55af7474cfb0 2023-06-23T18:21:40.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1bdffb700 1 -- 172.21.15.2:0/3152664602 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1d0065020 con 0x55af7474cfb0 2023-06-23T18:21:40.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1bdffb700 1 -- 172.21.15.2:0/3152664602 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd1d0098a80 con 0x55af7474cfb0 2023-06-23T18:21:40.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 10 client.15078 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:21:40.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 10 client.15078 resolve_mds: resolved name 'b' to daemon mds.b 2023-06-23T18:21:40.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 20 client.15078 populate_metadata read hostname 'smithi002' 2023-06-23T18:21:40.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55af747601c0 0x55af7475f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 4 client.15078 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:21:40.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1e8feab80 1 -- 172.21.15.2:0/3152664602 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af74695e60 con 0x55af747601c0 2023-06-23T18:21:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.687+0000 7fd1d659c700 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55af747601c0 0x55af7475f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.688+0000 7fd1d659c700 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55af747601c0 0x55af7475f920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64300aed0b7c1e85 server_cookie=dadc533082454e81 in_seq=0 out_seq=0 2023-06-23T18:21:40.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.688+0000 7fd1bdffb700 10 client.15078 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:21:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1bdffb700 1 -- 172.21.15.2:0/3152664602 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af74695e60 con 0x55af747601c0 2023-06-23T18:21:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1bdffb700 10 client.15078 handle_command_reply: tid=0 2023-06-23T18:21:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 client.15078 shutdown 2023-06-23T18:21:40.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 -- 172.21.15.2:0/3152664602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 msgr2=0x55af7475ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7475ca80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00a14b0 tx=0x7fd1d00b9130 comp rx=0 tx=0).stop 2023-06-23T18:21:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 -- 172.21.15.2:0/3152664602 shutdown_connections 2023-06-23T18:21:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55af747601c0 0x55af7475f920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474cfb0 0x55af7475ca80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1bdffb700 0 client.15078 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:21:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474bc30 0x55af74755500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/3152664602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af74755a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 -- 172.21.15.2:0/3152664602 >> 172.21.15.2:0/3152664602 conn(0x55af747413a0 msgr2=0x55af74745c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 -- 172.21.15.2:0/3152664602 shutdown_connections 2023-06-23T18:21:40.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 1 -- 172.21.15.2:0/3152664602 wait complete. 2023-06-23T18:21:40.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.689+0000 7fd1e8feab80 20 client.15078 trim_cache size 0 max 16384 2023-06-23T18:21:40.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1e8feab80 1 Processor -- start 2023-06-23T18:21:40.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1e8feab80 1 -- start start 2023-06-23T18:21:40.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74754210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7475c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474cfb0 0x55af747532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af74695e60 con 0x55af7474bc30 2023-06-23T18:21:40.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af74646470 con 0x55af7474c580 2023-06-23T18:21:40.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7452cde0 con 0x55af7474cfb0 2023-06-23T18:21:40.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74754210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74754210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53332/0 (socket says 172.21.15.2:53332) 2023-06-23T18:21:40.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d659c700 1 -- 172.21.15.2:0/1693530712 learned_addr learned my addr 172.21.15.2:0/1693530712 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d6d9d700 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474cfb0 0x55af747532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7475c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d659c700 1 -- 172.21.15.2:0/1693530712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474cfb0 msgr2=0x55af747532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d659c700 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474cfb0 0x55af747532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d659c700 1 -- 172.21.15.2:0/1693530712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 msgr2=0x55af7475c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d659c700 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7475c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.707+0000 7fd1d659c700 1 -- 172.21.15.2:0/1693530712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af744b3c10 con 0x55af7474bc30 2023-06-23T18:21:40.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1d659c700 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74754210 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8005cd0 tx=0x7fd1c8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8655df9ac66e29a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1c57fa700 1 -- 172.21.15.2:0/1693530712 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c802b040 con 0x55af7474bc30 2023-06-23T18:21:40.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1c57fa700 1 -- 172.21.15.2:0/1693530712 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1c802c040 con 0x55af7474bc30 2023-06-23T18:21:40.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1c57fa700 1 -- 172.21.15.2:0/1693530712 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80048f0 con 0x55af7474bc30 2023-06-23T18:21:40.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 -- 172.21.15.2:0/1693530712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 msgr2=0x55af74754210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74754210 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8005cd0 tx=0x7fd1c8020000 comp rx=0 tx=0).stop 2023-06-23T18:21:40.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 -- 172.21.15.2:0/1693530712 shutdown_connections 2023-06-23T18:21:40.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474cfb0 0x55af747532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74754210 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/1693530712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474c580 0x55af7475c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 -- 172.21.15.2:0/1693530712 >> 172.21.15.2:0/1693530712 conn(0x55af747413a0 msgr2=0x55af74768f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 -- 172.21.15.2:0/1693530712 shutdown_connections 2023-06-23T18:21:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.708+0000 7fd1e8feab80 1 -- 172.21.15.2:0/1693530712 wait complete. 2023-06-23T18:21:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 1 -- start start 2023-06-23T18:21:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 1 Processor -- start 2023-06-23T18:21:40.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74757e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474c580 0x55af747583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474cfb0 0x55af7475b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7475b8e0 con 0x55af7474bc30 2023-06-23T18:21:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7475ba50 con 0x55af7474cfb0 2023-06-23T18:21:40.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af7475bc10 con 0x55af7474c580 2023-06-23T18:21:40.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1e8feab80 10 client.0 _ll_register_callbacks cb 0x55af74642210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:21:40.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74757e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74757e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53338/0 (socket says 172.21.15.2:53338) 2023-06-23T18:21:40.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 -- 172.21.15.2:0/2048407759 learned_addr learned my addr 172.21.15.2:0/2048407759 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:40.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474c580 0x55af747583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 -- 172.21.15.2:0/2048407759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474c580 msgr2=0x55af747583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474c580 0x55af747583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 -- 172.21.15.2:0/2048407759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474cfb0 msgr2=0x55af7475b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:21:40.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474cfb0 0x55af7475b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 -- 172.21.15.2:0/2048407759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d8107810 con 0x55af7474bc30 2023-06-23T18:21:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474c580 0x55af747583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1d659c700 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74757e90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009ce0 tx=0x7fd1c800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58fc6e6ae5acd44 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.709+0000 7fd1bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:21:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1bdffb700 1 -- 172.21.15.2:0/2048407759 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c802b070 con 0x55af7474bc30 2023-06-23T18:21:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2048407759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd1d00048b0 con 0x55af7474bc30 2023-06-23T18:21:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1bdffb700 1 -- 172.21.15.2:0/2048407759 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd1c8035500 con 0x55af7474bc30 2023-06-23T18:21:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1bdffb700 1 -- 172.21.15.2:0/2048407759 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1c8036ef0 con 0x55af7474bc30 2023-06-23T18:21:40.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 10 client.15084 fetch_fsmap learned FSMap version 22 2023-06-23T18:21:40.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2048407759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55af744f1200 con 0x55af7474bc30 2023-06-23T18:21:40.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1bdffb700 1 -- 172.21.15.2:0/2048407759 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd1c804c020 con 0x55af7474bc30 2023-06-23T18:21:40.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1bdffb700 1 -- 172.21.15.2:0/2048407759 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd1c8003b10 con 0x55af7474bc30 2023-06-23T18:21:40.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 10 client.15084 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:21:40.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2023-06-23T18:21:40.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 20 client.15084 populate_metadata read hostname 'smithi002' 2023-06-23T18:21:40.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55af747b94a0 0x55af747b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:40.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 4 client.15084 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-06-23T18:21:40.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2048407759 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55af74658000 con 0x55af747b94a0 2023-06-23T18:21:40.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.710+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55af747b94a0 0x55af747b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:40.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1bdffb700 10 client.15084 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:21:40.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1d5d9b700 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55af747b94a0 0x55af747b6050 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db15c98f7a742332 server_cookie=7b350df9c37ebee4 in_seq=0 out_seq=0 2023-06-23T18:21:40.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1bdffb700 1 -- 172.21.15.2:0/2048407759 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55af74658000 con 0x55af747b94a0 2023-06-23T18:21:40.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1bdffb700 10 client.15084 handle_command_reply: tid=0 2023-06-23T18:21:40.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 client.15084 shutdown 2023-06-23T18:21:40.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2048407759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 msgr2=0x55af74757e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74757e90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009ce0 tx=0x7fd1c800bc60 comp rx=0 tx=0).stop 2023-06-23T18:21:40.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2048407759 shutdown_connections 2023-06-23T18:21:40.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55af747b94a0 0x55af747b6050 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55af7474cfb0 0x55af7475b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1bdffb700 0 client.15084 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:21:40.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55af7474bc30 0x55af74757e90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 --2- 172.21.15.2:0/2048407759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55af7474c580 0x55af747583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2048407759 >> 172.21.15.2:0/2048407759 conn(0x55af747413a0 msgr2=0x55af7474fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2048407759 shutdown_connections 2023-06-23T18:21:40.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 1 -- 172.21.15.2:0/2048407759 wait complete. 2023-06-23T18:21:40.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.711+0000 7fd1e8feab80 20 client.15084 trim_cache size 0 max 16384 2023-06-23T18:21:40.751 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:21:40.751 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:21:40.751 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2023-06-23T18:21:40.751 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24533, 2023-06-23T18:21:40.752 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-06-23T18:21:40.752 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-06-23T18:21:40.752 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-06-23T18:21:40.752 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 56.274231463, 2023-06-23T18:21:40.752 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 22, 2023-06-23T18:21:40.752 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 84, 2023-06-23T18:21:40.752 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 82, 2023-06-23T18:21:40.753 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 60.186351174000002 2023-06-23T18:21:40.753 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:21:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 -- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd1c00a11e0 msgr2=0x7fd1c00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd1c00a11e0 0x7fd1c00a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800b5c0 tx=0x7fd1c8005960 comp rx=0 tx=0).stop 2023-06-23T18:21:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 -- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d8100620 msgr2=0x7fd1d819cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:40.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d8100620 0x7fd1d819cde0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0007ee0 tx=0x7fd1d000d3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 -- 172.21.15.2:0/3845201474 shutdown_connections 2023-06-23T18:21:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd1d8100620 0x7fd1d819cde0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd1c00a11e0 0x7fd1c00a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1d80feaa0 0x7fd1d8195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 --2- 172.21.15.2:0/3845201474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1d80ffc90 0x7fd1d8195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 -- 172.21.15.2:0/3845201474 >> 172.21.15.2:0/3845201474 conn(0x7fd1d80fa0c0 msgr2=0x7fd1d80fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:40.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 -- 172.21.15.2:0/3845201474 shutdown_connections 2023-06-23T18:21:40.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:40.712+0000 7fd1dce65700 1 -- 172.21.15.2:0/3845201474 wait complete. 2023-06-23T18:21:40.756 INFO:tasks.cephadm:Setup complete, yielding 2023-06-23T18:21:40.756 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-06-23T18:21:40.766 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2023-06-23T18:21:40.766 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch status' 2023-06-23T18:21:41.324 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:41.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:41 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/373086637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:41 smithi082 ceph-mon[97889]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-06-23T18:21:41.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:41 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/373086637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:41.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:41 smithi017 ceph-mon[97472]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-06-23T18:21:41.533 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:41 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/373086637' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:21:41.533 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:41 smithi002 ceph-mon[95947]: pgmap v286: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 2 op/s 2023-06-23T18:21:42.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.531+0000 7f519f7ee700 1 -- 172.21.15.2:0/3264892719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 msgr2=0x7f5198102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:42.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.531+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3264892719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 0x7f5198102a10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5194009990 tx=0x7f5194009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.531+0000 7f519f7ee700 1 -- 172.21.15.2:0/3264892719 shutdown_connections 2023-06-23T18:21:42.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3264892719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5198105870 0x7f5198107c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3264892719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 0x7f5198102a10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3264892719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5198102f50 0x7f5198105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 -- 172.21.15.2:0/3264892719 >> 172.21.15.2:0/3264892719 conn(0x7f51980fa070 msgr2=0x7f51980fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:42.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 -- 172.21.15.2:0/3264892719 shutdown_connections 2023-06-23T18:21:42.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 -- 172.21.15.2:0/3264892719 wait complete. 2023-06-23T18:21:42.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 Processor -- start 2023-06-23T18:21:42.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 -- start start 2023-06-23T18:21:42.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 0x7f51980fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:42.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.532+0000 7f519f7ee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5198102f50 0x7f51980fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:42.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519f7ee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5198105870 0x7f51980feb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:42.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519f7ee700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980ff2a0 con 0x7f5198100630 2023-06-23T18:21:42.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519f7ee700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980ff410 con 0x7f5198105870 2023-06-23T18:21:42.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519f7ee700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519819f840 con 0x7f5198102f50 2023-06-23T18:21:42.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 0x7f51980fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:42.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 0x7f51980fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53350/0 (socket says 172.21.15.2:53350) 2023-06-23T18:21:42.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 -- 172.21.15.2:0/3759028347 learned_addr learned my addr 172.21.15.2:0/3759028347 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:42.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519cd89700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5198102f50 0x7f51980fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:42.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519dd8b700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5198105870 0x7f51980feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:42.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 -- 172.21.15.2:0/3759028347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5198102f50 msgr2=0x7f51980fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:42.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5198102f50 0x7f51980fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 -- 172.21.15.2:0/3759028347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5198105870 msgr2=0x7f51980feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:42.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5198105870 0x7f51980feb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 -- 172.21.15.2:0/3759028347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940095f0 con 0x7f5198100630 2023-06-23T18:21:42.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519d58a700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 0x7f51980fff90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51940098e0 tx=0x7f5194004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:42.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f518e7fc700 1 -- 172.21.15.2:0/3759028347 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519401d070 con 0x7f5198100630 2023-06-23T18:21:42.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f518e7fc700 1 -- 172.21.15.2:0/3759028347 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f519400be30 con 0x7f5198100630 2023-06-23T18:21:42.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f518e7fc700 1 -- 172.21.15.2:0/3759028347 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194022e00 con 0x7f5198100630 2023-06-23T18:21:42.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519819fac0 con 0x7f5198100630 2023-06-23T18:21:42.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.533+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519819fe80 con 0x7f5198100630 2023-06-23T18:21:42.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.534+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51980fbbe0 con 0x7f5198100630 2023-06-23T18:21:42.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.535+0000 7f518e7fc700 1 -- 172.21.15.2:0/3759028347 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f519400b9a0 con 0x7f5198100630 2023-06-23T18:21:42.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.536+0000 7f518e7fc700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51840a11e0 0x7f51840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:42.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.536+0000 7f519cd89700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51840a11e0 0x7f51840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:42.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.536+0000 7f518e7fc700 1 -- 172.21.15.2:0/3759028347 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f51940ce370 con 0x7f5198100630 2023-06-23T18:21:42.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.537+0000 7f519cd89700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51840a11e0 0x7f51840a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5188005950 tx=0x7f518800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:42.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.539+0000 7f518e7fc700 1 -- 172.21.15.2:0/3759028347 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f51940979f0 con 0x7f5198100630 2023-06-23T18:21:42.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.754+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5198060910 con 0x7f51840a11e0 2023-06-23T18:21:42.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.755+0000 7f518e7fc700 1 -- 172.21.15.2:0/3759028347 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5198060910 con 0x7f51840a11e0 2023-06-23T18:21:42.760 INFO:teuthology.orchestra.run.smithi002.stdout:Backend: cephadm 2023-06-23T18:21:42.760 INFO:teuthology.orchestra.run.smithi002.stdout:Available: Yes 2023-06-23T18:21:42.761 INFO:teuthology.orchestra.run.smithi002.stdout:Paused: No 2023-06-23T18:21:42.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51840a11e0 msgr2=0x7f51840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:42.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51840a11e0 0x7f51840a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5188005950 tx=0x7f518800b410 comp rx=0 tx=0).stop 2023-06-23T18:21:42.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 msgr2=0x7f51980fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:42.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 0x7f51980fff90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51940098e0 tx=0x7f5194004730 comp rx=0 tx=0).stop 2023-06-23T18:21:42.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 shutdown_connections 2023-06-23T18:21:42.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f51840a11e0 0x7f51840a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5198105870 0x7f51980feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5198100630 0x7f51980fff90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 --2- 172.21.15.2:0/3759028347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5198102f50 0x7f51980fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:42.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 >> 172.21.15.2:0/3759028347 conn(0x7f51980fa070 msgr2=0x7f5198101290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:42.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 shutdown_connections 2023-06-23T18:21:42.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:42.758+0000 7f519f7ee700 1 -- 172.21.15.2:0/3759028347 wait complete. 2023-06-23T18:21:43.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:42 smithi002 ceph-mon[95947]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.0 KiB/s, 1 objects/s recovering 2023-06-23T18:21:43.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:42 smithi002 ceph-mon[95947]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:42 smithi082 ceph-mon[97889]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.0 KiB/s, 1 objects/s recovering 2023-06-23T18:21:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:42 smithi082 ceph-mon[97889]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:43.189 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch ps' 2023-06-23T18:21:43.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:42 smithi017 ceph-mon[97472]: pgmap v287: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 7.0 KiB/s, 1 objects/s recovering 2023-06-23T18:21:43.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:42 smithi017 ceph-mon[97472]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:43.753 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 -- 172.21.15.2:0/1325273353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 msgr2=0x7f3170100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:45.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 --2- 172.21.15.2:0/1325273353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 0x7f3170100090 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3160009990 tx=0x7f3160009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 -- 172.21.15.2:0/1325273353 shutdown_connections 2023-06-23T18:21:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 --2- 172.21.15.2:0/1325273353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31701005d0 0x7f3170106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 --2- 172.21.15.2:0/1325273353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31700fea40 0x7f31700fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 --2- 172.21.15.2:0/1325273353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 0x7f3170100090 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 -- 172.21.15.2:0/1325273353 >> 172.21.15.2:0/1325273353 conn(0x7f31700fa070 msgr2=0x7f31700fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:45.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 -- 172.21.15.2:0/1325273353 shutdown_connections 2023-06-23T18:21:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.162+0000 7f3177634700 1 -- 172.21.15.2:0/1325273353 wait complete. 2023-06-23T18:21:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3177634700 1 Processor -- start 2023-06-23T18:21:45.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3177634700 1 -- start start 2023-06-23T18:21:45.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3177634700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31700fea40 0x7f31701945e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3177634700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 0x7f3170194b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3177634700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31701005d0 0x7f317019bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3177634700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019c170 con 0x7f31700ffc40 2023-06-23T18:21:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3177634700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019c2b0 con 0x7f31700fea40 2023-06-23T18:21:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3177634700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317019c440 con 0x7f31701005d0 2023-06-23T18:21:45.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 0x7f3170194b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:45.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 0x7f3170194b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53364/0 (socket says 172.21.15.2:53364) 2023-06-23T18:21:45.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 -- 172.21.15.2:0/2961050019 learned_addr learned my addr 172.21.15.2:0/2961050019 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:45.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3175bd1700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31701005d0 0x7f317019bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:45.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f31753d0700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31700fea40 0x7f31701945e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:45.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 -- 172.21.15.2:0/2961050019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31701005d0 msgr2=0x7f317019bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:45.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31701005d0 0x7f317019bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 -- 172.21.15.2:0/2961050019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31700fea40 msgr2=0x7f31701945e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:45.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31700fea40 0x7f31701945e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 -- 172.21.15.2:0/2961050019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31600095f0 con 0x7f31700ffc40 2023-06-23T18:21:45.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.163+0000 7f3174bcf700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 0x7f3170194b20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f31600059d0 tx=0x7f3160004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:45.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.164+0000 7f31667fc700 1 -- 172.21.15.2:0/2961050019 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316001d070 con 0x7f31700ffc40 2023-06-23T18:21:45.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.164+0000 7f31667fc700 1 -- 172.21.15.2:0/2961050019 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3160004080 con 0x7f31700ffc40 2023-06-23T18:21:45.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.164+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317019c6c0 con 0x7f31700ffc40 2023-06-23T18:21:45.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.164+0000 7f31667fc700 1 -- 172.21.15.2:0/2961050019 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3160022c50 con 0x7f31700ffc40 2023-06-23T18:21:45.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.164+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317019cb30 con 0x7f31700ffc40 2023-06-23T18:21:45.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.165+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3170106040 con 0x7f31700ffc40 2023-06-23T18:21:45.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.166+0000 7f31667fc700 1 -- 172.21.15.2:0/2961050019 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3160022470 con 0x7f31700ffc40 2023-06-23T18:21:45.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.166+0000 7f31667fc700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f315c0a1260 0x7f315c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:45.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.167+0000 7f31753d0700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f315c0a1260 0x7f315c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:45.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.167+0000 7f31667fc700 1 -- 172.21.15.2:0/2961050019 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f31600cdca0 con 0x7f31700ffc40 2023-06-23T18:21:45.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.167+0000 7f31753d0700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f315c0a1260 0x7f315c0a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f316c005950 tx=0x7f316c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:45.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.171+0000 7f31667fc700 1 -- 172.21.15.2:0/2961050019 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f316005c770 con 0x7f31700ffc40 2023-06-23T18:21:45.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:45 smithi002 ceph-mon[95947]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 7.0 KiB/s, 1 objects/s recovering 2023-06-23T18:21:45.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:45 smithi082 ceph-mon[97889]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 7.0 KiB/s, 1 objects/s recovering 2023-06-23T18:21:45.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.380+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3170060910 con 0x7f315c0a1260 2023-06-23T18:21:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.400+0000 7f31667fc700 1 -- 172.21.15.2:0/2961050019 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f3170060910 con 0x7f315c0a1260 2023-06-23T18:21:45.406 INFO:teuthology.orchestra.run.smithi002.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-06-23T18:21:45.406 INFO:teuthology.orchestra.run.smithi002.stdout:mds.a smithi002 running (67s) 57s ago 67s 13.5M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 6c4b0e04ae42 2023-06-23T18:21:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:mds.b smithi017 running (65s) 57s ago 64s 16.7M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 d32e7bd8abcd 2023-06-23T18:21:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:mds.c smithi082 running (77s) 58s ago 76s 15.8M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 6e552866e880 2023-06-23T18:21:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:mds.d smithi002 running (84s) 57s ago 83s 14.4M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 83dec644cca9 2023-06-23T18:21:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:mds.e smithi017 running (81s) 57s ago 81s 14.4M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 469345b6da28 2023-06-23T18:21:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:mds.f smithi082 running (79s) 58s ago 79s 13.5M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 03d890d4768b 2023-06-23T18:21:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:mds.g smithi002 running (70s) 57s ago 69s 13.2M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 b015140c5f56 2023-06-23T18:21:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:mds.h smithi017 running (74s) 57s ago 74s 13.4M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 f0778750c8dd 2023-06-23T18:21:45.407 INFO:teuthology.orchestra.run.smithi002.stdout:mds.i smithi082 running (72s) 58s ago 71s 14.2M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 85759418013a 2023-06-23T18:21:45.408 INFO:teuthology.orchestra.run.smithi002.stdout:mds.j smithi002 running (86s) 57s ago 85s 15.8M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 82c5d9966d7d 2023-06-23T18:21:45.408 INFO:teuthology.orchestra.run.smithi002.stdout:mds.k smithi017 running (88s) 57s ago 88s 14.2M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 465a2a6199a4 2023-06-23T18:21:45.408 INFO:teuthology.orchestra.run.smithi002.stdout:mds.l smithi082 running (90s) 58s ago 90s 18.7M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 eec95ac97356 2023-06-23T18:21:45.408 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.x smithi002 *:9283,8765 running (7m) 57s ago 7m 510M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 816801aad85c 2023-06-23T18:21:45.408 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.y smithi017 *:8443,8765 running (6m) 57s ago 6m 448M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 57366a7a68ec 2023-06-23T18:21:45.408 INFO:teuthology.orchestra.run.smithi002.stdout:mgr.z smithi082 *:8443,8765 running (6m) 58s ago 6m 448M - 18.0.0-4479-g8627af3e 2a05a4fe3a20 49520cddde6b 2023-06-23T18:21:45.408 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a smithi002 running (8m) 57s ago 8m 86.3M 2048M 18.0.0-4479-g8627af3e 2a05a4fe3a20 b4bd2bd2d707 2023-06-23T18:21:45.409 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b smithi017 running (6m) 57s ago 6m 61.9M 2048M 18.0.0-4479-g8627af3e 2a05a4fe3a20 23368a99f9d1 2023-06-23T18:21:45.409 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c smithi082 running (6m) 58s ago 6m 60.7M 2048M 18.0.0-4479-g8627af3e 2a05a4fe3a20 33104cbb816a 2023-06-23T18:21:45.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 smithi002 running (5m) 57s ago 5m 62.0M 4293M 18.0.0-4479-g8627af3e 2a05a4fe3a20 41f79cc5739d 2023-06-23T18:21:45.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 smithi017 running (5m) 57s ago 5m 67.9M 4293M 18.0.0-4479-g8627af3e 2a05a4fe3a20 b7b8c7f32939 2023-06-23T18:21:45.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 smithi082 running (5m) 58s ago 5m 63.5M 4292M 18.0.0-4479-g8627af3e 2a05a4fe3a20 21a2804ecf71 2023-06-23T18:21:45.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 smithi002 running (4m) 57s ago 4m 59.8M 4293M 18.0.0-4479-g8627af3e 2a05a4fe3a20 002248aff8f2 2023-06-23T18:21:45.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 smithi017 running (4m) 57s ago 4m 55.9M 4293M 18.0.0-4479-g8627af3e 2a05a4fe3a20 9182a9f165ef 2023-06-23T18:21:45.409 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 smithi082 running (3m) 58s ago 3m 61.0M 4292M 18.0.0-4479-g8627af3e 2a05a4fe3a20 1804a8797a80 2023-06-23T18:21:45.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 smithi002 running (3m) 57s ago 3m 51.7M 4293M 18.0.0-4479-g8627af3e 2a05a4fe3a20 a715ad1ace64 2023-06-23T18:21:45.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 smithi017 running (3m) 57s ago 3m 56.9M 4293M 18.0.0-4479-g8627af3e 2a05a4fe3a20 828e906bc69f 2023-06-23T18:21:45.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 smithi082 running (2m) 58s ago 2m 54.9M 4292M 18.0.0-4479-g8627af3e 2a05a4fe3a20 6b941a97f777 2023-06-23T18:21:45.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 smithi002 running (2m) 57s ago 2m 49.2M 4293M 18.0.0-4479-g8627af3e 2a05a4fe3a20 a08c64415be2 2023-06-23T18:21:45.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 smithi017 running (2m) 57s ago 2m 53.1M 4293M 18.0.0-4479-g8627af3e 2a05a4fe3a20 f24a2d48e332 2023-06-23T18:21:45.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 smithi082 running (109s) 58s ago 109s 46.5M 4292M 18.0.0-4479-g8627af3e 2a05a4fe3a20 87566b9804ea 2023-06-23T18:21:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f315c0a1260 msgr2=0x7f315c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f315c0a1260 0x7f315c0a3710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f316c005950 tx=0x7f316c0058e0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 msgr2=0x7f3170194b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 0x7f3170194b20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f31600059d0 tx=0x7f3160004730 comp rx=0 tx=0).stop 2023-06-23T18:21:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 shutdown_connections 2023-06-23T18:21:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31701005d0 0x7f317019bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f315c0a1260 0x7f315c0a3710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31700fea40 0x7f31701945e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 --2- 172.21.15.2:0/2961050019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31700ffc40 0x7f3170194b20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 >> 172.21.15.2:0/2961050019 conn(0x7f31700fa070 msgr2=0x7f3170103180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 shutdown_connections 2023-06-23T18:21:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:45.403+0000 7f3177634700 1 -- 172.21.15.2:0/2961050019 wait complete. 2023-06-23T18:21:45.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:45 smithi017 ceph-mon[97472]: pgmap v288: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 7.0 KiB/s, 1 objects/s recovering 2023-06-23T18:21:45.923 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch ls' 2023-06-23T18:21:46.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:46 smithi002 ceph-mon[95947]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:46.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:46 smithi082 ceph-mon[97889]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:46.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:46 smithi017 ceph-mon[97472]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:46.487 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:47.324 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:47 smithi002 ceph-mon[95947]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 objects/s recovering 2023-06-23T18:21:47.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:47 smithi082 ceph-mon[97889]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 objects/s recovering 2023-06-23T18:21:47.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:47 smithi017 ceph-mon[97472]: pgmap v289: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s; 5.8 KiB/s, 1 objects/s recovering 2023-06-23T18:21:47.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 -- 172.21.15.2:0/3561802029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7841058b0 msgr2=0x7fd784107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:47.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 --2- 172.21.15.2:0/3561802029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7841058b0 0x7fd784107c90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00b0a0 tx=0x7fd77c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:47.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 -- 172.21.15.2:0/3561802029 shutdown_connections 2023-06-23T18:21:47.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 --2- 172.21.15.2:0/3561802029 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd7841058b0 0x7fd784107c90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:47.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 --2- 172.21.15.2:0/3561802029 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd784100670 0x7fd784102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:47.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 --2- 172.21.15.2:0/3561802029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd784102f90 0x7fd784105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:47.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 -- 172.21.15.2:0/3561802029 >> 172.21.15.2:0/3561802029 conn(0x7fd7840fa070 msgr2=0x7fd7840fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:47.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 -- 172.21.15.2:0/3561802029 shutdown_connections 2023-06-23T18:21:47.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.777+0000 7fd78b305700 1 -- 172.21.15.2:0/3561802029 wait complete. 2023-06-23T18:21:47.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd78b305700 1 Processor -- start 2023-06-23T18:21:47.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd78b305700 1 -- start start 2023-06-23T18:21:47.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd78b305700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd784100670 0x7fd78410dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:47.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd78b305700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784102f90 0x7fd78410e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:47.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd78b305700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7841058b0 0x7fd78410a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:47.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd78b305700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78410af80 con 0x7fd784102f90 2023-06-23T18:21:47.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd78b305700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78410b0f0 con 0x7fd7841058b0 2023-06-23T18:21:47.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd78b305700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78410b2b0 con 0x7fd784100670 2023-06-23T18:21:47.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7888a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784102f90 0x7fd78410e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:47.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7888a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784102f90 0x7fd78410e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53378/0 (socket says 172.21.15.2:53378) 2023-06-23T18:21:47.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7888a0700 1 -- 172.21.15.2:0/967768051 learned_addr learned my addr 172.21.15.2:0/967768051 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:47.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7890a1700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd784100670 0x7fd78410dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:47.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7898a2700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7841058b0 0x7fd78410a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:47.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7888a0700 1 -- 172.21.15.2:0/967768051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd784100670 msgr2=0x7fd78410dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:47.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7888a0700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd784100670 0x7fd78410dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:47.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7888a0700 1 -- 172.21.15.2:0/967768051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7841058b0 msgr2=0x7fd78410a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:47.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7888a0700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7841058b0 0x7fd78410a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:47.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.778+0000 7fd7888a0700 1 -- 172.21.15.2:0/967768051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c009d00 con 0x7fd784102f90 2023-06-23T18:21:47.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.779+0000 7fd7898a2700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7841058b0 0x7fd78410a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:47.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.779+0000 7fd7888a0700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784102f90 0x7fd78410e1a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd77400d950 tx=0x7fd77400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:47.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.779+0000 7fd77a7fc700 1 -- 172.21.15.2:0/967768051 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774009940 con 0x7fd784102f90 2023-06-23T18:21:47.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.779+0000 7fd77a7fc700 1 -- 172.21.15.2:0/967768051 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd774010460 con 0x7fd784102f90 2023-06-23T18:21:47.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.779+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7841a3ff0 con 0x7fd784102f90 2023-06-23T18:21:47.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.779+0000 7fd77a7fc700 1 -- 172.21.15.2:0/967768051 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77400f620 con 0x7fd784102f90 2023-06-23T18:21:47.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.779+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7841a44c0 con 0x7fd784102f90 2023-06-23T18:21:47.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.781+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7840ff390 con 0x7fd784102f90 2023-06-23T18:21:47.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.781+0000 7fd77a7fc700 1 -- 172.21.15.2:0/967768051 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd7740105d0 con 0x7fd784102f90 2023-06-23T18:21:47.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.782+0000 7fd77a7fc700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd7700a1260 0x7fd7700a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:47.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.782+0000 7fd7890a1700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd7700a1260 0x7fd7700a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:47.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.782+0000 7fd77a7fc700 1 -- 172.21.15.2:0/967768051 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd7740cc850 con 0x7fd784102f90 2023-06-23T18:21:47.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.783+0000 7fd7890a1700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd7700a1260 0x7fd7700a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd780005950 tx=0x7fd78000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:47.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:47.786+0000 7fd77a7fc700 1 -- 172.21.15.2:0/967768051 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fd774095e40 con 0x7fd784102f90 2023-06-23T18:21:48.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.027+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7841022e0 con 0x7fd7700a1260 2023-06-23T18:21:48.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.034+0000 7fd77a7fc700 1 -- 172.21.15.2:0/967768051 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fd7841022e0 con 0x7fd7700a1260 2023-06-23T18:21:48.040 INFO:teuthology.orchestra.run.smithi002.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-06-23T18:21:48.040 INFO:teuthology.orchestra.run.smithi002.stdout:mds.all 12/12 60s ago 95s smithi002=a;smithi002=d;smithi002=g;smithi002=j;smithi017=b;smithi017=e;smithi017=h;smithi017=k;smithi082=c;smithi082=f;smithi082=i;smithi082=l;count:12 2023-06-23T18:21:48.040 INFO:teuthology.orchestra.run.smithi002.stdout:mgr 3/3 60s ago 6m smithi002=x;smithi017=y;smithi082=z;count:3 2023-06-23T18:21:48.040 INFO:teuthology.orchestra.run.smithi002.stdout:mon 3/3 60s ago 6m smithi002:172.21.15.2=a;smithi017:172.21.15.17=b;smithi082:172.21.15.82=c;count:3 2023-06-23T18:21:48.041 INFO:teuthology.orchestra.run.smithi002.stdout:osd 12 60s ago - 2023-06-23T18:21:48.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd7700a1260 msgr2=0x7fd7700a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:48.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd7700a1260 0x7fd7700a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd780005950 tx=0x7fd78000a300 comp rx=0 tx=0).stop 2023-06-23T18:21:48.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784102f90 msgr2=0x7fd78410e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:48.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784102f90 0x7fd78410e1a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd77400d950 tx=0x7fd77400dc60 comp rx=0 tx=0).stop 2023-06-23T18:21:48.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 shutdown_connections 2023-06-23T18:21:48.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd7700a1260 0x7fd7700a3710 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:48.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7841058b0 0x7fd78410a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:48.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd784100670 0x7fd78410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:48.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 --2- 172.21.15.2:0/967768051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd784102f90 0x7fd78410e1a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:48.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.036+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 >> 172.21.15.2:0/967768051 conn(0x7fd7840fa070 msgr2=0x7fd784105100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:48.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.037+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 shutdown_connections 2023-06-23T18:21:48.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:48.037+0000 7fd78b305700 1 -- 172.21.15.2:0/967768051 wait complete. 2023-06-23T18:21:48.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:48 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:21:48.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:48 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:21:48.442 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:48 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:21:48.443 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch host ls' 2023-06-23T18:21:49.122 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:49.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:49 smithi082 ceph-mon[97889]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:49.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:49 smithi082 ceph-mon[97889]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 5.3 KiB/s, 1 objects/s recovering 2023-06-23T18:21:49.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:49 smithi017 ceph-mon[97472]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:49.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:49 smithi017 ceph-mon[97472]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 5.3 KiB/s, 1 objects/s recovering 2023-06-23T18:21:49.491 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:49 smithi002 ceph-mon[95947]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:49.492 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:49 smithi002 ceph-mon[95947]: pgmap v290: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 5.3 KiB/s, 1 objects/s recovering 2023-06-23T18:21:50.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 -- 172.21.15.2:0/3678943097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8100dd0 msgr2=0x7fcca8101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:50.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 --2- 172.21.15.2:0/3678943097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8100dd0 0x7fcca8101220 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009990 tx=0x7fcc9c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 -- 172.21.15.2:0/3678943097 shutdown_connections 2023-06-23T18:21:50.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 --2- 172.21.15.2:0/3678943097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca8101760 0x7fcca8108020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 --2- 172.21.15.2:0/3678943097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcca80ffbd0 0x7fcca80fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 --2- 172.21.15.2:0/3678943097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8100dd0 0x7fcca8101220 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 -- 172.21.15.2:0/3678943097 >> 172.21.15.2:0/3678943097 conn(0x7fcca80fb1e0 msgr2=0x7fcca80fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:50.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 -- 172.21.15.2:0/3678943097 shutdown_connections 2023-06-23T18:21:50.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.488+0000 7fccaea63700 1 -- 172.21.15.2:0/3678943097 wait complete. 2023-06-23T18:21:50.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.489+0000 7fccaea63700 1 Processor -- start 2023-06-23T18:21:50.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccaea63700 1 -- start start 2023-06-23T18:21:50.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccaea63700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca80ffbd0 0x7fcca81957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:50.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccaea63700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcca8100dd0 0x7fcca8195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:50.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccaea63700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8101760 0x7fcca819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:50.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccaea63700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca819d340 con 0x7fcca8101760 2023-06-23T18:21:50.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccaea63700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca819d480 con 0x7fcca8100dd0 2023-06-23T18:21:50.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccaea63700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca819d610 con 0x7fcca80ffbd0 2023-06-23T18:21:50.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fcca7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca80ffbd0 0x7fcca81957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:50.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccad000700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8101760 0x7fcca819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:50.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fcca7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca80ffbd0 0x7fcca81957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:53726/0 (socket says 172.21.15.2:53726) 2023-06-23T18:21:50.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fcca7fff700 1 -- 172.21.15.2:0/3837463927 learned_addr learned my addr 172.21.15.2:0/3837463927 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:50.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fcca77fe700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcca8100dd0 0x7fcca8195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:50.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccad000700 1 -- 172.21.15.2:0/3837463927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca80ffbd0 msgr2=0x7fcca81957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:50.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccad000700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca80ffbd0 0x7fcca81957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccad000700 1 -- 172.21.15.2:0/3837463927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcca8100dd0 msgr2=0x7fcca8195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:50.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccad000700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcca8100dd0 0x7fcca8195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.490+0000 7fccad000700 1 -- 172.21.15.2:0/3837463927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c0095f0 con 0x7fcca8101760 2023-06-23T18:21:50.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.491+0000 7fcca77fe700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcca8100dd0 0x7fcca8195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:21:50.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.491+0000 7fcca7fff700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca80ffbd0 0x7fcca81957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:21:50.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.491+0000 7fccad000700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8101760 0x7fcca819cd70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcca0007f40 tx=0x7fcca000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:50.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.491+0000 7fcca57fa700 1 -- 172.21.15.2:0/3837463927 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0009ea0 con 0x7fcca8101760 2023-06-23T18:21:50.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.491+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca819d8f0 con 0x7fcca8101760 2023-06-23T18:21:50.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.491+0000 7fcca57fa700 1 -- 172.21.15.2:0/3837463927 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcca000f040 con 0x7fcca8101760 2023-06-23T18:21:50.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.491+0000 7fcca57fa700 1 -- 172.21.15.2:0/3837463927 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca0004280 con 0x7fcca8101760 2023-06-23T18:21:50.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.491+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca819ddf0 con 0x7fcca8101760 2023-06-23T18:21:50.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.492+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca8106bf0 con 0x7fcca8101760 2023-06-23T18:21:50.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.494+0000 7fcca57fa700 1 -- 172.21.15.2:0/3837463927 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fcca000db20 con 0x7fcca8101760 2023-06-23T18:21:50.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.495+0000 7fcca57fa700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc900a11e0 0x7fcc900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:50.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.495+0000 7fcca7fff700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc900a11e0 0x7fcc900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:50.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.495+0000 7fcca57fa700 1 -- 172.21.15.2:0/3837463927 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcca00cccd0 con 0x7fcca8101760 2023-06-23T18:21:50.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.495+0000 7fcca7fff700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc900a11e0 0x7fcc900a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc98007fd0 tx=0x7fcc98007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:50.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.498+0000 7fcca57fa700 1 -- 172.21.15.2:0/3837463927 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcca0096280 con 0x7fcca8101760 2023-06-23T18:21:50.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.719+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcca8060910 con 0x7fcc900a11e0 2023-06-23T18:21:50.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.720+0000 7fcca57fa700 1 -- 172.21.15.2:0/3837463927 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+187 (secure 0 0 0) 0x7fcca8060910 con 0x7fcc900a11e0 2023-06-23T18:21:50.726 INFO:teuthology.orchestra.run.smithi002.stdout:HOST ADDR LABELS STATUS 2023-06-23T18:21:50.726 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 172.21.15.2 2023-06-23T18:21:50.726 INFO:teuthology.orchestra.run.smithi002.stdout:smithi017 172.21.15.17 2023-06-23T18:21:50.726 INFO:teuthology.orchestra.run.smithi002.stdout:smithi082 172.21.15.82 2023-06-23T18:21:50.726 INFO:teuthology.orchestra.run.smithi002.stdout:3 hosts in cluster 2023-06-23T18:21:50.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc900a11e0 msgr2=0x7fcc900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:50.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc900a11e0 0x7fcc900a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc98007fd0 tx=0x7fcc98007480 comp rx=0 tx=0).stop 2023-06-23T18:21:50.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8101760 msgr2=0x7fcca819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:50.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8101760 0x7fcca819cd70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcca0007f40 tx=0x7fcca000d3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 shutdown_connections 2023-06-23T18:21:50.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcca8101760 0x7fcca819cd70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca80ffbd0 0x7fcca81957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcc900a11e0 0x7fcc900a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 --2- 172.21.15.2:0/3837463927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcca8100dd0 0x7fcca8195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:50.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 >> 172.21.15.2:0/3837463927 conn(0x7fcca80fb1e0 msgr2=0x7fcca80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:50.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 shutdown_connections 2023-06-23T18:21:50.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:50.723+0000 7fccaea63700 1 -- 172.21.15.2:0/3837463927 wait complete. 2023-06-23T18:21:51.140 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- bash -c 'ceph orch device ls' 2023-06-23T18:21:51.625 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:51 smithi002 ceph-mon[95947]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:51 smithi002 ceph-mon[95947]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4.7 KiB/s, 1 objects/s recovering 2023-06-23T18:21:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:51 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:51 smithi082 ceph-mon[97889]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:51 smithi082 ceph-mon[97889]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4.7 KiB/s, 1 objects/s recovering 2023-06-23T18:21:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:51 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:51 smithi017 ceph-mon[97472]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:51.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:51 smithi017 ceph-mon[97472]: pgmap v291: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4.7 KiB/s, 1 objects/s recovering 2023-06-23T18:21:51.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:51.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:51 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:52.702 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:21:52.702 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:21:52.702 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:21:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:21:52.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 -- 172.21.15.2:0/2925065255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a400ffb50 msgr2=0x7f3a400fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/2925065255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a400ffb50 0x7f3a400fff60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009990 tx=0x7f3a30009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 -- 172.21.15.2:0/2925065255 shutdown_connections 2023-06-23T18:21:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/2925065255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a401016e0 0x7f3a40107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/2925065255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a400ffb50 0x7f3a400fff60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:52.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/2925065255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a40100d50 0x7f3a401011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:52.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 -- 172.21.15.2:0/2925065255 >> 172.21.15.2:0/2925065255 conn(0x7f3a400fb160 msgr2=0x7f3a400fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:52.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 -- 172.21.15.2:0/2925065255 shutdown_connections 2023-06-23T18:21:52.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.886+0000 7f3a45c5f700 1 -- 172.21.15.2:0/2925065255 wait complete. 2023-06-23T18:21:52.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.887+0000 7f3a45c5f700 1 Processor -- start 2023-06-23T18:21:52.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.887+0000 7f3a45c5f700 1 -- start start 2023-06-23T18:21:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a45c5f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a400ffb50 0x7f3a401979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a45c5f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a40100d50 0x7f3a40197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a45c5f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a401016e0 0x7f3a4019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a45c5f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4019f580 con 0x7f3a40100d50 2023-06-23T18:21:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a45c5f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4019f6c0 con 0x7f3a401016e0 2023-06-23T18:21:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a45c5f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4019f850 con 0x7f3a400ffb50 2023-06-23T18:21:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a40100d50 0x7f3a40197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a40100d50 0x7f3a40197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53402/0 (socket says 172.21.15.2:53402) 2023-06-23T18:21:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 -- 172.21.15.2:0/239857172 learned_addr learned my addr 172.21.15.2:0/239857172 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3ffff700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a401016e0 0x7f3a4019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3f7fe700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a400ffb50 0x7f3a401979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 -- 172.21.15.2:0/239857172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a400ffb50 msgr2=0x7f3a401979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a400ffb50 0x7f3a401979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 -- 172.21.15.2:0/239857172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a401016e0 msgr2=0x7f3a4019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a401016e0 0x7f3a4019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:52.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 -- 172.21.15.2:0/239857172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300095f0 con 0x7f3a40100d50 2023-06-23T18:21:52.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3effd700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a40100d50 0x7f3a40197f30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400d980 tx=0x7f3a3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:52.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3cff9700 1 -- 172.21.15.2:0/239857172 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a340098d0 con 0x7f3a40100d50 2023-06-23T18:21:52.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3cff9700 1 -- 172.21.15.2:0/239857172 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a34010460 con 0x7f3a40100d50 2023-06-23T18:21:52.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a3cff9700 1 -- 172.21.15.2:0/239857172 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3400f620 con 0x7f3a40100d50 2023-06-23T18:21:52.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.888+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4019fb30 con 0x7f3a40100d50 2023-06-23T18:21:52.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.889+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4019ffe0 con 0x7f3a40100d50 2023-06-23T18:21:52.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.889+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a40192100 con 0x7f3a40100d50 2023-06-23T18:21:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.891+0000 7f3a3cff9700 1 -- 172.21.15.2:0/239857172 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3a34009a30 con 0x7f3a40100d50 2023-06-23T18:21:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.891+0000 7f3a3cff9700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a280a11e0 0x7f3a280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.891+0000 7f3a3f7fe700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a280a11e0 0x7f3a280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.891+0000 7f3a3cff9700 1 -- 172.21.15.2:0/239857172 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3a340cc820 con 0x7f3a40100d50 2023-06-23T18:21:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.892+0000 7f3a3f7fe700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a280a11e0 0x7f3a280a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000b5c0 tx=0x7f3a30005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:52.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:52.895+0000 7f3a3cff9700 1 -- 172.21.15.2:0/239857172 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3a34095e10 con 0x7f3a40100d50 2023-06-23T18:21:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:21:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:21:52.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:21:53.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.102+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a40060910 con 0x7f3a280a11e0 2023-06-23T18:21:53.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.104+0000 7f3a3cff9700 1 -- 172.21.15.2:0/239857172 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f3a40060910 con 0x7f3a280a11e0 2023-06-23T18:21:53.110 INFO:teuthology.orchestra.run.smithi002.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-06-23T18:21:53.110 INFO:teuthology.orchestra.run.smithi002.stdout:smithi002 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EV480DGN 447G No 2m ago LVM detected, locked 2023-06-23T18:21:53.110 INFO:teuthology.orchestra.run.smithi002.stdout:smithi017 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7473003Q480DGN 447G No 2m ago LVM detected, locked 2023-06-23T18:21:53.110 INFO:teuthology.orchestra.run.smithi002.stdout:smithi082 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400SD400BGN 372G No 111s ago LVM detected, locked 2023-06-23T18:21:53.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a280a11e0 msgr2=0x7f3a280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:53.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a280a11e0 0x7f3a280a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a3000b5c0 tx=0x7f3a30005fb0 comp rx=0 tx=0).stop 2023-06-23T18:21:53.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a40100d50 msgr2=0x7f3a40197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:53.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a40100d50 0x7f3a40197f30 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400d980 tx=0x7f3a3400dc90 comp rx=0 tx=0).stop 2023-06-23T18:21:53.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 shutdown_connections 2023-06-23T18:21:53.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a280a11e0 0x7f3a280a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:53.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a401016e0 0x7f3a4019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:53.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a400ffb50 0x7f3a401979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:53.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 --2- 172.21.15.2:0/239857172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a40100d50 0x7f3a40197f30 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:53.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 >> 172.21.15.2:0/239857172 conn(0x7f3a400fb160 msgr2=0x7f3a401066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:53.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 shutdown_connections 2023-06-23T18:21:53.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:53.106+0000 7f3a45c5f700 1 -- 172.21.15.2:0/239857172 wait complete. 2023-06-23T18:21:53.576 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-06-23T18:21:53.586 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi002.front.sepia.ceph.com 2023-06-23T18:21:53.586 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- bash -c 'ceph fs dump' 2023-06-23T18:21:53.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:53 smithi002 ceph-mon[95947]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4.7 KiB/s, 1 objects/s recovering 2023-06-23T18:21:53.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:53 smithi082 ceph-mon[97889]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4.7 KiB/s, 1 objects/s recovering 2023-06-23T18:21:53.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:53 smithi017 ceph-mon[97472]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4.7 KiB/s, 1 objects/s recovering 2023-06-23T18:21:54.148 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:54.666 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:54 smithi002 ceph-mon[95947]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:54 smithi082 ceph-mon[97889]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:54.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:54 smithi017 ceph-mon[97472]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-06-23T18:21:55.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 -- 172.21.15.2:0/2952470471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0ffc40 msgr2=0x7fa43c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:55.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 --2- 172.21.15.2:0/2952470471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0ffc40 0x7fa43c100090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa428009990 tx=0x7fa428009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 -- 172.21.15.2:0/2952470471 shutdown_connections 2023-06-23T18:21:55.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 --2- 172.21.15.2:0/2952470471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa43c1005d0 0x7fa43c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 --2- 172.21.15.2:0/2952470471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa43c0fea40 0x7fa43c0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 --2- 172.21.15.2:0/2952470471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0ffc40 0x7fa43c100090 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 -- 172.21.15.2:0/2952470471 >> 172.21.15.2:0/2952470471 conn(0x7fa43c0fa070 msgr2=0x7fa43c0fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 -- 172.21.15.2:0/2952470471 shutdown_connections 2023-06-23T18:21:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.425+0000 7fa442d0a700 1 -- 172.21.15.2:0/2952470471 wait complete. 2023-06-23T18:21:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa442d0a700 1 Processor -- start 2023-06-23T18:21:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa442d0a700 1 -- start start 2023-06-23T18:21:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa442d0a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0fea40 0x7fa43c068af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa442d0a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa43c0ffc40 0x7fa43c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa442d0a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa43c1005d0 0x7fa43c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa442d0a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c19d270 con 0x7fa43c0fea40 2023-06-23T18:21:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa442d0a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c19d3e0 con 0x7fa43c0ffc40 2023-06-23T18:21:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa442d0a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c19d5a0 con 0x7fa43c1005d0 2023-06-23T18:21:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa440aa6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0fea40 0x7fa43c068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa440aa6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0fea40 0x7fa43c068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53412/0 (socket says 172.21.15.2:53412) 2023-06-23T18:21:55.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa440aa6700 1 -- 172.21.15.2:0/428676315 learned_addr learned my addr 172.21.15.2:0/428676315 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:55.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa4412a7700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa43c1005d0 0x7fa43c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:55.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa43bfff700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa43c0ffc40 0x7fa43c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa440aa6700 1 -- 172.21.15.2:0/428676315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa43c1005d0 msgr2=0x7fa43c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa440aa6700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa43c1005d0 0x7fa43c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa440aa6700 1 -- 172.21.15.2:0/428676315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa43c0ffc40 msgr2=0x7fa43c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.426+0000 7fa440aa6700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa43c0ffc40 0x7fa43c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.427+0000 7fa440aa6700 1 -- 172.21.15.2:0/428676315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4280095f0 con 0x7fa43c0fea40 2023-06-23T18:21:55.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.427+0000 7fa440aa6700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0fea40 0x7fa43c068af0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa43000c970 tx=0x7fa43000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:55.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.427+0000 7fa439ffb700 1 -- 172.21.15.2:0/428676315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430004220 con 0x7fa43c0fea40 2023-06-23T18:21:55.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.427+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c19d880 con 0x7fa43c0fea40 2023-06-23T18:21:55.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.427+0000 7fa439ffb700 1 -- 172.21.15.2:0/428676315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa430007d30 con 0x7fa43c0fea40 2023-06-23T18:21:55.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.427+0000 7fa439ffb700 1 -- 172.21.15.2:0/428676315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430003b30 con 0x7fa43c0fea40 2023-06-23T18:21:55.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.427+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c19dd80 con 0x7fa43c0fea40 2023-06-23T18:21:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.428+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c10a0b0 con 0x7fa43c0fea40 2023-06-23T18:21:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.429+0000 7fa439ffb700 1 -- 172.21.15.2:0/428676315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fa430003c90 con 0x7fa43c0fea40 2023-06-23T18:21:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.430+0000 7fa439ffb700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa42c0a11e0 0x7fa42c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:55.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.430+0000 7fa43bfff700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa42c0a11e0 0x7fa42c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:55.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.430+0000 7fa439ffb700 1 -- 172.21.15.2:0/428676315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa4300cca60 con 0x7fa43c0fea40 2023-06-23T18:21:55.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.430+0000 7fa43bfff700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa42c0a11e0 0x7fa42c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa42800b5c0 tx=0x7fa428005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:55.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.433+0000 7fa439ffb700 1 -- 172.21.15.2:0/428676315 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa430096050 con 0x7fa43c0fea40 2023-06-23T18:21:55.535 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:55 smithi002 ceph-mon[95947]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.682+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa43c1059c0 con 0x7fa43c0fea40 2023-06-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.683+0000 7fa439ffb700 1 -- 172.21.15.2:0/428676315 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+2879 (secure 0 0 0) 0x7fa43001d070 con 0x7fa43c0fea40 2023-06-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi002.stdout:e22 2023-06-23T18:21:55.689 INFO:teuthology.orchestra.run.smithi002.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-06-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi002.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi002.stdout:legacy client fscid: 1 2023-06-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem 'cephfs' (1) 2023-06-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi002.stdout:fs_name cephfs 2023-06-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 22 2023-06-23T18:21:55.690 INFO:teuthology.orchestra.run.smithi002.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-06-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi002.stdout:created 2023-06-23T18:20:44.434596+0000 2023-06-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2023-06-23T18:20:49.041282+0000 2023-06-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi002.stdout:tableserver 0 2023-06-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi002.stdout:root 0 2023-06-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi002.stdout:session_timeout 300 2023-06-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi002.stdout:session_autoclose 300 2023-06-23T18:21:55.691 INFO:teuthology.orchestra.run.smithi002.stdout:max_file_size 1099511627776 2023-06-23T18:21:55.692 INFO:teuthology.orchestra.run.smithi002.stdout:max_xattr_size 65536 2023-06-23T18:21:55.692 INFO:teuthology.orchestra.run.smithi002.stdout:required_client_features {} 2023-06-23T18:21:55.692 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure 0 2023-06-23T18:21:55.693 INFO:teuthology.orchestra.run.smithi002.stdout:last_failure_osd_epoch 0 2023-06-23T18:21:55.693 INFO:teuthology.orchestra.run.smithi002.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-06-23T18:21:55.693 INFO:teuthology.orchestra.run.smithi002.stdout:max_mds 3 2023-06-23T18:21:55.693 INFO:teuthology.orchestra.run.smithi002.stdout:in 0,1,2 2023-06-23T18:21:55.693 INFO:teuthology.orchestra.run.smithi002.stdout:up {0=24533,1=24526,2=24505} 2023-06-23T18:21:55.693 INFO:teuthology.orchestra.run.smithi002.stdout:failed 2023-06-23T18:21:55.693 INFO:teuthology.orchestra.run.smithi002.stdout:damaged 2023-06-23T18:21:55.694 INFO:teuthology.orchestra.run.smithi002.stdout:stopped 2023-06-23T18:21:55.694 INFO:teuthology.orchestra.run.smithi002.stdout:data_pools [3] 2023-06-23T18:21:55.694 INFO:teuthology.orchestra.run.smithi002.stdout:metadata_pool 2 2023-06-23T18:21:55.694 INFO:teuthology.orchestra.run.smithi002.stdout:inline_data disabled 2023-06-23T18:21:55.694 INFO:teuthology.orchestra.run.smithi002.stdout:balancer 2023-06-23T18:21:55.694 INFO:teuthology.orchestra.run.smithi002.stdout:bal_rank_mask -1 2023-06-23T18:21:55.694 INFO:teuthology.orchestra.run.smithi002.stdout:standby_count_wanted 0 2023-06-23T18:21:55.694 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b{0:24533} state up:active seq 2 addr [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.h{0:24497} state up:standby-replay seq 1 addr [v2:172.21.15.17:6836/2388271800,v1:172.21.15.17:6837/2388271800] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.i{1:24526} state up:active seq 5 addr [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.f{1:24493} state up:standby-replay seq 1 addr [v2:172.21.15.82:6834/2287539900,v1:172.21.15.82:6835/2287539900] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c{2:24505} state up:active seq 6 addr [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.e{2:24467} state up:standby-replay seq 1 addr [v2:172.21.15.17:6834/2425660103,v1:172.21.15.17:6835/2425660103] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:55.695 INFO:teuthology.orchestra.run.smithi002.stdout:Standby daemons: 2023-06-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.2:6834/1508985058,v1:172.21.15.2:6835/1508985058] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.2:6836/3317629497,v1:172.21.15.2:6837/3317629497] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.g{-1:14616} state up:standby seq 1 addr [v2:172.21.15.2:6838/3063991639,v1:172.21.15.2:6839/3063991639] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a{-1:14628} state up:standby seq 1 addr [v2:172.21.15.2:6840/2758814728,v1:172.21.15.2:6841/2758814728] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.82:6832/2827087691,v1:172.21.15.82:6833/2827087691] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.696 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.k{-1:24443} state up:standby seq 1 addr [v2:172.21.15.17:6832/719153877,v1:172.21.15.17:6833/719153877] compat {c=[1],r=[1],i=[7ff]}] 2023-06-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa42c0a11e0 msgr2=0x7fa42c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:55.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa42c0a11e0 0x7fa42c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa42800b5c0 tx=0x7fa428005fb0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0fea40 msgr2=0x7fa43c068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0fea40 0x7fa43c068af0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fa43000c970 tx=0x7fa43000cc80 comp rx=0 tx=0).stop 2023-06-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 shutdown_connections 2023-06-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa43c1005d0 0x7fa43c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa43c0fea40 0x7fa43c068af0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa42c0a11e0 0x7fa42c0a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 --2- 172.21.15.2:0/428676315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa43c0ffc40 0x7fa43c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 >> 172.21.15.2:0/428676315 conn(0x7fa43c0fa070 msgr2=0x7fa43c0fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 shutdown_connections 2023-06-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:55.685+0000 7fa442d0a700 1 -- 172.21.15.2:0/428676315 wait complete. 2023-06-23T18:21:55.699 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:21:55.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:55 smithi082 ceph-mon[97889]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:55.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:55 smithi017 ceph-mon[97472]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:21:56.097 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- bash -c 'ceph osd dump' 2023-06-23T18:21:56.661 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:21:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:56 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/428676315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-06-23T18:21:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:56 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/428676315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-06-23T18:21:56.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:56 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/428676315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-06-23T18:21:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:57 smithi002 ceph-mon[95947]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:57.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:57 smithi082 ceph-mon[97889]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:57.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 -- 172.21.15.2:0/1077569386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40ffc40 msgr2=0x7fcfd4100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:57.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/1077569386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40ffc40 0x7fcfd4100090 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009990 tx=0x7fcfbc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:57.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 -- 172.21.15.2:0/1077569386 shutdown_connections 2023-06-23T18:21:57.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/1077569386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfd41005d0 0x7fcfd4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:57.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/1077569386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd40fea40 0x7fcfd40fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:57.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/1077569386 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40ffc40 0x7fcfd4100090 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:57.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 -- 172.21.15.2:0/1077569386 >> 172.21.15.2:0/1077569386 conn(0x7fcfd40fa070 msgr2=0x7fcfd40fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:57.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 -- 172.21.15.2:0/1077569386 shutdown_connections 2023-06-23T18:21:57.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.845+0000 7fcfd8a50700 1 -- 172.21.15.2:0/1077569386 wait complete. 2023-06-23T18:21:57.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.846+0000 7fcfd8a50700 1 Processor -- start 2023-06-23T18:21:57.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.846+0000 7fcfd8a50700 1 -- start start 2023-06-23T18:21:57.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.846+0000 7fcfd8a50700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40fea40 0x7fcfd4194620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:57.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.846+0000 7fcfd8a50700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfd40ffc40 0x7fcfd4194b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:57.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.846+0000 7fcfd8a50700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd41005d0 0x7fcfd419bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:57.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.846+0000 7fcfd8a50700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd419c1b0 con 0x7fcfd40fea40 2023-06-23T18:21:57.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.846+0000 7fcfd8a50700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd419c2f0 con 0x7fcfd40ffc40 2023-06-23T18:21:57.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.846+0000 7fcfd8a50700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd419c480 con 0x7fcfd41005d0 2023-06-23T18:21:57.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40fea40 0x7fcfd4194620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:57.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40fea40 0x7fcfd4194620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53426/0 (socket says 172.21.15.2:53426) 2023-06-23T18:21:57.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 -- 172.21.15.2:0/3403382515 learned_addr learned my addr 172.21.15.2:0/3403382515 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:57.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd1d9b700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfd40ffc40 0x7fcfd4194b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:57.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd2d9d700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd41005d0 0x7fcfd419bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:57.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 -- 172.21.15.2:0/3403382515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd41005d0 msgr2=0x7fcfd419bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:57.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd41005d0 0x7fcfd419bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:57.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 -- 172.21.15.2:0/3403382515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfd40ffc40 msgr2=0x7fcfd4194b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:57.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfd40ffc40 0x7fcfd4194b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:57.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 -- 172.21.15.2:0/3403382515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0095f0 con 0x7fcfd40fea40 2023-06-23T18:21:57.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd259c700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40fea40 0x7fcfd4194620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400d9b0 tx=0x7fcfc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:57.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfcb7fe700 1 -- 172.21.15.2:0/3403382515 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc400b950 con 0x7fcfd40fea40 2023-06-23T18:21:57.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd419c760 con 0x7fcfd40fea40 2023-06-23T18:21:57.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfcb7fe700 1 -- 172.21.15.2:0/3403382515 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfc4007620 con 0x7fcfd40fea40 2023-06-23T18:21:57.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfcb7fe700 1 -- 172.21.15.2:0/3403382515 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4010ea0 con 0x7fcfd40fea40 2023-06-23T18:21:57.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.847+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd419cc60 con 0x7fcfd40fea40 2023-06-23T18:21:57.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.848+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd4106090 con 0x7fcfd40fea40 2023-06-23T18:21:57.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.849+0000 7fcfcb7fe700 1 -- 172.21.15.2:0/3403382515 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fcfc400bab0 con 0x7fcfd40fea40 2023-06-23T18:21:57.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.850+0000 7fcfcb7fe700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfc00a11e0 0x7fcfc00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:57.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.850+0000 7fcfcb7fe700 1 -- 172.21.15.2:0/3403382515 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfc40cc8e0 con 0x7fcfd40fea40 2023-06-23T18:21:57.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.850+0000 7fcfd1d9b700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfc00a11e0 0x7fcfc00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:57.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.850+0000 7fcfd1d9b700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfc00a11e0 0x7fcfc00a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b5c0 tx=0x7fcfbc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:57.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:57.854+0000 7fcfcb7fe700 1 -- 172.21.15.2:0/3403382515 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcfc4095ed0 con 0x7fcfd40fea40 2023-06-23T18:21:57.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:57 smithi017 ceph-mon[97472]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:21:58.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.056+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fcfd402ca80 con 0x7fcfd40fea40 2023-06-23T18:21:58.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.057+0000 7fcfcb7fe700 1 -- 172.21.15.2:0/3403382515 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) v1 ==== 56+0+5242 (secure 0 0 0) 0x7fcfc40994f0 con 0x7fcfd40fea40 2023-06-23T18:21:58.063 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 84 2023-06-23T18:21:58.063 INFO:teuthology.orchestra.run.smithi002.stdout:fsid 902733cc-11f1-11ee-9b2b-001a4aab830c 2023-06-23T18:21:58.063 INFO:teuthology.orchestra.run.smithi002.stdout:created 2023-06-23T18:13:38.741731+0000 2023-06-23T18:21:58.063 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2023-06-23T18:21:38.066041+0000 2023-06-23T18:21:58.063 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-06-23T18:21:58.064 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 26 2023-06-23T18:21:58.064 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2023-06-23T18:21:58.064 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2023-06-23T18:21:58.064 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2023-06-23T18:21:58.065 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2023-06-23T18:21:58.065 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client luminous 2023-06-23T18:21:58.065 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release reef 2023-06-23T18:21:58.065 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2023-06-23T18:21:58.065 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-06-23T18:21:58.065 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-06-23T18:21:58.065 INFO:teuthology.orchestra.run.smithi002.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-06-23T18:21:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 12 2023-06-23T18:21:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6802/2079965032,v1:172.21.15.2:6803/2079965032] [v2:172.21.15.2:6804/2079965032,v1:172.21.15.2:6805/2079965032] exists,up 9d2907d6-0f38-4387-afe6-dda12616124d 2023-06-23T18:21:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 13 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6800/85914245,v1:172.21.15.17:6801/85914245] [v2:172.21.15.17:6802/85914245,v1:172.21.15.17:6803/85914245] exists,up b3d5d141-29f5-4a3e-b854-4ca05bf85d90 2023-06-23T18:21:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 18 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6800/625655423,v1:172.21.15.82:6801/625655423] [v2:172.21.15.82:6802/625655423,v1:172.21.15.82:6803/625655423] exists,up c62ee3f3-654e-4a1f-b47c-6dd9b9585db0 2023-06-23T18:21:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 25 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6810/1640369824,v1:172.21.15.2:6811/1640369824] [v2:172.21.15.2:6812/1640369824,v1:172.21.15.2:6813/1640369824] exists,up d0c433bc-2983-4dac-a88c-27d6d0060480 2023-06-23T18:21:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 33 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6808/2671149119,v1:172.21.15.17:6809/2671149119] [v2:172.21.15.17:6810/2671149119,v1:172.21.15.17:6811/2671149119] exists,up 820c84c1-7f7a-4c03-8380-3dffe8d9c3fd 2023-06-23T18:21:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6808/102458807,v1:172.21.15.82:6809/102458807] [v2:172.21.15.82:6810/102458807,v1:172.21.15.82:6811/102458807] exists,up eac48eb4-7436-4849-bff3-ccd457ee594a 2023-06-23T18:21:58.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 46 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6818/617424670,v1:172.21.15.2:6819/617424670] [v2:172.21.15.2:6820/617424670,v1:172.21.15.2:6821/617424670] exists,up 9e320af6-3d7b-4934-9f7a-2860876ee5a0 2023-06-23T18:21:58.067 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 51 up_thru 80 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6816/3002114056,v1:172.21.15.17:6817/3002114056] [v2:172.21.15.17:6818/3002114056,v1:172.21.15.17:6819/3002114056] exists,up d5b16bb7-b550-4400-846a-0d321e8af4c7 2023-06-23T18:21:58.067 INFO:teuthology.orchestra.run.smithi002.stdout:osd.8 up in weight 1 up_from 58 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6816/1305476760,v1:172.21.15.82:6817/1305476760] [v2:172.21.15.82:6818/1305476760,v1:172.21.15.82:6819/1305476760] exists,up 4e7ac9f9-4c02-4639-ba72-7af18127d2eb 2023-06-23T18:21:58.067 INFO:teuthology.orchestra.run.smithi002.stdout:osd.9 up in weight 1 up_from 65 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6826/1361624360,v1:172.21.15.2:6827/1361624360] [v2:172.21.15.2:6828/1361624360,v1:172.21.15.2:6829/1361624360] exists,up 9013ccc7-bc53-477d-8a23-2806313db552 2023-06-23T18:21:58.067 INFO:teuthology.orchestra.run.smithi002.stdout:osd.10 up in weight 1 up_from 70 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.17:6824/3783526814,v1:172.21.15.17:6825/3783526814] [v2:172.21.15.17:6826/3783526814,v1:172.21.15.17:6827/3783526814] exists,up 9ab48541-33f0-44f9-8d30-727fd67ecbe4 2023-06-23T18:21:58.067 INFO:teuthology.orchestra.run.smithi002.stdout:osd.11 up in weight 1 up_from 77 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.82:6824/3300059850,v1:172.21.15.82:6825/3300059850] [v2:172.21.15.82:6826/3300059850,v1:172.21.15.82:6827/3300059850] exists,up f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77 2023-06-23T18:21:58.067 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.a [2,11] 2023-06-23T18:21:58.069 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.10 [8,11] 2023-06-23T18:21:58.069 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 2.14 [2,4] 2023-06-23T18:21:58.070 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.6 [4,7] 2023-06-23T18:21:58.070 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.11 [4,8] 2023-06-23T18:21:58.070 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.1a [5,6] 2023-06-23T18:21:58.070 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.27 [3,9] 2023-06-23T18:21:58.070 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.36 [4,9] 2023-06-23T18:21:58.070 INFO:teuthology.orchestra.run.smithi002.stdout:pg_upmap_items 3.38 [3,9] 2023-06-23T18:21:58.070 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2916792555 expires 2023-06-24T18:14:36.653577+0000 2023-06-23T18:21:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/3628945044 expires 2023-06-24T18:14:36.653577+0000 2023-06-23T18:21:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/3628945044 expires 2023-06-24T18:14:36.653577+0000 2023-06-23T18:21:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/2835880554 expires 2023-06-24T18:14:36.653577+0000 2023-06-23T18:21:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1541129198 expires 2023-06-24T18:14:09.903724+0000 2023-06-23T18:21:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1477099684 expires 2023-06-24T18:14:09.903724+0000 2023-06-23T18:21:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/595372279 expires 2023-06-24T18:14:09.903724+0000 2023-06-23T18:21:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6800/647669756 expires 2023-06-24T18:14:09.903724+0000 2023-06-23T18:21:58.071 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:6801/647669756 expires 2023-06-24T18:14:09.903724+0000 2023-06-23T18:21:58.072 INFO:teuthology.orchestra.run.smithi002.stdout:blocklist 172.21.15.2:0/1251506010 expires 2023-06-24T18:14:36.653577+0000 2023-06-23T18:21:58.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfc00a11e0 msgr2=0x7fcfc00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfc00a11e0 0x7fcfc00a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00b5c0 tx=0x7fcfbc005fb0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40fea40 msgr2=0x7fcfd4194620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40fea40 0x7fcfd4194620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400d9b0 tx=0x7fcfc400dcc0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 shutdown_connections 2023-06-23T18:21:58.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd41005d0 0x7fcfd419bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfd40fea40 0x7fcfd4194620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfc00a11e0 0x7fcfc00a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 --2- 172.21.15.2:0/3403382515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfd40ffc40 0x7fcfd4194b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 >> 172.21.15.2:0/3403382515 conn(0x7fcfd40fa070 msgr2=0x7fcfd41031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:58.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 shutdown_connections 2023-06-23T18:21:58.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.059+0000 7fcfd8a50700 1 -- 172.21.15.2:0/3403382515 wait complete. 2023-06-23T18:21:58.408 INFO:teuthology.run_tasks:Running task fs.ready... 2023-06-23T18:21:58.425 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:21:58.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 -- 172.21.15.2:0/1237657734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80fea70 msgr2=0x7f26f80fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 --2- 172.21.15.2:0/1237657734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80fea70 0x7f26f80fee80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009990 tx=0x7f26ec009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 -- 172.21.15.2:0/1237657734 shutdown_connections 2023-06-23T18:21:58.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 --2- 172.21.15.2:0/1237657734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26f81005f0 0x7f26f8106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 --2- 172.21.15.2:0/1237657734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80fea70 0x7f26f80fee80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 --2- 172.21.15.2:0/1237657734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26f80ffc60 0x7f26f81000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 -- 172.21.15.2:0/1237657734 >> 172.21.15.2:0/1237657734 conn(0x7f26f80fa090 msgr2=0x7f26f80fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:58.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 -- 172.21.15.2:0/1237657734 shutdown_connections 2023-06-23T18:21:58.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.575+0000 7f26feb02700 1 -- 172.21.15.2:0/1237657734 wait complete. 2023-06-23T18:21:58.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26feb02700 1 Processor -- start 2023-06-23T18:21:58.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26feb02700 1 -- start start 2023-06-23T18:21:58.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26feb02700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26f80fea70 0x7f26f8195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:58.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26feb02700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80ffc60 0x7f26f8195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:58.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26feb02700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26f81005f0 0x7f26f819cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:58.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26feb02700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819d3b0 con 0x7f26f80ffc60 2023-06-23T18:21:58.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26feb02700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819d4f0 con 0x7f26f80fea70 2023-06-23T18:21:58.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26feb02700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f819d680 con 0x7f26f81005f0 2023-06-23T18:21:58.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80ffc60 0x7f26f8195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:58.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26f7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80ffc60 0x7f26f8195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53440/0 (socket says 172.21.15.2:53440) 2023-06-23T18:21:58.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26f7fff700 1 -- 172.21.15.2:0/237596332 learned_addr learned my addr 172.21.15.2:0/237596332 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:58.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26fc89e700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26f80fea70 0x7f26f8195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:58.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.576+0000 7f26fd09f700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26f81005f0 0x7f26f819cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:58.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f7fff700 1 -- 172.21.15.2:0/237596332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26f81005f0 msgr2=0x7f26f819cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f7fff700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26f81005f0 0x7f26f819cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f7fff700 1 -- 172.21.15.2:0/237596332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26f80fea70 msgr2=0x7f26f8195830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f7fff700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26f80fea70 0x7f26f8195830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f7fff700 1 -- 172.21.15.2:0/237596332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0095f0 con 0x7f26f80ffc60 2023-06-23T18:21:58.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26fd09f700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26f81005f0 0x7f26f819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:58.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f7fff700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80ffc60 0x7f26f8195d70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e9a0 tx=0x7f26e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:58.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f5ffb700 1 -- 172.21.15.2:0/237596332 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4009790 con 0x7f26f80ffc60 2023-06-23T18:21:58.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f5ffb700 1 -- 172.21.15.2:0/237596332 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f26e400cbd0 con 0x7f26f80ffc60 2023-06-23T18:21:58.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f819d960 con 0x7f26f80ffc60 2023-06-23T18:21:58.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26f5ffb700 1 -- 172.21.15.2:0/237596332 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e4018680 con 0x7f26f80ffc60 2023-06-23T18:21:58.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.577+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f819dec0 con 0x7f26f80ffc60 2023-06-23T18:21:58.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.578+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f810a140 con 0x7f26f80ffc60 2023-06-23T18:21:58.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.579+0000 7f26f5ffb700 1 -- 172.21.15.2:0/237596332 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f26e400cd40 con 0x7f26f80ffc60 2023-06-23T18:21:58.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.580+0000 7f26f5ffb700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26e80a1260 0x7f26e80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:58.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.580+0000 7f26f5ffb700 1 -- 172.21.15.2:0/237596332 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f26e4014070 con 0x7f26f80ffc60 2023-06-23T18:21:58.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.580+0000 7f26fc89e700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26e80a1260 0x7f26e80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:58.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.580+0000 7f26fc89e700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26e80a1260 0x7f26e80a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0098e0 tx=0x7f26ec00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:58.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.583+0000 7f26f5ffb700 1 -- 172.21.15.2:0/237596332 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f26e40965f0 con 0x7f26f80ffc60 2023-06-23T18:21:58.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.783+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f810a330 con 0x7f26f80ffc60 2023-06-23T18:21:58.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.785+0000 7f26f5ffb700 1 -- 172.21.15.2:0/237596332 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f26e4026020 con 0x7f26f80ffc60 2023-06-23T18:21:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:21:58.791 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:21:58.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26e80a1260 msgr2=0x7f26e80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26e80a1260 0x7f26e80a3710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f26ec0098e0 tx=0x7f26ec00b260 comp rx=0 tx=0).stop 2023-06-23T18:21:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80ffc60 msgr2=0x7f26f8195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80ffc60 0x7f26f8195d70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f26e400e9a0 tx=0x7f26e400ecb0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 shutdown_connections 2023-06-23T18:21:58.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26f81005f0 0x7f26f819cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f26e80a1260 0x7f26e80a3710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26f80fea70 0x7f26f8195830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 --2- 172.21.15.2:0/237596332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26f80ffc60 0x7f26f8195d70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 >> 172.21.15.2:0/237596332 conn(0x7f26f80fa090 msgr2=0x7f26f80faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:58.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 shutdown_connections 2023-06-23T18:21:58.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.787+0000 7f26feb02700 1 -- 172.21.15.2:0/237596332 wait complete. 2023-06-23T18:21:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:58 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3403382515' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-06-23T18:21:58.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:21:58.806 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:21:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:58 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3403382515' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-06-23T18:21:58.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 -- 172.21.15.2:0/4045003338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0ffc80 msgr2=0x7ffbdc1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/4045003338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0ffc80 0x7ffbdc1000d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009990 tx=0x7ffbc4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 -- 172.21.15.2:0/4045003338 shutdown_connections 2023-06-23T18:21:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/4045003338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbdc100610 0x7ffbdc106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/4045003338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbdc0fea90 0x7ffbdc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/4045003338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0ffc80 0x7ffbdc1000d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 -- 172.21.15.2:0/4045003338 >> 172.21.15.2:0/4045003338 conn(0x7ffbdc0fa090 msgr2=0x7ffbdc0fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:58.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 -- 172.21.15.2:0/4045003338 shutdown_connections 2023-06-23T18:21:58.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.931+0000 7ffbe0a57700 1 -- 172.21.15.2:0/4045003338 wait complete. 2023-06-23T18:21:58.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbe0a57700 1 Processor -- start 2023-06-23T18:21:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbe0a57700 1 -- start start 2023-06-23T18:21:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbe0a57700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0fea90 0x7ffbdc069050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbe0a57700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbdc0ffc80 0x7ffbdc19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbe0a57700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbdc100610 0x7ffbdc19cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbe0a57700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19d350 con 0x7ffbdc0fea90 2023-06-23T18:21:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbe0a57700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19d4c0 con 0x7ffbdc0ffc80 2023-06-23T18:21:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbe0a57700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbdc19d680 con 0x7ffbdc100610 2023-06-23T18:21:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0fea90 0x7ffbdc069050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0fea90 0x7ffbdc069050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53452/0 (socket says 172.21.15.2:53452) 2023-06-23T18:21:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 -- 172.21.15.2:0/2481192003 learned_addr learned my addr 172.21.15.2:0/2481192003 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbd9d9b700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbdc0ffc80 0x7ffbdc19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbdad9d700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbdc100610 0x7ffbdc19cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 -- 172.21.15.2:0/2481192003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbdc100610 msgr2=0x7ffbdc19cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbdc100610 0x7ffbdc19cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 -- 172.21.15.2:0/2481192003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbdc0ffc80 msgr2=0x7ffbdc19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbdc0ffc80 0x7ffbdc19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 -- 172.21.15.2:0/2481192003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40095f0 con 0x7ffbdc0fea90 2023-06-23T18:21:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbd9d9b700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbdc0ffc80 0x7ffbdc19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.932+0000 7ffbda59c700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0fea90 0x7ffbdc069050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00e970 tx=0x7ffbcc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.933+0000 7ffbd37fe700 1 -- 172.21.15.2:0/2481192003 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc009790 con 0x7ffbdc0fea90 2023-06-23T18:21:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.933+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbdc19d960 con 0x7ffbdc0fea90 2023-06-23T18:21:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.933+0000 7ffbd37fe700 1 -- 172.21.15.2:0/2481192003 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffbcc0044b0 con 0x7ffbdc0fea90 2023-06-23T18:21:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.933+0000 7ffbd37fe700 1 -- 172.21.15.2:0/2481192003 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc003dd0 con 0x7ffbdc0fea90 2023-06-23T18:21:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.933+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbdc19de70 con 0x7ffbdc0fea90 2023-06-23T18:21:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.934+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbdc10a160 con 0x7ffbdc0fea90 2023-06-23T18:21:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.935+0000 7ffbd37fe700 1 -- 172.21.15.2:0/2481192003 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ffbcc003f30 con 0x7ffbdc0fea90 2023-06-23T18:21:58.945 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:58 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3403382515' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-06-23T18:21:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.936+0000 7ffbd37fe700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffbc80a11e0 0x7ffbc80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.936+0000 7ffbd9d9b700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffbc80a11e0 0x7ffbc80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.936+0000 7ffbd37fe700 1 -- 172.21.15.2:0/2481192003 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffbcc014070 con 0x7ffbdc0fea90 2023-06-23T18:21:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.936+0000 7ffbd9d9b700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffbc80a11e0 0x7ffbc80a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098e0 tx=0x7ffbc4004740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:58.939+0000 7ffbd37fe700 1 -- 172.21.15.2:0/2481192003 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ffbcc096490 con 0x7ffbdc0fea90 2023-06-23T18:21:59.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.137+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbdc1059d0 con 0x7ffbdc0fea90 2023-06-23T18:21:59.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.137+0000 7ffbd37fe700 1 -- 172.21.15.2:0/2481192003 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbcc099ab0 con 0x7ffbdc0fea90 2023-06-23T18:21:59.144 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:21:59.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.139+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffbc80a11e0 msgr2=0x7ffbc80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.139+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffbc80a11e0 0x7ffbc80a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098e0 tx=0x7ffbc4004740 comp rx=0 tx=0).stop 2023-06-23T18:21:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.139+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0fea90 msgr2=0x7ffbdc069050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.139+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0fea90 0x7ffbdc069050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00e970 tx=0x7ffbcc00ed30 comp rx=0 tx=0).stop 2023-06-23T18:21:59.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.140+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 shutdown_connections 2023-06-23T18:21:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.140+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbdc100610 0x7ffbdc19cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.140+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbdc0fea90 0x7ffbdc069050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.140+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffbc80a11e0 0x7ffbc80a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.140+0000 7ffbe0a57700 1 --2- 172.21.15.2:0/2481192003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbdc0ffc80 0x7ffbdc19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.140+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 >> 172.21.15.2:0/2481192003 conn(0x7ffbdc0fa090 msgr2=0x7ffbdc0fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:59.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.140+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 shutdown_connections 2023-06-23T18:21:59.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.140+0000 7ffbe0a57700 1 -- 172.21.15.2:0/2481192003 wait complete. 2023-06-23T18:21:59.157 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:21:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 -- 172.21.15.2:0/3184922856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 msgr2=0x7f30b00fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/3184922856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 0x7f30b00fff70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009990 tx=0x7f30a0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 -- 172.21.15.2:0/3184922856 shutdown_connections 2023-06-23T18:21:59.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/3184922856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30b01017a0 0x7f30b0108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/3184922856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 0x7f30b00fff70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/3184922856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30b0100df0 0x7f30b0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 -- 172.21.15.2:0/3184922856 >> 172.21.15.2:0/3184922856 conn(0x7f30b00fb180 msgr2=0x7f30b00fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 -- 172.21.15.2:0/3184922856 shutdown_connections 2023-06-23T18:21:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 -- 172.21.15.2:0/3184922856 wait complete. 2023-06-23T18:21:59.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 Processor -- start 2023-06-23T18:21:59.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.283+0000 7f30b5a5a700 1 -- start start 2023-06-23T18:21:59.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b5a5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 0x7f30b0197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b5a5a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30b0100df0 0x7f30b0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b5a5a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30b01017a0 0x7f30b019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b5a5a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b019f5c0 con 0x7f30b00ffb60 2023-06-23T18:21:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b5a5a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b019f700 con 0x7f30b01017a0 2023-06-23T18:21:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b5a5a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b019f890 con 0x7f30b0100df0 2023-06-23T18:21:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 0x7f30b0197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 0x7f30b0197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53464/0 (socket says 172.21.15.2:53464) 2023-06-23T18:21:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 -- 172.21.15.2:0/2712260632 learned_addr learned my addr 172.21.15.2:0/2712260632 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 -- 172.21.15.2:0/2712260632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30b0100df0 msgr2=0x7f30b0197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30b0100df0 0x7f30b0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 -- 172.21.15.2:0/2712260632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30b01017a0 msgr2=0x7f30b019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30b01017a0 0x7f30b019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 -- 172.21.15.2:0/2712260632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a00095f0 con 0x7f30b00ffb60 2023-06-23T18:21:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30aeffd700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 0x7f30b0197a40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f30a0005910 tx=0x7f30a0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b4a58700 1 -- 172.21.15.2:0/2712260632 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a001d070 con 0x7f30b00ffb60 2023-06-23T18:21:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b4a58700 1 -- 172.21.15.2:0/2712260632 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30a0004040 con 0x7f30b00ffb60 2023-06-23T18:21:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b019fb10 con 0x7f30b00ffb60 2023-06-23T18:21:59.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.284+0000 7f30b4a58700 1 -- 172.21.15.2:0/2712260632 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0005290 con 0x7f30b00ffb60 2023-06-23T18:21:59.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.285+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b019ff30 con 0x7f30b00ffb60 2023-06-23T18:21:59.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.286+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b010b230 con 0x7f30b00ffb60 2023-06-23T18:21:59.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.287+0000 7f30b4a58700 1 -- 172.21.15.2:0/2712260632 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f30a00047e0 con 0x7f30b00ffb60 2023-06-23T18:21:59.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.287+0000 7f30b4a58700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f309c0a11e0 0x7f309c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.287+0000 7f30b4a58700 1 -- 172.21.15.2:0/2712260632 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f30a00cdba0 con 0x7f30b00ffb60 2023-06-23T18:21:59.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.287+0000 7f30a67fc700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f309c0a11e0 0x7f309c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.288+0000 7f30a67fc700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f309c0a11e0 0x7f309c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f309800a990 tx=0x7f3098005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:59.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.292+0000 7f30b4a58700 1 -- 172.21.15.2:0/2712260632 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f30a0097220 con 0x7f30b00ffb60 2023-06-23T18:21:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.492+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30b01a0540 con 0x7f30b00ffb60 2023-06-23T18:21:59.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.493+0000 7f30b4a58700 1 -- 172.21.15.2:0/2712260632 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30a00a0020 con 0x7f30b00ffb60 2023-06-23T18:21:59.499 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:21:59.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f309c0a11e0 msgr2=0x7f309c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f309c0a11e0 0x7f309c0a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f309800a990 tx=0x7f3098005c80 comp rx=0 tx=0).stop 2023-06-23T18:21:59.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 msgr2=0x7f30b0197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 0x7f30b0197a40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f30a0005910 tx=0x7f30a0005ef0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 shutdown_connections 2023-06-23T18:21:59.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f309c0a11e0 0x7f309c0a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30b01017a0 0x7f30b019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f30b00ffb60 0x7f30b0197a40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 --2- 172.21.15.2:0/2712260632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30b0100df0 0x7f30b0197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 >> 172.21.15.2:0/2712260632 conn(0x7f30b00fb180 msgr2=0x7f30b00fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:59.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 shutdown_connections 2023-06-23T18:21:59.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.495+0000 7f30b5a5a700 1 -- 172.21.15.2:0/2712260632 wait complete. 2023-06-23T18:21:59.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:21:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.638+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/1872902578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 msgr2=0x7f1d78102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.638+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/1872902578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 0x7f1d78102ac0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b0a0 tx=0x7f1d6800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.638+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/1872902578 shutdown_connections 2023-06-23T18:21:59.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.638+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/1872902578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78105920 0x7f1d78107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.638+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/1872902578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 0x7f1d78102ac0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.638+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/1872902578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d78103000 0x7f1d781053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.638+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/1872902578 >> 172.21.15.2:0/1872902578 conn(0x7f1d780fa0c0 msgr2=0x7f1d780fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:59.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/1872902578 shutdown_connections 2023-06-23T18:21:59.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/1872902578 wait complete. 2023-06-23T18:21:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 Processor -- start 2023-06-23T18:21:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 -- start start 2023-06-23T18:21:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 0x7f1d781959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d78103000 0x7f1d78195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78105920 0x7f1d7819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7819d530 con 0x7f1d781006e0 2023-06-23T18:21:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7819d670 con 0x7f1d78105920 2023-06-23T18:21:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7fc3a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7819d800 con 0x7f1d78103000 2023-06-23T18:21:59.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7d9d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 0x7f1d781959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7d9d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 0x7f1d781959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53478/0 (socket says 172.21.15.2:53478) 2023-06-23T18:21:59.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7d9d6700 1 -- 172.21.15.2:0/764432069 learned_addr learned my addr 172.21.15.2:0/764432069 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:59.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.639+0000 7f1d7e1d7700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78105920 0x7f1d7819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7d9d6700 1 -- 172.21.15.2:0/764432069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d78103000 msgr2=0x7f1d78195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7d1d5700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d78103000 0x7f1d78195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7d9d6700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d78103000 0x7f1d78195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7d9d6700 1 -- 172.21.15.2:0/764432069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78105920 msgr2=0x7f1d7819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7d9d6700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78105920 0x7f1d7819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7d9d6700 1 -- 172.21.15.2:0/764432069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68009d00 con 0x7f1d781006e0 2023-06-23T18:21:59.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7d1d5700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d78103000 0x7f1d78195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:59.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7d9d6700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 0x7f1d781959b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d68007a30 tx=0x7f1d68004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d76ffd700 1 -- 172.21.15.2:0/764432069 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6800e040 con 0x7f1d781006e0 2023-06-23T18:21:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7819da80 con 0x7f1d781006e0 2023-06-23T18:21:59.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d76ffd700 1 -- 172.21.15.2:0/764432069 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d680094c0 con 0x7f1d781006e0 2023-06-23T18:21:59.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d76ffd700 1 -- 172.21.15.2:0/764432069 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6801db90 con 0x7f1d781006e0 2023-06-23T18:21:59.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.640+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7819df20 con 0x7f1d781006e0 2023-06-23T18:21:59.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.641+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d780fbc30 con 0x7f1d781006e0 2023-06-23T18:21:59.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.642+0000 7f1d76ffd700 1 -- 172.21.15.2:0/764432069 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1d68005450 con 0x7f1d781006e0 2023-06-23T18:21:59.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.643+0000 7f1d76ffd700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1d600a11e0 0x7f1d600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.643+0000 7f1d7d1d5700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1d600a11e0 0x7f1d600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.643+0000 7f1d76ffd700 1 -- 172.21.15.2:0/764432069 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d680cea40 con 0x7f1d781006e0 2023-06-23T18:21:59.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.643+0000 7f1d7d1d5700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1d600a11e0 0x7f1d600a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c005fd0 tx=0x7f1d6c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:59.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.646+0000 7f1d76ffd700 1 -- 172.21.15.2:0/764432069 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1d68098080 con 0x7f1d781006e0 2023-06-23T18:21:59.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:59 smithi002 ceph-mon[95947]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-06-23T18:21:59.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:59 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/237596332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:59.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:59 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2481192003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:59.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:21:59 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2712260632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:59.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:59 smithi082 ceph-mon[97889]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-06-23T18:21:59.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:59 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/237596332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:59.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:59 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2481192003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:59.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:21:59 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2712260632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:59.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.847+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1d7802cc90 con 0x7f1d781006e0 2023-06-23T18:21:59.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.848+0000 7f1d76ffd700 1 -- 172.21.15.2:0/764432069 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1d7802cc90 con 0x7f1d781006e0 2023-06-23T18:21:59.854 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:21:59.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1d600a11e0 msgr2=0x7f1d600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1d600a11e0 0x7f1d600a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c005fd0 tx=0x7f1d6c005ee0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 msgr2=0x7f1d781959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 0x7f1d781959b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1d68007a30 tx=0x7f1d68004db0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 shutdown_connections 2023-06-23T18:21:59.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1d600a11e0 0x7f1d600a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d78105920 0x7f1d7819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1d781006e0 0x7f1d781959b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 --2- 172.21.15.2:0/764432069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d78103000 0x7f1d78195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 >> 172.21.15.2:0/764432069 conn(0x7f1d780fa0c0 msgr2=0x7f1d78101340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:59.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 shutdown_connections 2023-06-23T18:21:59.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.850+0000 7f1d7fc3a700 1 -- 172.21.15.2:0/764432069 wait complete. 2023-06-23T18:21:59.868 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:21:59.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:59 smithi017 ceph-mon[97472]: pgmap v295: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-06-23T18:21:59.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:59 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/237596332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:21:59.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:59 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2481192003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:21:59.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:21:59 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2712260632' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:21:59.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 -- 172.21.15.2:0/386946949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 msgr2=0x7f8a90106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 --2- 172.21.15.2:0/386946949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 0x7f8a90106ea0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800b0a0 tx=0x7f8a8800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 -- 172.21.15.2:0/386946949 shutdown_connections 2023-06-23T18:21:59.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 --2- 172.21.15.2:0/386946949 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 0x7f8a90106ea0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 --2- 172.21.15.2:0/386946949 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a900fea60 0x7f8a900fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 --2- 172.21.15.2:0/386946949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a900ffc50 0x7f8a901000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 -- 172.21.15.2:0/386946949 >> 172.21.15.2:0/386946949 conn(0x7f8a900fa0e0 msgr2=0x7f8a900fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:21:59.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 -- 172.21.15.2:0/386946949 shutdown_connections 2023-06-23T18:21:59.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.988+0000 7f8a98362700 1 -- 172.21.15.2:0/386946949 wait complete. 2023-06-23T18:21:59.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a98362700 1 Processor -- start 2023-06-23T18:21:59.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a98362700 1 -- start start 2023-06-23T18:21:59.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a98362700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a900fea60 0x7f8a90068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a98362700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a900ffc50 0x7f8a9019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a98362700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 0x7f8a9019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:21:59.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a98362700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019d3f0 con 0x7f8a901005e0 2023-06-23T18:21:59.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a98362700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019d560 con 0x7f8a900ffc50 2023-06-23T18:21:59.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a98362700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9019d720 con 0x7f8a900fea60 2023-06-23T18:21:59.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 0x7f8a9019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 0x7f8a9019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53496/0 (socket says 172.21.15.2:53496) 2023-06-23T18:21:59.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 -- 172.21.15.2:0/3244912859 learned_addr learned my addr 172.21.15.2:0/3244912859 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:21:59.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a960fe700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a900fea60 0x7f8a90068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a958fd700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a900ffc50 0x7f8a9019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:21:59.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 -- 172.21.15.2:0/3244912859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a900fea60 msgr2=0x7f8a90068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a900fea60 0x7f8a90068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 -- 172.21.15.2:0/3244912859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a900ffc50 msgr2=0x7f8a9019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:21:59.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a900ffc50 0x7f8a9019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:21:59.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 -- 172.21.15.2:0/3244912859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88009d00 con 0x7f8a901005e0 2023-06-23T18:21:59.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a958fd700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a900ffc50 0x7f8a9019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:21:59.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a960fe700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a900fea60 0x7f8a90068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:21:59.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a968ff700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 0x7f8a9019cc20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8a880095e0 tx=0x7f8a8800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:21:59.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a877fe700 1 -- 172.21.15.2:0/3244912859 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a8800e040 con 0x7f8a901005e0 2023-06-23T18:22:00.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.989+0000 7f8a877fe700 1 -- 172.21.15.2:0/3244912859 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8a88003c00 con 0x7f8a901005e0 2023-06-23T18:22:00.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.990+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9019d9a0 con 0x7f8a901005e0 2023-06-23T18:22:00.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.990+0000 7f8a877fe700 1 -- 172.21.15.2:0/3244912859 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a880052a0 con 0x7f8a901005e0 2023-06-23T18:22:00.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.990+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9019de20 con 0x7f8a901005e0 2023-06-23T18:22:00.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.990+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a90106050 con 0x7f8a901005e0 2023-06-23T18:22:00.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.992+0000 7f8a877fe700 1 -- 172.21.15.2:0/3244912859 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f8a88005610 con 0x7f8a901005e0 2023-06-23T18:22:00.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.993+0000 7f8a877fe700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a7c0a11e0 0x7f8a7c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.993+0000 7f8a877fe700 1 -- 172.21.15.2:0/3244912859 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8a880ce5d0 con 0x7f8a901005e0 2023-06-23T18:22:00.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.993+0000 7f8a960fe700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a7c0a11e0 0x7f8a7c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.993+0000 7f8a960fe700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a7c0a11e0 0x7f8a7c0a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a900ffab0 tx=0x7f8a8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:00.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:21:59.996+0000 7f8a877fe700 1 -- 172.21.15.2:0/3244912859 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8a88097c10 con 0x7f8a901005e0 2023-06-23T18:22:00.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.239+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a9002c810 con 0x7f8a901005e0 2023-06-23T18:22:00.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.240+0000 7f8a877fe700 1 -- 172.21.15.2:0/3244912859 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7f8a8809b230 con 0x7f8a901005e0 2023-06-23T18:22:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:00.247 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:22:00.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a7c0a11e0 msgr2=0x7f8a7c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a7c0a11e0 0x7f8a7c0a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a900ffab0 tx=0x7f8a8c009260 comp rx=0 tx=0).stop 2023-06-23T18:22:00.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 msgr2=0x7f8a9019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 0x7f8a9019cc20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8a880095e0 tx=0x7f8a8800bb00 comp rx=0 tx=0).stop 2023-06-23T18:22:00.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 shutdown_connections 2023-06-23T18:22:00.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8a901005e0 0x7f8a9019cc20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8a900fea60 0x7f8a90068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8a7c0a11e0 0x7f8a7c0a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 --2- 172.21.15.2:0/3244912859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a900ffc50 0x7f8a9019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 >> 172.21.15.2:0/3244912859 conn(0x7f8a900fa0e0 msgr2=0x7f8a90103190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:00.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.242+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 shutdown_connections 2023-06-23T18:22:00.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.243+0000 7f8a98362700 1 -- 172.21.15.2:0/3244912859 wait complete. 2023-06-23T18:22:00.252 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:22:00.261 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:22:00.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/2676737764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 msgr2=0x7fc99c105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/2676737764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 0x7fc99c105340 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc990009990 tx=0x7fc990009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/2676737764 shutdown_connections 2023-06-23T18:22:00.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/2676737764 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc99c105880 0x7fc99c107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/2676737764 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c100640 0x7fc99c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/2676737764 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 0x7fc99c105340 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/2676737764 >> 172.21.15.2:0/2676737764 conn(0x7fc99c0fa0e0 msgr2=0x7fc99c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/2676737764 shutdown_connections 2023-06-23T18:22:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.382+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/2676737764 wait complete. 2023-06-23T18:22:00.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc9a1bfd700 1 Processor -- start 2023-06-23T18:22:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc9a1bfd700 1 -- start start 2023-06-23T18:22:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc9a1bfd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc99c100640 0x7fc99c1957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc9a1bfd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 0x7fc99c195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc9a1bfd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c105880 0x7fc99c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc9a1bfd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c19d320 con 0x7fc99c102f60 2023-06-23T18:22:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc9a1bfd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c19d460 con 0x7fc99c100640 2023-06-23T18:22:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc9a1bfd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c19d5f0 con 0x7fc99c105880 2023-06-23T18:22:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc99affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 0x7fc99c195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc99affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 0x7fc99c195ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53508/0 (socket says 172.21.15.2:53508) 2023-06-23T18:22:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc99affd700 1 -- 172.21.15.2:0/568552704 learned_addr learned my addr 172.21.15.2:0/568552704 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.383+0000 7fc99b7fe700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc99c100640 0x7fc99c1957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc99bfff700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c105880 0x7fc99c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc99affd700 1 -- 172.21.15.2:0/568552704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c105880 msgr2=0x7fc99c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc99affd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c105880 0x7fc99c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc99affd700 1 -- 172.21.15.2:0/568552704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc99c100640 msgr2=0x7fc99c1957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc99affd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc99c100640 0x7fc99c1957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc99affd700 1 -- 172.21.15.2:0/568552704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900095f0 con 0x7fc99c102f60 2023-06-23T18:22:00.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc99affd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 0x7fc99c195ce0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc990006010 tx=0x7fc990003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:00.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc998ff9700 1 -- 172.21.15.2:0/568552704 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99001d070 con 0x7fc99c102f60 2023-06-23T18:22:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c19d870 con 0x7fc99c102f60 2023-06-23T18:22:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc998ff9700 1 -- 172.21.15.2:0/568552704 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc990004640 con 0x7fc99c102f60 2023-06-23T18:22:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc998ff9700 1 -- 172.21.15.2:0/568552704 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99000f460 con 0x7fc99c102f60 2023-06-23T18:22:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.384+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c19dc90 con 0x7fc99c102f60 2023-06-23T18:22:00.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.385+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c0fcc60 con 0x7fc99c102f60 2023-06-23T18:22:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.386+0000 7fc998ff9700 1 -- 172.21.15.2:0/568552704 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc990022470 con 0x7fc99c102f60 2023-06-23T18:22:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.387+0000 7fc998ff9700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9840a11e0 0x7fc9840a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.387+0000 7fc998ff9700 1 -- 172.21.15.2:0/568552704 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc9900ce1a0 con 0x7fc99c102f60 2023-06-23T18:22:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.387+0000 7fc99b7fe700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9840a11e0 0x7fc9840a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.387+0000 7fc99b7fe700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9840a11e0 0x7fc9840a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005950 tx=0x7fc98c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:00.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.391+0000 7fc998ff9700 1 -- 172.21.15.2:0/568552704 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc990097820 con 0x7fc99c102f60 2023-06-23T18:22:00.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.589+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc99c02c9f0 con 0x7fc99c102f60 2023-06-23T18:22:00.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.591+0000 7fc998ff9700 1 -- 172.21.15.2:0/568552704 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7fc990022720 con 0x7fc99c102f60 2023-06-23T18:22:00.596 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:22:00.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9840a11e0 msgr2=0x7fc9840a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9840a11e0 0x7fc9840a3690 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc98c005950 tx=0x7fc98c00a300 comp rx=0 tx=0).stop 2023-06-23T18:22:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 msgr2=0x7fc99c195ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 0x7fc99c195ce0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc990006010 tx=0x7fc990003a50 comp rx=0 tx=0).stop 2023-06-23T18:22:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 shutdown_connections 2023-06-23T18:22:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc99c105880 0x7fc99c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9840a11e0 0x7fc9840a3690 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc99c100640 0x7fc99c1957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 --2- 172.21.15.2:0/568552704 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc99c102f60 0x7fc99c195ce0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 >> 172.21.15.2:0/568552704 conn(0x7fc99c0fa0e0 msgr2=0x7fc99c101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 shutdown_connections 2023-06-23T18:22:00.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.593+0000 7fc9a1bfd700 1 -- 172.21.15.2:0/568552704 wait complete. 2023-06-23T18:22:00.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:22:00.612 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:22:00.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 -- 172.21.15.2:0/2258917468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff454100600 msgr2=0x7ff454106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 --2- 172.21.15.2:0/2258917468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff454100600 0x7ff454106ec0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff44c00b0a0 tx=0x7ff44c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 -- 172.21.15.2:0/2258917468 shutdown_connections 2023-06-23T18:22:00.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 --2- 172.21.15.2:0/2258917468 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff454100600 0x7ff454106ec0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 --2- 172.21.15.2:0/2258917468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4540fea80 0x7ff4540fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 --2- 172.21.15.2:0/2258917468 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4540ffc70 0x7ff4541000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 -- 172.21.15.2:0/2258917468 >> 172.21.15.2:0/2258917468 conn(0x7ff4540fa0e0 msgr2=0x7ff4540fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:00.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 -- 172.21.15.2:0/2258917468 shutdown_connections 2023-06-23T18:22:00.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.736+0000 7ff45baee700 1 -- 172.21.15.2:0/2258917468 wait complete. 2023-06-23T18:22:00.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45baee700 1 Processor -- start 2023-06-23T18:22:00.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45baee700 1 -- start start 2023-06-23T18:22:00.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45baee700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4540fea80 0x7ff454195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45baee700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4540ffc70 0x7ff454195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45baee700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454100600 0x7ff45419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45baee700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45419d490 con 0x7ff4540fea80 2023-06-23T18:22:00.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45baee700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45419d5d0 con 0x7ff4540ffc70 2023-06-23T18:22:00.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45baee700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45419d760 con 0x7ff454100600 2023-06-23T18:22:00.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4540fea80 0x7ff454195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4540fea80 0x7ff454195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53520/0 (socket says 172.21.15.2:53520) 2023-06-23T18:22:00.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 -- 172.21.15.2:0/1426091145 learned_addr learned my addr 172.21.15.2:0/1426091145 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:00.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff459089700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4540ffc70 0x7ff454195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45a08b700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454100600 0x7ff45419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 -- 172.21.15.2:0/1426091145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454100600 msgr2=0x7ff45419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454100600 0x7ff45419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 -- 172.21.15.2:0/1426091145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4540ffc70 msgr2=0x7ff454195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4540ffc70 0x7ff454195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 -- 172.21.15.2:0/1426091145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c009d00 con 0x7ff4540fea80 2023-06-23T18:22:00.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.737+0000 7ff45988a700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4540fea80 0x7ff454195960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff45000d980 tx=0x7ff45000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:00.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.738+0000 7ff44affd700 1 -- 172.21.15.2:0/1426091145 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45000de60 con 0x7ff4540fea80 2023-06-23T18:22:00.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.738+0000 7ff44affd700 1 -- 172.21.15.2:0/1426091145 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4500044b0 con 0x7ff4540fea80 2023-06-23T18:22:00.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.738+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff45419da40 con 0x7ff4540fea80 2023-06-23T18:22:00.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.738+0000 7ff44affd700 1 -- 172.21.15.2:0/1426091145 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450003b40 con 0x7ff4540fea80 2023-06-23T18:22:00.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.738+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff45419df50 con 0x7ff4540fea80 2023-06-23T18:22:00.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.739+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff454106b00 con 0x7ff4540fea80 2023-06-23T18:22:00.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.740+0000 7ff44affd700 1 -- 172.21.15.2:0/1426091145 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff450003ca0 con 0x7ff4540fea80 2023-06-23T18:22:00.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.740+0000 7ff44affd700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4400a11e0 0x7ff4400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:00.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.740+0000 7ff459089700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4400a11e0 0x7ff4400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:00.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.740+0000 7ff44affd700 1 -- 172.21.15.2:0/1426091145 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff4500cc7b0 con 0x7ff4540fea80 2023-06-23T18:22:00.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.741+0000 7ff459089700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4400a11e0 0x7ff4400a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff444005fd0 tx=0x7ff444005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:00.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.744+0000 7ff44affd700 1 -- 172.21.15.2:0/1426091145 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff450095da0 con 0x7ff4540fea80 2023-06-23T18:22:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:00 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/764432069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:00 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3244912859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:00 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/764432069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:00.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:00 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3244912859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:00.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:00 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/764432069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:00.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:00 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3244912859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:00.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.944+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff45402ca90 con 0x7ff4540fea80 2023-06-23T18:22:00.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.944+0000 7ff44affd700 1 -- 172.21.15.2:0/1426091145 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff450009940 con 0x7ff4540fea80 2023-06-23T18:22:00.950 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:22:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4400a11e0 msgr2=0x7ff4400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4400a11e0 0x7ff4400a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff444005fd0 tx=0x7ff444005f60 comp rx=0 tx=0).stop 2023-06-23T18:22:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4540fea80 msgr2=0x7ff454195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4540fea80 0x7ff454195960 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff45000d980 tx=0x7ff45000dd40 comp rx=0 tx=0).stop 2023-06-23T18:22:00.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 shutdown_connections 2023-06-23T18:22:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff454100600 0x7ff45419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4540fea80 0x7ff454195960 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4400a11e0 0x7ff4400a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 --2- 172.21.15.2:0/1426091145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4540ffc70 0x7ff454195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.946+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 >> 172.21.15.2:0/1426091145 conn(0x7ff4540fa0e0 msgr2=0x7ff4540fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.947+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 shutdown_connections 2023-06-23T18:22:00.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:00.947+0000 7ff45baee700 1 -- 172.21.15.2:0/1426091145 wait complete. 2023-06-23T18:22:00.964 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:22:01.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 -- 172.21.15.2:0/1921996873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 msgr2=0x7f390c0fe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 --2- 172.21.15.2:0/1921996873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 0x7f390c0fe030 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009990 tx=0x7f38fc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 -- 172.21.15.2:0/1921996873 shutdown_connections 2023-06-23T18:22:01.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 --2- 172.21.15.2:0/1921996873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390c0fe570 0x7f390c108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 --2- 172.21.15.2:0/1921996873 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c101170 0x7f390c101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 --2- 172.21.15.2:0/1921996873 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 0x7f390c0fe030 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 -- 172.21.15.2:0/1921996873 >> 172.21.15.2:0/1921996873 conn(0x7f390c0f9850 msgr2=0x7f390c0fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:01.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 -- 172.21.15.2:0/1921996873 shutdown_connections 2023-06-23T18:22:01.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.088+0000 7f3911181700 1 -- 172.21.15.2:0/1921996873 wait complete. 2023-06-23T18:22:01.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f3911181700 1 Processor -- start 2023-06-23T18:22:01.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f3911181700 1 -- start start 2023-06-23T18:22:01.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f3911181700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 0x7f390c1958f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f3911181700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c0fe570 0x7f390c195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f3911181700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390c101170 0x7f390c19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f3911181700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c19d470 con 0x7f390c0fdbe0 2023-06-23T18:22:01.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f3911181700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c19d5b0 con 0x7f390c0fe570 2023-06-23T18:22:01.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f3911181700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c19d740 con 0x7f390c101170 2023-06-23T18:22:01.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f390ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 0x7f390c1958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f390ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 0x7f390c1958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53534/0 (socket says 172.21.15.2:53534) 2023-06-23T18:22:01.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f390ad9d700 1 -- 172.21.15.2:0/2090488862 learned_addr learned my addr 172.21.15.2:0/2090488862 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:01.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f390b59e700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390c101170 0x7f390c19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.089+0000 7f390a59c700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c0fe570 0x7f390c195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f390ad9d700 1 -- 172.21.15.2:0/2090488862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390c101170 msgr2=0x7f390c19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f390ad9d700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390c101170 0x7f390c19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f390ad9d700 1 -- 172.21.15.2:0/2090488862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c0fe570 msgr2=0x7f390c195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f390ad9d700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c0fe570 0x7f390c195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f390ad9d700 1 -- 172.21.15.2:0/2090488862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0095f0 con 0x7f390c0fdbe0 2023-06-23T18:22:01.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f390a59c700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c0fe570 0x7f390c195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:01.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f390ad9d700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 0x7f390c1958f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b500 tx=0x7f38f400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:01.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f3903fff700 1 -- 172.21.15.2:0/2090488862 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4011830 con 0x7f390c0fdbe0 2023-06-23T18:22:01.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c19da20 con 0x7f390c0fdbe0 2023-06-23T18:22:01.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f3903fff700 1 -- 172.21.15.2:0/2090488862 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38f4011e20 con 0x7f390c0fdbe0 2023-06-23T18:22:01.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f3903fff700 1 -- 172.21.15.2:0/2090488862 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38f4010dd0 con 0x7f390c0fdbe0 2023-06-23T18:22:01.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.090+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c19df30 con 0x7f390c0fdbe0 2023-06-23T18:22:01.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.091+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c1073a0 con 0x7f390c0fdbe0 2023-06-23T18:22:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.092+0000 7f3903fff700 1 -- 172.21.15.2:0/2090488862 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f38f4011990 con 0x7f390c0fdbe0 2023-06-23T18:22:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.092+0000 7f3903fff700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38f80a11e0 0x7f38f80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.093+0000 7f390a59c700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38f80a11e0 0x7f38f80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.093+0000 7f3903fff700 1 -- 172.21.15.2:0/2090488862 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f38f40cc400 con 0x7f390c0fdbe0 2023-06-23T18:22:01.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.093+0000 7f390a59c700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38f80a11e0 0x7f38f80a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0059d0 tx=0x7f38fc005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:01.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.097+0000 7f3903fff700 1 -- 172.21.15.2:0/2090488862 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f38f4095b00 con 0x7f390c0fdbe0 2023-06-23T18:22:01.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.298+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f390c02ca60 con 0x7f390c0fdbe0 2023-06-23T18:22:01.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.298+0000 7f3903fff700 1 -- 172.21.15.2:0/2090488862 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38f4099120 con 0x7f390c0fdbe0 2023-06-23T18:22:01.304 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:22:01.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38f80a11e0 msgr2=0x7f38f80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38f80a11e0 0x7f38f80a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0059d0 tx=0x7f38fc005940 comp rx=0 tx=0).stop 2023-06-23T18:22:01.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 msgr2=0x7f390c1958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 0x7f390c1958f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f38f400b500 tx=0x7f38f400b810 comp rx=0 tx=0).stop 2023-06-23T18:22:01.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 shutdown_connections 2023-06-23T18:22:01.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f390c101170 0x7f390c19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f390c0fdbe0 0x7f390c1958f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38f80a11e0 0x7f38f80a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 --2- 172.21.15.2:0/2090488862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f390c0fe570 0x7f390c195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.300+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 >> 172.21.15.2:0/2090488862 conn(0x7f390c0f9850 msgr2=0x7f390c1044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:01.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.301+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 shutdown_connections 2023-06-23T18:22:01.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.301+0000 7f3911181700 1 -- 172.21.15.2:0/2090488862 wait complete. 2023-06-23T18:22:01.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:22:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- 172.21.15.2:0/1543845686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e24100620 msgr2=0x7f2e24106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 --2- 172.21.15.2:0/1543845686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e24100620 0x7f2e24106ee0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00b0a0 tx=0x7f2e1c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- 172.21.15.2:0/1543845686 shutdown_connections 2023-06-23T18:22:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 --2- 172.21.15.2:0/1543845686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e24100620 0x7f2e24106ee0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 --2- 172.21.15.2:0/1543845686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e240feaa0 0x7f2e240feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 --2- 172.21.15.2:0/1543845686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ffc90 0x7f2e241000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- 172.21.15.2:0/1543845686 >> 172.21.15.2:0/1543845686 conn(0x7f2e240fa0c0 msgr2=0x7f2e240fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- 172.21.15.2:0/1543845686 shutdown_connections 2023-06-23T18:22:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- 172.21.15.2:0/1543845686 wait complete. 2023-06-23T18:22:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 Processor -- start 2023-06-23T18:22:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- start start 2023-06-23T18:22:01.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e240feaa0 0x7f2e24195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e240ffc90 0x7f2e24195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e24100620 0x7f2e2419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2419d400 con 0x7f2e240ffc90 2023-06-23T18:22:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2419d540 con 0x7f2e240feaa0 2023-06-23T18:22:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.444+0000 7f2e2be03700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2419d6d0 con 0x7f2e24100620 2023-06-23T18:22:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e240ffc90 0x7f2e24195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e240ffc90 0x7f2e24195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53550/0 (socket says 172.21.15.2:53550) 2023-06-23T18:22:01.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 -- 172.21.15.2:0/3699311464 learned_addr learned my addr 172.21.15.2:0/3699311464 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e29b9f700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e240feaa0 0x7f2e24195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2a3a0700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e24100620 0x7f2e2419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 -- 172.21.15.2:0/3699311464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e24100620 msgr2=0x7f2e2419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e24100620 0x7f2e2419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 -- 172.21.15.2:0/3699311464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e240feaa0 msgr2=0x7f2e24195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e240feaa0 0x7f2e24195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 -- 172.21.15.2:0/3699311464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c009d00 con 0x7f2e240ffc90 2023-06-23T18:22:01.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e29b9f700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e240feaa0 0x7f2e24195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:01.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2939e700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e240ffc90 0x7f2e24195dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400d980 tx=0x7f2e1400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:01.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e1affd700 1 -- 172.21.15.2:0/3699311464 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1400bb60 con 0x7f2e240ffc90 2023-06-23T18:22:01.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e1affd700 1 -- 172.21.15.2:0/3699311464 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2e14007640 con 0x7f2e240ffc90 2023-06-23T18:22:01.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2419d9b0 con 0x7f2e240ffc90 2023-06-23T18:22:01.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e1affd700 1 -- 172.21.15.2:0/3699311464 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e14004d60 con 0x7f2e240ffc90 2023-06-23T18:22:01.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.445+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2419dec0 con 0x7f2e240ffc90 2023-06-23T18:22:01.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.446+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2410a130 con 0x7f2e240ffc90 2023-06-23T18:22:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.447+0000 7f2e1affd700 1 -- 172.21.15.2:0/3699311464 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2e14004020 con 0x7f2e240ffc90 2023-06-23T18:22:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.448+0000 7f2e1affd700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e100a11e0 0x7f2e100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.448+0000 7f2e29b9f700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e100a11e0 0x7f2e100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.448+0000 7f2e1affd700 1 -- 172.21.15.2:0/3699311464 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2e140cced0 con 0x7f2e240ffc90 2023-06-23T18:22:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.448+0000 7f2e29b9f700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e100a11e0 0x7f2e100a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2e240ffaf0 tx=0x7f2e2000a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:01.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.451+0000 7f2e1affd700 1 -- 172.21.15.2:0/3699311464 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2e140964c0 con 0x7f2e240ffc90 2023-06-23T18:22:01.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.651+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e2419e470 con 0x7f2e240ffc90 2023-06-23T18:22:01.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.652+0000 7f2e1affd700 1 -- 172.21.15.2:0/3699311464 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e140160a0 con 0x7f2e240ffc90 2023-06-23T18:22:01.657 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:22:01.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e100a11e0 msgr2=0x7f2e100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e100a11e0 0x7f2e100a3690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2e240ffaf0 tx=0x7f2e2000a680 comp rx=0 tx=0).stop 2023-06-23T18:22:01.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e240ffc90 msgr2=0x7f2e24195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e240ffc90 0x7f2e24195dc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2e1400d980 tx=0x7f2e1400dd40 comp rx=0 tx=0).stop 2023-06-23T18:22:01.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 shutdown_connections 2023-06-23T18:22:01.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e24100620 0x7f2e2419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2e100a11e0 0x7f2e100a3690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e240feaa0 0x7f2e24195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 --2- 172.21.15.2:0/3699311464 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2e240ffc90 0x7f2e24195dc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 >> 172.21.15.2:0/3699311464 conn(0x7f2e240fa0c0 msgr2=0x7f2e240685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:01.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 shutdown_connections 2023-06-23T18:22:01.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.654+0000 7f2e2be03700 1 -- 172.21.15.2:0/3699311464 wait complete. 2023-06-23T18:22:01.671 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:01 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/568552704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:01.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:01 smithi002 ceph-mon[95947]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:22:01.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:01 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1426091145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:01.672 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:01 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2090488862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:01.674 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:22:01.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 -- 172.21.15.2:0/2297897406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0102f00 msgr2=0x7fc8b01052e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/2297897406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0102f00 0x7fc8b01052e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009990 tx=0x7fc8a4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 -- 172.21.15.2:0/2297897406 shutdown_connections 2023-06-23T18:22:01.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/2297897406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b0105820 0x7fc8b0107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/2297897406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8b01005e0 0x7fc8b01029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/2297897406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0102f00 0x7fc8b01052e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 -- 172.21.15.2:0/2297897406 >> 172.21.15.2:0/2297897406 conn(0x7fc8b00fa0e0 msgr2=0x7fc8b00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:01.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 -- 172.21.15.2:0/2297897406 shutdown_connections 2023-06-23T18:22:01.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.794+0000 7fc8b65fe700 1 -- 172.21.15.2:0/2297897406 wait complete. 2023-06-23T18:22:01.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b65fe700 1 Processor -- start 2023-06-23T18:22:01.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b65fe700 1 -- start start 2023-06-23T18:22:01.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b65fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8b01005e0 0x7fc8b01957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b65fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b0102f00 0x7fc8b0195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b65fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0105820 0x7fc8b019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b65fe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019d330 con 0x7fc8b0105820 2023-06-23T18:22:01.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b65fe700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019d470 con 0x7fc8b0102f00 2023-06-23T18:22:01.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b65fe700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b019d600 con 0x7fc8b01005e0 2023-06-23T18:22:01.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b4b9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0105820 0x7fc8b019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b4b9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0105820 0x7fc8b019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53566/0 (socket says 172.21.15.2:53566) 2023-06-23T18:22:01.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8b4b9b700 1 -- 172.21.15.2:0/3789228916 learned_addr learned my addr 172.21.15.2:0/3789228916 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:01.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8affff700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8b01005e0 0x7fc8b01957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.795+0000 7fc8af7fe700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b0102f00 0x7fc8b0195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8b4b9b700 1 -- 172.21.15.2:0/3789228916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8b01005e0 msgr2=0x7fc8b01957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8b4b9b700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8b01005e0 0x7fc8b01957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8b4b9b700 1 -- 172.21.15.2:0/3789228916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b0102f00 msgr2=0x7fc8b0195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:01.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8b4b9b700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b0102f00 0x7fc8b0195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:01.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8b4b9b700 1 -- 172.21.15.2:0/3789228916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40095f0 con 0x7fc8b0105820 2023-06-23T18:22:01.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8b4b9b700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0105820 0x7fc8b019cd60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800ada0 tx=0x7fc8a800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:01.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8ad7fa700 1 -- 172.21.15.2:0/3789228916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a800f070 con 0x7fc8b0105820 2023-06-23T18:22:01.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8ad7fa700 1 -- 172.21.15.2:0/3789228916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8a8007b30 con 0x7fc8b0105820 2023-06-23T18:22:01.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b019d8e0 con 0x7fc8b0105820 2023-06-23T18:22:01.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8ad7fa700 1 -- 172.21.15.2:0/3789228916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8005280 con 0x7fc8b0105820 2023-06-23T18:22:01.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.796+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b019ddf0 con 0x7fc8b0105820 2023-06-23T18:22:01.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.797+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b00fcc60 con 0x7fc8b0105820 2023-06-23T18:22:01.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.799+0000 7fc8ad7fa700 1 -- 172.21.15.2:0/3789228916 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc8a800a090 con 0x7fc8b0105820 2023-06-23T18:22:01.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.799+0000 7fc8ad7fa700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8980a11e0 0x7fc8980a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:01.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.800+0000 7fc8affff700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8980a11e0 0x7fc8980a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:01.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.800+0000 7fc8ad7fa700 1 -- 172.21.15.2:0/3789228916 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc8a80cc5c0 con 0x7fc8b0105820 2023-06-23T18:22:01.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.800+0000 7fc8affff700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8980a11e0 0x7fc8980a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0005950 tx=0x7fc8a000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:01.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:01.803+0000 7fc8ad7fa700 1 -- 172.21.15.2:0/3789228916 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fc8a8095c40 con 0x7fc8b0105820 2023-06-23T18:22:01.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:01 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/568552704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:01.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:01 smithi082 ceph-mon[97889]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:22:01.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:01 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1426091145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:01.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:01 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2090488862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:01.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:01 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/568552704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:01.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:01 smithi017 ceph-mon[97472]: pgmap v296: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:22:01.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:01 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1426091145' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:01.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:01 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2090488862' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:02.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.050+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8b002cca0 con 0x7fc8b0105820 2023-06-23T18:22:02.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.051+0000 7fc8ad7fa700 1 -- 172.21.15.2:0/3789228916 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7fc8a8099260 con 0x7fc8b0105820 2023-06-23T18:22:02.057 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:02.058 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:22:02.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8980a11e0 msgr2=0x7fc8980a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8980a11e0 0x7fc8980a3690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0005950 tx=0x7fc8a000a300 comp rx=0 tx=0).stop 2023-06-23T18:22:02.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0105820 msgr2=0x7fc8b019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0105820 0x7fc8b019cd60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc8a800ada0 tx=0x7fc8a800c330 comp rx=0 tx=0).stop 2023-06-23T18:22:02.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 shutdown_connections 2023-06-23T18:22:02.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8b0105820 0x7fc8b019cd60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8b01005e0 0x7fc8b01957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8980a11e0 0x7fc8980a3690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 --2- 172.21.15.2:0/3789228916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8b0102f00 0x7fc8b0195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 >> 172.21.15.2:0/3789228916 conn(0x7fc8b00fa0e0 msgr2=0x7fc8b0101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 shutdown_connections 2023-06-23T18:22:02.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.053+0000 7fc8b65fe700 1 -- 172.21.15.2:0/3789228916 wait complete. 2023-06-23T18:22:02.060 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:22:02.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:22:02.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 -- 172.21.15.2:0/3742078016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00ff3f0 msgr2=0x7f88b00ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 --2- 172.21.15.2:0/3742078016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00ff3f0 0x7f88b00ff840 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009990 tx=0x7f88a4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 -- 172.21.15.2:0/3742078016 shutdown_connections 2023-06-23T18:22:02.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 --2- 172.21.15.2:0/3742078016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88b00ffd80 0x7f88b0106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 --2- 172.21.15.2:0/3742078016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b00fe200 0x7f88b00fe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 --2- 172.21.15.2:0/3742078016 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00ff3f0 0x7f88b00ff840 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 -- 172.21.15.2:0/3742078016 >> 172.21.15.2:0/3742078016 conn(0x7f88b00f97e0 msgr2=0x7f88b00fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 -- 172.21.15.2:0/3742078016 shutdown_connections 2023-06-23T18:22:02.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.193+0000 7f88b4c72700 1 -- 172.21.15.2:0/3742078016 wait complete. 2023-06-23T18:22:02.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.194+0000 7f88b4c72700 1 Processor -- start 2023-06-23T18:22:02.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.194+0000 7f88b4c72700 1 -- start start 2023-06-23T18:22:02.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88b4c72700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00fe200 0x7f88b0195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88b4c72700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88b00ff3f0 0x7f88b0195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88b4c72700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b00ffd80 0x7f88b019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88b4c72700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b019d390 con 0x7f88b00fe200 2023-06-23T18:22:02.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88b4c72700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b019d4d0 con 0x7f88b00ff3f0 2023-06-23T18:22:02.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88b4c72700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b019d660 con 0x7f88b00ffd80 2023-06-23T18:22:02.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00fe200 0x7f88b0195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00fe200 0x7f88b0195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53576/0 (socket says 172.21.15.2:53576) 2023-06-23T18:22:02.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 -- 172.21.15.2:0/1866046363 learned_addr learned my addr 172.21.15.2:0/1866046363 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88add9b700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88b00ff3f0 0x7f88b0195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 -- 172.21.15.2:0/1866046363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b00ffd80 msgr2=0x7f88b019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b00ffd80 0x7f88b019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 -- 172.21.15.2:0/1866046363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88b00ff3f0 msgr2=0x7f88b0195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88b00ff3f0 0x7f88b0195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 -- 172.21.15.2:0/1866046363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a40095f0 con 0x7f88b00fe200 2023-06-23T18:22:02.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88ae59c700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00fe200 0x7f88b0195810 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88a000d980 tx=0x7f88a000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f889f7fe700 1 -- 172.21.15.2:0/1866046363 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a000de60 con 0x7f88b00fe200 2023-06-23T18:22:02.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b019d940 con 0x7f88b00fe200 2023-06-23T18:22:02.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.195+0000 7f889f7fe700 1 -- 172.21.15.2:0/1866046363 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88a00044b0 con 0x7f88b00fe200 2023-06-23T18:22:02.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.196+0000 7f889f7fe700 1 -- 172.21.15.2:0/1866046363 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0003b40 con 0x7f88b00fe200 2023-06-23T18:22:02.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.196+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b019dea0 con 0x7f88b00fe200 2023-06-23T18:22:02.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.197+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0105810 con 0x7f88b00fe200 2023-06-23T18:22:02.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.198+0000 7f889f7fe700 1 -- 172.21.15.2:0/1866046363 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f88a0004870 con 0x7f88b00fe200 2023-06-23T18:22:02.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.198+0000 7f889f7fe700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88980a11e0 0x7f88980a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.198+0000 7f88add9b700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88980a11e0 0x7f88980a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.198+0000 7f889f7fe700 1 -- 172.21.15.2:0/1866046363 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f88a00ccb50 con 0x7f88b00fe200 2023-06-23T18:22:02.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.199+0000 7f88add9b700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88980a11e0 0x7f88980a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88a40095c0 tx=0x7f88a4009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.202+0000 7f889f7fe700 1 -- 172.21.15.2:0/1866046363 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f88a0096140 con 0x7f88b00fe200 2023-06-23T18:22:02.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.406+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88b002c810 con 0x7f88b00fe200 2023-06-23T18:22:02.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.407+0000 7f889f7fe700 1 -- 172.21.15.2:0/1866046363 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f88a0009940 con 0x7f88b00fe200 2023-06-23T18:22:02.413 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:02.413 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:22:02.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88980a11e0 msgr2=0x7f88980a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88980a11e0 0x7f88980a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88a40095c0 tx=0x7f88a4009550 comp rx=0 tx=0).stop 2023-06-23T18:22:02.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00fe200 msgr2=0x7f88b0195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00fe200 0x7f88b0195810 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88a000d980 tx=0x7f88a000dd40 comp rx=0 tx=0).stop 2023-06-23T18:22:02.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 shutdown_connections 2023-06-23T18:22:02.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b00ffd80 0x7f88b019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88b00fe200 0x7f88b0195810 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88980a11e0 0x7f88980a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 --2- 172.21.15.2:0/1866046363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88b00ff3f0 0x7f88b0195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 >> 172.21.15.2:0/1866046363 conn(0x7f88b00f97e0 msgr2=0x7f88b0102950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 shutdown_connections 2023-06-23T18:22:02.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.409+0000 7f88b4c72700 1 -- 172.21.15.2:0/1866046363 wait complete. 2023-06-23T18:22:02.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-06-23T18:20:44.434596+0000', 'modified': '2023-06-23T18:20:49.041282+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24533, 'mds_1': 24526, 'mds_2': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24467': {'gid': 24467, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6835/2425660103', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6834', 'nonce': 2425660103}, {'type': 'v1', 'addr': '172.21.15.17:6835', 'nonce': 2425660103}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24493': {'gid': 24493, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.82:6835/2287539900', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6834', 'nonce': 2287539900}, {'type': 'v1', 'addr': '172.21.15.82:6835', 'nonce': 2287539900}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24497': {'gid': 24497, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.17:6837/2388271800', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6836', 'nonce': 2388271800}, {'type': 'v1', 'addr': '172.21.15.17:6837', 'nonce': 2388271800}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24505': {'gid': 24505, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.82:6837/2740251056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 2740251056}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2740251056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24526': {'gid': 24526, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.82:6839/2781114494', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6838', 'nonce': 2781114494}, {'type': 'v1', 'addr': '172.21.15.82:6839', 'nonce': 2781114494}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24533': {'gid': 24533, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.17:6839/4283937693', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6838', 'nonce': 4283937693}, {'type': 'v1', 'addr': '172.21.15.17:6839', 'nonce': 4283937693}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-06-23T18:22:02.432 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-06-23T18:22:02.433 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-06-23T18:22:02.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 -- 172.21.15.2:0/968012021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0ffc90 msgr2=0x7f146c1000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 --2- 172.21.15.2:0/968012021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0ffc90 0x7f146c1000e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f145c009990 tx=0x7f145c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 -- 172.21.15.2:0/968012021 shutdown_connections 2023-06-23T18:22:02.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 --2- 172.21.15.2:0/968012021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c100620 0x7f146c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 --2- 172.21.15.2:0/968012021 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c0feaa0 0x7f146c0feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 --2- 172.21.15.2:0/968012021 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0ffc90 0x7f146c1000e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 -- 172.21.15.2:0/968012021 >> 172.21.15.2:0/968012021 conn(0x7f146c0fa0c0 msgr2=0x7f146c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 -- 172.21.15.2:0/968012021 shutdown_connections 2023-06-23T18:22:02.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.555+0000 7f14736cb700 1 -- 172.21.15.2:0/968012021 wait complete. 2023-06-23T18:22:02.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f14736cb700 1 Processor -- start 2023-06-23T18:22:02.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f14736cb700 1 -- start start 2023-06-23T18:22:02.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f14736cb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0feaa0 0x7f146c0690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f14736cb700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ffc90 0x7f146c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f14736cb700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c100620 0x7f146c19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f14736cb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c19d3b0 con 0x7f146c0feaa0 2023-06-23T18:22:02.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f14736cb700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c19d520 con 0x7f146c100620 2023-06-23T18:22:02.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f14736cb700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c19d6e0 con 0x7f146c0ffc90 2023-06-23T18:22:02.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f1471467700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0feaa0 0x7f146c0690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f1471467700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0feaa0 0x7f146c0690f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53590/0 (socket says 172.21.15.2:53590) 2023-06-23T18:22:02.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f1471467700 1 -- 172.21.15.2:0/988523340 learned_addr learned my addr 172.21.15.2:0/988523340 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f1470c66700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ffc90 0x7f146c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.556+0000 7f1471c68700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c100620 0x7f146c19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f1471467700 1 -- 172.21.15.2:0/988523340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ffc90 msgr2=0x7f146c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f1471467700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ffc90 0x7f146c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f1471467700 1 -- 172.21.15.2:0/988523340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c100620 msgr2=0x7f146c19cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f1471467700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c100620 0x7f146c19cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f1471467700 1 -- 172.21.15.2:0/988523340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c0095f0 con 0x7f146c0feaa0 2023-06-23T18:22:02.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f1471467700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0feaa0 0x7f146c0690f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f146800e970 tx=0x7f146800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f14627fc700 1 -- 172.21.15.2:0/988523340 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468009790 con 0x7f146c0feaa0 2023-06-23T18:22:02.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f14736cb700 1 -- 172.21.15.2:0/988523340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c19d9c0 con 0x7f146c0feaa0 2023-06-23T18:22:02.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f14627fc700 1 -- 172.21.15.2:0/988523340 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f146800c7e0 con 0x7f146c0feaa0 2023-06-23T18:22:02.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f14627fc700 1 -- 172.21.15.2:0/988523340 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468003bd0 con 0x7f146c0feaa0 2023-06-23T18:22:02.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.557+0000 7f14736cb700 1 -- 172.21.15.2:0/988523340 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c19dea0 con 0x7f146c0feaa0 2023-06-23T18:22:02.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.559+0000 7f14627fc700 1 -- 172.21.15.2:0/988523340 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f146800c950 con 0x7f146c0feaa0 2023-06-23T18:22:02.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.560+0000 7f14627fc700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14580a11e0 0x7f14580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.560+0000 7f1470c66700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14580a11e0 0x7f14580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.560+0000 7f14627fc700 1 -- 172.21.15.2:0/988523340 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1468014070 con 0x7f146c0feaa0 2023-06-23T18:22:02.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.560+0000 7f1470c66700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14580a11e0 0x7f14580a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f145c006010 tx=0x7f145c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f147f850b80 1 Processor -- start 2023-06-23T18:22:02.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f147f850b80 1 -- start start 2023-06-23T18:22:02.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f147f850b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ec30 0x55eca7b9f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f147f850b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7b9fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f147f850b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f147f850b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7aa3c60 con 0x55eca7b9f580 2023-06-23T18:22:02.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f147f850b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7af69f0 con 0x55eca7b9ffb0 2023-06-23T18:22:02.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f147f850b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7aefe80 con 0x55eca7b9ec30 2023-06-23T18:22:02.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1470c66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7b9fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1470c66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7b9fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53600/0 (socket says 172.21.15.2:53600) 2023-06-23T18:22:02.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1470c66700 1 -- 172.21.15.2:0/1765738169 learned_addr learned my addr 172.21.15.2:0/1765738169 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1471467700 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ec30 0x55eca7b9f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1470c66700 1 -- 172.21.15.2:0/1765738169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ec30 msgr2=0x55eca7b9f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1470c66700 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ec30 0x55eca7b9f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1471c68700 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1470c66700 1 -- 172.21.15.2:0/1765738169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 msgr2=0x55eca7ba0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1470c66700 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.562+0000 7f1470c66700 1 -- 172.21.15.2:0/1765738169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca79715e0 con 0x55eca7b9f580 2023-06-23T18:22:02.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.563+0000 7f1470c66700 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7b9fa20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f145c015000 tx=0x7f145c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447b121945d75311 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.563+0000 7f14577fe700 1 -- 172.21.15.2:0/1765738169 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c003d10 con 0x55eca7b9f580 2023-06-23T18:22:02.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.563+0000 7f14577fe700 1 -- 172.21.15.2:0/1765738169 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f145c01e040 con 0x55eca7b9f580 2023-06-23T18:22:02.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.563+0000 7f14577fe700 1 -- 172.21.15.2:0/1765738169 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c0047f0 con 0x55eca7b9f580 2023-06-23T18:22:02.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.563+0000 7f147f850b80 1 -- 172.21.15.2:0/1765738169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 msgr2=0x55eca7b9fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.563+0000 7f147f850b80 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7b9fa20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f145c015000 tx=0x7f145c020000 comp rx=0 tx=0).stop 2023-06-23T18:22:02.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.563+0000 7f147f850b80 1 -- 172.21.15.2:0/1765738169 shutdown_connections 2023-06-23T18:22:02.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.563+0000 7f147f850b80 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.564+0000 7f147f850b80 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ec30 0x55eca7b9f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.564+0000 7f147f850b80 1 --2- 172.21.15.2:0/1765738169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7b9fa20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.564+0000 7f147f850b80 1 -- 172.21.15.2:0/1765738169 >> 172.21.15.2:0/1765738169 conn(0x55eca7b943a0 msgr2=0x55eca7b994d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.564+0000 7f147f850b80 1 -- 172.21.15.2:0/1765738169 shutdown_connections 2023-06-23T18:22:02.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.564+0000 7f147f850b80 1 -- 172.21.15.2:0/1765738169 wait complete. 2023-06-23T18:22:02.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.564+0000 7f147f850b80 1 -- start start 2023-06-23T18:22:02.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.564+0000 7f147f850b80 1 Processor -- start 2023-06-23T18:22:02.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ec30 0x55eca7ba84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 0x55eca7ba8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 0x55eca7bafa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7a8fa70 con 0x55eca7b9ec30 2023-06-23T18:22:02.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7a8f770 con 0x55eca7b9ffb0 2023-06-23T18:22:02.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca79738d0 con 0x55eca7b9f580 2023-06-23T18:22:02.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ec30 0x55eca7ba84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 10 client.0 _ll_register_callbacks cb 0x55eca7a95210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:02.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ec30 0x55eca7ba84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53604/0 (socket says 172.21.15.2:53604) 2023-06-23T18:22:02.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 -- 172.21.15.2:0/641807989 learned_addr learned my addr 172.21.15.2:0/641807989 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1470c66700 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 0x55eca7ba8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 -- 172.21.15.2:0/641807989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 msgr2=0x55eca7ba8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 0x55eca7ba8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 -- 172.21.15.2:0/641807989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 msgr2=0x55eca7bafa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 0x55eca7bafa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 -- 172.21.15.2:0/641807989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c107810 con 0x55eca7b9ec30 2023-06-23T18:22:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1471467700 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ec30 0x55eca7ba84e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f146800cc00 tx=0x7f146800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28882c1ae5ef3dae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1454ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1454ff9700 1 -- 172.21.15.2:0/641807989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146809a3a0 con 0x55eca7b9ec30 2023-06-23T18:22:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1454ff9700 1 -- 172.21.15.2:0/641807989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f14680a2390 con 0x55eca7b9ec30 2023-06-23T18:22:02.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 1 -- 172.21.15.2:0/641807989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eca7b91b60 con 0x55eca7b9ec30 2023-06-23T18:22:02.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f1454ff9700 1 -- 172.21.15.2:0/641807989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14680b8250 con 0x55eca7b9ec30 2023-06-23T18:22:02.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 10 client.15204 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:02.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.565+0000 7f147f850b80 1 -- 172.21.15.2:0/641807989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55eca791dd90 con 0x55eca7b9ec30 2023-06-23T18:22:02.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f1454ff9700 1 -- 172.21.15.2:0/641807989 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eca7b91b60 con 0x55eca7b9ec30 2023-06-23T18:22:02.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f1454ff9700 1 -- 172.21.15.2:0/641807989 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14680a1f00 con 0x55eca7b9ec30 2023-06-23T18:22:02.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f147f850b80 10 client.15204 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f147f850b80 10 client.15204 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-23T18:22:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f147f850b80 20 client.15204 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f147f850b80 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eca7bb3750 0x55eca7bb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f147f850b80 4 client.15204 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f147f850b80 1 -- 172.21.15.2:0/641807989 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eca7ae8e60 con 0x55eca7bb3750 2023-06-23T18:22:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.566+0000 7f1470c66700 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eca7bb3750 0x55eca7bb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.567+0000 7f1454ff9700 10 client.15204 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:22:02.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.567+0000 7f1470c66700 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eca7bb3750 0x55eca7bb2b70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=817799eda058d44 server_cookie=7057db406e991da0 in_seq=0 out_seq=0 2023-06-23T18:22:02.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.567+0000 7f1454ff9700 1 -- 172.21.15.2:0/641807989 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eca7ae8e60 con 0x55eca7bb3750 2023-06-23T18:22:02.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.567+0000 7f1454ff9700 10 client.15204 handle_command_reply: tid=0 2023-06-23T18:22:02.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.567+0000 7f147f850b80 1 client.15204 shutdown 2023-06-23T18:22:02.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.567+0000 7f147f850b80 1 -- 172.21.15.2:0/641807989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ec30 msgr2=0x55eca7ba84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.567+0000 7f147f850b80 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ec30 0x55eca7ba84e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f146800cc00 tx=0x7f146800cd00 comp rx=0 tx=0).stop 2023-06-23T18:22:02.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 1 -- 172.21.15.2:0/641807989 shutdown_connections 2023-06-23T18:22:02.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eca7bb3750 0x55eca7bb2b70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ffb0 0x55eca7bafa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ec30 0x55eca7ba84e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 1 --2- 172.21.15.2:0/641807989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 0x55eca7ba8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 1 -- 172.21.15.2:0/641807989 >> 172.21.15.2:0/641807989 conn(0x55eca7b943a0 msgr2=0x55eca7b98c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f1454ff9700 0 client.15204 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:22:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 1 -- 172.21.15.2:0/641807989 shutdown_connections 2023-06-23T18:22:02.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 1 -- 172.21.15.2:0/641807989 wait complete. 2023-06-23T18:22:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.568+0000 7f147f850b80 20 client.15204 trim_cache size 0 max 16384 2023-06-23T18:22:02.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.585+0000 7f147f850b80 1 Processor -- start 2023-06-23T18:22:02.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f147f850b80 1 -- start start 2023-06-23T18:22:02.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f147f850b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 0x55eca7bb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f147f850b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 0x55eca7bafa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f147f850b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f147f850b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7ae8e60 con 0x55eca7b9ffb0 2023-06-23T18:22:02.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f147f850b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7b91a70 con 0x55eca7b9ec30 2023-06-23T18:22:02.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f147f850b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7abe790 con 0x55eca7b9f580 2023-06-23T18:22:02.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53616/0 (socket says 172.21.15.2:53616) 2023-06-23T18:22:02.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 -- 172.21.15.2:0/1679459793 learned_addr learned my addr 172.21.15.2:0/1679459793 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471467700 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 0x55eca7bb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1470c66700 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 0x55eca7bafa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 -- 172.21.15.2:0/1679459793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 msgr2=0x55eca7bafa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 0x55eca7bafa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 -- 172.21.15.2:0/1679459793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 msgr2=0x55eca7bb2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 0x55eca7bb2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 -- 172.21.15.2:0/1679459793 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca7906c10 con 0x55eca7b9ffb0 2023-06-23T18:22:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.586+0000 7f1471c68700 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba2ed0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1464009490 tx=0x7f14640077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed3036703bf714a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f1457fff700 1 -- 172.21.15.2:0/1679459793 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146400e040 con 0x55eca7b9ffb0 2023-06-23T18:22:02.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f1457fff700 1 -- 172.21.15.2:0/1679459793 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f146400f040 con 0x55eca7b9ffb0 2023-06-23T18:22:02.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f1457fff700 1 -- 172.21.15.2:0/1679459793 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1464016c20 con 0x55eca7b9ffb0 2023-06-23T18:22:02.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 -- 172.21.15.2:0/1679459793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ffb0 msgr2=0x55eca7ba2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba2ed0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1464009490 tx=0x7f14640077e0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 -- 172.21.15.2:0/1679459793 shutdown_connections 2023-06-23T18:22:02.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9ffb0 0x55eca7ba2ed0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 0x55eca7bb2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 --2- 172.21.15.2:0/1679459793 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9f580 0x55eca7bafa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 -- 172.21.15.2:0/1679459793 >> 172.21.15.2:0/1679459793 conn(0x55eca7b943a0 msgr2=0x55eca7ba7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 -- 172.21.15.2:0/1679459793 shutdown_connections 2023-06-23T18:22:02.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 -- 172.21.15.2:0/1679459793 wait complete. 2023-06-23T18:22:02.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.587+0000 7f147f850b80 1 -- start start 2023-06-23T18:22:02.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f147f850b80 1 Processor -- start 2023-06-23T18:22:02.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f147f850b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 0x55eca7baa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f147f850b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7baadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f147f850b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ffb0 0x55eca7c07610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f147f850b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7bab4e0 con 0x55eca7b9f580 2023-06-23T18:22:02.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f147f850b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7c07b50 con 0x55eca7b9ec30 2023-06-23T18:22:02.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f147f850b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca7c07d10 con 0x55eca7b9ffb0 2023-06-23T18:22:02.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f147f850b80 10 client.0 _ll_register_callbacks cb 0x55eca7a95210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:02.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1470c66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7baadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1470c66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7baadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53620/0 (socket says 172.21.15.2:53620) 2023-06-23T18:22:02.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1470c66700 1 -- 172.21.15.2:0/1581735187 learned_addr learned my addr 172.21.15.2:0/1581735187 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1471467700 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 0x55eca7baa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1471c68700 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ffb0 0x55eca7c07610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1470c66700 1 -- 172.21.15.2:0/1581735187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ffb0 msgr2=0x55eca7c07610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1470c66700 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ffb0 0x55eca7c07610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1470c66700 1 -- 172.21.15.2:0/1581735187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 msgr2=0x55eca7baa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1470c66700 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 0x55eca7baa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.588+0000 7f1470c66700 1 -- 172.21.15.2:0/1581735187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1464009170 con 0x55eca7b9f580 2023-06-23T18:22:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f1470c66700 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7baadf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f145c009990 tx=0x7f145c00b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee52adf132fd8fc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f1454ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f1454ff9700 1 -- 172.21.15.2:0/1581735187 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c03c0b0 con 0x55eca7b9f580 2023-06-23T18:22:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 1 -- 172.21.15.2:0/1581735187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f145c01e040 con 0x55eca7b9f580 2023-06-23T18:22:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f1454ff9700 1 -- 172.21.15.2:0/1581735187 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f145c03c210 con 0x55eca7b9f580 2023-06-23T18:22:02.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f1454ff9700 1 -- 172.21.15.2:0/1581735187 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f145c03c3d0 con 0x55eca7b9f580 2023-06-23T18:22:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 10 client.15210 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 1 -- 172.21.15.2:0/1581735187 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55eca7944200 con 0x55eca7b9f580 2023-06-23T18:22:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f1454ff9700 1 -- 172.21.15.2:0/1581735187 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f145c01e040 con 0x55eca7b9f580 2023-06-23T18:22:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f1454ff9700 1 -- 172.21.15.2:0/1581735187 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f145c004500 con 0x55eca7b9f580 2023-06-23T18:22:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 10 client.15210 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 10 client.15210 resolve_mds: resolved name 'c' to daemon mds.c 2023-06-23T18:22:02.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 20 client.15210 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eca7c0c5e0 0x55eca7c08f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 4 client.15210 mds_command: new command op to 24505 tid=0[{"prefix": "status"}] 2023-06-23T18:22:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.589+0000 7f147f850b80 1 -- 172.21.15.2:0/1581735187 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55eca7aab000 con 0x55eca7c0c5e0 2023-06-23T18:22:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f1471467700 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eca7c0c5e0 0x55eca7c08f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f1454ff9700 10 client.15210 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:22:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f1471467700 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eca7c0c5e0 0x55eca7c08f40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d6c54a530a1eaa5 server_cookie=f391f9869ae5c05b in_seq=0 out_seq=0 2023-06-23T18:22:02.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f1454ff9700 1 -- 172.21.15.2:0/1581735187 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55eca7aab000 con 0x55eca7c0c5e0 2023-06-23T18:22:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f1454ff9700 10 client.15210 handle_command_reply: tid=0 2023-06-23T18:22:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 client.15210 shutdown 2023-06-23T18:22:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 -- 172.21.15.2:0/1581735187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 msgr2=0x55eca7baadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7baadf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f145c009990 tx=0x7f145c00b2d0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 -- 172.21.15.2:0/1581735187 shutdown_connections 2023-06-23T18:22:02.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eca7c0c5e0 0x55eca7c08f40 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eca7b9ffb0 0x55eca7c07610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f1454ff9700 0 client.15210 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:22:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eca7b9ec30 0x55eca7baa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 --2- 172.21.15.2:0/1581735187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eca7b9f580 0x55eca7baadf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 -- 172.21.15.2:0/1581735187 >> 172.21.15.2:0/1581735187 conn(0x55eca7b943a0 msgr2=0x55eca7bb2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 -- 172.21.15.2:0/1581735187 shutdown_connections 2023-06-23T18:22:02.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.590+0000 7f147f850b80 1 -- 172.21.15.2:0/1581735187 wait complete. 2023-06-23T18:22:02.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f147f850b80 20 client.15210 trim_cache size 0 max 16384 2023-06-23T18:22:02.607 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:02.607 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:22:02.607 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 2, 2023-06-23T18:22:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24505, 2023-06-23T18:22:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-06-23T18:22:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-06-23T18:22:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-06-23T18:22:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 74.099618917000001, 2023-06-23T18:22:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 22, 2023-06-23T18:22:02.608 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 84, 2023-06-23T18:22:02.609 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 82, 2023-06-23T18:22:02.609 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 94.071342670000007 2023-06-23T18:22:02.609 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:02.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 -- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14580a11e0 msgr2=0x7f14580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14580a11e0 0x7f14580a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f145c006010 tx=0x7f145c00b260 comp rx=0 tx=0).stop 2023-06-23T18:22:02.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 -- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0feaa0 msgr2=0x7f146c0690f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0feaa0 0x7f146c0690f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f146800e970 tx=0x7f146800ec80 comp rx=0 tx=0).stop 2023-06-23T18:22:02.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 -- 172.21.15.2:0/988523340 shutdown_connections 2023-06-23T18:22:02.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14580a11e0 0x7f14580a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c100620 0x7f146c19cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f146c0feaa0 0x7f146c0690f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 --2- 172.21.15.2:0/988523340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ffc90 0x7f146c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 -- 172.21.15.2:0/988523340 >> 172.21.15.2:0/988523340 conn(0x7f146c0fa0c0 msgr2=0x7f146c0fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 -- 172.21.15.2:0/988523340 shutdown_connections 2023-06-23T18:22:02.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.591+0000 7f14736cb700 1 -- 172.21.15.2:0/988523340 wait complete. 2023-06-23T18:22:02.612 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-06-23T18:22:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 -- 172.21.15.2:0/2519616433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00fea60 msgr2=0x7fbfb00fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/2519616433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00fea60 0x7fbfb00fee70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009990 tx=0x7fbfa4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 -- 172.21.15.2:0/2519616433 shutdown_connections 2023-06-23T18:22:02.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/2519616433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb01006a0 0x7fbfb0106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/2519616433 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00fea60 0x7fbfb00fee70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/2519616433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb0100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 -- 172.21.15.2:0/2519616433 >> 172.21.15.2:0/2519616433 conn(0x7fbfb00fa080 msgr2=0x7fbfb00fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 -- 172.21.15.2:0/2519616433 shutdown_connections 2023-06-23T18:22:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.733+0000 7fbfb6f12700 1 -- 172.21.15.2:0/2519616433 wait complete. 2023-06-23T18:22:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb6f12700 1 Processor -- start 2023-06-23T18:22:02.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb6f12700 1 -- start start 2023-06-23T18:22:02.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb6f12700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb00694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb6f12700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb6f12700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb01006a0 0x7fbfb019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb6f12700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb019d3b0 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb6f12700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb019d520 con 0x7fbfb00fea60 2023-06-23T18:22:02.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb6f12700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb019d6e0 con 0x7fbfb01006a0 2023-06-23T18:22:02.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfaffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfaffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53634/0 (socket says 172.21.15.2:53634) 2023-06-23T18:22:02.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfaffff700 1 -- 172.21.15.2:0/3430823076 learned_addr learned my addr 172.21.15.2:0/3430823076 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb00694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb54af700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb01006a0 0x7fbfb019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfaffff700 1 -- 172.21.15.2:0/3430823076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb01006a0 msgr2=0x7fbfb019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfaffff700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb01006a0 0x7fbfb019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfaffff700 1 -- 172.21.15.2:0/3430823076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 msgr2=0x7fbfb00694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfaffff700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb00694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfaffff700 1 -- 172.21.15.2:0/3430823076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa40095f0 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb54af700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb01006a0 0x7fbfb019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:02.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.734+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb00694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.735+0000 7fbfaffff700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb019a590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00d950 tx=0x7fbf9c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.735+0000 7fbfadffb700 1 -- 172.21.15.2:0/3430823076 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c009b40 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.735+0000 7fbfadffb700 1 -- 172.21.15.2:0/3430823076 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf9c009ca0 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.735+0000 7fbfb6f12700 1 -- 172.21.15.2:0/3430823076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb019d9c0 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.735+0000 7fbfadffb700 1 -- 172.21.15.2:0/3430823076 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00f620 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.735+0000 7fbfb6f12700 1 -- 172.21.15.2:0/3430823076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb019ded0 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.738+0000 7fbfadffb700 1 -- 172.21.15.2:0/3430823076 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fbf9c010460 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.738+0000 7fbfadffb700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfa00a11e0 0x7fbfa00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.738+0000 7fbfc3097b80 1 Processor -- start 2023-06-23T18:22:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.738+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfa00a11e0 0x7fbfa00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.738+0000 7fbfadffb700 1 -- 172.21.15.2:0/3430823076 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbf9c0cce00 con 0x7fbfb00ffcf0 2023-06-23T18:22:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfc3097b80 1 -- start start 2023-06-23T18:22:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86521c30 0x55dc86522040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522580 0x55dc86522a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522fb0 0x55dc86523420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc86426c60 con 0x55dc86522580 2023-06-23T18:22:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc864799f0 con 0x55dc86522fb0 2023-06-23T18:22:02.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc86472e80 con 0x55dc86521c30 2023-06-23T18:22:02.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522580 0x55dc86522a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522580 0x55dc86522a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53640/0 (socket says 172.21.15.2:53640) 2023-06-23T18:22:02.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 -- 172.21.15.2:0/2726904173 learned_addr learned my addr 172.21.15.2:0/2726904173 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfa00a11e0 0x7fbfa00a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400b5c0 tx=0x7fbfa4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfb54af700 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522fb0 0x55dc86523420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86521c30 0x55dc86522040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 -- 172.21.15.2:0/2726904173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86521c30 msgr2=0x55dc86522040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86521c30 0x55dc86522040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 -- 172.21.15.2:0/2726904173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522fb0 msgr2=0x55dc86523420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522fb0 0x55dc86523420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 -- 172.21.15.2:0/2726904173 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc862f45e0 con 0x55dc86522580 2023-06-23T18:22:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86521c30 0x55dc86522040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.739+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522580 0x55dc86522a20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c010ab0 tx=0x7fbf9c099c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ebc9afe0602198 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbf9affd700 1 -- 172.21.15.2:0/2726904173 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c09e0e0 con 0x55dc86522580 2023-06-23T18:22:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbf9affd700 1 -- 172.21.15.2:0/2726904173 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf9c0a2500 con 0x55dc86522580 2023-06-23T18:22:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbf9affd700 1 -- 172.21.15.2:0/2726904173 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c09a550 con 0x55dc86522580 2023-06-23T18:22:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2726904173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522580 msgr2=0x55dc86522a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522580 0x55dc86522a20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c010ab0 tx=0x7fbf9c099c70 comp rx=0 tx=0).stop 2023-06-23T18:22:02.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2726904173 shutdown_connections 2023-06-23T18:22:02.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522fb0 0x55dc86523420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86521c30 0x55dc86522040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2726904173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522580 0x55dc86522a20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2726904173 >> 172.21.15.2:0/2726904173 conn(0x55dc865173a0 msgr2=0x55dc8651c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2726904173 shutdown_connections 2023-06-23T18:22:02.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.740+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2726904173 wait complete. 2023-06-23T18:22:02.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.741+0000 7fbfc3097b80 1 -- start start 2023-06-23T18:22:02.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.741+0000 7fbfc3097b80 1 Processor -- start 2023-06-23T18:22:02.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.741+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.741+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 0x55dc8652ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.741+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 0x55dc86532a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc86412a70 con 0x55dc86521c30 2023-06-23T18:22:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc86412770 con 0x55dc86522580 2023-06-23T18:22:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc862f68d0 con 0x55dc86522fb0 2023-06-23T18:22:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfc3097b80 10 client.0 _ll_register_callbacks cb 0x55dc86418210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53646/0 (socket says 172.21.15.2:53646) 2023-06-23T18:22:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 -- 172.21.15.2:0/2078398379 learned_addr learned my addr 172.21.15.2:0/2078398379 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb54af700 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 0x55dc86532a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 0x55dc8652ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 -- 172.21.15.2:0/2078398379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 msgr2=0x55dc86532a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 0x55dc86532a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 -- 172.21.15.2:0/2078398379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 msgr2=0x55dc8652ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 0x55dc8652ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 -- 172.21.15.2:0/2078398379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb019ded0 con 0x55dc86521c30 2023-06-23T18:22:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652b4e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40038a0 tx=0x7fbfa401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9183fa0e995992f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbf8ffff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbf8ffff700 1 -- 172.21.15.2:0/2078398379 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa402b040 con 0x55dc86521c30 2023-06-23T18:22:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2078398379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc86514b60 con 0x55dc86521c30 2023-06-23T18:22:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbf8ffff700 1 -- 172.21.15.2:0/2078398379 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfa402f420 con 0x55dc86521c30 2023-06-23T18:22:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.742+0000 7fbf8ffff700 1 -- 172.21.15.2:0/2078398379 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfa4004220 con 0x55dc86521c30 2023-06-23T18:22:02.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfc3097b80 10 client.15222 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2078398379 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55dc862a0d90 con 0x55dc86521c30 2023-06-23T18:22:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbf8ffff700 1 -- 172.21.15.2:0/2078398379 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dc86514b60 con 0x55dc86521c30 2023-06-23T18:22:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbf8ffff700 1 -- 172.21.15.2:0/2078398379 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbfa4024410 con 0x55dc86521c30 2023-06-23T18:22:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfc3097b80 10 client.15222 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfc3097b80 10 client.15222 resolve_mds: resolved name 'i' to daemon mds.i 2023-06-23T18:22:02.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfc3097b80 20 client.15222 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55dc865367b0 0x55dc86535bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfc3097b80 4 client.15222 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2078398379 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc8646be60 con 0x55dc865367b0 2023-06-23T18:22:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.743+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55dc865367b0 0x55dc86535bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.744+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55dc865367b0 0x55dc86535bb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c407f900bece470 server_cookie=9b6e4235d9c33096 in_seq=0 out_seq=0 2023-06-23T18:22:02.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.744+0000 7fbf8ffff700 10 client.15222 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:22:02.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.744+0000 7fbf8ffff700 1 -- 172.21.15.2:0/2078398379 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dc8646be60 con 0x55dc865367b0 2023-06-23T18:22:02.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.744+0000 7fbf8ffff700 10 client.15222 handle_command_reply: tid=0 2023-06-23T18:22:02.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.744+0000 7fbfc3097b80 1 client.15222 shutdown 2023-06-23T18:22:02.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2078398379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 msgr2=0x55dc8652b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652b4e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40038a0 tx=0x7fbfa401a050 comp rx=0 tx=0).stop 2023-06-23T18:22:02.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2078398379 shutdown_connections 2023-06-23T18:22:02.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55dc865367b0 0x55dc86535bb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 0x55dc86532a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbf8ffff700 0 client.15222 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:22:02.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652b4e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2078398379 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 0x55dc8652ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2078398379 >> 172.21.15.2:0/2078398379 conn(0x55dc865173a0 msgr2=0x55dc8651bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2078398379 shutdown_connections 2023-06-23T18:22:02.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2078398379 wait complete. 2023-06-23T18:22:02.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.745+0000 7fbfc3097b80 20 client.15222 trim_cache size 0 max 16384 2023-06-23T18:22:02.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfc3097b80 1 Processor -- start 2023-06-23T18:22:02.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfc3097b80 1 -- start start 2023-06-23T18:22:02.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86521c30 0x55dc86535bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522580 0x55dc865360f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522fb0 0x55dc865329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8652b0b0 con 0x55dc86522fb0 2023-06-23T18:22:02.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc8646be60 con 0x55dc86521c30 2023-06-23T18:22:02.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc86519420 con 0x55dc86522580 2023-06-23T18:22:02.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522fb0 0x55dc865329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522fb0 0x55dc865329f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53658/0 (socket says 172.21.15.2:53658) 2023-06-23T18:22:02.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 -- 172.21.15.2:0/2701269344 learned_addr learned my addr 172.21.15.2:0/2701269344 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86521c30 0x55dc86535bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522580 0x55dc865360f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 -- 172.21.15.2:0/2701269344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522580 msgr2=0x55dc865360f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522580 0x55dc865360f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 -- 172.21.15.2:0/2701269344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86521c30 msgr2=0x55dc86535bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86521c30 0x55dc86535bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 -- 172.21.15.2:0/2701269344 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc86289c10 con 0x55dc86522fb0 2023-06-23T18:22:02.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86521c30 0x55dc86535bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:02.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522580 0x55dc865360f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:02.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.769+0000 7fbfb54af700 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522fb0 0x55dc865329f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ea0 tx=0x7fbfa80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90061a9441600bb4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbf9b7fe700 1 -- 172.21.15.2:0/2701269344 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8007e30 con 0x55dc86522fb0 2023-06-23T18:22:02.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbf9b7fe700 1 -- 172.21.15.2:0/2701269344 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfa8004bb0 con 0x55dc86522fb0 2023-06-23T18:22:02.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbf9b7fe700 1 -- 172.21.15.2:0/2701269344 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80052c0 con 0x55dc86522fb0 2023-06-23T18:22:02.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2701269344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522fb0 msgr2=0x55dc865329f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522fb0 0x55dc865329f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ea0 tx=0x7fbfa80077e0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2701269344 shutdown_connections 2023-06-23T18:22:02.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86522fb0 0x55dc865329f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86521c30 0x55dc86535bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/2701269344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522580 0x55dc865360f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2701269344 >> 172.21.15.2:0/2701269344 conn(0x55dc865173a0 msgr2=0x55dc8652a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2701269344 shutdown_connections 2023-06-23T18:22:02.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 -- 172.21.15.2:0/2701269344 wait complete. 2023-06-23T18:22:02.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.770+0000 7fbfc3097b80 1 -- start start 2023-06-23T18:22:02.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfc3097b80 1 Processor -- start 2023-06-23T18:22:02.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 0x55dc8652e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfc3097b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 0x55dc865311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc865316f0 con 0x55dc86521c30 2023-06-23T18:22:02.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc86531860 con 0x55dc86522580 2023-06-23T18:22:02.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfc3097b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc86531a20 con 0x55dc86522fb0 2023-06-23T18:22:02.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfc3097b80 10 client.0 _ll_register_callbacks cb 0x55dc86418210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:02.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb4cae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb4cae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53660/0 (socket says 172.21.15.2:53660) 2023-06-23T18:22:02.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb4cae700 1 -- 172.21.15.2:0/1593153870 learned_addr learned my addr 172.21.15.2:0/1593153870 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb54af700 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 0x55dc865311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 0x55dc8652e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb4cae700 1 -- 172.21.15.2:0/1593153870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 msgr2=0x55dc865311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 0x55dc865311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb4cae700 1 -- 172.21.15.2:0/1593153870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 msgr2=0x55dc8652e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 0x55dc8652e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.771+0000 7fbfb4cae700 1 -- 172.21.15.2:0/1593153870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa8009b50 con 0x55dc86521c30 2023-06-23T18:22:02.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfb4cae700 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652dc50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40098e0 tx=0x7fbfa4024640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48568549af38748b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbf8ffff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:02.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbf8ffff700 1 -- 172.21.15.2:0/1593153870 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa402b070 con 0x55dc86521c30 2023-06-23T18:22:02.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbf8ffff700 1 -- 172.21.15.2:0/1593153870 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfa4024b10 con 0x55dc86521c30 2023-06-23T18:22:02.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 1 -- 172.21.15.2:0/1593153870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf9c0a2500 con 0x55dc86521c30 2023-06-23T18:22:02.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbf8ffff700 1 -- 172.21.15.2:0/1593153870 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfa4004220 con 0x55dc86521c30 2023-06-23T18:22:02.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 10 client.15228 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:02.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 1 -- 172.21.15.2:0/1593153870 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55dc862c7200 con 0x55dc86521c30 2023-06-23T18:22:02.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbf8ffff700 1 -- 172.21.15.2:0/1593153870 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfa4051020 con 0x55dc86521c30 2023-06-23T18:22:02.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbf8ffff700 1 -- 172.21.15.2:0/1593153870 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbfa4017bd0 con 0x55dc86521c30 2023-06-23T18:22:02.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 10 client.15228 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:02.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 10 client.15228 resolve_mds: resolved name 'i' to daemon mds.i 2023-06-23T18:22:02.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 20 client.15228 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:02.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55dc8658f960 0x55dc8658c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 4 client.15228 mds_command: new command op to 24526 tid=0[{"prefix": "status"}] 2023-06-23T18:22:02.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.772+0000 7fbfc3097b80 1 -- 172.21.15.2:0/1593153870 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dc8642e000 con 0x55dc8658f960 2023-06-23T18:22:02.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55dc8658f960 0x55dc8658c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbf8ffff700 10 client.15228 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:22:02.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55dc8658f960 0x55dc8658c660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef8407e4cc383f04 server_cookie=7180ffec5241bc89 in_seq=0 out_seq=0 2023-06-23T18:22:02.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbf8ffff700 1 -- 172.21.15.2:0/1593153870 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55dc8642e000 con 0x55dc8658f960 2023-06-23T18:22:02.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbf8ffff700 10 client.15228 handle_command_reply: tid=0 2023-06-23T18:22:02.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 client.15228 shutdown 2023-06-23T18:22:02.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 -- 172.21.15.2:0/1593153870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 msgr2=0x55dc8652dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652dc50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbfa40098e0 tx=0x7fbfa4024640 comp rx=0 tx=0).stop 2023-06-23T18:22:02.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 -- 172.21.15.2:0/1593153870 shutdown_connections 2023-06-23T18:22:02.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55dc8658f960 0x55dc8658c660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dc86522fb0 0x55dc865311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbf8ffff700 0 client.15228 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:22:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dc86521c30 0x55dc8652dc50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 --2- 172.21.15.2:0/1593153870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dc86522580 0x55dc8652e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.773+0000 7fbfc3097b80 1 -- 172.21.15.2:0/1593153870 >> 172.21.15.2:0/1593153870 conn(0x55dc865173a0 msgr2=0x55dc86535bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfc3097b80 1 -- 172.21.15.2:0/1593153870 shutdown_connections 2023-06-23T18:22:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfc3097b80 1 -- 172.21.15.2:0/1593153870 wait complete. 2023-06-23T18:22:02.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfc3097b80 20 client.15228 trim_cache size 0 max 16384 2023-06-23T18:22:02.800 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:22:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 1, 2023-06-23T18:22:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24526, 2023-06-23T18:22:02.800 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-06-23T18:22:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-06-23T18:22:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-06-23T18:22:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 75.285042200999996, 2023-06-23T18:22:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 22, 2023-06-23T18:22:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 84, 2023-06-23T18:22:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 82, 2023-06-23T18:22:02.801 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 89.363565984000005 2023-06-23T18:22:02.801 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:02.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 -- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfa00a11e0 msgr2=0x7fbfa00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfa00a11e0 0x7fbfa00a3690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400b5c0 tx=0x7fbfa4005fb0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 -- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00ffcf0 msgr2=0x7fbfb019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb019a590 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00d950 tx=0x7fbf9c00dd10 comp rx=0 tx=0).stop 2023-06-23T18:22:02.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 -- 172.21.15.2:0/3430823076 shutdown_connections 2023-06-23T18:22:02.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb01006a0 0x7fbfb019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfa00a11e0 0x7fbfa00a3690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb00694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 --2- 172.21.15.2:0/3430823076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb019a590 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 -- 172.21.15.2:0/3430823076 >> 172.21.15.2:0/3430823076 conn(0x7fbfb00fa080 msgr2=0x7fbfb0103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 -- 172.21.15.2:0/3430823076 shutdown_connections 2023-06-23T18:22:02.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.774+0000 7fbfb6f12700 1 -- 172.21.15.2:0/3430823076 wait complete. 2023-06-23T18:22:02.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:02 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3699311464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:02.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:02 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3789228916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:02.805 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:02 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1866046363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:02.806 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-06-23T18:22:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:02 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3699311464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:02.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:02 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3789228916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:02.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:02 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1866046363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:02.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.927+0000 7f0c2e611700 1 -- 172.21.15.2:0/2932125943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 msgr2=0x7f0c28106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.927+0000 7f0c2e611700 1 --2- 172.21.15.2:0/2932125943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 0x7f0c28106f60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000b0a0 tx=0x7f0c2000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.927+0000 7f0c2e611700 1 -- 172.21.15.2:0/2932125943 shutdown_connections 2023-06-23T18:22:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.927+0000 7f0c2e611700 1 --2- 172.21.15.2:0/2932125943 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 0x7f0c28106f60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.927+0000 7f0c2e611700 1 --2- 172.21.15.2:0/2932125943 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c280fea80 0x7f0c280fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.927+0000 7f0c2e611700 1 --2- 172.21.15.2:0/2932125943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c280ffd10 0x7f0c28100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.927+0000 7f0c2e611700 1 -- 172.21.15.2:0/2932125943 >> 172.21.15.2:0/2932125943 conn(0x7f0c280fa0c0 msgr2=0x7f0c280fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.927+0000 7f0c2e611700 1 -- 172.21.15.2:0/2932125943 shutdown_connections 2023-06-23T18:22:02.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 -- 172.21.15.2:0/2932125943 wait complete. 2023-06-23T18:22:02.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 Processor -- start 2023-06-23T18:22:02.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 -- start start 2023-06-23T18:22:02.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c280fea80 0x7f0c28102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c280ffd10 0x7f0c281010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 0x7f0c28101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28101dd0 con 0x7f0c281006c0 2023-06-23T18:22:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28101f40 con 0x7f0c280ffd10 2023-06-23T18:22:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.928+0000 7f0c2e611700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c2819f940 con 0x7f0c280fea80 2023-06-23T18:22:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 0x7f0c28101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 0x7f0c28101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53678/0 (socket says 172.21.15.2:53678) 2023-06-23T18:22:02.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 -- 172.21.15.2:0/870059443 learned_addr learned my addr 172.21.15.2:0/870059443 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c277fe700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c280ffd10 0x7f0c281010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c27fff700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c280fea80 0x7f0c28102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 -- 172.21.15.2:0/870059443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c280fea80 msgr2=0x7f0c28102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c280fea80 0x7f0c28102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 -- 172.21.15.2:0/870059443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c280ffd10 msgr2=0x7f0c281010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c280ffd10 0x7f0c281010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 -- 172.21.15.2:0/870059443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c20009d00 con 0x7f0c281006c0 2023-06-23T18:22:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c27fff700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c280fea80 0x7f0c28102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:02.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 0x7f0c28101630 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0c200062a0 tx=0x7f0c20004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c257fa700 1 -- 172.21.15.2:0/870059443 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2000e040 con 0x7f0c281006c0 2023-06-23T18:22:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2e611700 1 -- 172.21.15.2:0/870059443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2819fbc0 con 0x7f0c281006c0 2023-06-23T18:22:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c257fa700 1 -- 172.21.15.2:0/870059443 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c20003e90 con 0x7f0c281006c0 2023-06-23T18:22:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c257fa700 1 -- 172.21.15.2:0/870059443 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c20004020 con 0x7f0c281006c0 2023-06-23T18:22:02.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.929+0000 7f0c2e611700 1 -- 172.21.15.2:0/870059443 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c281a0040 con 0x7f0c281006c0 2023-06-23T18:22:02.946 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:02 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3699311464' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:02.946 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:02 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3789228916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:02.946 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:02 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1866046363' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:02.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.931+0000 7f0c257fa700 1 -- 172.21.15.2:0/870059443 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f0c20003a00 con 0x7f0c281006c0 2023-06-23T18:22:02.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.932+0000 7f0c257fa700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0c100a11e0 0x7f0c100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.932+0000 7f0c27fff700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0c100a11e0 0x7f0c100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.932+0000 7f0c257fa700 1 -- 172.21.15.2:0/870059443 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0c200ce430 con 0x7f0c281006c0 2023-06-23T18:22:02.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c27fff700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0c100a11e0 0x7f0c100a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005fd0 tx=0x7f0c18005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c3a796b80 1 Processor -- start 2023-06-23T18:22:02.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c3a796b80 1 -- start start 2023-06-23T18:22:02.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd8e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8e580 0x55f81bd8ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8efb0 0x55f81bd8f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bc92c60 con 0x55f81bd8dc30 2023-06-23T18:22:02.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bce59f0 con 0x55f81bd8e580 2023-06-23T18:22:02.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bcdee80 con 0x55f81bd8efb0 2023-06-23T18:22:02.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c27fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c27fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd8e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53682/0 (socket says 172.21.15.2:53682) 2023-06-23T18:22:02.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c27fff700 1 -- 172.21.15.2:0/3605131602 learned_addr learned my addr 172.21.15.2:0/3605131602 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8efb0 0x55f81bd8f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.933+0000 7f0c277fe700 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8e580 0x55f81bd8ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c27fff700 1 -- 172.21.15.2:0/3605131602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8efb0 msgr2=0x55f81bd8f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c27fff700 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8efb0 0x55f81bd8f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c27fff700 1 -- 172.21.15.2:0/3605131602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8e580 msgr2=0x55f81bd8ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c27fff700 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8e580 0x55f81bd8ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c27fff700 1 -- 172.21.15.2:0/3605131602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f81bb605e0 con 0x55f81bd8dc30 2023-06-23T18:22:02.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c277fe700 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8e580 0x55f81bd8ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:02.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c27fff700 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd8e040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800ff30 tx=0x7f0c18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff867f84b025c3b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c0e7fc700 1 -- 172.21.15.2:0/3605131602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c18011bb0 con 0x55f81bd8dc30 2023-06-23T18:22:02.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c0e7fc700 1 -- 172.21.15.2:0/3605131602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c18011d10 con 0x55f81bd8dc30 2023-06-23T18:22:02.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.934+0000 7f0c0e7fc700 1 -- 172.21.15.2:0/3605131602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1801d610 con 0x55f81bd8dc30 2023-06-23T18:22:02.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3605131602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 msgr2=0x55f81bd8e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd8e040 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800ff30 tx=0x7f0c18012000 comp rx=0 tx=0).stop 2023-06-23T18:22:02.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3605131602 shutdown_connections 2023-06-23T18:22:02.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8efb0 0x55f81bd8f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd8e040 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/3605131602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8e580 0x55f81bd8ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3605131602 >> 172.21.15.2:0/3605131602 conn(0x55f81bd833a0 msgr2=0x55f81bd884d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3605131602 shutdown_connections 2023-06-23T18:22:02.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3605131602 wait complete. 2023-06-23T18:22:02.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 -- start start 2023-06-23T18:22:02.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.935+0000 7f0c3a796b80 1 Processor -- start 2023-06-23T18:22:02.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd97500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bd97a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bc7ea70 con 0x55f81bd8dc30 2023-06-23T18:22:02.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bc7e770 con 0x55f81bd8efb0 2023-06-23T18:22:02.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bb628d0 con 0x55f81bd8e580 2023-06-23T18:22:02.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c3a796b80 10 client.0 _ll_register_callbacks cb 0x55f81bc84210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:02.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd97500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd97500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53688/0 (socket says 172.21.15.2:53688) 2023-06-23T18:22:02.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 -- 172.21.15.2:0/2948597602 learned_addr learned my addr 172.21.15.2:0/2948597602 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c277fe700 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bd97a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 -- 172.21.15.2:0/2948597602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 msgr2=0x55f81bd97a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bd97a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 -- 172.21.15.2:0/2948597602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 msgr2=0x55f81bd9ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 -- 172.21.15.2:0/2948597602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1800fbe0 con 0x55f81bd8dc30 2023-06-23T18:22:02.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c27fff700 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd97500 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c1801e680 tx=0x7f0c1801e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e1b024e661ebe5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c077fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c077fe700 1 -- 172.21.15.2:0/2948597602 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1802d070 con 0x55f81bd8dc30 2023-06-23T18:22:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c077fe700 1 -- 172.21.15.2:0/2948597602 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0c1800bdd0 con 0x55f81bd8dc30 2023-06-23T18:22:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.936+0000 7f0c3a796b80 1 -- 172.21.15.2:0/2948597602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f81bd80b60 con 0x55f81bd8dc30 2023-06-23T18:22:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c077fe700 1 -- 172.21.15.2:0/2948597602 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c18011bb0 con 0x55f81bd8dc30 2023-06-23T18:22:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c3a796b80 10 client.15240 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c3a796b80 1 -- 172.21.15.2:0/2948597602 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55f81bb0cd90 con 0x55f81bd8dc30 2023-06-23T18:22:02.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c077fe700 1 -- 172.21.15.2:0/2948597602 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f81bd80b60 con 0x55f81bd8dc30 2023-06-23T18:22:02.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c077fe700 1 -- 172.21.15.2:0/2948597602 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0c18027450 con 0x55f81bd8dc30 2023-06-23T18:22:02.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c3a796b80 10 client.15240 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:02.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c3a796b80 10 client.15240 resolve_mds: resolved name 'b' to daemon mds.b 2023-06-23T18:22:02.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c3a796b80 20 client.15240 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:02.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f81bda2690 0x55f81bda1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c3a796b80 4 client.15240 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:02.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c3a796b80 1 -- 172.21.15.2:0/2948597602 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f81bcd7e60 con 0x55f81bda2690 2023-06-23T18:22:02.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.937+0000 7f0c277fe700 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f81bda2690 0x55f81bda1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.938+0000 7f0c077fe700 10 client.15240 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:02.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.938+0000 7f0c277fe700 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f81bda2690 0x55f81bda1a90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59c9edee4957510 server_cookie=21f697bf80ef81fe in_seq=0 out_seq=0 2023-06-23T18:22:02.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c077fe700 1 -- 172.21.15.2:0/2948597602 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f81bcd7e60 con 0x55f81bda2690 2023-06-23T18:22:02.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c077fe700 10 client.15240 handle_command_reply: tid=0 2023-06-23T18:22:02.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 client.15240 shutdown 2023-06-23T18:22:02.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 -- 172.21.15.2:0/2948597602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 msgr2=0x55f81bd97500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd97500 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0c1801e680 tx=0x7f0c1801e760 comp rx=0 tx=0).stop 2023-06-23T18:22:02.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 -- 172.21.15.2:0/2948597602 shutdown_connections 2023-06-23T18:22:02.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f81bda2690 0x55f81bda1a90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd97500 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/2948597602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bd97a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c077fe700 0 client.15240 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:02.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 -- 172.21.15.2:0/2948597602 >> 172.21.15.2:0/2948597602 conn(0x55f81bd833a0 msgr2=0x55f81bd87c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 -- 172.21.15.2:0/2948597602 shutdown_connections 2023-06-23T18:22:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 1 -- 172.21.15.2:0/2948597602 wait complete. 2023-06-23T18:22:02.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.939+0000 7f0c3a796b80 20 client.15240 trim_cache size 0 max 16384 2023-06-23T18:22:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c3a796b80 1 Processor -- start 2023-06-23T18:22:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c3a796b80 1 -- start start 2023-06-23T18:22:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8dc30 0x55f81bda1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bda1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bcd7e60 con 0x55f81bd8efb0 2023-06-23T18:22:02.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bcad790 con 0x55f81bd8dc30 2023-06-23T18:22:02.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bb6ede0 con 0x55f81bd8e580 2023-06-23T18:22:02.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c2cbae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c2cbae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53700/0 (socket says 172.21.15.2:53700) 2023-06-23T18:22:02.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c2cbae700 1 -- 172.21.15.2:0/3803309101 learned_addr learned my addr 172.21.15.2:0/3803309101 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c277fe700 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bda1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c27fff700 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8dc30 0x55f81bda1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c2cbae700 1 -- 172.21.15.2:0/3803309101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 msgr2=0x55f81bda1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bda1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c2cbae700 1 -- 172.21.15.2:0/3803309101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8dc30 msgr2=0x55f81bda1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8dc30 0x55f81bda1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c2cbae700 1 -- 172.21.15.2:0/3803309101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f81baf5c10 con 0x55f81bd8efb0 2023-06-23T18:22:02.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.957+0000 7f0c27fff700 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8dc30 0x55f81bda1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:02.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c2009b700 tx=0x7f0c20086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a241c41c8f0b05 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c0effd700 1 -- 172.21.15.2:0/3803309101 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2009b0d0 con 0x55f81bd8efb0 2023-06-23T18:22:02.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c0effd700 1 -- 172.21.15.2:0/3803309101 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c2001d690 con 0x55f81bd8efb0 2023-06-23T18:22:02.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c0effd700 1 -- 172.21.15.2:0/3803309101 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c200a3b30 con 0x55f81bd8efb0 2023-06-23T18:22:02.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3803309101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8efb0 msgr2=0x55f81bd9ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0c2009b700 tx=0x7f0c20086040 comp rx=0 tx=0).stop 2023-06-23T18:22:02.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3803309101 shutdown_connections 2023-06-23T18:22:02.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8efb0 0x55f81bd9ea40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8dc30 0x55f81bda1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/3803309101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bda1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3803309101 >> 172.21.15.2:0/3803309101 conn(0x55f81bd833a0 msgr2=0x55f81bd96270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3803309101 shutdown_connections 2023-06-23T18:22:02.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.958+0000 7f0c3a796b80 1 -- 172.21.15.2:0/3803309101 wait complete. 2023-06-23T18:22:02.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 1 -- start start 2023-06-23T18:22:02.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 1 Processor -- start 2023-06-23T18:22:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd99cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bd9a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bdf64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bd9a810 con 0x55f81bd8dc30 2023-06-23T18:22:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bdf6a30 con 0x55f81bd8efb0 2023-06-23T18:22:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f81bdf6b70 con 0x55f81bd8e580 2023-06-23T18:22:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c3a796b80 10 client.0 _ll_register_callbacks cb 0x55f81bc84210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:02.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c27fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd99cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c27fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd99cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53702/0 (socket says 172.21.15.2:53702) 2023-06-23T18:22:02.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c27fff700 1 -- 172.21.15.2:0/717962491 learned_addr learned my addr 172.21.15.2:0/717962491 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:02.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bdf64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.959+0000 7f0c277fe700 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bd9a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c2cbae700 1 -- 172.21.15.2:0/717962491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 msgr2=0x55f81bd9a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bd9a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c2cbae700 1 -- 172.21.15.2:0/717962491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 msgr2=0x55f81bd99cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:02.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd99cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c2cbae700 1 -- 172.21.15.2:0/717962491 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c281a0040 con 0x55f81bd8efb0 2023-06-23T18:22:02.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c27fff700 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd99cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:02.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c2cbae700 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bdf64f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55f81bd8ef80 tx=0x7f0c200099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae08ff23e92bec1d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:02.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c077fe700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:22:02.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c077fe700 1 -- 172.21.15.2:0/717962491 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2009b0d0 con 0x55f81bd8efb0 2023-06-23T18:22:02.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c3a796b80 1 -- 172.21.15.2:0/717962491 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c18011d10 con 0x55f81bd8efb0 2023-06-23T18:22:02.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c077fe700 1 -- 172.21.15.2:0/717962491 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0c20003cb0 con 0x55f81bd8efb0 2023-06-23T18:22:02.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c077fe700 1 -- 172.21.15.2:0/717962491 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c2001c3e0 con 0x55f81bd8efb0 2023-06-23T18:22:02.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c3a796b80 10 client.24902 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:02.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c3a796b80 1 -- 172.21.15.2:0/717962491 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55f81bb33200 con 0x55f81bd8efb0 2023-06-23T18:22:02.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.960+0000 7f0c077fe700 1 -- 172.21.15.2:0/717962491 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c20065080 con 0x55f81bd8efb0 2023-06-23T18:22:02.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c077fe700 1 -- 172.21.15.2:0/717962491 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0c2009ac60 con 0x55f81bd8efb0 2023-06-23T18:22:02.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c3a796b80 10 client.24902 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:02.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c3a796b80 10 client.24902 resolve_mds: resolved name 'b' to daemon mds.b 2023-06-23T18:22:02.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c3a796b80 20 client.24902 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:02.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f81bdfb1a0 0x55f81bdf7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:02.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c3a796b80 4 client.24902 mds_command: new command op to 24533 tid=0[{"prefix": "status"}] 2023-06-23T18:22:02.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c3a796b80 1 -- 172.21.15.2:0/717962491 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f81bc9a000 con 0x55f81bdfb1a0 2023-06-23T18:22:02.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c27fff700 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f81bdfb1a0 0x55f81bdf7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:02.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c077fe700 10 client.24902 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:02.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c27fff700 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f81bdfb1a0 0x55f81bdf7ce0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5c2b7670e5a854 server_cookie=45a45cf43d150890 in_seq=0 out_seq=0 2023-06-23T18:22:02.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c077fe700 1 -- 172.21.15.2:0/717962491 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f81bc9a000 con 0x55f81bdfb1a0 2023-06-23T18:22:02.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c077fe700 10 client.24902 handle_command_reply: tid=0 2023-06-23T18:22:02.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.961+0000 7f0c3a796b80 1 client.24902 shutdown 2023-06-23T18:22:02.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 -- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 msgr2=0x55f81bdf64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bdf64f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x55f81bd8ef80 tx=0x7f0c200099e0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 -- 172.21.15.2:0/717962491 shutdown_connections 2023-06-23T18:22:02.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f81bdfb1a0 0x55f81bdf7ce0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f81bd8efb0 0x55f81bdf64f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f81bd8dc30 0x55f81bd99cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 --2- 172.21.15.2:0/717962491 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f81bd8e580 0x55f81bd9a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c077fe700 0 client.24902 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:02.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 -- 172.21.15.2:0/717962491 >> 172.21.15.2:0/717962491 conn(0x55f81bd833a0 msgr2=0x55f81bd91dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 -- 172.21.15.2:0/717962491 shutdown_connections 2023-06-23T18:22:02.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 1 -- 172.21.15.2:0/717962491 wait complete. 2023-06-23T18:22:02.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c3a796b80 20 client.24902 trim_cache size 0 max 16384 2023-06-23T18:22:02.980 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "902733cc-11f1-11ee-9b2b-001a4aab830c", 2023-06-23T18:22:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2023-06-23T18:22:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 24533, 2023-06-23T18:22:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-06-23T18:22:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-06-23T18:22:02.980 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-06-23T18:22:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 78.525680872999999, 2023-06-23T18:22:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 22, 2023-06-23T18:22:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 84, 2023-06-23T18:22:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 82, 2023-06-23T18:22:02.981 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 82.437798411000003 2023-06-23T18:22:02.981 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:02.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 -- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0c100a11e0 msgr2=0x7f0c100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0c100a11e0 0x7f0c100a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c18005fd0 tx=0x7f0c18005f60 comp rx=0 tx=0).stop 2023-06-23T18:22:02.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 -- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 msgr2=0x7f0c28101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:02.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 0x7f0c28101630 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f0c200062a0 tx=0x7f0c20004d00 comp rx=0 tx=0).stop 2023-06-23T18:22:02.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 -- 172.21.15.2:0/870059443 shutdown_connections 2023-06-23T18:22:02.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0c281006c0 0x7f0c28101630 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c280fea80 0x7f0c28102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0c100a11e0 0x7f0c100a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 --2- 172.21.15.2:0/870059443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c280ffd10 0x7f0c281010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:02.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.962+0000 7f0c2e611700 1 -- 172.21.15.2:0/870059443 >> 172.21.15.2:0/870059443 conn(0x7f0c280fa0c0 msgr2=0x7f0c28068a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:02.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.963+0000 7f0c2e611700 1 -- 172.21.15.2:0/870059443 shutdown_connections 2023-06-23T18:22:02.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:02.963+0000 7f0c2e611700 1 -- 172.21.15.2:0/870059443 wait complete. 2023-06-23T18:22:02.984 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-06-23T18:22:03.022 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-06-23T18:22:03.022 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-06-23T18:22:03.023 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2023-06-23T18:22:03.023 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-06-23T18:22:03.024 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-06-23T18:22:03.024 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-06-23T18:22:03.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:03.025 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-06-23T18:22:03.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:03.043 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-06-23T18:22:03.112 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2023-06-23T18:22:03.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T18:22:03.114 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-06-23T18:22:03.114 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph fs ls 2023-06-23T18:22:03.714 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:22:03.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:03 smithi082 ceph-mon[97889]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-06-23T18:22:03.905 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:03 smithi002 ceph-mon[95947]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-06-23T18:22:03.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:03 smithi017 ceph-mon[97472]: pgmap v297: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-06-23T18:22:04.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 -- 172.21.15.2:0/3711414405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 msgr2=0x7fb40c10e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:04.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 --2- 172.21.15.2:0/3711414405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 0x7fb40c10e8d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb40400b0a0 tx=0x7fb40400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:04.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 -- 172.21.15.2:0/3711414405 shutdown_connections 2023-06-23T18:22:04.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 --2- 172.21.15.2:0/3711414405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 0x7fb40c10e8d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:04.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 --2- 172.21.15.2:0/3711414405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb40c0fe940 0x7fb40c0fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:04.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 --2- 172.21.15.2:0/3711414405 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb40c0ff250 0x7fb40c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:04.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 -- 172.21.15.2:0/3711414405 >> 172.21.15.2:0/3711414405 conn(0x7fb40c075b30 msgr2=0x7fb40c075f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:04.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 -- 172.21.15.2:0/3711414405 shutdown_connections 2023-06-23T18:22:04.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.923+0000 7fb4134d3700 1 -- 172.21.15.2:0/3711414405 wait complete. 2023-06-23T18:22:04.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb4134d3700 1 Processor -- start 2023-06-23T18:22:04.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb4134d3700 1 -- start start 2023-06-23T18:22:04.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb4134d3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb40c0fe940 0x7fb40c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:04.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb4134d3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb40c0ff250 0x7fb40c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:04.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb4134d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 0x7fb40c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:04.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb4134d3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c19ceb0 con 0x7fb40c10c4f0 2023-06-23T18:22:04.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb4134d3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c19d020 con 0x7fb40c0ff250 2023-06-23T18:22:04.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb4134d3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c19d1e0 con 0x7fb40c0fe940 2023-06-23T18:22:04.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb411a70700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 0x7fb40c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb411a70700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 0x7fb40c19c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53718/0 (socket says 172.21.15.2:53718) 2023-06-23T18:22:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb411a70700 1 -- 172.21.15.2:0/2051346617 learned_addr learned my addr 172.21.15.2:0/2051346617 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb41126f700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb40c0fe940 0x7fb40c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb410a6e700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb40c0ff250 0x7fb40c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb411a70700 1 -- 172.21.15.2:0/2051346617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb40c0fe940 msgr2=0x7fb40c197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb411a70700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb40c0fe940 0x7fb40c197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb411a70700 1 -- 172.21.15.2:0/2051346617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb40c0ff250 msgr2=0x7fb40c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb411a70700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb40c0ff250 0x7fb40c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb411a70700 1 -- 172.21.15.2:0/2051346617 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404009d00 con 0x7fb40c10c4f0 2023-06-23T18:22:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb41126f700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb40c0fe940 0x7fb40c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.924+0000 7fb410a6e700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb40c0ff250 0x7fb40c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.925+0000 7fb411a70700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 0x7fb40c19c8a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb40400d010 tx=0x7fb404009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.925+0000 7fb4027fc700 1 -- 172.21.15.2:0/2051346617 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40400e040 con 0x7fb40c10c4f0 2023-06-23T18:22:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.925+0000 7fb4027fc700 1 -- 172.21.15.2:0/2051346617 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb40401db10 con 0x7fb40c10c4f0 2023-06-23T18:22:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.925+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c19d460 con 0x7fb40c10c4f0 2023-06-23T18:22:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.925+0000 7fb4027fc700 1 -- 172.21.15.2:0/2051346617 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404005790 con 0x7fb40c10c4f0 2023-06-23T18:22:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.925+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c1a4d20 con 0x7fb40c10c4f0 2023-06-23T18:22:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.926+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40c04e1d0 con 0x7fb40c10c4f0 2023-06-23T18:22:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.927+0000 7fb4027fc700 1 -- 172.21.15.2:0/2051346617 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb40401dc80 con 0x7fb40c10c4f0 2023-06-23T18:22:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.927+0000 7fb4027fc700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3f80a1260 0x7fb3f80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.927+0000 7fb41126f700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3f80a1260 0x7fb3f80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.927+0000 7fb4027fc700 1 -- 172.21.15.2:0/2051346617 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb4040ce500 con 0x7fb40c10c4f0 2023-06-23T18:22:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.928+0000 7fb41126f700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3f80a1260 0x7fb3f80a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc007900 tx=0x7fb3fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:04.932+0000 7fb4027fc700 1 -- 172.21.15.2:0/2051346617 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fb404097b40 con 0x7fb40c10c4f0 2023-06-23T18:22:05.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.178+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb40c1a4f70 con 0x7fb40c10c4f0 2023-06-23T18:22:05.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.178+0000 7fb4027fc700 1 -- 172.21.15.2:0/2051346617 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v22) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb40409b160 con 0x7fb40c10c4f0 2023-06-23T18:22:05.184 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-06-23T18:22:05.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3f80a1260 msgr2=0x7fb3f80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:05.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3f80a1260 0x7fb3f80a3710 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc007900 tx=0x7fb3fc008040 comp rx=0 tx=0).stop 2023-06-23T18:22:05.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 msgr2=0x7fb40c19c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:05.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 0x7fb40c19c8a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb40400d010 tx=0x7fb404009570 comp rx=0 tx=0).stop 2023-06-23T18:22:05.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 shutdown_connections 2023-06-23T18:22:05.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb40c10c4f0 0x7fb40c19c8a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:05.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb40c0fe940 0x7fb40c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:05.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3f80a1260 0x7fb3f80a3710 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:05.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 --2- 172.21.15.2:0/2051346617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb40c0ff250 0x7fb40c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:05.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.180+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 >> 172.21.15.2:0/2051346617 conn(0x7fb40c075b30 msgr2=0x7fb40c0fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:05.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.181+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 shutdown_connections 2023-06-23T18:22:05.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:05.181+0000 7fb4134d3700 1 -- 172.21.15.2:0/2051346617 wait complete. 2023-06-23T18:22:05.322 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:05 smithi002 ceph-mon[95947]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:22:05.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:05 smithi082 ceph-mon[97889]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:22:05.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:05 smithi017 ceph-mon[97472]: pgmap v298: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:22:05.567 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-06-23T18:22:05.567 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-06-23T18:22:05.568 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2023-06-23T18:22:05.568 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-06-23T18:22:05.568 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-06-23T18:22:05.568 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-06-23T18:22:05.568 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-06-23T18:22:05.569 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-06-23T18:22:05.569 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-06-23T18:22:05.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:05.570 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2023-06-23T18:22:05.589 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-06-23T18:22:05.589 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-06-23T18:22:05.589 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2023-06-23T18:22:05.589 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:22:05.589 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2023-06-23T18:22:05.590 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:22:05.590 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:22:05.590 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:22:05.590 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:22:05.590 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:22:05.590 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-06-23T18:22:05.590 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:22:05.591 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-06-23T18:22:05.591 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 41882sec preferred_lft 41882sec 2023-06-23T18:22:05.591 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2023-06-23T18:22:05.591 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-06-23T18:22:05.591 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-06-23T18:22:05.591 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2023-06-23T18:22:05.592 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-06-23T18:22:05.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:05.592 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-06-23T18:22:05.593 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-06-23T18:22:05.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2023-06-23T18:22:05.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2023-06-23T18:22:05.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2023-06-23T18:22:05.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-06-23T18:22:05.594 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-06-23T18:22:05.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:05.766 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-06-23T18:22:05.794 INFO:teuthology.orchestra.run.smithi002.stdout:1 2023-06-23T18:22:05.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:05.796 DEBUG:teuthology.orchestra.run.smithi002:> route 2023-06-23T18:22:05.880 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2023-06-23T18:22:05.881 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-06-23T18:22:05.881 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-06-23T18:22:05.881 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-06-23T18:22:05.881 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-06-23T18:22:05.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:05.882 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-06-23T18:22:05.882 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-06-23T18:22:05.882 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-06-23T18:22:05.883 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-06-23T18:22:05.883 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-06-23T18:22:05.883 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-06-23T18:22:06.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:06.121 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-06-23T18:22:06.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:06.144 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2023-06-23T18:22:06.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:06.205 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-06-23T18:22:06.206 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-06-23T18:22:06.206 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--tmp-cephfs 2023-06-23T18:22:06.206 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-06-23T18:22:06.207 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-06-23T18:22:06.287 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:06 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2051346617' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-06-23T18:22:06.303 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-06-23T18:22:06.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:06.305 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-06-23T18:22:06.305 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-06-23T18:22:06.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-06-23T18:22:06.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-06-23T18:22:06.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-06-23T18:22:06.308 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-06-23T18:22:06.308 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-06-23T18:22:06.309 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-06-23T18:22:06.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:06 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2051346617' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-06-23T18:22:06.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:06 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2051346617' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-06-23T18:22:06.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:06.547 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-06-23T18:22:06.548 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-06-23T18:22:06.548 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2023-06-23T18:22:06.548 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2023-06-23T18:22:06.548 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-06-23T18:22:06.649 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2023-06-23T18:22:06.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:22:06.649 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /tmp/cephfs 2023-06-23T18:22:06.667 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/tmp/cephfs' 2023-06-23T18:22:06.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-06-23T18:22:06.668 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /tmp/cephfs 2023-06-23T18:22:06.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2023-06-23T18:22:06.808 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:22:06.808 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:22:06.877 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:22:06.878 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:22:06.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:06.879 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2023-06-23T18:22:06.936 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-06-23T18:22:06.936 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/_nogroup/sv_0/de777b9e-6e5e-433e-950a-99a8a51ea828 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2023-06-23T18:22:06.979 DEBUG:teuthology.orchestra.run.smithi002:> sudo modprobe fuse 2023-06-23T18:22:07.007 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-06-23T18:22:07.007 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-06-23T18:22:07.075 INFO:teuthology.orchestra.run.smithi002.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-06-23T18:22:07.076 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-06-23T18:22:07.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:07.077 DEBUG:teuthology.orchestra.run.smithi002:> ls /sys/fs/fuse/connections 2023-06-23T18:22:07.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:2023-06-23T18:22:07.094+0000 7f93de1045c0 -1 init, newargv = 0x55c1e3d61320 newargc=17 2023-06-23T18:22:07.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[143755]: starting ceph client 2023-06-23T18:22:07.115 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[143755]: starting fuse 2023-06-23T18:22:07.118 INFO:teuthology.orchestra.run.smithi002.stdout:124 2023-06-23T18:22:07.118 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-06-23T18:22:07.119 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-06-23T18:22:07.119 DEBUG:teuthology.orchestra.run.smithi002:> import glob 2023-06-23T18:22:07.119 DEBUG:teuthology.orchestra.run.smithi002:> import re 2023-06-23T18:22:07.120 DEBUG:teuthology.orchestra.run.smithi002:> import os 2023-06-23T18:22:07.120 DEBUG:teuthology.orchestra.run.smithi002:> import subprocess 2023-06-23T18:22:07.120 DEBUG:teuthology.orchestra.run.smithi002:> 2023-06-23T18:22:07.120 DEBUG:teuthology.orchestra.run.smithi002:> def _find_admin_socket(client_name): 2023-06-23T18:22:07.121 DEBUG:teuthology.orchestra.run.smithi002:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-06-23T18:22:07.121 DEBUG:teuthology.orchestra.run.smithi002:> files = glob.glob(asok_path) 2023-06-23T18:22:07.121 DEBUG:teuthology.orchestra.run.smithi002:> mountpoint = "/tmp/cephfs" 2023-06-23T18:22:07.121 DEBUG:teuthology.orchestra.run.smithi002:> 2023-06-23T18:22:07.121 DEBUG:teuthology.orchestra.run.smithi002:> # Given a non-glob path, it better be there 2023-06-23T18:22:07.122 DEBUG:teuthology.orchestra.run.smithi002:> if "*" not in asok_path: 2023-06-23T18:22:07.122 DEBUG:teuthology.orchestra.run.smithi002:> assert(len(files) == 1) 2023-06-23T18:22:07.122 DEBUG:teuthology.orchestra.run.smithi002:> return files[0] 2023-06-23T18:22:07.122 DEBUG:teuthology.orchestra.run.smithi002:> 2023-06-23T18:22:07.123 DEBUG:teuthology.orchestra.run.smithi002:> for f in files: 2023-06-23T18:22:07.123 DEBUG:teuthology.orchestra.run.smithi002:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-06-23T18:22:07.123 DEBUG:teuthology.orchestra.run.smithi002:> if os.path.exists("/proc/{0}".format(pid)): 2023-06-23T18:22:07.123 DEBUG:teuthology.orchestra.run.smithi002:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-06-23T18:22:07.124 DEBUG:teuthology.orchestra.run.smithi002:> contents = proc_f.read() 2023-06-23T18:22:07.124 DEBUG:teuthology.orchestra.run.smithi002:> if mountpoint in contents: 2023-06-23T18:22:07.124 DEBUG:teuthology.orchestra.run.smithi002:> return f 2023-06-23T18:22:07.124 DEBUG:teuthology.orchestra.run.smithi002:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-06-23T18:22:07.125 DEBUG:teuthology.orchestra.run.smithi002:> 2023-06-23T18:22:07.125 DEBUG:teuthology.orchestra.run.smithi002:> print(_find_admin_socket("client.0")) 2023-06-23T18:22:07.125 DEBUG:teuthology.orchestra.run.smithi002:> ' 2023-06-23T18:22:07.244 INFO:teuthology.orchestra.run.smithi002.stdout:/var/run/ceph/ceph-client.0.143755.asok 2023-06-23T18:22:07.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:07 smithi082 ceph-mon[97889]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:22:07.425 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:07 smithi002 ceph-mon[95947]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:22:07.427 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.143755.asok 2023-06-23T18:22:07.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:07.429 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.143755.asok status 2023-06-23T18:22:07.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:07 smithi017 ceph-mon[97472]: pgmap v299: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:22:07.590 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "metadata": { 2023-06-23T18:22:07.590 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_sha1": "8627af3e0adcb765a3c249fcc209cba9f4873e1b", 2023-06-23T18:22:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_version": "ceph version 18.0.0-4479-g8627af3e (8627af3e0adcb765a3c249fcc209cba9f4873e1b) reef (dev)", 2023-06-23T18:22:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "entity_id": "0", 2023-06-23T18:22:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": "smithi002", 2023-06-23T18:22:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "mount_point": "/tmp/cephfs", 2023-06-23T18:22:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "pid": "143755", 2023-06-23T18:22:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: "root": "/volumes/_nogroup/sv_0/de777b9e-6e5e-433e-950a-99a8a51ea828" 2023-06-23T18:22:07.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:22:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_count": 0, 2023-06-23T18:22:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "dentry_pinned_count": 0, 2023-06-23T18:22:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 15258, 2023-06-23T18:22:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "inst": { 2023-06-23T18:22:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": { 2023-06-23T18:22:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "client", 2023-06-23T18:22:07.592 INFO:teuthology.orchestra.run.smithi002.stdout: "num": 15258 2023-06-23T18:22:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:22:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2023-06-23T18:22:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2023-06-23T18:22:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:22:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3807440325 2023-06-23T18:22:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:22:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:22:07.593 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": { 2023-06-23T18:22:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2023-06-23T18:22:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "192.168.0.1:0", 2023-06-23T18:22:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3807440325 2023-06-23T18:22:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:22:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "inst_str": "client.15258 192.168.0.1:0/3807440325", 2023-06-23T18:22:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "addr_str": "192.168.0.1:0/3807440325", 2023-06-23T18:22:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "inode_count": 5, 2023-06-23T18:22:07.594 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_epoch": 22, 2023-06-23T18:22:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch": 84, 2023-06-23T18:22:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_epoch_barrier": 0, 2023-06-23T18:22:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklisted": false, 2023-06-23T18:22:07.595 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs" 2023-06-23T18:22:07.595 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:07.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:07.600 DEBUG:teuthology.orchestra.run.smithi002:> stat --file-system '--printf=%T 2023-06-23T18:22:07.600 DEBUG:teuthology.orchestra.run.smithi002:> ' -- /tmp/cephfs 2023-06-23T18:22:07.619 INFO:teuthology.orchestra.run.smithi002.stdout:fuseblk 2023-06-23T18:22:07.619 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2023-06-23T18:22:07.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:22:07.620 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /tmp/cephfs 2023-06-23T18:22:07.689 INFO:teuthology.run_tasks:Running task check-counter... 2023-06-23T18:22:07.703 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-06-23T18:22:07.742 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:22:07.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.863+0000 7f7c139de700 1 -- 172.21.15.2:0/3983849195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 msgr2=0x7f7c0c106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:07.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.863+0000 7f7c139de700 1 --2- 172.21.15.2:0/3983849195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 0x7f7c0c106ea0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b0a0 tx=0x7f7c0400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:07.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.863+0000 7f7c139de700 1 -- 172.21.15.2:0/3983849195 shutdown_connections 2023-06-23T18:22:07.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.863+0000 7f7c139de700 1 --2- 172.21.15.2:0/3983849195 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 0x7f7c0c106ea0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:07.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.863+0000 7f7c139de700 1 --2- 172.21.15.2:0/3983849195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c0c0fea60 0x7f7c0c0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:07.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 --2- 172.21.15.2:0/3983849195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c0ffc50 0x7f7c0c1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:07.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 -- 172.21.15.2:0/3983849195 >> 172.21.15.2:0/3983849195 conn(0x7f7c0c0fa0e0 msgr2=0x7f7c0c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:07.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 -- 172.21.15.2:0/3983849195 shutdown_connections 2023-06-23T18:22:07.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 -- 172.21.15.2:0/3983849195 wait complete. 2023-06-23T18:22:07.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 Processor -- start 2023-06-23T18:22:07.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 -- start start 2023-06-23T18:22:07.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c0fea60 0x7f7c0c068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:07.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c0c0ffc50 0x7f7c0c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:07.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 0x7f7c0c19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:07.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c19d440 con 0x7f7c0c1005e0 2023-06-23T18:22:07.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c19d5b0 con 0x7f7c0c0ffc50 2023-06-23T18:22:07.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.864+0000 7f7c139de700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c19d770 con 0x7f7c0c0fea60 2023-06-23T18:22:07.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 0x7f7c0c19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:07.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 0x7f7c0c19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53762/0 (socket says 172.21.15.2:53762) 2023-06-23T18:22:07.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 -- 172.21.15.2:0/3263222045 learned_addr learned my addr 172.21.15.2:0/3263222045 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:07.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c10f79700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c0c0ffc50 0x7f7c0c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:07.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c1177a700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c0fea60 0x7f7c0c068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:07.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 -- 172.21.15.2:0/3263222045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c0fea60 msgr2=0x7f7c0c068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:07.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c0fea60 0x7f7c0c068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:07.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 -- 172.21.15.2:0/3263222045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c0c0ffc50 msgr2=0x7f7c0c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:07.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c0c0ffc50 0x7f7c0c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:07.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 -- 172.21.15.2:0/3263222045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c04009d00 con 0x7f7c0c1005e0 2023-06-23T18:22:07.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c1177a700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c0fea60 0x7f7c0c068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:07.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c11f7b700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 0x7f7c0c19cc70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b070 tx=0x7f7c04004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:07.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c027fc700 1 -- 172.21.15.2:0/3263222045 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400e040 con 0x7f7c0c1005e0 2023-06-23T18:22:07.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c027fc700 1 -- 172.21.15.2:0/3263222045 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7c04003e90 con 0x7f7c0c1005e0 2023-06-23T18:22:07.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0c19d9f0 con 0x7f7c0c1005e0 2023-06-23T18:22:07.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c027fc700 1 -- 172.21.15.2:0/3263222045 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c04004020 con 0x7f7c0c1005e0 2023-06-23T18:22:07.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.865+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c19dec0 con 0x7f7c0c1005e0 2023-06-23T18:22:07.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.866+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c106050 con 0x7f7c0c1005e0 2023-06-23T18:22:07.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.867+0000 7f7c027fc700 1 -- 172.21.15.2:0/3263222045 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7c04003a00 con 0x7f7c0c1005e0 2023-06-23T18:22:07.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.868+0000 7f7c027fc700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7bf80a11d0 0x7f7bf80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:07.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.868+0000 7f7c1177a700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7bf80a11d0 0x7f7bf80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:07.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.868+0000 7f7c027fc700 1 -- 172.21.15.2:0/3263222045 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7c040ce5c0 con 0x7f7c0c1005e0 2023-06-23T18:22:07.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.869+0000 7f7c1177a700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7bf80a11d0 0x7f7bf80a3680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005fd0 tx=0x7f7c08005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:07.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:07.872+0000 7f7c027fc700 1 -- 172.21.15.2:0/3263222045 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7c04097da0 con 0x7f7c0c1005e0 2023-06-23T18:22:08.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.073+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c0c02c9e0 con 0x7f7c0c1005e0 2023-06-23T18:22:08.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.074+0000 7f7c027fc700 1 -- 172.21.15.2:0/3263222045 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f7c04011390 con 0x7f7c0c1005e0 2023-06-23T18:22:08.081 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:08.081 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:22:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7bf80a11d0 msgr2=0x7f7bf80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7bf80a11d0 0x7f7bf80a3680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005fd0 tx=0x7f7c08005f00 comp rx=0 tx=0).stop 2023-06-23T18:22:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 msgr2=0x7f7c0c19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 0x7f7c0c19cc70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400b070 tx=0x7f7c04004c80 comp rx=0 tx=0).stop 2023-06-23T18:22:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 shutdown_connections 2023-06-23T18:22:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7c0c1005e0 0x7f7c0c19cc70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c0fea60 0x7f7c0c068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7bf80a11d0 0x7f7bf80a3680 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.076+0000 7f7c139de700 1 --2- 172.21.15.2:0/3263222045 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c0c0ffc50 0x7f7c0c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.077+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 >> 172.21.15.2:0/3263222045 conn(0x7f7c0c0fa0e0 msgr2=0x7f7c0c103190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.077+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 shutdown_connections 2023-06-23T18:22:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.077+0000 7f7c139de700 1 -- 172.21.15.2:0/3263222045 wait complete. 2023-06-23T18:22:08.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:22:08.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:22:08.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.219+0000 7f05b1e66700 1 -- 172.21.15.2:0/3979547710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac069730 msgr2=0x7f05ac1026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.219+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3979547710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac069730 0x7f05ac1026c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009990 tx=0x7f05a0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 -- 172.21.15.2:0/3979547710 shutdown_connections 2023-06-23T18:22:08.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3979547710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05ac105710 0x7f05ac107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3979547710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac069730 0x7f05ac1026c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3979547710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05ac102c00 0x7f05ac105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 -- 172.21.15.2:0/3979547710 >> 172.21.15.2:0/3979547710 conn(0x7f05ac0fa030 msgr2=0x7f05ac0fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 -- 172.21.15.2:0/3979547710 shutdown_connections 2023-06-23T18:22:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 -- 172.21.15.2:0/3979547710 wait complete. 2023-06-23T18:22:08.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 Processor -- start 2023-06-23T18:22:08.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.220+0000 7f05b1e66700 1 -- start start 2023-06-23T18:22:08.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05b1e66700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05ac069730 0x7f05ac100020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05b1e66700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05ac102c00 0x7f05ac0fe660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05b1e66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac105710 0x7f05ac0feba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05b1e66700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0ff340 con 0x7f05ac105710 2023-06-23T18:22:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05b1e66700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0ff4b0 con 0x7f05ac102c00 2023-06-23T18:22:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05b1e66700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac19f7e0 con 0x7f05ac069730 2023-06-23T18:22:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac105710 0x7f05ac0feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac105710 0x7f05ac0feba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53776/0 (socket says 172.21.15.2:53776) 2023-06-23T18:22:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 -- 172.21.15.2:0/3399581049 learned_addr learned my addr 172.21.15.2:0/3399581049 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05aaffd700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05ac102c00 0x7f05ac0fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05ab7fe700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05ac069730 0x7f05ac100020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 -- 172.21.15.2:0/3399581049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05ac069730 msgr2=0x7f05ac100020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05ac069730 0x7f05ac100020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 -- 172.21.15.2:0/3399581049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05ac102c00 msgr2=0x7f05ac0fe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05ac102c00 0x7f05ac0fe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 -- 172.21.15.2:0/3399581049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a00095f0 con 0x7f05ac105710 2023-06-23T18:22:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05abfff700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac105710 0x7f05ac0feba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f05a4007d70 tx=0x7f05a400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05a8ff9700 1 -- 172.21.15.2:0/3399581049 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400ad40 con 0x7f05ac105710 2023-06-23T18:22:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05a8ff9700 1 -- 172.21.15.2:0/3399581049 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05a40128b0 con 0x7f05ac105710 2023-06-23T18:22:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac19fac0 con 0x7f05ac105710 2023-06-23T18:22:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.221+0000 7f05a8ff9700 1 -- 172.21.15.2:0/3399581049 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a40115a0 con 0x7f05ac105710 2023-06-23T18:22:08.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.222+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac19ff80 con 0x7f05ac105710 2023-06-23T18:22:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.222+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac0fbba0 con 0x7f05ac105710 2023-06-23T18:22:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.223+0000 7f05a8ff9700 1 -- 172.21.15.2:0/3399581049 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f05a4012a20 con 0x7f05ac105710 2023-06-23T18:22:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.224+0000 7f05a8ff9700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05940a1150 0x7f05940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.224+0000 7f05a8ff9700 1 -- 172.21.15.2:0/3399581049 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f05a40ccbc0 con 0x7f05ac105710 2023-06-23T18:22:08.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.224+0000 7f05ab7fe700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05940a1150 0x7f05940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.225+0000 7f05ab7fe700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05940a1150 0x7f05940a3600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f05a000b5c0 tx=0x7f05a0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:08.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.228+0000 7f05a8ff9700 1 -- 172.21.15.2:0/3399581049 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f05a4096310 con 0x7f05ac105710 2023-06-23T18:22:08.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:08 smithi082 ceph-mon[97889]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-06-23T18:22:08.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:08 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3263222045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:08.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.427+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05ac0610e0 con 0x7f05ac105710 2023-06-23T18:22:08.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.428+0000 7f05a8ff9700 1 -- 172.21.15.2:0/3399581049 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05a4099930 con 0x7f05ac105710 2023-06-23T18:22:08.434 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:22:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05940a1150 msgr2=0x7f05940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05940a1150 0x7f05940a3600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f05a000b5c0 tx=0x7f05a0005fb0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac105710 msgr2=0x7f05ac0feba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac105710 0x7f05ac0feba0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f05a4007d70 tx=0x7f05a400a340 comp rx=0 tx=0).stop 2023-06-23T18:22:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 shutdown_connections 2023-06-23T18:22:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f05ac105710 0x7f05ac0feba0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05ac069730 0x7f05ac100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05940a1150 0x7f05940a3600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 --2- 172.21.15.2:0/3399581049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05ac102c00 0x7f05ac0fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 >> 172.21.15.2:0/3399581049 conn(0x7f05ac0fa030 msgr2=0x7f05ac100f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.430+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 shutdown_connections 2023-06-23T18:22:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.431+0000 7f05b1e66700 1 -- 172.21.15.2:0/3399581049 wait complete. 2023-06-23T18:22:08.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:08 smithi017 ceph-mon[97472]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-06-23T18:22:08.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:08 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3263222045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:08.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:08 smithi002 ceph-mon[95947]: mgrmap e18: x(active, since 7m), standbys: z, y 2023-06-23T18:22:08.447 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:08 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3263222045' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:08.448 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:22:08.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 -- 172.21.15.2:0/1112813990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1300ffc70 msgr2=0x7ff1301000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 --2- 172.21.15.2:0/1112813990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1300ffc70 0x7ff1301000c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff124009990 tx=0x7ff124009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 -- 172.21.15.2:0/1112813990 shutdown_connections 2023-06-23T18:22:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 --2- 172.21.15.2:0/1112813990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff130100600 0x7ff130106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 --2- 172.21.15.2:0/1112813990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1300fea80 0x7ff1300fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 --2- 172.21.15.2:0/1112813990 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff1300ffc70 0x7ff1301000c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 -- 172.21.15.2:0/1112813990 >> 172.21.15.2:0/1112813990 conn(0x7ff1300fa0e0 msgr2=0x7ff1300fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:08.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 -- 172.21.15.2:0/1112813990 shutdown_connections 2023-06-23T18:22:08.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.571+0000 7ff138007700 1 -- 172.21.15.2:0/1112813990 wait complete. 2023-06-23T18:22:08.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff138007700 1 Processor -- start 2023-06-23T18:22:08.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff138007700 1 -- start start 2023-06-23T18:22:08.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff138007700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1300fea80 0x7ff130102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff138007700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1300ffc70 0x7ff130101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff138007700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff130100600 0x7ff130101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff138007700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130101d30 con 0x7ff130100600 2023-06-23T18:22:08.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff138007700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130101ea0 con 0x7ff1300ffc70 2023-06-23T18:22:08.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff138007700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13019f800 con 0x7ff1300fea80 2023-06-23T18:22:08.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff1365a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff130100600 0x7ff130101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff1365a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff130100600 0x7ff130101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53790/0 (socket says 172.21.15.2:53790) 2023-06-23T18:22:08.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff135da3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1300fea80 0x7ff130102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff1355a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1300ffc70 0x7ff130101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.572+0000 7ff1365a4700 1 -- 172.21.15.2:0/2448944904 learned_addr learned my addr 172.21.15.2:0/2448944904 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:08.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff1365a4700 1 -- 172.21.15.2:0/2448944904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1300fea80 msgr2=0x7ff130102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff1365a4700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1300fea80 0x7ff130102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff1365a4700 1 -- 172.21.15.2:0/2448944904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1300ffc70 msgr2=0x7ff130101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff1365a4700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1300ffc70 0x7ff130101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff1365a4700 1 -- 172.21.15.2:0/2448944904 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1240095f0 con 0x7ff130100600 2023-06-23T18:22:08.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff1365a4700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff130100600 0x7ff130101590 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff128007ee0 tx=0x7ff12800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:08.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff122ffd700 1 -- 172.21.15.2:0/2448944904 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128017070 con 0x7ff130100600 2023-06-23T18:22:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13019fae0 con 0x7ff130100600 2023-06-23T18:22:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff122ffd700 1 -- 172.21.15.2:0/2448944904 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff12800f040 con 0x7ff130100600 2023-06-23T18:22:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff122ffd700 1 -- 172.21.15.2:0/2448944904 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128005570 con 0x7ff130100600 2023-06-23T18:22:08.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.573+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13019fff0 con 0x7ff130100600 2023-06-23T18:22:08.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.574+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13010a100 con 0x7ff130100600 2023-06-23T18:22:08.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.575+0000 7ff122ffd700 1 -- 172.21.15.2:0/2448944904 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff12800db20 con 0x7ff130100600 2023-06-23T18:22:08.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.576+0000 7ff122ffd700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff11c0a1150 0x7ff11c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.576+0000 7ff122ffd700 1 -- 172.21.15.2:0/2448944904 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff1280cc710 con 0x7ff130100600 2023-06-23T18:22:08.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.576+0000 7ff135da3700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff11c0a1150 0x7ff11c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.576+0000 7ff135da3700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff11c0a1150 0x7ff11c0a3600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff1300ffad0 tx=0x7ff12c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:08.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.580+0000 7ff122ffd700 1 -- 172.21.15.2:0/2448944904 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff128095e20 con 0x7ff130100600 2023-06-23T18:22:08.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.776+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1301a0600 con 0x7ff130100600 2023-06-23T18:22:08.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.777+0000 7ff122ffd700 1 -- 172.21.15.2:0/2448944904 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff128099440 con 0x7ff130100600 2023-06-23T18:22:08.783 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:22:08.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff11c0a1150 msgr2=0x7ff11c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff11c0a1150 0x7ff11c0a3600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff1300ffad0 tx=0x7ff12c009260 comp rx=0 tx=0).stop 2023-06-23T18:22:08.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff130100600 msgr2=0x7ff130101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff130100600 0x7ff130101590 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff128007ee0 tx=0x7ff12800d3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 shutdown_connections 2023-06-23T18:22:08.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff130100600 0x7ff130101590 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff1300fea80 0x7ff130102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff11c0a1150 0x7ff11c0a3600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 --2- 172.21.15.2:0/2448944904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1300ffc70 0x7ff130101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 >> 172.21.15.2:0/2448944904 conn(0x7ff1300fa0e0 msgr2=0x7ff130067800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:08.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 shutdown_connections 2023-06-23T18:22:08.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.779+0000 7ff138007700 1 -- 172.21.15.2:0/2448944904 wait complete. 2023-06-23T18:22:08.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:22:08.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 -- 172.21.15.2:0/749764575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780fea80 msgr2=0x7fe4780fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/749764575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780fea80 0x7fe4780fee90 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe468009990 tx=0x7fe468009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 -- 172.21.15.2:0/749764575 shutdown_connections 2023-06-23T18:22:08.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/749764575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe478100600 0x7fe478106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/749764575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780fea80 0x7fe4780fee90 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/749764575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4780ffc70 0x7fe4781000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 -- 172.21.15.2:0/749764575 >> 172.21.15.2:0/749764575 conn(0x7fe4780fa0e0 msgr2=0x7fe4780fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:08.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 -- 172.21.15.2:0/749764575 shutdown_connections 2023-06-23T18:22:08.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.921+0000 7fe47dc1e700 1 -- 172.21.15.2:0/749764575 wait complete. 2023-06-23T18:22:08.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe47dc1e700 1 Processor -- start 2023-06-23T18:22:08.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe47dc1e700 1 -- start start 2023-06-23T18:22:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe47dc1e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780fea80 0x7fe478068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe47dc1e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780ffc70 0x7fe47819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe47dc1e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe478100600 0x7fe47819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe47dc1e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47819d2e0 con 0x7fe4780ffc70 2023-06-23T18:22:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe47dc1e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47819d450 con 0x7fe4780fea80 2023-06-23T18:22:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe47dc1e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47819d610 con 0x7fe478100600 2023-06-23T18:22:08.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe476ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780ffc70 0x7fe47819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe476ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780ffc70 0x7fe47819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53802/0 (socket says 172.21.15.2:53802) 2023-06-23T18:22:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe476ffd700 1 -- 172.21.15.2:0/291318466 learned_addr learned my addr 172.21.15.2:0/291318466 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe4777fe700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780fea80 0x7fe478068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe477fff700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe478100600 0x7fe47819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe476ffd700 1 -- 172.21.15.2:0/291318466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe478100600 msgr2=0x7fe47819cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe476ffd700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe478100600 0x7fe47819cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe476ffd700 1 -- 172.21.15.2:0/291318466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780fea80 msgr2=0x7fe478068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:08.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe476ffd700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780fea80 0x7fe478068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:08.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.922+0000 7fe476ffd700 1 -- 172.21.15.2:0/291318466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4680095f0 con 0x7fe4780ffc70 2023-06-23T18:22:08.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.923+0000 7fe476ffd700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780ffc70 0x7fe47819a4b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe46000d950 tx=0x7fe46000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:08.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.923+0000 7fe474ff9700 1 -- 172.21.15.2:0/291318466 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46000f890 con 0x7fe4780ffc70 2023-06-23T18:22:08.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.923+0000 7fe474ff9700 1 -- 172.21.15.2:0/291318466 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe46000fe80 con 0x7fe4780ffc70 2023-06-23T18:22:08.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.923+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47819d8f0 con 0x7fe4780ffc70 2023-06-23T18:22:08.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.923+0000 7fe474ff9700 1 -- 172.21.15.2:0/291318466 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46000e3b0 con 0x7fe4780ffc70 2023-06-23T18:22:08.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.923+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47819de00 con 0x7fe4780ffc70 2023-06-23T18:22:08.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.924+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe478105a30 con 0x7fe4780ffc70 2023-06-23T18:22:08.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.925+0000 7fe474ff9700 1 -- 172.21.15.2:0/291318466 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe460010460 con 0x7fe4780ffc70 2023-06-23T18:22:08.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.926+0000 7fe474ff9700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe4640a1150 0x7fe4640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:08.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.926+0000 7fe4777fe700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe4640a1150 0x7fe4640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:08.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.926+0000 7fe474ff9700 1 -- 172.21.15.2:0/291318466 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe4600cc6b0 con 0x7fe4780ffc70 2023-06-23T18:22:08.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.926+0000 7fe4777fe700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe4640a1150 0x7fe4640a3600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe468006010 tx=0x7fe46800b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:08.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:08.931+0000 7fe474ff9700 1 -- 172.21.15.2:0/291318466 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe460095e00 con 0x7fe4780ffc70 2023-06-23T18:22:09.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.132+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe47802cca0 con 0x7fe4780ffc70 2023-06-23T18:22:09.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.133+0000 7fe474ff9700 1 -- 172.21.15.2:0/291318466 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe460099420 con 0x7fe4780ffc70 2023-06-23T18:22:09.138 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:22:09.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe4640a1150 msgr2=0x7fe4640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe4640a1150 0x7fe4640a3600 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe468006010 tx=0x7fe46800b2a0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780ffc70 msgr2=0x7fe47819a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780ffc70 0x7fe47819a4b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe46000d950 tx=0x7fe46000dc60 comp rx=0 tx=0).stop 2023-06-23T18:22:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 shutdown_connections 2023-06-23T18:22:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe478100600 0x7fe47819cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe4640a1150 0x7fe4640a3600 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4780fea80 0x7fe478068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 --2- 172.21.15.2:0/291318466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4780ffc70 0x7fe47819a4b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 >> 172.21.15.2:0/291318466 conn(0x7fe4780fa0e0 msgr2=0x7fe4780fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 shutdown_connections 2023-06-23T18:22:09.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.135+0000 7fe47dc1e700 1 -- 172.21.15.2:0/291318466 wait complete. 2023-06-23T18:22:09.153 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:22:09.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 -- 172.21.15.2:0/3855665176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e740fea60 msgr2=0x7f3e740fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 --2- 172.21.15.2:0/3855665176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e740fea60 0x7f3e740fee70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e70009990 tx=0x7f3e70009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 -- 172.21.15.2:0/3855665176 shutdown_connections 2023-06-23T18:22:09.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 --2- 172.21.15.2:0/3855665176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e741005e0 0x7f3e74106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 --2- 172.21.15.2:0/3855665176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e740fea60 0x7f3e740fee70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 --2- 172.21.15.2:0/3855665176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e740ffc50 0x7f3e741000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 -- 172.21.15.2:0/3855665176 >> 172.21.15.2:0/3855665176 conn(0x7f3e740fa0e0 msgr2=0x7f3e740fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:09.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 -- 172.21.15.2:0/3855665176 shutdown_connections 2023-06-23T18:22:09.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.275+0000 7f3e7c306700 1 -- 172.21.15.2:0/3855665176 wait complete. 2023-06-23T18:22:09.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7c306700 1 Processor -- start 2023-06-23T18:22:09.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7c306700 1 -- start start 2023-06-23T18:22:09.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7c306700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e740fea60 0x7f3e741946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7c306700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e740ffc50 0x7f3e74194c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7c306700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e741005e0 0x7f3e7419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7c306700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7419c260 con 0x7f3e741005e0 2023-06-23T18:22:09.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7c306700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7419c3a0 con 0x7f3e740ffc50 2023-06-23T18:22:09.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7c306700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7419c530 con 0x7f3e740fea60 2023-06-23T18:22:09.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7a8a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e741005e0 0x7f3e7419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7a8a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e741005e0 0x7f3e7419bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53816/0 (socket says 172.21.15.2:53816) 2023-06-23T18:22:09.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7a8a3700 1 -- 172.21.15.2:0/2311634841 learned_addr learned my addr 172.21.15.2:0/2311634841 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:09.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e798a1700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e740ffc50 0x7f3e74194c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.276+0000 7f3e7a0a2700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e740fea60 0x7f3e741946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e7a8a3700 1 -- 172.21.15.2:0/2311634841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e740fea60 msgr2=0x7f3e741946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e7a8a3700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e740fea60 0x7f3e741946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e7a8a3700 1 -- 172.21.15.2:0/2311634841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e740ffc50 msgr2=0x7f3e74194c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e7a8a3700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e740ffc50 0x7f3e74194c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e7a8a3700 1 -- 172.21.15.2:0/2311634841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700095f0 con 0x7f3e741005e0 2023-06-23T18:22:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e7a8a3700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e741005e0 0x7f3e7419bc90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c007f10 tx=0x7f3e6c00f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:09.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e6b7fe700 1 -- 172.21.15.2:0/2311634841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c003a00 con 0x7f3e741005e0 2023-06-23T18:22:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e6b7fe700 1 -- 172.21.15.2:0/2311634841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e6c004020 con 0x7f3e741005e0 2023-06-23T18:22:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e74067800 con 0x7f3e741005e0 2023-06-23T18:22:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e6b7fe700 1 -- 172.21.15.2:0/2311634841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0126a0 con 0x7f3e741005e0 2023-06-23T18:22:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.277+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e74067d10 con 0x7f3e741005e0 2023-06-23T18:22:09.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.278+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7410a0e0 con 0x7f3e741005e0 2023-06-23T18:22:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.279+0000 7f3e6b7fe700 1 -- 172.21.15.2:0/2311634841 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3e6c004190 con 0x7f3e741005e0 2023-06-23T18:22:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.280+0000 7f3e6b7fe700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3e600a1150 0x7f3e600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.280+0000 7f3e6b7fe700 1 -- 172.21.15.2:0/2311634841 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3e6c0ced80 con 0x7f3e741005e0 2023-06-23T18:22:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.280+0000 7f3e7a0a2700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3e600a1150 0x7f3e600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.280+0000 7f3e7a0a2700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3e600a1150 0x7f3e600a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e70006010 tx=0x7f3e7000b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:09.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.284+0000 7f3e6b7fe700 1 -- 172.21.15.2:0/2311634841 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3e6c0984d0 con 0x7f3e741005e0 2023-06-23T18:22:09.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:09 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3399581049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:09.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:09 smithi082 ceph-mon[97889]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 596 B/s wr, 7 op/s 2023-06-23T18:22:09.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:09 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2448944904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:09.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:09 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3399581049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:09.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:09 smithi017 ceph-mon[97472]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 596 B/s wr, 7 op/s 2023-06-23T18:22:09.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:09 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2448944904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:09.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.484+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e7410a2d0 con 0x7f3e741005e0 2023-06-23T18:22:09.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.485+0000 7f3e6b7fe700 1 -- 172.21.15.2:0/2311634841 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f3e6c021090 con 0x7f3e741005e0 2023-06-23T18:22:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:09.490 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:22:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3e600a1150 msgr2=0x7f3e600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3e600a1150 0x7f3e600a3600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e70006010 tx=0x7f3e7000b2a0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e741005e0 msgr2=0x7f3e7419bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e741005e0 0x7f3e7419bc90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c007f10 tx=0x7f3e6c00f5c0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 shutdown_connections 2023-06-23T18:22:09.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e741005e0 0x7f3e7419bc90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e740fea60 0x7f3e741946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3e600a1150 0x7f3e600a3600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 --2- 172.21.15.2:0/2311634841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e740ffc50 0x7f3e74194c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 >> 172.21.15.2:0/2311634841 conn(0x7f3e740fa0e0 msgr2=0x7f3e740fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.487+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 shutdown_connections 2023-06-23T18:22:09.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.488+0000 7f3e7c306700 1 -- 172.21.15.2:0/2311634841 wait complete. 2023-06-23T18:22:09.503 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:09 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3399581049' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:09.503 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:09 smithi002 ceph-mon[95947]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 596 B/s wr, 7 op/s 2023-06-23T18:22:09.503 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:09 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2448944904' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:09.504 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:22:09.505 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:22:09.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 -- 172.21.15.2:0/908055446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0ffc70 msgr2=0x7f053c1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 --2- 172.21.15.2:0/908055446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0ffc70 0x7f053c1000c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0538009990 tx=0x7f0538009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 -- 172.21.15.2:0/908055446 shutdown_connections 2023-06-23T18:22:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 --2- 172.21.15.2:0/908055446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f053c100600 0x7f053c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 --2- 172.21.15.2:0/908055446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f053c0fea80 0x7f053c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 --2- 172.21.15.2:0/908055446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0ffc70 0x7f053c1000c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 -- 172.21.15.2:0/908055446 >> 172.21.15.2:0/908055446 conn(0x7f053c0fa0e0 msgr2=0x7f053c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:09.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 -- 172.21.15.2:0/908055446 shutdown_connections 2023-06-23T18:22:09.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.629+0000 7f0544345700 1 -- 172.21.15.2:0/908055446 wait complete. 2023-06-23T18:22:09.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f0544345700 1 Processor -- start 2023-06-23T18:22:09.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f0544345700 1 -- start start 2023-06-23T18:22:09.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f0544345700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0fea80 0x7f053c068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f0544345700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f053c0ffc70 0x7f053c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f0544345700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f053c100600 0x7f053c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f0544345700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c19f510 con 0x7f053c0fea80 2023-06-23T18:22:09.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f0544345700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c19f680 con 0x7f053c100600 2023-06-23T18:22:09.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f0544345700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c19f840 con 0x7f053c0ffc70 2023-06-23T18:22:09.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05420e1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0fea80 0x7f053c068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05420e1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0fea80 0x7f053c068310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53824/0 (socket says 172.21.15.2:53824) 2023-06-23T18:22:09.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05420e1700 1 -- 172.21.15.2:0/2817826401 learned_addr learned my addr 172.21.15.2:0/2817826401 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:09.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05418e0700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f053c0ffc70 0x7f053c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05428e2700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f053c100600 0x7f053c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05420e1700 1 -- 172.21.15.2:0/2817826401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f053c0ffc70 msgr2=0x7f053c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05420e1700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f053c0ffc70 0x7f053c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05420e1700 1 -- 172.21.15.2:0/2817826401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f053c100600 msgr2=0x7f053c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05420e1700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f053c100600 0x7f053c19ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.630+0000 7f05420e1700 1 -- 172.21.15.2:0/2817826401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380095f0 con 0x7f053c0fea80 2023-06-23T18:22:09.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.631+0000 7f05420e1700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0fea80 0x7f053c068310 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f052c00d980 tx=0x7f052c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:09.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.631+0000 7f05337fe700 1 -- 172.21.15.2:0/2817826401 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c004020 con 0x7f053c0fea80 2023-06-23T18:22:09.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.631+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c19fb20 con 0x7f053c0fea80 2023-06-23T18:22:09.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.631+0000 7f05337fe700 1 -- 172.21.15.2:0/2817826401 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f052c009730 con 0x7f053c0fea80 2023-06-23T18:22:09.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.631+0000 7f05337fe700 1 -- 172.21.15.2:0/2817826401 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c003a00 con 0x7f053c0fea80 2023-06-23T18:22:09.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.631+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c1a0030 con 0x7f053c0fea80 2023-06-23T18:22:09.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.632+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c10a100 con 0x7f053c0fea80 2023-06-23T18:22:09.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.633+0000 7f05337fe700 1 -- 172.21.15.2:0/2817826401 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f052c0098a0 con 0x7f053c0fea80 2023-06-23T18:22:09.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.634+0000 7f05337fe700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05280a11d0 0x7f05280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.634+0000 7f05418e0700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05280a11d0 0x7f05280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.634+0000 7f05337fe700 1 -- 172.21.15.2:0/2817826401 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f052c0cc9d0 con 0x7f053c0fea80 2023-06-23T18:22:09.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.634+0000 7f05418e0700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05280a11d0 0x7f05280a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f053800b2e0 tx=0x7f0538004000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:09.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.637+0000 7f05337fe700 1 -- 172.21.15.2:0/2817826401 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f052c0960e0 con 0x7f053c0fea80 2023-06-23T18:22:09.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.838+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f053c105a00 con 0x7f053c0fea80 2023-06-23T18:22:09.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.839+0000 7f05337fe700 1 -- 172.21.15.2:0/2817826401 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f052c099700 con 0x7f053c0fea80 2023-06-23T18:22:09.844 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:22:09.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05280a11d0 msgr2=0x7f05280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05280a11d0 0x7f05280a3680 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f053800b2e0 tx=0x7f0538004000 comp rx=0 tx=0).stop 2023-06-23T18:22:09.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0fea80 msgr2=0x7f053c068310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0fea80 0x7f053c068310 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f052c00d980 tx=0x7f052c00dc90 comp rx=0 tx=0).stop 2023-06-23T18:22:09.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 shutdown_connections 2023-06-23T18:22:09.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f05280a11d0 0x7f05280a3680 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f053c100600 0x7f053c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f053c0fea80 0x7f053c068310 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 --2- 172.21.15.2:0/2817826401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f053c0ffc70 0x7f053c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 >> 172.21.15.2:0/2817826401 conn(0x7f053c0fa0e0 msgr2=0x7f053c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:09.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 shutdown_connections 2023-06-23T18:22:09.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.841+0000 7f0544345700 1 -- 172.21.15.2:0/2817826401 wait complete. 2023-06-23T18:22:09.858 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:22:09.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 -- 172.21.15.2:0/2953212340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00fea80 msgr2=0x7fabd00fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 --2- 172.21.15.2:0/2953212340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00fea80 0x7fabd00fee90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009990 tx=0x7fabc0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 -- 172.21.15.2:0/2953212340 shutdown_connections 2023-06-23T18:22:09.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 --2- 172.21.15.2:0/2953212340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd0100600 0x7fabd0106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 --2- 172.21.15.2:0/2953212340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00fea80 0x7fabd00fee90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 --2- 172.21.15.2:0/2953212340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabd00ffc70 0x7fabd01000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 -- 172.21.15.2:0/2953212340 >> 172.21.15.2:0/2953212340 conn(0x7fabd00fa0e0 msgr2=0x7fabd00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:09.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 -- 172.21.15.2:0/2953212340 shutdown_connections 2023-06-23T18:22:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 -- 172.21.15.2:0/2953212340 wait complete. 2023-06-23T18:22:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 Processor -- start 2023-06-23T18:22:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.983+0000 7fabd7998700 1 -- start start 2023-06-23T18:22:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd7998700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabd00fea80 0x7fabd0068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd7998700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00ffc70 0x7fabd019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd7998700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd0100600 0x7fabd019cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd7998700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019d310 con 0x7fabd00ffc70 2023-06-23T18:22:09.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd7998700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019d480 con 0x7fabd0100600 2023-06-23T18:22:09.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd7998700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd019d640 con 0x7fabd00fea80 2023-06-23T18:22:09.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00ffc70 0x7fabd019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00ffc70 0x7fabd019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53840/0 (socket says 172.21.15.2:53840) 2023-06-23T18:22:09.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 -- 172.21.15.2:0/2930004080 learned_addr learned my addr 172.21.15.2:0/2930004080 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:09.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd5734700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabd00fea80 0x7fabd0068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 -- 172.21.15.2:0/2930004080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabd00fea80 msgr2=0x7fabd0068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabd00fea80 0x7fabd0068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 -- 172.21.15.2:0/2930004080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd0100600 msgr2=0x7fabd019cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:09.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd5f35700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd0100600 0x7fabd019cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd0100600 0x7fabd019cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:09.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 -- 172.21.15.2:0/2930004080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc00095f0 con 0x7fabd00ffc70 2023-06-23T18:22:09.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabd4f33700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00ffc70 0x7fabd019a4f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00e9a0 tx=0x7fabcc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:09.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.984+0000 7fabc67fc700 1 -- 172.21.15.2:0/2930004080 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc009790 con 0x7fabd00ffc70 2023-06-23T18:22:09.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.985+0000 7fabc67fc700 1 -- 172.21.15.2:0/2930004080 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fabcc00cbd0 con 0x7fabd00ffc70 2023-06-23T18:22:09.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.985+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd019d920 con 0x7fabd00ffc70 2023-06-23T18:22:09.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.985+0000 7fabc67fc700 1 -- 172.21.15.2:0/2930004080 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc018680 con 0x7fabd00ffc70 2023-06-23T18:22:09.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.985+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd019de80 con 0x7fabd00ffc70 2023-06-23T18:22:09.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.985+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd0105a30 con 0x7fabd00ffc70 2023-06-23T18:22:09.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.987+0000 7fabc67fc700 1 -- 172.21.15.2:0/2930004080 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fabcc00cd40 con 0x7fabd00ffc70 2023-06-23T18:22:09.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.987+0000 7fabc67fc700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fabbc0a11d0 0x7fabbc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:09.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.987+0000 7fabd5734700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fabbc0a11d0 0x7fabbc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:09.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.987+0000 7fabc67fc700 1 -- 172.21.15.2:0/2930004080 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fabcc014070 con 0x7fabd00ffc70 2023-06-23T18:22:10.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.988+0000 7fabd5734700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fabbc0a11d0 0x7fabbc0a3680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fabc0006010 tx=0x7fabc000b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:10.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:09.991+0000 7fabc67fc700 1 -- 172.21.15.2:0/2930004080 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fabcc0965c0 con 0x7fabd00ffc70 2023-06-23T18:22:10.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.191+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fabd002c9a0 con 0x7fabd00ffc70 2023-06-23T18:22:10.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.192+0000 7fabc67fc700 1 -- 172.21.15.2:0/2930004080 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabcc0260a0 con 0x7fabd00ffc70 2023-06-23T18:22:10.197 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:22:10.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fabbc0a11d0 msgr2=0x7fabbc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fabbc0a11d0 0x7fabbc0a3680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fabc0006010 tx=0x7fabc000b2a0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00ffc70 msgr2=0x7fabd019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00ffc70 0x7fabd019a4f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00e9a0 tx=0x7fabcc00ecb0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 shutdown_connections 2023-06-23T18:22:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fabbc0a11d0 0x7fabbc0a3680 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabd0100600 0x7fabd019cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabd00fea80 0x7fabd0068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 --2- 172.21.15.2:0/2930004080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fabd00ffc70 0x7fabd019a4f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 >> 172.21.15.2:0/2930004080 conn(0x7fabd00fa0e0 msgr2=0x7fabd00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 shutdown_connections 2023-06-23T18:22:10.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.194+0000 7fabd7998700 1 -- 172.21.15.2:0/2930004080 wait complete. 2023-06-23T18:22:10.209 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:10 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/291318466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:10.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:10 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2311634841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:10.210 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:10 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2817826401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:10.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:22:10.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 -- 172.21.15.2:0/2661040346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1740100600 msgr2=0x7f1740106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 --2- 172.21.15.2:0/2661040346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1740100600 0x7f1740106ec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f173800b0a0 tx=0x7f173800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 -- 172.21.15.2:0/2661040346 shutdown_connections 2023-06-23T18:22:10.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 --2- 172.21.15.2:0/2661040346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1740100600 0x7f1740106ec0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 --2- 172.21.15.2:0/2661040346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17400fea80 0x7f17400fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 --2- 172.21.15.2:0/2661040346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17400ffc70 0x7f17401000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 -- 172.21.15.2:0/2661040346 >> 172.21.15.2:0/2661040346 conn(0x7f17400fa0e0 msgr2=0x7f17400fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:10.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 -- 172.21.15.2:0/2661040346 shutdown_connections 2023-06-23T18:22:10.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.335+0000 7f1745f74700 1 -- 172.21.15.2:0/2661040346 wait complete. 2023-06-23T18:22:10.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.336+0000 7f1745f74700 1 Processor -- start 2023-06-23T18:22:10.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.336+0000 7f1745f74700 1 -- start start 2023-06-23T18:22:10.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.336+0000 7f1745f74700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17400fea80 0x7f1740068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:10.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.336+0000 7f1745f74700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17400ffc70 0x7f174019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:10.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f1745f74700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1740100600 0x7f174019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:10.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f1745f74700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174019d3c0 con 0x7f17400ffc70 2023-06-23T18:22:10.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f1745f74700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174019d530 con 0x7f17400fea80 2023-06-23T18:22:10.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f1745f74700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174019d6f0 con 0x7f1740100600 2023-06-23T18:22:10.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17400ffc70 0x7f174019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:10.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17400ffc70 0x7f174019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53854/0 (socket says 172.21.15.2:53854) 2023-06-23T18:22:10.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 -- 172.21.15.2:0/391789535 learned_addr learned my addr 172.21.15.2:0/391789535 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:10.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173f7fe700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17400fea80 0x7f1740068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:10.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173ffff700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1740100600 0x7f174019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:10.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 -- 172.21.15.2:0/391789535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1740100600 msgr2=0x7f174019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1740100600 0x7f174019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 -- 172.21.15.2:0/391789535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17400fea80 msgr2=0x7f1740068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17400fea80 0x7f1740068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 -- 172.21.15.2:0/391789535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1738009d00 con 0x7f17400ffc70 2023-06-23T18:22:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173effd700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17400ffc70 0x7f174019a590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f173000d950 tx=0x7f173000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173cff9700 1 -- 172.21.15.2:0/391789535 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000f890 con 0x7f17400ffc70 2023-06-23T18:22:10.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173cff9700 1 -- 172.21.15.2:0/391789535 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f173000fe80 con 0x7f17400ffc70 2023-06-23T18:22:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f174019d9d0 con 0x7f17400ffc70 2023-06-23T18:22:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f173cff9700 1 -- 172.21.15.2:0/391789535 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000e3b0 con 0x7f17400ffc70 2023-06-23T18:22:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.337+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174019dee0 con 0x7f17400ffc70 2023-06-23T18:22:10.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.338+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1740105fb0 con 0x7f17400ffc70 2023-06-23T18:22:10.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:10 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/291318466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:10.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:10 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2311634841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:10.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:10 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2817826401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.339+0000 7f173cff9700 1 -- 172.21.15.2:0/391789535 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1730010460 con 0x7f17400ffc70 2023-06-23T18:22:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.340+0000 7f173cff9700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f17280a1150 0x7f17280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.340+0000 7f173f7fe700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f17280a1150 0x7f17280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:10.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.340+0000 7f173cff9700 1 -- 172.21.15.2:0/391789535 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f17300cc6f0 con 0x7f17400ffc70 2023-06-23T18:22:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.341+0000 7f173f7fe700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f17280a1150 0x7f17280a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f17400ffad0 tx=0x7f1734009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:10.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.345+0000 7f173cff9700 1 -- 172.21.15.2:0/391789535 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f1730095e00 con 0x7f17400ffc70 2023-06-23T18:22:10.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:10 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/291318466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:10.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:10 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2311634841' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:10.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:10 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2817826401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:10.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.546+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1740061110 con 0x7f17400ffc70 2023-06-23T18:22:10.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.547+0000 7f173cff9700 1 -- 172.21.15.2:0/391789535 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1730099420 con 0x7f17400ffc70 2023-06-23T18:22:10.551 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:22:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f17280a1150 msgr2=0x7f17280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f17280a1150 0x7f17280a3600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f17400ffad0 tx=0x7f1734009260 comp rx=0 tx=0).stop 2023-06-23T18:22:10.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17400ffc70 msgr2=0x7f174019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17400ffc70 0x7f174019a590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f173000d950 tx=0x7f173000dc60 comp rx=0 tx=0).stop 2023-06-23T18:22:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 shutdown_connections 2023-06-23T18:22:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1740100600 0x7f174019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f17280a1150 0x7f17280a3600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f17400fea80 0x7f1740068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 --2- 172.21.15.2:0/391789535 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17400ffc70 0x7f174019a590 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 >> 172.21.15.2:0/391789535 conn(0x7f17400fa0e0 msgr2=0x7f17401030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:10.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 shutdown_connections 2023-06-23T18:22:10.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.549+0000 7f1745f74700 1 -- 172.21.15.2:0/391789535 wait complete. 2023-06-23T18:22:10.565 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-06-23T18:22:10.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:22:10.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 -- 172.21.15.2:0/227799927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 msgr2=0x7fe36c0feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 --2- 172.21.15.2:0/227799927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 0x7fe36c0feeb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe360009990 tx=0x7fe360009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 -- 172.21.15.2:0/227799927 shutdown_connections 2023-06-23T18:22:10.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 --2- 172.21.15.2:0/227799927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe36c100620 0x7fe36c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 --2- 172.21.15.2:0/227799927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 0x7fe36c0feeb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 --2- 172.21.15.2:0/227799927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe36c0ffc90 0x7fe36c1000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 -- 172.21.15.2:0/227799927 >> 172.21.15.2:0/227799927 conn(0x7fe36c0fa0c0 msgr2=0x7fe36c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:10.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 -- 172.21.15.2:0/227799927 shutdown_connections 2023-06-23T18:22:10.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.686+0000 7fe372ba9700 1 -- 172.21.15.2:0/227799927 wait complete. 2023-06-23T18:22:10.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe372ba9700 1 Processor -- start 2023-06-23T18:22:10.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe372ba9700 1 -- start start 2023-06-23T18:22:10.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe372ba9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 0x7fe36c069090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:10.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe372ba9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe36c0ffc90 0x7fe36c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:10.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe372ba9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe36c100620 0x7fe36c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:10.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe372ba9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c19d2a0 con 0x7fe36c0feaa0 2023-06-23T18:22:10.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe372ba9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c19d410 con 0x7fe36c0ffc90 2023-06-23T18:22:10.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe372ba9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c19d5d0 con 0x7fe36c100620 2023-06-23T18:22:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe370945700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 0x7fe36c069090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe370945700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 0x7fe36c069090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53866/0 (socket says 172.21.15.2:53866) 2023-06-23T18:22:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe370945700 1 -- 172.21.15.2:0/3132778017 learned_addr learned my addr 172.21.15.2:0/3132778017 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe36bfff700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe36c0ffc90 0x7fe36c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe371146700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe36c100620 0x7fe36c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe370945700 1 -- 172.21.15.2:0/3132778017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe36c100620 msgr2=0x7fe36c19cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe370945700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe36c100620 0x7fe36c19cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe370945700 1 -- 172.21.15.2:0/3132778017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe36c0ffc90 msgr2=0x7fe36c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe370945700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe36c0ffc90 0x7fe36c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe370945700 1 -- 172.21.15.2:0/3132778017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600095f0 con 0x7fe36c0feaa0 2023-06-23T18:22:10.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe36bfff700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe36c0ffc90 0x7fe36c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:10.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.687+0000 7fe371146700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe36c100620 0x7fe36c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:10.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.688+0000 7fe370945700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 0x7fe36c069090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b5c0 tx=0x7fe360005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:10.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.688+0000 7fe369ffb700 1 -- 172.21.15.2:0/3132778017 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36001d070 con 0x7fe36c0feaa0 2023-06-23T18:22:10.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.688+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c19d850 con 0x7fe36c0feaa0 2023-06-23T18:22:10.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.688+0000 7fe369ffb700 1 -- 172.21.15.2:0/3132778017 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe360004080 con 0x7fe36c0feaa0 2023-06-23T18:22:10.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.688+0000 7fe369ffb700 1 -- 172.21.15.2:0/3132778017 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe360022cd0 con 0x7fe36c0feaa0 2023-06-23T18:22:10.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.688+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c19dcd0 con 0x7fe36c0feaa0 2023-06-23T18:22:10.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.689+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c105980 con 0x7fe36c0feaa0 2023-06-23T18:22:10.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.690+0000 7fe369ffb700 1 -- 172.21.15.2:0/3132778017 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe360022470 con 0x7fe36c0feaa0 2023-06-23T18:22:10.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.691+0000 7fe369ffb700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe35c0a1150 0x7fe35c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:10.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.691+0000 7fe369ffb700 1 -- 172.21.15.2:0/3132778017 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3600ce5e0 con 0x7fe36c0feaa0 2023-06-23T18:22:10.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.691+0000 7fe36bfff700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe35c0a1150 0x7fe35c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:10.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.691+0000 7fe36bfff700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe35c0a1150 0x7fe35c0a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe358005fd0 tx=0x7fe358005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:10.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.694+0000 7fe369ffb700 1 -- 172.21.15.2:0/3132778017 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe360097dc0 con 0x7fe36c0feaa0 2023-06-23T18:22:10.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.938+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe36c02c810 con 0x7fe36c0feaa0 2023-06-23T18:22:10.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.938+0000 7fe369ffb700 1 -- 172.21.15.2:0/3132778017 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7fe360027090 con 0x7fe36c0feaa0 2023-06-23T18:22:10.943 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:10.944 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:22:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe35c0a1150 msgr2=0x7fe35c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe35c0a1150 0x7fe35c0a3600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe358005fd0 tx=0x7fe358005e30 comp rx=0 tx=0).stop 2023-06-23T18:22:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 msgr2=0x7fe36c069090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 0x7fe36c069090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe36000b5c0 tx=0x7fe360005ef0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 shutdown_connections 2023-06-23T18:22:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe36c100620 0x7fe36c19cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe36c0feaa0 0x7fe36c069090 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe35c0a1150 0x7fe35c0a3600 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 --2- 172.21.15.2:0/3132778017 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe36c0ffc90 0x7fe36c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 >> 172.21.15.2:0/3132778017 conn(0x7fe36c0fa0c0 msgr2=0x7fe36c0fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:10.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 shutdown_connections 2023-06-23T18:22:10.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:10.941+0000 7fe372ba9700 1 -- 172.21.15.2:0/3132778017 wait complete. 2023-06-23T18:22:10.947 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:22:10.958 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-06-23T18:22:10.958 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-06-23T18:22:10.958 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:22:11.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 -- 172.21.15.2:0/350573303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15840ffb90 msgr2=0x7f15840fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 --2- 172.21.15.2:0/350573303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15840ffb90 0x7f15840fffa0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1574009990 tx=0x7f1574009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 -- 172.21.15.2:0/350573303 shutdown_connections 2023-06-23T18:22:11.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 --2- 172.21.15.2:0/350573303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1584101710 0x7f1584107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 --2- 172.21.15.2:0/350573303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15840ffb90 0x7f15840fffa0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 --2- 172.21.15.2:0/350573303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1584100d80 0x7f15841011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 -- 172.21.15.2:0/350573303 >> 172.21.15.2:0/350573303 conn(0x7f15840fb190 msgr2=0x7f15840fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:11.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 -- 172.21.15.2:0/350573303 shutdown_connections 2023-06-23T18:22:11.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.080+0000 7f158a050700 1 -- 172.21.15.2:0/350573303 wait complete. 2023-06-23T18:22:11.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f158a050700 1 Processor -- start 2023-06-23T18:22:11.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f158a050700 1 -- start start 2023-06-23T18:22:11.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f158a050700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ffb90 0x7f15841957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f158a050700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1584100d80 0x7f1584195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f158a050700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1584101710 0x7f158419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f158a050700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158419d360 con 0x7f1584101710 2023-06-23T18:22:11.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f158a050700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158419d4a0 con 0x7f1584100d80 2023-06-23T18:22:11.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f158a050700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158419d630 con 0x7f15840ffb90 2023-06-23T18:22:11.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1583fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1584101710 0x7f158419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1583fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1584101710 0x7f158419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53884/0 (socket says 172.21.15.2:53884) 2023-06-23T18:22:11.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1583fff700 1 -- 172.21.15.2:0/2217620632 learned_addr learned my addr 172.21.15.2:0/2217620632 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:11.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f15837fe700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ffb90 0x7f15841957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1582ffd700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1584100d80 0x7f1584195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1583fff700 1 -- 172.21.15.2:0/2217620632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ffb90 msgr2=0x7f15841957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1583fff700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ffb90 0x7f15841957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1583fff700 1 -- 172.21.15.2:0/2217620632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1584100d80 msgr2=0x7f1584195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1583fff700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1584100d80 0x7f1584195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1583fff700 1 -- 172.21.15.2:0/2217620632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15740095f0 con 0x7f1584101710 2023-06-23T18:22:11.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f15837fe700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ffb90 0x7f15841957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:11.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.081+0000 7f1582ffd700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1584100d80 0x7f1584195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:11.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.082+0000 7f1583fff700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1584101710 0x7f158419cd90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f157c00ef30 tx=0x7f157c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:11.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.082+0000 7f1580ff9700 1 -- 172.21.15.2:0/2217620632 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c018070 con 0x7f1584101710 2023-06-23T18:22:11.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.082+0000 7f1580ff9700 1 -- 172.21.15.2:0/2217620632 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f157c00f040 con 0x7f1584101710 2023-06-23T18:22:11.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.082+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158419d8b0 con 0x7f1584101710 2023-06-23T18:22:11.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.082+0000 7f1580ff9700 1 -- 172.21.15.2:0/2217620632 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c003d00 con 0x7f1584101710 2023-06-23T18:22:11.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.082+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158419dd70 con 0x7f1584101710 2023-06-23T18:22:11.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.083+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158410b210 con 0x7f1584101710 2023-06-23T18:22:11.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.084+0000 7f1580ff9700 1 -- 172.21.15.2:0/2217620632 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f157c00caa0 con 0x7f1584101710 2023-06-23T18:22:11.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.084+0000 7f1580ff9700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15700a1150 0x7f15700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.085+0000 7f1580ff9700 1 -- 172.21.15.2:0/2217620632 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f157c0cd040 con 0x7f1584101710 2023-06-23T18:22:11.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.085+0000 7f15837fe700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15700a1150 0x7f15700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.085+0000 7f15837fe700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15700a1150 0x7f15700a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1574006010 tx=0x7f1574005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:11.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.088+0000 7f1580ff9700 1 -- 172.21.15.2:0/2217620632 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f157c0967e0 con 0x7f1584101710 2023-06-23T18:22:11.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.286+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1584060910 con 0x7f15700a1150 2023-06-23T18:22:11.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:11 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2930004080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:11.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:11 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/391789535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:11.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:11 smithi002 ceph-mon[95947]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2023-06-23T18:22:11.305 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:11 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3132778017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:11.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.299+0000 7f1580ff9700 1 -- 172.21.15.2:0/2217620632 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344209 (secure 0 0 0) 0x7f1584060910 con 0x7f15700a1150 2023-06-23T18:22:11.310 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:11.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15700a1150 msgr2=0x7f15700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15700a1150 0x7f15700a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1574006010 tx=0x7f1574005960 comp rx=0 tx=0).stop 2023-06-23T18:22:11.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1584101710 msgr2=0x7f158419cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1584101710 0x7f158419cd90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f157c00ef30 tx=0x7f157c00c450 comp rx=0 tx=0).stop 2023-06-23T18:22:11.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 shutdown_connections 2023-06-23T18:22:11.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1584101710 0x7f158419cd90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15840ffb90 0x7f15841957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f15700a1150 0x7f15700a3600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 --2- 172.21.15.2:0/2217620632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1584100d80 0x7f1584195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 >> 172.21.15.2:0/2217620632 conn(0x7f15840fb190 msgr2=0x7f15840fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:11.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.302+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 shutdown_connections 2023-06-23T18:22:11.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.303+0000 7f158a050700 1 -- 172.21.15.2:0/2217620632 wait complete. 2023-06-23T18:22:11.315 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-06-23T18:22:11.319 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-06-23T18:22:10.757253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653855,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":618,"num_read_kb":420,"num_write":345,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1238346,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":637,"ondisk_log_size":637,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334516,"kb_used_data":11092,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607308,"statfs":{"total":1151940427776,"available":1151597883392,"internally_reserved":0,"allocated":11358208,"data_stored":7207136,"data_compressed":186066,"data_compressed_allocated":245760,"data_compressed_original":2019328,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,24,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1839,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":33,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006963"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785637+0000","last_change":"2023-06-23T18:20:44.466650+0000","last_active":"2023-06-23T18:22:06.785637+0000","last_peered":"2023-06-23T18:22:06.785637+0000","last_clean":"2023-06-23T18:22:06.785637+0000","last_became_active":"2023-06-23T18:20:44.466425+0000","last_became_peered":"2023-06-23T18:20:44.466425+0000","last_unstale":"2023-06-23T18:22:06.785637+0000","last_undegraded":"2023-06-23T18:22:06.785637+0000","last_fullsized":"2023-06-23T18:22:06.785637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:30:41.271809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080017+0000","last_change":"2023-06-23T18:20:44.465762+0000","last_active":"2023-06-23T18:21:38.080017+0000","last_peered":"2023-06-23T18:21:38.080017+0000","last_clean":"2023-06-23T18:21:38.080017+0000","last_became_active":"2023-06-23T18:20:44.463592+0000","last_became_peered":"2023-06-23T18:20:44.463592+0000","last_unstale":"2023-06-23T18:21:38.080017+0000","last_undegraded":"2023-06-23T18:21:38.080017+0000","last_fullsized":"2023-06-23T18:21:38.080017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:04:03.928905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079784+0000","last_change":"2023-06-23T18:20:44.464162+0000","last_active":"2023-06-23T18:21:38.079784+0000","last_peered":"2023-06-23T18:21:38.079784+0000","last_clean":"2023-06-23T18:21:38.079784+0000","last_became_active":"2023-06-23T18:20:44.463854+0000","last_became_peered":"2023-06-23T18:20:44.463854+0000","last_unstale":"2023-06-23T18:21:38.079784+0000","last_undegraded":"2023-06-23T18:21:38.079784+0000","last_fullsized":"2023-06-23T18:21:38.079784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:52:10.241558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.777090+0000","last_change":"2023-06-23T18:20:44.462923+0000","last_active":"2023-06-23T18:22:06.777090+0000","last_peered":"2023-06-23T18:22:06.777090+0000","last_clean":"2023-06-23T18:22:06.777090+0000","last_became_active":"2023-06-23T18:20:44.462511+0000","last_became_peered":"2023-06-23T18:20:44.462511+0000","last_unstale":"2023-06-23T18:22:06.777090+0000","last_undegraded":"2023-06-23T18:22:06.777090+0000","last_fullsized":"2023-06-23T18:22:06.777090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:41:07.780308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.775980+0000","last_change":"2023-06-23T18:20:44.454727+0000","last_active":"2023-06-23T18:22:06.775980+0000","last_peered":"2023-06-23T18:22:06.775980+0000","last_clean":"2023-06-23T18:22:06.775980+0000","last_became_active":"2023-06-23T18:20:44.454418+0000","last_became_peered":"2023-06-23T18:20:44.454418+0000","last_unstale":"2023-06-23T18:22:06.775980+0000","last_undegraded":"2023-06-23T18:22:06.775980+0000","last_fullsized":"2023-06-23T18:22:06.775980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:06:57.921477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790102+0000","last_change":"2023-06-23T18:20:44.460636+0000","last_active":"2023-06-23T18:22:06.790102+0000","last_peered":"2023-06-23T18:22:06.790102+0000","last_clean":"2023-06-23T18:22:06.790102+0000","last_became_active":"2023-06-23T18:20:44.460305+0000","last_became_peered":"2023-06-23T18:20:44.460305+0000","last_unstale":"2023-06-23T18:22:06.790102+0000","last_undegraded":"2023-06-23T18:22:06.790102+0000","last_fullsized":"2023-06-23T18:22:06.790102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:08:27.871589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785720+0000","last_change":"2023-06-23T18:20:44.463552+0000","last_active":"2023-06-23T18:22:06.785720+0000","last_peered":"2023-06-23T18:22:06.785720+0000","last_clean":"2023-06-23T18:22:06.785720+0000","last_became_active":"2023-06-23T18:20:44.463187+0000","last_became_peered":"2023-06-23T18:20:44.463187+0000","last_unstale":"2023-06-23T18:22:06.785720+0000","last_undegraded":"2023-06-23T18:22:06.785720+0000","last_fullsized":"2023-06-23T18:22:06.785720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:16:36.563197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790515+0000","last_change":"2023-06-23T18:21:38.083209+0000","last_active":"2023-06-23T18:22:06.790515+0000","last_peered":"2023-06-23T18:22:06.790515+0000","last_clean":"2023-06-23T18:22:06.790515+0000","last_became_active":"2023-06-23T18:21:38.082912+0000","last_became_peered":"2023-06-23T18:21:38.082912+0000","last_unstale":"2023-06-23T18:22:06.790515+0000","last_undegraded":"2023-06-23T18:22:06.790515+0000","last_fullsized":"2023-06-23T18:22:06.790515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:54:52.291671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778393+0000","last_change":"2023-06-23T18:20:44.461199+0000","last_active":"2023-06-23T18:22:06.778393+0000","last_peered":"2023-06-23T18:22:06.778393+0000","last_clean":"2023-06-23T18:22:06.778393+0000","last_became_active":"2023-06-23T18:20:44.460891+0000","last_became_peered":"2023-06-23T18:20:44.460891+0000","last_unstale":"2023-06-23T18:22:06.778393+0000","last_undegraded":"2023-06-23T18:22:06.778393+0000","last_fullsized":"2023-06-23T18:22:06.778393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:35.270311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.791756+0000","last_change":"2023-06-23T18:21:38.084799+0000","last_active":"2023-06-23T18:22:06.791756+0000","last_peered":"2023-06-23T18:22:06.791756+0000","last_clean":"2023-06-23T18:22:06.791756+0000","last_became_active":"2023-06-23T18:21:38.084336+0000","last_became_peered":"2023-06-23T18:21:38.084336+0000","last_unstale":"2023-06-23T18:22:06.791756+0000","last_undegraded":"2023-06-23T18:22:06.791756+0000","last_fullsized":"2023-06-23T18:22:06.791756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:09:38.488565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789987+0000","last_change":"2023-06-23T18:20:44.453337+0000","last_active":"2023-06-23T18:22:06.789987+0000","last_peered":"2023-06-23T18:22:06.789987+0000","last_clean":"2023-06-23T18:22:06.789987+0000","last_became_active":"2023-06-23T18:20:44.452925+0000","last_became_peered":"2023-06-23T18:20:44.452925+0000","last_unstale":"2023-06-23T18:22:06.789987+0000","last_undegraded":"2023-06-23T18:22:06.789987+0000","last_fullsized":"2023-06-23T18:22:06.789987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:39:25.750095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.774400+0000","last_change":"2023-06-23T18:20:44.456710+0000","last_active":"2023-06-23T18:22:06.774400+0000","last_peered":"2023-06-23T18:22:06.774400+0000","last_clean":"2023-06-23T18:22:06.774400+0000","last_became_active":"2023-06-23T18:20:44.456411+0000","last_became_peered":"2023-06-23T18:20:44.456411+0000","last_unstale":"2023-06-23T18:22:06.774400+0000","last_undegraded":"2023-06-23T18:22:06.774400+0000","last_fullsized":"2023-06-23T18:22:06.774400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:42:13.109612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072919+0000","last_change":"2023-06-23T18:20:44.453586+0000","last_active":"2023-06-23T18:21:38.072919+0000","last_peered":"2023-06-23T18:21:38.072919+0000","last_clean":"2023-06-23T18:21:38.072919+0000","last_became_active":"2023-06-23T18:20:44.453239+0000","last_became_peered":"2023-06-23T18:20:44.453239+0000","last_unstale":"2023-06-23T18:21:38.072919+0000","last_undegraded":"2023-06-23T18:21:38.072919+0000","last_fullsized":"2023-06-23T18:21:38.072919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:36:37.353139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079914+0000","last_change":"2023-06-23T18:20:44.465627+0000","last_active":"2023-06-23T18:21:38.079914+0000","last_peered":"2023-06-23T18:21:38.079914+0000","last_clean":"2023-06-23T18:21:38.079914+0000","last_became_active":"2023-06-23T18:20:44.464568+0000","last_became_peered":"2023-06-23T18:20:44.464568+0000","last_unstale":"2023-06-23T18:21:38.079914+0000","last_undegraded":"2023-06-23T18:21:38.079914+0000","last_fullsized":"2023-06-23T18:21:38.079914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:15:39.010793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079452+0000","last_change":"2023-06-23T18:20:44.464169+0000","last_active":"2023-06-23T18:21:38.079452+0000","last_peered":"2023-06-23T18:21:38.079452+0000","last_clean":"2023-06-23T18:21:38.079452+0000","last_became_active":"2023-06-23T18:20:44.463867+0000","last_became_peered":"2023-06-23T18:20:44.463867+0000","last_unstale":"2023-06-23T18:21:38.079452+0000","last_undegraded":"2023-06-23T18:21:38.079452+0000","last_fullsized":"2023-06-23T18:21:38.079452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:12:31.642855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792291+0000","last_change":"2023-06-23T18:20:44.467101+0000","last_active":"2023-06-23T18:22:06.792291+0000","last_peered":"2023-06-23T18:22:06.792291+0000","last_clean":"2023-06-23T18:22:06.792291+0000","last_became_active":"2023-06-23T18:20:44.466788+0000","last_became_peered":"2023-06-23T18:20:44.466788+0000","last_unstale":"2023-06-23T18:22:06.792291+0000","last_undegraded":"2023-06-23T18:22:06.792291+0000","last_fullsized":"2023-06-23T18:22:06.792291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:41:04.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785439+0000","last_change":"2023-06-23T18:20:44.466055+0000","last_active":"2023-06-23T18:22:06.785439+0000","last_peered":"2023-06-23T18:22:06.785439+0000","last_clean":"2023-06-23T18:22:06.785439+0000","last_became_active":"2023-06-23T18:20:44.465608+0000","last_became_peered":"2023-06-23T18:20:44.465608+0000","last_unstale":"2023-06-23T18:22:06.785439+0000","last_undegraded":"2023-06-23T18:22:06.785439+0000","last_fullsized":"2023-06-23T18:22:06.785439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:16:31.985669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790712+0000","last_change":"2023-06-23T18:20:44.459414+0000","last_active":"2023-06-23T18:22:06.790712+0000","last_peered":"2023-06-23T18:22:06.790712+0000","last_clean":"2023-06-23T18:22:06.790712+0000","last_became_active":"2023-06-23T18:20:44.459014+0000","last_became_peered":"2023-06-23T18:20:44.459014+0000","last_unstale":"2023-06-23T18:22:06.790712+0000","last_undegraded":"2023-06-23T18:22:06.790712+0000","last_fullsized":"2023-06-23T18:22:06.790712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:16.590974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.774664+0000","last_change":"2023-06-23T18:20:44.446784+0000","last_active":"2023-06-23T18:22:06.774664+0000","last_peered":"2023-06-23T18:22:06.774664+0000","last_clean":"2023-06-23T18:22:06.774664+0000","last_became_active":"2023-06-23T18:20:44.446464+0000","last_became_peered":"2023-06-23T18:20:44.446464+0000","last_unstale":"2023-06-23T18:22:06.774664+0000","last_undegraded":"2023-06-23T18:22:06.774664+0000","last_fullsized":"2023-06-23T18:22:06.774664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:35:12.334055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075653+0000","last_change":"2023-06-23T18:20:44.039326+0000","last_active":"2023-06-23T18:21:38.075653+0000","last_peered":"2023-06-23T18:21:38.075653+0000","last_clean":"2023-06-23T18:21:38.075653+0000","last_became_active":"2023-06-23T18:20:44.038787+0000","last_became_peered":"2023-06-23T18:20:44.038787+0000","last_unstale":"2023-06-23T18:21:38.075653+0000","last_undegraded":"2023-06-23T18:21:38.075653+0000","last_fullsized":"2023-06-23T18:21:38.075653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:49:48.612922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081147+0000","last_change":"2023-06-23T18:20:44.465269+0000","last_active":"2023-06-23T18:21:38.081147+0000","last_peered":"2023-06-23T18:21:38.081147+0000","last_clean":"2023-06-23T18:21:38.081147+0000","last_became_active":"2023-06-23T18:20:44.464973+0000","last_became_peered":"2023-06-23T18:20:44.464973+0000","last_unstale":"2023-06-23T18:21:38.081147+0000","last_undegraded":"2023-06-23T18:21:38.081147+0000","last_fullsized":"2023-06-23T18:21:38.081147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:09:55.905699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081281+0000","last_change":"2023-06-23T18:20:44.037175+0000","last_active":"2023-06-23T18:21:38.081281+0000","last_peered":"2023-06-23T18:21:38.081281+0000","last_clean":"2023-06-23T18:21:38.081281+0000","last_became_active":"2023-06-23T18:20:44.036788+0000","last_became_peered":"2023-06-23T18:20:44.036788+0000","last_unstale":"2023-06-23T18:21:38.081281+0000","last_undegraded":"2023-06-23T18:21:38.081281+0000","last_fullsized":"2023-06-23T18:21:38.081281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:19:02.349726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080329+0000","last_change":"2023-06-23T18:20:44.463344+0000","last_active":"2023-06-23T18:21:38.080329+0000","last_peered":"2023-06-23T18:21:38.080329+0000","last_clean":"2023-06-23T18:21:38.080329+0000","last_became_active":"2023-06-23T18:20:44.462649+0000","last_became_peered":"2023-06-23T18:20:44.462649+0000","last_unstale":"2023-06-23T18:21:38.080329+0000","last_undegraded":"2023-06-23T18:21:38.080329+0000","last_fullsized":"2023-06-23T18:21:38.080329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:36:25.801756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076445+0000","last_change":"2023-06-23T18:20:44.045525+0000","last_active":"2023-06-23T18:21:38.076445+0000","last_peered":"2023-06-23T18:21:38.076445+0000","last_clean":"2023-06-23T18:21:38.076445+0000","last_became_active":"2023-06-23T18:20:44.045212+0000","last_became_peered":"2023-06-23T18:20:44.045212+0000","last_unstale":"2023-06-23T18:21:38.076445+0000","last_undegraded":"2023-06-23T18:21:38.076445+0000","last_fullsized":"2023-06-23T18:21:38.076445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:25.401039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790279+0000","last_change":"2023-06-23T18:20:44.461071+0000","last_active":"2023-06-23T18:22:06.790279+0000","last_peered":"2023-06-23T18:22:06.790279+0000","last_clean":"2023-06-23T18:22:06.790279+0000","last_became_active":"2023-06-23T18:20:44.460726+0000","last_became_peered":"2023-06-23T18:20:44.460726+0000","last_unstale":"2023-06-23T18:22:06.790279+0000","last_undegraded":"2023-06-23T18:22:06.790279+0000","last_fullsized":"2023-06-23T18:22:06.790279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:04:57.655489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076111+0000","last_change":"2023-06-23T18:20:44.037162+0000","last_active":"2023-06-23T18:21:38.076111+0000","last_peered":"2023-06-23T18:21:38.076111+0000","last_clean":"2023-06-23T18:21:38.076111+0000","last_became_active":"2023-06-23T18:20:44.036809+0000","last_became_peered":"2023-06-23T18:20:44.036809+0000","last_unstale":"2023-06-23T18:21:38.076111+0000","last_undegraded":"2023-06-23T18:21:38.076111+0000","last_fullsized":"2023-06-23T18:21:38.076111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:05:40.391806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.786602+0000","last_change":"2023-06-23T18:20:44.463525+0000","last_active":"2023-06-23T18:22:06.786602+0000","last_peered":"2023-06-23T18:22:06.786602+0000","last_clean":"2023-06-23T18:22:06.786602+0000","last_became_active":"2023-06-23T18:20:44.463245+0000","last_became_peered":"2023-06-23T18:20:44.463245+0000","last_unstale":"2023-06-23T18:22:06.786602+0000","last_undegraded":"2023-06-23T18:22:06.786602+0000","last_fullsized":"2023-06-23T18:22:06.786602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:29:11.041855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":108,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:08.167164+0000","last_change":"2023-06-23T18:20:44.044415+0000","last_active":"2023-06-23T18:22:08.167164+0000","last_peered":"2023-06-23T18:22:08.167164+0000","last_clean":"2023-06-23T18:22:08.167164+0000","last_became_active":"2023-06-23T18:20:44.044085+0000","last_became_peered":"2023-06-23T18:20:44.044085+0000","last_unstale":"2023-06-23T18:22:08.167164+0000","last_undegraded":"2023-06-23T18:22:08.167164+0000","last_fullsized":"2023-06-23T18:22:08.167164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:43:24.443528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1057,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778243+0000","last_change":"2023-06-23T18:21:38.077968+0000","last_active":"2023-06-23T18:22:06.778243+0000","last_peered":"2023-06-23T18:22:06.778243+0000","last_clean":"2023-06-23T18:22:06.778243+0000","last_became_active":"2023-06-23T18:21:38.077519+0000","last_became_peered":"2023-06-23T18:21:38.077519+0000","last_unstale":"2023-06-23T18:22:06.778243+0000","last_undegraded":"2023-06-23T18:22:06.778243+0000","last_fullsized":"2023-06-23T18:22:06.778243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:03:50.715602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076205+0000","last_change":"2023-06-23T18:20:44.039604+0000","last_active":"2023-06-23T18:21:38.076205+0000","last_peered":"2023-06-23T18:21:38.076205+0000","last_clean":"2023-06-23T18:21:38.076205+0000","last_became_active":"2023-06-23T18:20:44.039317+0000","last_became_peered":"2023-06-23T18:20:44.039317+0000","last_unstale":"2023-06-23T18:21:38.076205+0000","last_undegraded":"2023-06-23T18:21:38.076205+0000","last_fullsized":"2023-06-23T18:21:38.076205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:57:55.720236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780609+0000","last_change":"2023-06-23T18:20:44.465748+0000","last_active":"2023-06-23T18:22:06.780609+0000","last_peered":"2023-06-23T18:22:06.780609+0000","last_clean":"2023-06-23T18:22:06.780609+0000","last_became_active":"2023-06-23T18:20:44.465430+0000","last_became_peered":"2023-06-23T18:20:44.465430+0000","last_unstale":"2023-06-23T18:22:06.780609+0000","last_undegraded":"2023-06-23T18:22:06.780609+0000","last_fullsized":"2023-06-23T18:22:06.780609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:29:38.870085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072805+0000","last_change":"2023-06-23T18:20:44.034388+0000","last_active":"2023-06-23T18:21:38.072805+0000","last_peered":"2023-06-23T18:21:38.072805+0000","last_clean":"2023-06-23T18:21:38.072805+0000","last_became_active":"2023-06-23T18:20:44.034067+0000","last_became_peered":"2023-06-23T18:20:44.034067+0000","last_unstale":"2023-06-23T18:21:38.072805+0000","last_undegraded":"2023-06-23T18:21:38.072805+0000","last_fullsized":"2023-06-23T18:21:38.072805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:21:51.273582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080025+0000","last_change":"2023-06-23T18:20:44.453280+0000","last_active":"2023-06-23T18:21:38.080025+0000","last_peered":"2023-06-23T18:21:38.080025+0000","last_clean":"2023-06-23T18:21:38.080025+0000","last_became_active":"2023-06-23T18:20:44.452980+0000","last_became_peered":"2023-06-23T18:20:44.452980+0000","last_unstale":"2023-06-23T18:21:38.080025+0000","last_undegraded":"2023-06-23T18:21:38.080025+0000","last_fullsized":"2023-06-23T18:21:38.080025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:01:27.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.082028+0000","last_change":"2023-06-23T18:20:44.036165+0000","last_active":"2023-06-23T18:21:38.082028+0000","last_peered":"2023-06-23T18:21:38.082028+0000","last_clean":"2023-06-23T18:21:38.082028+0000","last_became_active":"2023-06-23T18:20:44.035313+0000","last_became_peered":"2023-06-23T18:20:44.035313+0000","last_unstale":"2023-06-23T18:21:38.082028+0000","last_undegraded":"2023-06-23T18:21:38.082028+0000","last_fullsized":"2023-06-23T18:21:38.082028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:15:36.463724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790545+0000","last_change":"2023-06-23T18:20:44.468358+0000","last_active":"2023-06-23T18:22:06.790545+0000","last_peered":"2023-06-23T18:22:06.790545+0000","last_clean":"2023-06-23T18:22:06.790545+0000","last_became_active":"2023-06-23T18:20:44.468032+0000","last_became_peered":"2023-06-23T18:20:44.468032+0000","last_unstale":"2023-06-23T18:22:06.790545+0000","last_undegraded":"2023-06-23T18:22:06.790545+0000","last_fullsized":"2023-06-23T18:22:06.790545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:23:56.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.101308+0000","last_change":"2023-06-23T18:20:44.037921+0000","last_active":"2023-06-23T18:22:07.101308+0000","last_peered":"2023-06-23T18:22:07.101308+0000","last_clean":"2023-06-23T18:22:07.101308+0000","last_became_active":"2023-06-23T18:20:44.037571+0000","last_became_peered":"2023-06-23T18:20:44.037571+0000","last_unstale":"2023-06-23T18:22:07.101308+0000","last_undegraded":"2023-06-23T18:22:07.101308+0000","last_fullsized":"2023-06-23T18:22:07.101308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:32:06.313509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792659+0000","last_change":"2023-06-23T18:20:44.460897+0000","last_active":"2023-06-23T18:22:06.792659+0000","last_peered":"2023-06-23T18:22:06.792659+0000","last_clean":"2023-06-23T18:22:06.792659+0000","last_became_active":"2023-06-23T18:20:44.460476+0000","last_became_peered":"2023-06-23T18:20:44.460476+0000","last_unstale":"2023-06-23T18:22:06.792659+0000","last_undegraded":"2023-06-23T18:22:06.792659+0000","last_fullsized":"2023-06-23T18:22:06.792659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:43:49.396858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073866+0000","last_change":"2023-06-23T18:20:44.033013+0000","last_active":"2023-06-23T18:21:38.073866+0000","last_peered":"2023-06-23T18:21:38.073866+0000","last_clean":"2023-06-23T18:21:38.073866+0000","last_became_active":"2023-06-23T18:20:44.032580+0000","last_became_peered":"2023-06-23T18:20:44.032580+0000","last_unstale":"2023-06-23T18:21:38.073866+0000","last_undegraded":"2023-06-23T18:21:38.073866+0000","last_fullsized":"2023-06-23T18:21:38.073866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:54.441960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080378+0000","last_change":"2023-06-23T18:20:44.463585+0000","last_active":"2023-06-23T18:21:38.080378+0000","last_peered":"2023-06-23T18:21:38.080378+0000","last_clean":"2023-06-23T18:21:38.080378+0000","last_became_active":"2023-06-23T18:20:44.462512+0000","last_became_peered":"2023-06-23T18:20:44.462512+0000","last_unstale":"2023-06-23T18:21:38.080378+0000","last_undegraded":"2023-06-23T18:21:38.080378+0000","last_fullsized":"2023-06-23T18:21:38.080378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:01:51.999262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081156+0000","last_change":"2023-06-23T18:20:44.037134+0000","last_active":"2023-06-23T18:21:38.081156+0000","last_peered":"2023-06-23T18:21:38.081156+0000","last_clean":"2023-06-23T18:21:38.081156+0000","last_became_active":"2023-06-23T18:20:44.036536+0000","last_became_peered":"2023-06-23T18:20:44.036536+0000","last_unstale":"2023-06-23T18:21:38.081156+0000","last_undegraded":"2023-06-23T18:21:38.081156+0000","last_fullsized":"2023-06-23T18:21:38.081156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:45:16.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785436+0000","last_change":"2023-06-23T18:20:44.454453+0000","last_active":"2023-06-23T18:22:06.785436+0000","last_peered":"2023-06-23T18:22:06.785436+0000","last_clean":"2023-06-23T18:22:06.785436+0000","last_became_active":"2023-06-23T18:20:44.454133+0000","last_became_peered":"2023-06-23T18:20:44.454133+0000","last_unstale":"2023-06-23T18:22:06.785436+0000","last_undegraded":"2023-06-23T18:22:06.785436+0000","last_fullsized":"2023-06-23T18:22:06.785436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:12:10.199634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":234,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.148559+0000","last_change":"2023-06-23T18:21:38.108843+0000","last_active":"2023-06-23T18:22:07.148559+0000","last_peered":"2023-06-23T18:22:07.148559+0000","last_clean":"2023-06-23T18:22:07.148559+0000","last_became_active":"2023-06-23T18:21:38.102073+0000","last_became_peered":"2023-06-23T18:21:38.102073+0000","last_unstale":"2023-06-23T18:22:07.148559+0000","last_undegraded":"2023-06-23T18:22:07.148559+0000","last_fullsized":"2023-06-23T18:22:07.148559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:17:04.550195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58230,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":134,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57498,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.787027+0000","last_change":"2023-06-23T18:20:44.456558+0000","last_active":"2023-06-23T18:22:06.787027+0000","last_peered":"2023-06-23T18:22:06.787027+0000","last_clean":"2023-06-23T18:22:06.787027+0000","last_became_active":"2023-06-23T18:20:44.456199+0000","last_became_peered":"2023-06-23T18:20:44.456199+0000","last_unstale":"2023-06-23T18:22:06.787027+0000","last_undegraded":"2023-06-23T18:22:06.787027+0000","last_fullsized":"2023-06-23T18:22:06.787027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:53:48.531640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080152+0000","last_change":"2023-06-23T18:20:44.034101+0000","last_active":"2023-06-23T18:21:38.080152+0000","last_peered":"2023-06-23T18:21:38.080152+0000","last_clean":"2023-06-23T18:21:38.080152+0000","last_became_active":"2023-06-23T18:20:44.033287+0000","last_became_peered":"2023-06-23T18:20:44.033287+0000","last_unstale":"2023-06-23T18:21:38.080152+0000","last_undegraded":"2023-06-23T18:21:38.080152+0000","last_fullsized":"2023-06-23T18:21:38.080152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:44:53.381086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072861+0000","last_change":"2023-06-23T18:20:44.446298+0000","last_active":"2023-06-23T18:21:38.072861+0000","last_peered":"2023-06-23T18:21:38.072861+0000","last_clean":"2023-06-23T18:21:38.072861+0000","last_became_active":"2023-06-23T18:20:44.445922+0000","last_became_peered":"2023-06-23T18:20:44.445922+0000","last_unstale":"2023-06-23T18:21:38.072861+0000","last_undegraded":"2023-06-23T18:21:38.072861+0000","last_fullsized":"2023-06-23T18:21:38.072861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:51.263367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080693+0000","last_change":"2023-06-23T18:20:44.035791+0000","last_active":"2023-06-23T18:21:38.080693+0000","last_peered":"2023-06-23T18:21:38.080693+0000","last_clean":"2023-06-23T18:21:38.080693+0000","last_became_active":"2023-06-23T18:20:44.035416+0000","last_became_peered":"2023-06-23T18:20:44.035416+0000","last_unstale":"2023-06-23T18:21:38.080693+0000","last_undegraded":"2023-06-23T18:21:38.080693+0000","last_fullsized":"2023-06-23T18:21:38.080693+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:49:57.755758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790515+0000","last_change":"2023-06-23T18:20:44.455474+0000","last_active":"2023-06-23T18:22:06.790515+0000","last_peered":"2023-06-23T18:22:06.790515+0000","last_clean":"2023-06-23T18:22:06.790515+0000","last_became_active":"2023-06-23T18:20:44.455098+0000","last_became_peered":"2023-06-23T18:20:44.455098+0000","last_unstale":"2023-06-23T18:22:06.790515+0000","last_undegraded":"2023-06-23T18:22:06.790515+0000","last_fullsized":"2023-06-23T18:22:06.790515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:15:19.154271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":107,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.164451+0000","last_change":"2023-06-23T18:20:44.030305+0000","last_active":"2023-06-23T18:22:07.164451+0000","last_peered":"2023-06-23T18:22:07.164451+0000","last_clean":"2023-06-23T18:22:07.164451+0000","last_became_active":"2023-06-23T18:20:44.029898+0000","last_became_peered":"2023-06-23T18:20:44.029898+0000","last_unstale":"2023-06-23T18:22:07.164451+0000","last_undegraded":"2023-06-23T18:22:07.164451+0000","last_fullsized":"2023-06-23T18:22:07.164451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:49:33.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1554,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790958+0000","last_change":"2023-06-23T18:20:44.451321+0000","last_active":"2023-06-23T18:22:06.790958+0000","last_peered":"2023-06-23T18:22:06.790958+0000","last_clean":"2023-06-23T18:22:06.790958+0000","last_became_active":"2023-06-23T18:20:44.450549+0000","last_became_peered":"2023-06-23T18:20:44.450549+0000","last_unstale":"2023-06-23T18:22:06.790958+0000","last_undegraded":"2023-06-23T18:22:06.790958+0000","last_fullsized":"2023-06-23T18:22:06.790958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:31:37.630032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.097392+0000","last_change":"2023-06-23T18:21:38.097392+0000","last_active":"2023-06-23T18:21:38.097392+0000","last_peered":"2023-06-23T18:21:38.097392+0000","last_clean":"2023-06-23T18:21:38.097392+0000","last_became_active":"2023-06-23T18:21:38.082554+0000","last_became_peered":"2023-06-23T18:21:38.082554+0000","last_unstale":"2023-06-23T18:21:38.097392+0000","last_undegraded":"2023-06-23T18:21:38.097392+0000","last_fullsized":"2023-06-23T18:21:38.097392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:24:59.677445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789557+0000","last_change":"2023-06-23T18:21:38.085114+0000","last_active":"2023-06-23T18:22:06.789557+0000","last_peered":"2023-06-23T18:22:06.789557+0000","last_clean":"2023-06-23T18:22:06.789557+0000","last_became_active":"2023-06-23T18:21:38.084575+0000","last_became_peered":"2023-06-23T18:21:38.084575+0000","last_unstale":"2023-06-23T18:22:06.789557+0000","last_undegraded":"2023-06-23T18:22:06.789557+0000","last_fullsized":"2023-06-23T18:22:06.789557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:27:44.436006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076256+0000","last_change":"2023-06-23T18:20:44.038961+0000","last_active":"2023-06-23T18:21:38.076256+0000","last_peered":"2023-06-23T18:21:38.076256+0000","last_clean":"2023-06-23T18:21:38.076256+0000","last_became_active":"2023-06-23T18:20:44.038674+0000","last_became_peered":"2023-06-23T18:20:44.038674+0000","last_unstale":"2023-06-23T18:21:38.076256+0000","last_undegraded":"2023-06-23T18:21:38.076256+0000","last_fullsized":"2023-06-23T18:21:38.076256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:37:52.894959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.776481+0000","last_change":"2023-06-23T18:20:44.461077+0000","last_active":"2023-06-23T18:22:06.776481+0000","last_peered":"2023-06-23T18:22:06.776481+0000","last_clean":"2023-06-23T18:22:06.776481+0000","last_became_active":"2023-06-23T18:20:44.460767+0000","last_became_peered":"2023-06-23T18:20:44.460767+0000","last_unstale":"2023-06-23T18:22:06.776481+0000","last_undegraded":"2023-06-23T18:22:06.776481+0000","last_fullsized":"2023-06-23T18:22:06.776481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:33:02.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075336+0000","last_change":"2023-06-23T18:20:44.042114+0000","last_active":"2023-06-23T18:21:38.075336+0000","last_peered":"2023-06-23T18:21:38.075336+0000","last_clean":"2023-06-23T18:21:38.075336+0000","last_became_active":"2023-06-23T18:20:44.041822+0000","last_became_peered":"2023-06-23T18:20:44.041822+0000","last_unstale":"2023-06-23T18:21:38.075336+0000","last_undegraded":"2023-06-23T18:21:38.075336+0000","last_fullsized":"2023-06-23T18:21:38.075336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:09:25.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789488+0000","last_change":"2023-06-23T18:20:44.452903+0000","last_active":"2023-06-23T18:22:06.789488+0000","last_peered":"2023-06-23T18:22:06.789488+0000","last_clean":"2023-06-23T18:22:06.789488+0000","last_became_active":"2023-06-23T18:20:44.452539+0000","last_became_peered":"2023-06-23T18:20:44.452539+0000","last_unstale":"2023-06-23T18:22:06.789488+0000","last_undegraded":"2023-06-23T18:22:06.789488+0000","last_fullsized":"2023-06-23T18:22:06.789488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:00:37.883811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073621+0000","last_change":"2023-06-23T18:20:44.032905+0000","last_active":"2023-06-23T18:21:38.073621+0000","last_peered":"2023-06-23T18:21:38.073621+0000","last_clean":"2023-06-23T18:21:38.073621+0000","last_became_active":"2023-06-23T18:20:44.032606+0000","last_became_peered":"2023-06-23T18:20:44.032606+0000","last_unstale":"2023-06-23T18:21:38.073621+0000","last_undegraded":"2023-06-23T18:21:38.073621+0000","last_fullsized":"2023-06-23T18:21:38.073621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:44:01.287033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789982+0000","last_change":"2023-06-23T18:20:44.465580+0000","last_active":"2023-06-23T18:22:06.789982+0000","last_peered":"2023-06-23T18:22:06.789982+0000","last_clean":"2023-06-23T18:22:06.789982+0000","last_became_active":"2023-06-23T18:20:44.465264+0000","last_became_peered":"2023-06-23T18:20:44.465264+0000","last_unstale":"2023-06-23T18:22:06.789982+0000","last_undegraded":"2023-06-23T18:22:06.789982+0000","last_fullsized":"2023-06-23T18:22:06.789982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:40:26.704880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778940+0000","last_change":"2023-06-23T18:20:44.464324+0000","last_active":"2023-06-23T18:22:06.778940+0000","last_peered":"2023-06-23T18:22:06.778940+0000","last_clean":"2023-06-23T18:22:06.778940+0000","last_became_active":"2023-06-23T18:20:44.464021+0000","last_became_peered":"2023-06-23T18:20:44.464021+0000","last_unstale":"2023-06-23T18:22:06.778940+0000","last_undegraded":"2023-06-23T18:22:06.778940+0000","last_fullsized":"2023-06-23T18:22:06.778940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:09.888384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:05.160742+0000","last_change":"2023-06-23T18:20:44.034309+0000","last_active":"2023-06-23T18:22:05.160742+0000","last_peered":"2023-06-23T18:22:05.160742+0000","last_clean":"2023-06-23T18:22:05.160742+0000","last_became_active":"2023-06-23T18:20:44.034007+0000","last_became_peered":"2023-06-23T18:20:44.034007+0000","last_unstale":"2023-06-23T18:22:05.160742+0000","last_undegraded":"2023-06-23T18:22:05.160742+0000","last_fullsized":"2023-06-23T18:22:05.160742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:50:17.056971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080879+0000","last_change":"2023-06-23T18:20:44.468220+0000","last_active":"2023-06-23T18:21:38.080879+0000","last_peered":"2023-06-23T18:21:38.080879+0000","last_clean":"2023-06-23T18:21:38.080879+0000","last_became_active":"2023-06-23T18:20:44.467725+0000","last_became_peered":"2023-06-23T18:20:44.467725+0000","last_unstale":"2023-06-23T18:21:38.080879+0000","last_undegraded":"2023-06-23T18:21:38.080879+0000","last_fullsized":"2023-06-23T18:21:38.080879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:20:06.380008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100500+0000","last_change":"2023-06-23T18:20:44.030921+0000","last_active":"2023-06-23T18:21:38.100500+0000","last_peered":"2023-06-23T18:21:38.100500+0000","last_clean":"2023-06-23T18:21:38.100500+0000","last_became_active":"2023-06-23T18:20:44.030362+0000","last_became_peered":"2023-06-23T18:20:44.030362+0000","last_unstale":"2023-06-23T18:21:38.100500+0000","last_undegraded":"2023-06-23T18:21:38.100500+0000","last_fullsized":"2023-06-23T18:21:38.100500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:07:01.761788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073517+0000","last_change":"2023-06-23T18:19:44.952262+0000","last_active":"2023-06-23T18:21:38.073517+0000","last_peered":"2023-06-23T18:21:38.073517+0000","last_clean":"2023-06-23T18:21:38.073517+0000","last_became_active":"2023-06-23T18:19:44.951803+0000","last_became_peered":"2023-06-23T18:19:44.951803+0000","last_unstale":"2023-06-23T18:21:38.073517+0000","last_undegraded":"2023-06-23T18:21:38.073517+0000","last_fullsized":"2023-06-23T18:21:38.073517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_clean_scrub_stamp":"2023-06-23T18:16:49.458922+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:43:25.254131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072640+0000","last_change":"2023-06-23T18:20:44.456602+0000","last_active":"2023-06-23T18:21:38.072640+0000","last_peered":"2023-06-23T18:21:38.072640+0000","last_clean":"2023-06-23T18:21:38.072640+0000","last_became_active":"2023-06-23T18:20:44.456249+0000","last_became_peered":"2023-06-23T18:20:44.456249+0000","last_unstale":"2023-06-23T18:21:38.072640+0000","last_undegraded":"2023-06-23T18:21:38.072640+0000","last_fullsized":"2023-06-23T18:21:38.072640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:50:46.341121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073728+0000","last_change":"2023-06-23T18:20:44.039100+0000","last_active":"2023-06-23T18:21:38.073728+0000","last_peered":"2023-06-23T18:21:38.073728+0000","last_clean":"2023-06-23T18:21:38.073728+0000","last_became_active":"2023-06-23T18:20:44.038801+0000","last_became_peered":"2023-06-23T18:20:44.038801+0000","last_unstale":"2023-06-23T18:21:38.073728+0000","last_undegraded":"2023-06-23T18:21:38.073728+0000","last_fullsized":"2023-06-23T18:21:38.073728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:01:20.046240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076298+0000","last_change":"2023-06-23T18:20:44.038037+0000","last_active":"2023-06-23T18:21:38.076298+0000","last_peered":"2023-06-23T18:21:38.076298+0000","last_clean":"2023-06-23T18:21:38.076298+0000","last_became_active":"2023-06-23T18:20:44.037732+0000","last_became_peered":"2023-06-23T18:20:44.037732+0000","last_unstale":"2023-06-23T18:21:38.076298+0000","last_undegraded":"2023-06-23T18:21:38.076298+0000","last_fullsized":"2023-06-23T18:21:38.076298+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:17:33.256219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073019+0000","last_change":"2023-06-23T18:20:44.459927+0000","last_active":"2023-06-23T18:21:38.073019+0000","last_peered":"2023-06-23T18:21:38.073019+0000","last_clean":"2023-06-23T18:21:38.073019+0000","last_became_active":"2023-06-23T18:20:44.459594+0000","last_became_peered":"2023-06-23T18:20:44.459594+0000","last_unstale":"2023-06-23T18:21:38.073019+0000","last_undegraded":"2023-06-23T18:21:38.073019+0000","last_fullsized":"2023-06-23T18:21:38.073019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:16:49.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075638+0000","last_change":"2023-06-23T18:20:44.039429+0000","last_active":"2023-06-23T18:21:38.075638+0000","last_peered":"2023-06-23T18:21:38.075638+0000","last_clean":"2023-06-23T18:21:38.075638+0000","last_became_active":"2023-06-23T18:20:44.038944+0000","last_became_peered":"2023-06-23T18:20:44.038944+0000","last_unstale":"2023-06-23T18:21:38.075638+0000","last_undegraded":"2023-06-23T18:21:38.075638+0000","last_fullsized":"2023-06-23T18:21:38.075638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:03.297820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080916+0000","last_change":"2023-06-23T18:20:44.458227+0000","last_active":"2023-06-23T18:21:38.080916+0000","last_peered":"2023-06-23T18:21:38.080916+0000","last_clean":"2023-06-23T18:21:38.080916+0000","last_became_active":"2023-06-23T18:20:44.457904+0000","last_became_peered":"2023-06-23T18:20:44.457904+0000","last_unstale":"2023-06-23T18:21:38.080916+0000","last_undegraded":"2023-06-23T18:21:38.080916+0000","last_fullsized":"2023-06-23T18:21:38.080916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:49:55.045014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100759+0000","last_change":"2023-06-23T18:20:44.034089+0000","last_active":"2023-06-23T18:21:38.100759+0000","last_peered":"2023-06-23T18:21:38.100759+0000","last_clean":"2023-06-23T18:21:38.100759+0000","last_became_active":"2023-06-23T18:20:44.033745+0000","last_became_peered":"2023-06-23T18:20:44.033745+0000","last_unstale":"2023-06-23T18:21:38.100759+0000","last_undegraded":"2023-06-23T18:21:38.100759+0000","last_fullsized":"2023-06-23T18:21:38.100759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:37:54.500817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.082533+0000","last_change":"2023-06-23T18:21:38.082533+0000","last_active":"2023-06-23T18:21:38.082533+0000","last_peered":"2023-06-23T18:21:38.082533+0000","last_clean":"2023-06-23T18:21:38.082533+0000","last_became_active":"2023-06-23T18:21:38.082114+0000","last_became_peered":"2023-06-23T18:21:38.082114+0000","last_unstale":"2023-06-23T18:21:38.082533+0000","last_undegraded":"2023-06-23T18:21:38.082533+0000","last_fullsized":"2023-06-23T18:21:38.082533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:03:31.751873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.078831+0000","last_change":"2023-06-23T18:20:44.030647+0000","last_active":"2023-06-23T18:21:38.078831+0000","last_peered":"2023-06-23T18:21:38.078831+0000","last_clean":"2023-06-23T18:21:38.078831+0000","last_became_active":"2023-06-23T18:20:44.030316+0000","last_became_peered":"2023-06-23T18:20:44.030316+0000","last_unstale":"2023-06-23T18:21:38.078831+0000","last_undegraded":"2023-06-23T18:21:38.078831+0000","last_fullsized":"2023-06-23T18:21:38.078831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:35:45.816400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.743921+0000","last_change":"2023-06-23T18:20:44.458365+0000","last_active":"2023-06-23T18:22:06.743921+0000","last_peered":"2023-06-23T18:22:06.743921+0000","last_clean":"2023-06-23T18:22:06.743921+0000","last_became_active":"2023-06-23T18:20:44.458039+0000","last_became_peered":"2023-06-23T18:20:44.458039+0000","last_unstale":"2023-06-23T18:22:06.743921+0000","last_undegraded":"2023-06-23T18:22:06.743921+0000","last_fullsized":"2023-06-23T18:22:06.743921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:02:29.952453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080611+0000","last_change":"2023-06-23T18:20:44.028908+0000","last_active":"2023-06-23T18:21:38.080611+0000","last_peered":"2023-06-23T18:21:38.080611+0000","last_clean":"2023-06-23T18:21:38.080611+0000","last_became_active":"2023-06-23T18:20:44.028551+0000","last_became_peered":"2023-06-23T18:20:44.028551+0000","last_unstale":"2023-06-23T18:21:38.080611+0000","last_undegraded":"2023-06-23T18:21:38.080611+0000","last_fullsized":"2023-06-23T18:21:38.080611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:55:58.763919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780847+0000","last_change":"2023-06-23T18:20:44.465300+0000","last_active":"2023-06-23T18:22:06.780847+0000","last_peered":"2023-06-23T18:22:06.780847+0000","last_clean":"2023-06-23T18:22:06.780847+0000","last_became_active":"2023-06-23T18:20:44.465008+0000","last_became_peered":"2023-06-23T18:20:44.465008+0000","last_unstale":"2023-06-23T18:22:06.780847+0000","last_undegraded":"2023-06-23T18:22:06.780847+0000","last_fullsized":"2023-06-23T18:22:06.780847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:37:20.321477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100292+0000","last_change":"2023-06-23T18:21:38.100292+0000","last_active":"2023-06-23T18:21:38.100292+0000","last_peered":"2023-06-23T18:21:38.100292+0000","last_clean":"2023-06-23T18:21:38.100292+0000","last_became_active":"2023-06-23T18:21:38.083373+0000","last_became_peered":"2023-06-23T18:21:38.083373+0000","last_unstale":"2023-06-23T18:21:38.100292+0000","last_undegraded":"2023-06-23T18:21:38.100292+0000","last_fullsized":"2023-06-23T18:21:38.100292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:44:38.453889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081058+0000","last_change":"2023-06-23T18:20:44.465127+0000","last_active":"2023-06-23T18:21:38.081058+0000","last_peered":"2023-06-23T18:21:38.081058+0000","last_clean":"2023-06-23T18:21:38.081058+0000","last_became_active":"2023-06-23T18:20:44.464768+0000","last_became_peered":"2023-06-23T18:20:44.464768+0000","last_unstale":"2023-06-23T18:21:38.081058+0000","last_undegraded":"2023-06-23T18:21:38.081058+0000","last_fullsized":"2023-06-23T18:21:38.081058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:07:36.805439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075973+0000","last_change":"2023-06-23T18:20:44.046245+0000","last_active":"2023-06-23T18:21:38.075973+0000","last_peered":"2023-06-23T18:21:38.075973+0000","last_clean":"2023-06-23T18:21:38.075973+0000","last_became_active":"2023-06-23T18:20:44.045945+0000","last_became_peered":"2023-06-23T18:20:44.045945+0000","last_unstale":"2023-06-23T18:21:38.075973+0000","last_undegraded":"2023-06-23T18:21:38.075973+0000","last_fullsized":"2023-06-23T18:21:38.075973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:53:58.962783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789675+0000","last_change":"2023-06-23T18:20:44.453520+0000","last_active":"2023-06-23T18:22:06.789675+0000","last_peered":"2023-06-23T18:22:06.789675+0000","last_clean":"2023-06-23T18:22:06.789675+0000","last_became_active":"2023-06-23T18:20:44.453214+0000","last_became_peered":"2023-06-23T18:20:44.453214+0000","last_unstale":"2023-06-23T18:22:06.789675+0000","last_undegraded":"2023-06-23T18:22:06.789675+0000","last_fullsized":"2023-06-23T18:22:06.789675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:40:14.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080295+0000","last_change":"2023-06-23T18:20:44.042526+0000","last_active":"2023-06-23T18:21:38.080295+0000","last_peered":"2023-06-23T18:21:38.080295+0000","last_clean":"2023-06-23T18:21:38.080295+0000","last_became_active":"2023-06-23T18:20:44.042249+0000","last_became_peered":"2023-06-23T18:20:44.042249+0000","last_unstale":"2023-06-23T18:21:38.080295+0000","last_undegraded":"2023-06-23T18:21:38.080295+0000","last_fullsized":"2023-06-23T18:21:38.080295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:59:17.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790121+0000","last_change":"2023-06-23T18:20:44.454645+0000","last_active":"2023-06-23T18:22:06.790121+0000","last_peered":"2023-06-23T18:22:06.790121+0000","last_clean":"2023-06-23T18:22:06.790121+0000","last_became_active":"2023-06-23T18:20:44.453309+0000","last_became_peered":"2023-06-23T18:20:44.453309+0000","last_unstale":"2023-06-23T18:22:06.790121+0000","last_undegraded":"2023-06-23T18:22:06.790121+0000","last_fullsized":"2023-06-23T18:22:06.790121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:15:24.080996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080976+0000","last_change":"2023-06-23T18:20:44.037681+0000","last_active":"2023-06-23T18:21:38.080976+0000","last_peered":"2023-06-23T18:21:38.080976+0000","last_clean":"2023-06-23T18:21:38.080976+0000","last_became_active":"2023-06-23T18:20:44.037323+0000","last_became_peered":"2023-06-23T18:20:44.037323+0000","last_unstale":"2023-06-23T18:21:38.080976+0000","last_undegraded":"2023-06-23T18:21:38.080976+0000","last_fullsized":"2023-06-23T18:21:38.080976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:49:34.898906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.787192+0000","last_change":"2023-06-23T18:20:44.456434+0000","last_active":"2023-06-23T18:22:06.787192+0000","last_peered":"2023-06-23T18:22:06.787192+0000","last_clean":"2023-06-23T18:22:06.787192+0000","last_became_active":"2023-06-23T18:20:44.456110+0000","last_became_peered":"2023-06-23T18:20:44.456110+0000","last_unstale":"2023-06-23T18:22:06.787192+0000","last_undegraded":"2023-06-23T18:22:06.787192+0000","last_fullsized":"2023-06-23T18:22:06.787192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:38:39.438073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073300+0000","last_change":"2023-06-23T18:20:44.040722+0000","last_active":"2023-06-23T18:21:38.073300+0000","last_peered":"2023-06-23T18:21:38.073300+0000","last_clean":"2023-06-23T18:21:38.073300+0000","last_became_active":"2023-06-23T18:20:44.040390+0000","last_became_peered":"2023-06-23T18:20:44.040390+0000","last_unstale":"2023-06-23T18:21:38.073300+0000","last_undegraded":"2023-06-23T18:21:38.073300+0000","last_fullsized":"2023-06-23T18:21:38.073300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:49:31.045068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792445+0000","last_change":"2023-06-23T18:20:44.455559+0000","last_active":"2023-06-23T18:22:06.792445+0000","last_peered":"2023-06-23T18:22:06.792445+0000","last_clean":"2023-06-23T18:22:06.792445+0000","last_became_active":"2023-06-23T18:20:44.455256+0000","last_became_peered":"2023-06-23T18:20:44.455256+0000","last_unstale":"2023-06-23T18:22:06.792445+0000","last_undegraded":"2023-06-23T18:22:06.792445+0000","last_fullsized":"2023-06-23T18:22:06.792445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:43:47.289820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.779277+0000","last_change":"2023-06-23T18:20:44.460403+0000","last_active":"2023-06-23T18:22:06.779277+0000","last_peered":"2023-06-23T18:22:06.779277+0000","last_clean":"2023-06-23T18:22:06.779277+0000","last_became_active":"2023-06-23T18:20:44.459928+0000","last_became_peered":"2023-06-23T18:20:44.459928+0000","last_unstale":"2023-06-23T18:22:06.779277+0000","last_undegraded":"2023-06-23T18:22:06.779277+0000","last_fullsized":"2023-06-23T18:22:06.779277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:37:08.283159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780411+0000","last_change":"2023-06-23T18:20:44.458050+0000","last_active":"2023-06-23T18:22:06.780411+0000","last_peered":"2023-06-23T18:22:06.780411+0000","last_clean":"2023-06-23T18:22:06.780411+0000","last_became_active":"2023-06-23T18:20:44.457733+0000","last_became_peered":"2023-06-23T18:20:44.457733+0000","last_unstale":"2023-06-23T18:22:06.780411+0000","last_undegraded":"2023-06-23T18:22:06.780411+0000","last_fullsized":"2023-06-23T18:22:06.780411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:43:34.360833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.776703+0000","last_change":"2023-06-23T18:20:44.456233+0000","last_active":"2023-06-23T18:22:06.776703+0000","last_peered":"2023-06-23T18:22:06.776703+0000","last_clean":"2023-06-23T18:22:06.776703+0000","last_became_active":"2023-06-23T18:20:44.455677+0000","last_became_peered":"2023-06-23T18:20:44.455677+0000","last_unstale":"2023-06-23T18:22:06.776703+0000","last_undegraded":"2023-06-23T18:22:06.776703+0000","last_fullsized":"2023-06-23T18:22:06.776703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:30:25.029586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080153+0000","last_change":"2023-06-23T18:20:44.469668+0000","last_active":"2023-06-23T18:21:38.080153+0000","last_peered":"2023-06-23T18:21:38.080153+0000","last_clean":"2023-06-23T18:21:38.080153+0000","last_became_active":"2023-06-23T18:20:44.466563+0000","last_became_peered":"2023-06-23T18:20:44.466563+0000","last_unstale":"2023-06-23T18:21:38.080153+0000","last_undegraded":"2023-06-23T18:21:38.080153+0000","last_fullsized":"2023-06-23T18:21:38.080153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:05:04.024545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073109+0000","last_change":"2023-06-23T18:20:44.456021+0000","last_active":"2023-06-23T18:21:38.073109+0000","last_peered":"2023-06-23T18:21:38.073109+0000","last_clean":"2023-06-23T18:21:38.073109+0000","last_became_active":"2023-06-23T18:20:44.455705+0000","last_became_peered":"2023-06-23T18:20:44.455705+0000","last_unstale":"2023-06-23T18:21:38.073109+0000","last_undegraded":"2023-06-23T18:21:38.073109+0000","last_fullsized":"2023-06-23T18:21:38.073109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:50:09.933725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.775420+0000","last_change":"2023-06-23T18:20:44.460359+0000","last_active":"2023-06-23T18:22:06.775420+0000","last_peered":"2023-06-23T18:22:06.775420+0000","last_clean":"2023-06-23T18:22:06.775420+0000","last_became_active":"2023-06-23T18:20:44.460034+0000","last_became_peered":"2023-06-23T18:20:44.460034+0000","last_unstale":"2023-06-23T18:22:06.775420+0000","last_undegraded":"2023-06-23T18:22:06.775420+0000","last_fullsized":"2023-06-23T18:22:06.775420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:39.654726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.786166+0000","last_change":"2023-06-23T18:20:44.455743+0000","last_active":"2023-06-23T18:22:06.786166+0000","last_peered":"2023-06-23T18:22:06.786166+0000","last_clean":"2023-06-23T18:22:06.786166+0000","last_became_active":"2023-06-23T18:20:44.455266+0000","last_became_peered":"2023-06-23T18:20:44.455266+0000","last_unstale":"2023-06-23T18:22:06.786166+0000","last_undegraded":"2023-06-23T18:22:06.786166+0000","last_fullsized":"2023-06-23T18:22:06.786166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:51:16.923982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.783014+0000","last_change":"2023-06-23T18:21:38.101164+0000","last_active":"2023-06-23T18:22:06.783014+0000","last_peered":"2023-06-23T18:22:06.783014+0000","last_clean":"2023-06-23T18:22:06.783014+0000","last_became_active":"2023-06-23T18:21:38.100818+0000","last_became_peered":"2023-06-23T18:21:38.100818+0000","last_unstale":"2023-06-23T18:22:06.783014+0000","last_undegraded":"2023-06-23T18:22:06.783014+0000","last_fullsized":"2023-06-23T18:22:06.783014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:54:39.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.777099+0000","last_change":"2023-06-23T18:20:44.465192+0000","last_active":"2023-06-23T18:22:06.777099+0000","last_peered":"2023-06-23T18:22:06.777099+0000","last_clean":"2023-06-23T18:22:06.777099+0000","last_became_active":"2023-06-23T18:20:44.464905+0000","last_became_peered":"2023-06-23T18:20:44.464905+0000","last_unstale":"2023-06-23T18:22:06.777099+0000","last_undegraded":"2023-06-23T18:22:06.777099+0000","last_fullsized":"2023-06-23T18:22:06.777099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:32:18.229559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080712+0000","last_change":"2023-06-23T18:20:44.458725+0000","last_active":"2023-06-23T18:21:38.080712+0000","last_peered":"2023-06-23T18:21:38.080712+0000","last_clean":"2023-06-23T18:21:38.080712+0000","last_became_active":"2023-06-23T18:20:44.458414+0000","last_became_peered":"2023-06-23T18:20:44.458414+0000","last_unstale":"2023-06-23T18:21:38.080712+0000","last_undegraded":"2023-06-23T18:21:38.080712+0000","last_fullsized":"2023-06-23T18:21:38.080712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:34:31.245389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792579+0000","last_change":"2023-06-23T18:20:44.465665+0000","last_active":"2023-06-23T18:22:06.792579+0000","last_peered":"2023-06-23T18:22:06.792579+0000","last_clean":"2023-06-23T18:22:06.792579+0000","last_became_active":"2023-06-23T18:20:44.465337+0000","last_became_peered":"2023-06-23T18:20:44.465337+0000","last_unstale":"2023-06-23T18:22:06.792579+0000","last_undegraded":"2023-06-23T18:22:06.792579+0000","last_fullsized":"2023-06-23T18:22:06.792579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:05:22.890126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072919+0000","last_change":"2023-06-23T18:20:44.455107+0000","last_active":"2023-06-23T18:21:38.072919+0000","last_peered":"2023-06-23T18:21:38.072919+0000","last_clean":"2023-06-23T18:21:38.072919+0000","last_became_active":"2023-06-23T18:20:44.454798+0000","last_became_peered":"2023-06-23T18:20:44.454798+0000","last_unstale":"2023-06-23T18:21:38.072919+0000","last_undegraded":"2023-06-23T18:21:38.072919+0000","last_fullsized":"2023-06-23T18:21:38.072919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:19:46.878268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780928+0000","last_change":"2023-06-23T18:20:44.461044+0000","last_active":"2023-06-23T18:22:06.780928+0000","last_peered":"2023-06-23T18:22:06.780928+0000","last_clean":"2023-06-23T18:22:06.780928+0000","last_became_active":"2023-06-23T18:20:44.460737+0000","last_became_peered":"2023-06-23T18:20:44.460737+0000","last_unstale":"2023-06-23T18:22:06.780928+0000","last_undegraded":"2023-06-23T18:22:06.780928+0000","last_fullsized":"2023-06-23T18:22:06.780928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:22:32.348419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":358,"ondisk_log_size":358,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281641,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63241,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":492,"num_read_kb":311,"num_write":162,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":57610,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":188757,"data_compressed":8925,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":437933,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.76000000000000001,"15min":0.80000000000000004},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.624,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.54700000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.873,"15min":0.873},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.48599999999999999},"min":{"1min":0.35399999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.371,"5min":0.317,"15min":0.317},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.54700000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83899999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.873,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.873}]},{"osd":6,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6580000000000004,"5min":5.6580000000000004,"15min":5.6580000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95899999999999996,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70499999999999996,"15min":0.69399999999999995},"min":{"1min":0.61499999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0069999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.82099999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.87,"5min":1.369,"15min":1.369},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.66900000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95399999999999996,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.71499999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.63100000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98599999999999999,"5min":1.579,"15min":1.579},"last":0.89600000000000002}]}]},{"osd":10,"up_from":70,"seq":300647710753,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1088504,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":11.565,"5min":11.565,"15min":11.565},"last":2.0329999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.298,"5min":4.298,"15min":4.298},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76500000000000001,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82999999999999996,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.745},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80400000000000005,"5min":2.782,"15min":2.782},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71699999999999997,"15min":0.73499999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.73199999999999998,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.55100000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.58899999999999997,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90700000000000003,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.89200000000000002,"15min":0.85799999999999998},"min":{"1min":0.51700000000000002,"5min":0.501,"15min":0.501},"max":{"1min":7.2240000000000002,"5min":7.2240000000000002,"15min":7.2240000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.79600000000000004,"15min":0.80900000000000005},"min":{"1min":0.54700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89800000000000002,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84599999999999997,"15min":0.85299999999999998},"min":{"1min":0.61799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93799999999999994,"5min":1.782,"15min":1.782},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86899999999999999,"15min":0.88600000000000001},"min":{"1min":0.61899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94399999999999995,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.88100000000000001,"5min":1.645,"15min":1.645},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.90400000000000003,"15min":0.92600000000000005},"min":{"1min":0.64200000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91900000000000004,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.66300000000000003}]},{"osd":11,"last update":"Fri Jun 23 18:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.90800000000000003},"min":{"1min":0.51500000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.97699999999999998,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96299999999999997,"5min":1.754,"15min":1.754},"last":0.85699999999999998}]}]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":438057,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.247},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44600000000000001,"15min":0.45200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.32800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.67100000000000004,"15min":0.69499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.72999999999999998,"15min":0.753},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.57399999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.309},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.622},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.75,"15min":0.77500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83899999999999997,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.497}]},{"osd":5,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.70599999999999996,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.373},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.034,"5min":1.04,"15min":1.04},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0669999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":1.397,"15min":1.397},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.81599999999999995,"15min":0.83399999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0960000000000001,"5min":1.661,"15min":1.661},"last":0.66500000000000004}]},{"osd":8,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0349999999999999,"5min":1.079,"15min":1.079},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.93000000000000005,"15min":0.97299999999999998},"min":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.91900000000000004,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.434}]},{"osd":10,"last update":"Fri Jun 23 18:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0029999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":437534,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.42099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.501,"15min":0.49399999999999999},"min":{"1min":0.39400000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.68500000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.47399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45600000000000002,"15min":0.45100000000000001},"min":{"1min":0.35999999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63800000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.25700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.745,"5min":0.752,"15min":0.752},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.42899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76200000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.62}]},{"osd":4,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.52600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85699999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.86899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.61399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83199999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.89400000000000002}]},{"osd":10,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.56000000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.55300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":0.998,"15min":0.998},"last":0.92500000000000004}]},{"osd":11,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.94799999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":1029957,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61399999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63800000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.48999999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84599999999999997,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84999999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81299999999999994,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.73399999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.379}]},{"osd":5,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65600000000000003,"15min":0.63500000000000001},"min":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80200000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66500000000000004,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.877,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.65000000000000002},"min":{"1min":0.49299999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89400000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.752},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89200000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.71999999999999997},"min":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92100000000000004,"5min":1.024,"15min":1.024},"last":0.55300000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.93999999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.79800000000000004}]},{"osd":10,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94999999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.627,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93600000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":11,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73099999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495665,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":437452,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.58899999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.28699999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.379}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.89400000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70799999999999996,"5min":1.105,"15min":1.105},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65400000000000003,"5min":1.141,"15min":1.141},"last":0.432},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72199999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.872,"15min":0.872},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77200000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68300000000000005,"15min":0.69699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.82999999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91800000000000004,"5min":1.03,"15min":1.03},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97199999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58299999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69199999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96799999999999997,"5min":1.008,"15min":1.008},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71699999999999997,"15min":0.73599999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.47699999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.878,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":495469,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.48499999999999999},"min":{"1min":0.34799999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.84999999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.625},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52500000000000002,"15min":0.498},"min":{"1min":0.35699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73999999999999999,"5min":1.218,"15min":1.218},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.623,"15min":0.54900000000000004},"min":{"1min":0.436,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":1.25,"15min":1.25},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.91300000000000003,"5min":1.343,"15min":1.343},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.42799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.93200000000000005,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.49099999999999999,"15min":0.44},"min":{"1min":0.32300000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.42399999999999999},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77000000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.66500000000000004,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98399999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64200000000000002,"15min":0.60399999999999998},"min":{"1min":0.41799999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.96799999999999997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84599999999999997,"5min":1.169,"15min":1.169},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.60099999999999998},"min":{"1min":0.53200000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.093,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.754}]},{"osd":7,"last update":"Fri Jun 23 18:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.72799999999999998,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.80100000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.82299999999999995,"15min":0.80300000000000005},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0820000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98899999999999999,"5min":1.319,"15min":1.319},"last":0.77300000000000002}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.876,"15min":0.88900000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1180000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96399999999999997,"5min":1.288,"15min":1.288},"last":0.81499999999999995}]},{"osd":10,"last update":"Fri Jun 23 18:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.85599999999999998,"15min":0.86499999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.046,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.93400000000000005}]},{"osd":11,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":438717,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.48099999999999998},"min":{"1min":0.39200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51400000000000001,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.503}]},{"osd":2,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56100000000000005,"15min":0.50700000000000001},"min":{"1min":0.39200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.84999999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.372},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.55000000000000004},"min":{"1min":0.439,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.55400000000000005},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.64100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80800000000000005,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.443},{"interface":"front","average":{"1min":0.501,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82499999999999996,"5min":1.073,"15min":1.073},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Jun 23 18:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56399999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.876,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.76400000000000001,"15min":0.79000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0449999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73699999999999999,"15min":0.748},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.83899999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63800000000000001}]},{"osd":11,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69399999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":439024,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.36399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67300000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.38300000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56799999999999995,"15min":0.53000000000000003},"min":{"1min":0.52300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.47599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:22:00 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.34300000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.52200000000000002,"15min":0.48199999999999998},"min":{"1min":0.40300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0069999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59799999999999998}]},{"osd":8,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90500000000000003,"5min":1.095,"15min":1.095},"last":0.71299999999999997}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.621},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63300000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.71299999999999997,"15min":0.69199999999999995},"min":{"1min":0.58199999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79400000000000004}]},{"osd":11,"last update":"Fri Jun 23 18:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.745}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":437769,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.502,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":12.246,"5min":12.246,"15min":12.246},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58099999999999996,"15min":0.54200000000000004},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":8.3569999999999993,"5min":8.3569999999999993,"15min":8.3569999999999993},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.59099999999999997},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:21:26 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.70699999999999996,"15min":0.59799999999999998},"min":{"1min":0.377,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":12.183999999999999,"5min":12.183999999999999,"15min":12.183999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.59099999999999997,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.443}]},{"osd":7,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.69199999999999995,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3239999999999998,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.501},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.48299999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79600000000000004,"15min":0.75600000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":8.2650000000000006,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2649999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":17.315999999999999,"5min":17.315999999999999,"15min":17.315999999999999},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":497079,"data_compressed":14732,"data_compressed_allocated":20480,"data_compressed_original":69632,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,24,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.51700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40799999999999997,"15min":0.378},"min":{"1min":0.309,"5min":0.255,"15min":0.255},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":1.6459999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41699999999999998,"15min":0.38900000000000001},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60999999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":4.6230000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60599999999999998,"15min":0.55200000000000005},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81599999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.52800000000000002},"min":{"1min":0.40500000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":4.6520000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.629}]},{"osd":5,"last update":"Fri Jun 23 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63800000000000001,"15min":0.60699999999999998},"min":{"1min":0.55200000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.63400000000000001,"15min":0.60599999999999998},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":4.5309999999999997}]},{"osd":6,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":1.129,"15min":1.129},"last":1.6299999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":1.7150000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.497,"5min":0.378,"15min":0.378},"max":{"1min":0.87,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":7.2290000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.623,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97599999999999998,"5min":0.998,"15min":0.998},"last":4.694}]},{"osd":8,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.503,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94899999999999995,"5min":1.202,"15min":1.202},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.79600000000000004,"15min":0.77200000000000002},"min":{"1min":0.65200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":4.585}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":3.6360000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":3.8889999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77500000000000002,"15min":0.76000000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.022,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":11.475},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.754,"15min":0.74299999999999999},"min":{"1min":0.65800000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96599999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":4.718}]},{"osd":11,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":1.6830000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71999999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":1029641,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47999999999999998,"15min":0.46100000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.747,"5min":0.875,"15min":0.875},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.73199999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.372}]},{"osd":1,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54800000000000004,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81299999999999994,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.39600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78000000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.437,"15min":0.41699999999999998},"min":{"1min":0.34799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.44900000000000001,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.44900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85799999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":5.399},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":5.4690000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88800000000000001,"5min":0.998,"15min":0.998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.629},"min":{"1min":0.505,"5min":0.374,"15min":0.374},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53600000000000003}]},{"osd":9,"last update":"Fri Jun 23 18:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.60499999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.74299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":59840,"data_compressed":8925,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:22:11.329 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:22:11.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:11 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2930004080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:11.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:11 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/391789535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:11.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:11 smithi082 ceph-mon[97889]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2023-06-23T18:22:11.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:11 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3132778017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:11.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:11 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2930004080' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:11.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:11 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/391789535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:11.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:11 smithi017 ceph-mon[97472]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2023-06-23T18:22:11.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:11 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3132778017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:11.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 -- 172.21.15.2:0/938921876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27601005e0 msgr2=0x7f2760106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 --2- 172.21.15.2:0/938921876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27601005e0 0x7f2760106ea0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f275800b0a0 tx=0x7f275800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 -- 172.21.15.2:0/938921876 shutdown_connections 2023-06-23T18:22:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 --2- 172.21.15.2:0/938921876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27601005e0 0x7f2760106ea0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 --2- 172.21.15.2:0/938921876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27600fea60 0x7f27600fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 --2- 172.21.15.2:0/938921876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27600ffc50 0x7f27601000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 -- 172.21.15.2:0/938921876 >> 172.21.15.2:0/938921876 conn(0x7f27600fa0e0 msgr2=0x7f27600fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 -- 172.21.15.2:0/938921876 shutdown_connections 2023-06-23T18:22:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.481+0000 7f276490c700 1 -- 172.21.15.2:0/938921876 wait complete. 2023-06-23T18:22:11.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.482+0000 7f276490c700 1 Processor -- start 2023-06-23T18:22:11.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.482+0000 7f276490c700 1 -- start start 2023-06-23T18:22:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.482+0000 7f276490c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27600fea60 0x7f2760068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.482+0000 7f276490c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ffc50 0x7f276019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.482+0000 7f276490c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27601005e0 0x7f276019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.482+0000 7f276490c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276019d3a0 con 0x7f27600ffc50 2023-06-23T18:22:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.482+0000 7f276490c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276019d510 con 0x7f27600fea60 2023-06-23T18:22:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.482+0000 7f276490c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276019d6d0 con 0x7f27601005e0 2023-06-23T18:22:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ffc50 0x7f276019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ffc50 0x7f276019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53896/0 (socket says 172.21.15.2:53896) 2023-06-23T18:22:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 -- 172.21.15.2:0/2308312598 learned_addr learned my addr 172.21.15.2:0/2308312598 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275e59c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27600fea60 0x7f2760068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275ed9d700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27601005e0 0x7f276019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 -- 172.21.15.2:0/2308312598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27601005e0 msgr2=0x7f276019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27601005e0 0x7f276019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 -- 172.21.15.2:0/2308312598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27600fea60 msgr2=0x7f2760068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27600fea60 0x7f2760068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 -- 172.21.15.2:0/2308312598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2758009d00 con 0x7f27600ffc50 2023-06-23T18:22:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f275dd9b700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ffc50 0x7f276019a570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f274800e9a0 tx=0x7f274800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f27577fe700 1 -- 172.21.15.2:0/2308312598 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748009790 con 0x7f27600ffc50 2023-06-23T18:22:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f27577fe700 1 -- 172.21.15.2:0/2308312598 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f274800cbd0 con 0x7f27600ffc50 2023-06-23T18:22:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f27577fe700 1 -- 172.21.15.2:0/2308312598 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748018680 con 0x7f27600ffc50 2023-06-23T18:22:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276019d9b0 con 0x7f27600ffc50 2023-06-23T18:22:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.483+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276019de70 con 0x7f27600ffc50 2023-06-23T18:22:11.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.484+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2760105ff0 con 0x7f27600ffc50 2023-06-23T18:22:11.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.485+0000 7f27577fe700 1 -- 172.21.15.2:0/2308312598 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f274800cd40 con 0x7f27600ffc50 2023-06-23T18:22:11.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.486+0000 7f27577fe700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f274c0a1150 0x7f274c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.486+0000 7f275e59c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f274c0a1150 0x7f274c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.486+0000 7f27577fe700 1 -- 172.21.15.2:0/2308312598 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2748014070 con 0x7f27600ffc50 2023-06-23T18:22:11.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.487+0000 7f275e59c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f274c0a1150 0x7f274c0a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f27600ffab0 tx=0x7f2750007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:11.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.490+0000 7f27577fe700 1 -- 172.21.15.2:0/2308312598 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2748096640 con 0x7f27600ffc50 2023-06-23T18:22:11.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.687+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2760060910 con 0x7f274c0a1150 2023-06-23T18:22:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.699+0000 7f27577fe700 1 -- 172.21.15.2:0/2308312598 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344209 (secure 0 0 0) 0x7f2760060910 con 0x7f274c0a1150 2023-06-23T18:22:11.707 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:11.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f274c0a1150 msgr2=0x7f274c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f274c0a1150 0x7f274c0a3600 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f27600ffab0 tx=0x7f2750007400 comp rx=0 tx=0).stop 2023-06-23T18:22:11.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ffc50 msgr2=0x7f276019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ffc50 0x7f276019a570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f274800e9a0 tx=0x7f274800ecb0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 shutdown_connections 2023-06-23T18:22:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27601005e0 0x7f276019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f274c0a1150 0x7f274c0a3600 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f27600fea60 0x7f2760068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 --2- 172.21.15.2:0/2308312598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f27600ffc50 0x7f276019a570 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 >> 172.21.15.2:0/2308312598 conn(0x7f27600fa0e0 msgr2=0x7f2760103130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 shutdown_connections 2023-06-23T18:22:11.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.703+0000 7f276490c700 1 -- 172.21.15.2:0/2308312598 wait complete. 2023-06-23T18:22:11.711 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-06-23T18:22:11.720 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-06-23T18:22:10.757253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653855,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":618,"num_read_kb":420,"num_write":345,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1238346,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":637,"ondisk_log_size":637,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334516,"kb_used_data":11092,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607308,"statfs":{"total":1151940427776,"available":1151597883392,"internally_reserved":0,"allocated":11358208,"data_stored":7207136,"data_compressed":186066,"data_compressed_allocated":245760,"data_compressed_original":2019328,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,24,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1839,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":33,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006963"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785637+0000","last_change":"2023-06-23T18:20:44.466650+0000","last_active":"2023-06-23T18:22:06.785637+0000","last_peered":"2023-06-23T18:22:06.785637+0000","last_clean":"2023-06-23T18:22:06.785637+0000","last_became_active":"2023-06-23T18:20:44.466425+0000","last_became_peered":"2023-06-23T18:20:44.466425+0000","last_unstale":"2023-06-23T18:22:06.785637+0000","last_undegraded":"2023-06-23T18:22:06.785637+0000","last_fullsized":"2023-06-23T18:22:06.785637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:30:41.271809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080017+0000","last_change":"2023-06-23T18:20:44.465762+0000","last_active":"2023-06-23T18:21:38.080017+0000","last_peered":"2023-06-23T18:21:38.080017+0000","last_clean":"2023-06-23T18:21:38.080017+0000","last_became_active":"2023-06-23T18:20:44.463592+0000","last_became_peered":"2023-06-23T18:20:44.463592+0000","last_unstale":"2023-06-23T18:21:38.080017+0000","last_undegraded":"2023-06-23T18:21:38.080017+0000","last_fullsized":"2023-06-23T18:21:38.080017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:04:03.928905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079784+0000","last_change":"2023-06-23T18:20:44.464162+0000","last_active":"2023-06-23T18:21:38.079784+0000","last_peered":"2023-06-23T18:21:38.079784+0000","last_clean":"2023-06-23T18:21:38.079784+0000","last_became_active":"2023-06-23T18:20:44.463854+0000","last_became_peered":"2023-06-23T18:20:44.463854+0000","last_unstale":"2023-06-23T18:21:38.079784+0000","last_undegraded":"2023-06-23T18:21:38.079784+0000","last_fullsized":"2023-06-23T18:21:38.079784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:52:10.241558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.777090+0000","last_change":"2023-06-23T18:20:44.462923+0000","last_active":"2023-06-23T18:22:06.777090+0000","last_peered":"2023-06-23T18:22:06.777090+0000","last_clean":"2023-06-23T18:22:06.777090+0000","last_became_active":"2023-06-23T18:20:44.462511+0000","last_became_peered":"2023-06-23T18:20:44.462511+0000","last_unstale":"2023-06-23T18:22:06.777090+0000","last_undegraded":"2023-06-23T18:22:06.777090+0000","last_fullsized":"2023-06-23T18:22:06.777090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:41:07.780308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.775980+0000","last_change":"2023-06-23T18:20:44.454727+0000","last_active":"2023-06-23T18:22:06.775980+0000","last_peered":"2023-06-23T18:22:06.775980+0000","last_clean":"2023-06-23T18:22:06.775980+0000","last_became_active":"2023-06-23T18:20:44.454418+0000","last_became_peered":"2023-06-23T18:20:44.454418+0000","last_unstale":"2023-06-23T18:22:06.775980+0000","last_undegraded":"2023-06-23T18:22:06.775980+0000","last_fullsized":"2023-06-23T18:22:06.775980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:06:57.921477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790102+0000","last_change":"2023-06-23T18:20:44.460636+0000","last_active":"2023-06-23T18:22:06.790102+0000","last_peered":"2023-06-23T18:22:06.790102+0000","last_clean":"2023-06-23T18:22:06.790102+0000","last_became_active":"2023-06-23T18:20:44.460305+0000","last_became_peered":"2023-06-23T18:20:44.460305+0000","last_unstale":"2023-06-23T18:22:06.790102+0000","last_undegraded":"2023-06-23T18:22:06.790102+0000","last_fullsized":"2023-06-23T18:22:06.790102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:08:27.871589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785720+0000","last_change":"2023-06-23T18:20:44.463552+0000","last_active":"2023-06-23T18:22:06.785720+0000","last_peered":"2023-06-23T18:22:06.785720+0000","last_clean":"2023-06-23T18:22:06.785720+0000","last_became_active":"2023-06-23T18:20:44.463187+0000","last_became_peered":"2023-06-23T18:20:44.463187+0000","last_unstale":"2023-06-23T18:22:06.785720+0000","last_undegraded":"2023-06-23T18:22:06.785720+0000","last_fullsized":"2023-06-23T18:22:06.785720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:16:36.563197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790515+0000","last_change":"2023-06-23T18:21:38.083209+0000","last_active":"2023-06-23T18:22:06.790515+0000","last_peered":"2023-06-23T18:22:06.790515+0000","last_clean":"2023-06-23T18:22:06.790515+0000","last_became_active":"2023-06-23T18:21:38.082912+0000","last_became_peered":"2023-06-23T18:21:38.082912+0000","last_unstale":"2023-06-23T18:22:06.790515+0000","last_undegraded":"2023-06-23T18:22:06.790515+0000","last_fullsized":"2023-06-23T18:22:06.790515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:54:52.291671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778393+0000","last_change":"2023-06-23T18:20:44.461199+0000","last_active":"2023-06-23T18:22:06.778393+0000","last_peered":"2023-06-23T18:22:06.778393+0000","last_clean":"2023-06-23T18:22:06.778393+0000","last_became_active":"2023-06-23T18:20:44.460891+0000","last_became_peered":"2023-06-23T18:20:44.460891+0000","last_unstale":"2023-06-23T18:22:06.778393+0000","last_undegraded":"2023-06-23T18:22:06.778393+0000","last_fullsized":"2023-06-23T18:22:06.778393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:35.270311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.791756+0000","last_change":"2023-06-23T18:21:38.084799+0000","last_active":"2023-06-23T18:22:06.791756+0000","last_peered":"2023-06-23T18:22:06.791756+0000","last_clean":"2023-06-23T18:22:06.791756+0000","last_became_active":"2023-06-23T18:21:38.084336+0000","last_became_peered":"2023-06-23T18:21:38.084336+0000","last_unstale":"2023-06-23T18:22:06.791756+0000","last_undegraded":"2023-06-23T18:22:06.791756+0000","last_fullsized":"2023-06-23T18:22:06.791756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:09:38.488565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789987+0000","last_change":"2023-06-23T18:20:44.453337+0000","last_active":"2023-06-23T18:22:06.789987+0000","last_peered":"2023-06-23T18:22:06.789987+0000","last_clean":"2023-06-23T18:22:06.789987+0000","last_became_active":"2023-06-23T18:20:44.452925+0000","last_became_peered":"2023-06-23T18:20:44.452925+0000","last_unstale":"2023-06-23T18:22:06.789987+0000","last_undegraded":"2023-06-23T18:22:06.789987+0000","last_fullsized":"2023-06-23T18:22:06.789987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:39:25.750095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.774400+0000","last_change":"2023-06-23T18:20:44.456710+0000","last_active":"2023-06-23T18:22:06.774400+0000","last_peered":"2023-06-23T18:22:06.774400+0000","last_clean":"2023-06-23T18:22:06.774400+0000","last_became_active":"2023-06-23T18:20:44.456411+0000","last_became_peered":"2023-06-23T18:20:44.456411+0000","last_unstale":"2023-06-23T18:22:06.774400+0000","last_undegraded":"2023-06-23T18:22:06.774400+0000","last_fullsized":"2023-06-23T18:22:06.774400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:42:13.109612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072919+0000","last_change":"2023-06-23T18:20:44.453586+0000","last_active":"2023-06-23T18:21:38.072919+0000","last_peered":"2023-06-23T18:21:38.072919+0000","last_clean":"2023-06-23T18:21:38.072919+0000","last_became_active":"2023-06-23T18:20:44.453239+0000","last_became_peered":"2023-06-23T18:20:44.453239+0000","last_unstale":"2023-06-23T18:21:38.072919+0000","last_undegraded":"2023-06-23T18:21:38.072919+0000","last_fullsized":"2023-06-23T18:21:38.072919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:36:37.353139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079914+0000","last_change":"2023-06-23T18:20:44.465627+0000","last_active":"2023-06-23T18:21:38.079914+0000","last_peered":"2023-06-23T18:21:38.079914+0000","last_clean":"2023-06-23T18:21:38.079914+0000","last_became_active":"2023-06-23T18:20:44.464568+0000","last_became_peered":"2023-06-23T18:20:44.464568+0000","last_unstale":"2023-06-23T18:21:38.079914+0000","last_undegraded":"2023-06-23T18:21:38.079914+0000","last_fullsized":"2023-06-23T18:21:38.079914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:15:39.010793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079452+0000","last_change":"2023-06-23T18:20:44.464169+0000","last_active":"2023-06-23T18:21:38.079452+0000","last_peered":"2023-06-23T18:21:38.079452+0000","last_clean":"2023-06-23T18:21:38.079452+0000","last_became_active":"2023-06-23T18:20:44.463867+0000","last_became_peered":"2023-06-23T18:20:44.463867+0000","last_unstale":"2023-06-23T18:21:38.079452+0000","last_undegraded":"2023-06-23T18:21:38.079452+0000","last_fullsized":"2023-06-23T18:21:38.079452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:12:31.642855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792291+0000","last_change":"2023-06-23T18:20:44.467101+0000","last_active":"2023-06-23T18:22:06.792291+0000","last_peered":"2023-06-23T18:22:06.792291+0000","last_clean":"2023-06-23T18:22:06.792291+0000","last_became_active":"2023-06-23T18:20:44.466788+0000","last_became_peered":"2023-06-23T18:20:44.466788+0000","last_unstale":"2023-06-23T18:22:06.792291+0000","last_undegraded":"2023-06-23T18:22:06.792291+0000","last_fullsized":"2023-06-23T18:22:06.792291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:41:04.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785439+0000","last_change":"2023-06-23T18:20:44.466055+0000","last_active":"2023-06-23T18:22:06.785439+0000","last_peered":"2023-06-23T18:22:06.785439+0000","last_clean":"2023-06-23T18:22:06.785439+0000","last_became_active":"2023-06-23T18:20:44.465608+0000","last_became_peered":"2023-06-23T18:20:44.465608+0000","last_unstale":"2023-06-23T18:22:06.785439+0000","last_undegraded":"2023-06-23T18:22:06.785439+0000","last_fullsized":"2023-06-23T18:22:06.785439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:16:31.985669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790712+0000","last_change":"2023-06-23T18:20:44.459414+0000","last_active":"2023-06-23T18:22:06.790712+0000","last_peered":"2023-06-23T18:22:06.790712+0000","last_clean":"2023-06-23T18:22:06.790712+0000","last_became_active":"2023-06-23T18:20:44.459014+0000","last_became_peered":"2023-06-23T18:20:44.459014+0000","last_unstale":"2023-06-23T18:22:06.790712+0000","last_undegraded":"2023-06-23T18:22:06.790712+0000","last_fullsized":"2023-06-23T18:22:06.790712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:16.590974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.774664+0000","last_change":"2023-06-23T18:20:44.446784+0000","last_active":"2023-06-23T18:22:06.774664+0000","last_peered":"2023-06-23T18:22:06.774664+0000","last_clean":"2023-06-23T18:22:06.774664+0000","last_became_active":"2023-06-23T18:20:44.446464+0000","last_became_peered":"2023-06-23T18:20:44.446464+0000","last_unstale":"2023-06-23T18:22:06.774664+0000","last_undegraded":"2023-06-23T18:22:06.774664+0000","last_fullsized":"2023-06-23T18:22:06.774664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:35:12.334055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075653+0000","last_change":"2023-06-23T18:20:44.039326+0000","last_active":"2023-06-23T18:21:38.075653+0000","last_peered":"2023-06-23T18:21:38.075653+0000","last_clean":"2023-06-23T18:21:38.075653+0000","last_became_active":"2023-06-23T18:20:44.038787+0000","last_became_peered":"2023-06-23T18:20:44.038787+0000","last_unstale":"2023-06-23T18:21:38.075653+0000","last_undegraded":"2023-06-23T18:21:38.075653+0000","last_fullsized":"2023-06-23T18:21:38.075653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:49:48.612922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081147+0000","last_change":"2023-06-23T18:20:44.465269+0000","last_active":"2023-06-23T18:21:38.081147+0000","last_peered":"2023-06-23T18:21:38.081147+0000","last_clean":"2023-06-23T18:21:38.081147+0000","last_became_active":"2023-06-23T18:20:44.464973+0000","last_became_peered":"2023-06-23T18:20:44.464973+0000","last_unstale":"2023-06-23T18:21:38.081147+0000","last_undegraded":"2023-06-23T18:21:38.081147+0000","last_fullsized":"2023-06-23T18:21:38.081147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:09:55.905699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081281+0000","last_change":"2023-06-23T18:20:44.037175+0000","last_active":"2023-06-23T18:21:38.081281+0000","last_peered":"2023-06-23T18:21:38.081281+0000","last_clean":"2023-06-23T18:21:38.081281+0000","last_became_active":"2023-06-23T18:20:44.036788+0000","last_became_peered":"2023-06-23T18:20:44.036788+0000","last_unstale":"2023-06-23T18:21:38.081281+0000","last_undegraded":"2023-06-23T18:21:38.081281+0000","last_fullsized":"2023-06-23T18:21:38.081281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:19:02.349726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080329+0000","last_change":"2023-06-23T18:20:44.463344+0000","last_active":"2023-06-23T18:21:38.080329+0000","last_peered":"2023-06-23T18:21:38.080329+0000","last_clean":"2023-06-23T18:21:38.080329+0000","last_became_active":"2023-06-23T18:20:44.462649+0000","last_became_peered":"2023-06-23T18:20:44.462649+0000","last_unstale":"2023-06-23T18:21:38.080329+0000","last_undegraded":"2023-06-23T18:21:38.080329+0000","last_fullsized":"2023-06-23T18:21:38.080329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:36:25.801756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076445+0000","last_change":"2023-06-23T18:20:44.045525+0000","last_active":"2023-06-23T18:21:38.076445+0000","last_peered":"2023-06-23T18:21:38.076445+0000","last_clean":"2023-06-23T18:21:38.076445+0000","last_became_active":"2023-06-23T18:20:44.045212+0000","last_became_peered":"2023-06-23T18:20:44.045212+0000","last_unstale":"2023-06-23T18:21:38.076445+0000","last_undegraded":"2023-06-23T18:21:38.076445+0000","last_fullsized":"2023-06-23T18:21:38.076445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:25.401039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790279+0000","last_change":"2023-06-23T18:20:44.461071+0000","last_active":"2023-06-23T18:22:06.790279+0000","last_peered":"2023-06-23T18:22:06.790279+0000","last_clean":"2023-06-23T18:22:06.790279+0000","last_became_active":"2023-06-23T18:20:44.460726+0000","last_became_peered":"2023-06-23T18:20:44.460726+0000","last_unstale":"2023-06-23T18:22:06.790279+0000","last_undegraded":"2023-06-23T18:22:06.790279+0000","last_fullsized":"2023-06-23T18:22:06.790279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:04:57.655489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076111+0000","last_change":"2023-06-23T18:20:44.037162+0000","last_active":"2023-06-23T18:21:38.076111+0000","last_peered":"2023-06-23T18:21:38.076111+0000","last_clean":"2023-06-23T18:21:38.076111+0000","last_became_active":"2023-06-23T18:20:44.036809+0000","last_became_peered":"2023-06-23T18:20:44.036809+0000","last_unstale":"2023-06-23T18:21:38.076111+0000","last_undegraded":"2023-06-23T18:21:38.076111+0000","last_fullsized":"2023-06-23T18:21:38.076111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:05:40.391806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.786602+0000","last_change":"2023-06-23T18:20:44.463525+0000","last_active":"2023-06-23T18:22:06.786602+0000","last_peered":"2023-06-23T18:22:06.786602+0000","last_clean":"2023-06-23T18:22:06.786602+0000","last_became_active":"2023-06-23T18:20:44.463245+0000","last_became_peered":"2023-06-23T18:20:44.463245+0000","last_unstale":"2023-06-23T18:22:06.786602+0000","last_undegraded":"2023-06-23T18:22:06.786602+0000","last_fullsized":"2023-06-23T18:22:06.786602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:29:11.041855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":108,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:08.167164+0000","last_change":"2023-06-23T18:20:44.044415+0000","last_active":"2023-06-23T18:22:08.167164+0000","last_peered":"2023-06-23T18:22:08.167164+0000","last_clean":"2023-06-23T18:22:08.167164+0000","last_became_active":"2023-06-23T18:20:44.044085+0000","last_became_peered":"2023-06-23T18:20:44.044085+0000","last_unstale":"2023-06-23T18:22:08.167164+0000","last_undegraded":"2023-06-23T18:22:08.167164+0000","last_fullsized":"2023-06-23T18:22:08.167164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:43:24.443528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1057,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778243+0000","last_change":"2023-06-23T18:21:38.077968+0000","last_active":"2023-06-23T18:22:06.778243+0000","last_peered":"2023-06-23T18:22:06.778243+0000","last_clean":"2023-06-23T18:22:06.778243+0000","last_became_active":"2023-06-23T18:21:38.077519+0000","last_became_peered":"2023-06-23T18:21:38.077519+0000","last_unstale":"2023-06-23T18:22:06.778243+0000","last_undegraded":"2023-06-23T18:22:06.778243+0000","last_fullsized":"2023-06-23T18:22:06.778243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:03:50.715602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076205+0000","last_change":"2023-06-23T18:20:44.039604+0000","last_active":"2023-06-23T18:21:38.076205+0000","last_peered":"2023-06-23T18:21:38.076205+0000","last_clean":"2023-06-23T18:21:38.076205+0000","last_became_active":"2023-06-23T18:20:44.039317+0000","last_became_peered":"2023-06-23T18:20:44.039317+0000","last_unstale":"2023-06-23T18:21:38.076205+0000","last_undegraded":"2023-06-23T18:21:38.076205+0000","last_fullsized":"2023-06-23T18:21:38.076205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:57:55.720236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780609+0000","last_change":"2023-06-23T18:20:44.465748+0000","last_active":"2023-06-23T18:22:06.780609+0000","last_peered":"2023-06-23T18:22:06.780609+0000","last_clean":"2023-06-23T18:22:06.780609+0000","last_became_active":"2023-06-23T18:20:44.465430+0000","last_became_peered":"2023-06-23T18:20:44.465430+0000","last_unstale":"2023-06-23T18:22:06.780609+0000","last_undegraded":"2023-06-23T18:22:06.780609+0000","last_fullsized":"2023-06-23T18:22:06.780609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:29:38.870085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072805+0000","last_change":"2023-06-23T18:20:44.034388+0000","last_active":"2023-06-23T18:21:38.072805+0000","last_peered":"2023-06-23T18:21:38.072805+0000","last_clean":"2023-06-23T18:21:38.072805+0000","last_became_active":"2023-06-23T18:20:44.034067+0000","last_became_peered":"2023-06-23T18:20:44.034067+0000","last_unstale":"2023-06-23T18:21:38.072805+0000","last_undegraded":"2023-06-23T18:21:38.072805+0000","last_fullsized":"2023-06-23T18:21:38.072805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:21:51.273582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080025+0000","last_change":"2023-06-23T18:20:44.453280+0000","last_active":"2023-06-23T18:21:38.080025+0000","last_peered":"2023-06-23T18:21:38.080025+0000","last_clean":"2023-06-23T18:21:38.080025+0000","last_became_active":"2023-06-23T18:20:44.452980+0000","last_became_peered":"2023-06-23T18:20:44.452980+0000","last_unstale":"2023-06-23T18:21:38.080025+0000","last_undegraded":"2023-06-23T18:21:38.080025+0000","last_fullsized":"2023-06-23T18:21:38.080025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:01:27.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.082028+0000","last_change":"2023-06-23T18:20:44.036165+0000","last_active":"2023-06-23T18:21:38.082028+0000","last_peered":"2023-06-23T18:21:38.082028+0000","last_clean":"2023-06-23T18:21:38.082028+0000","last_became_active":"2023-06-23T18:20:44.035313+0000","last_became_peered":"2023-06-23T18:20:44.035313+0000","last_unstale":"2023-06-23T18:21:38.082028+0000","last_undegraded":"2023-06-23T18:21:38.082028+0000","last_fullsized":"2023-06-23T18:21:38.082028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:15:36.463724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790545+0000","last_change":"2023-06-23T18:20:44.468358+0000","last_active":"2023-06-23T18:22:06.790545+0000","last_peered":"2023-06-23T18:22:06.790545+0000","last_clean":"2023-06-23T18:22:06.790545+0000","last_became_active":"2023-06-23T18:20:44.468032+0000","last_became_peered":"2023-06-23T18:20:44.468032+0000","last_unstale":"2023-06-23T18:22:06.790545+0000","last_undegraded":"2023-06-23T18:22:06.790545+0000","last_fullsized":"2023-06-23T18:22:06.790545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:23:56.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.101308+0000","last_change":"2023-06-23T18:20:44.037921+0000","last_active":"2023-06-23T18:22:07.101308+0000","last_peered":"2023-06-23T18:22:07.101308+0000","last_clean":"2023-06-23T18:22:07.101308+0000","last_became_active":"2023-06-23T18:20:44.037571+0000","last_became_peered":"2023-06-23T18:20:44.037571+0000","last_unstale":"2023-06-23T18:22:07.101308+0000","last_undegraded":"2023-06-23T18:22:07.101308+0000","last_fullsized":"2023-06-23T18:22:07.101308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:32:06.313509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792659+0000","last_change":"2023-06-23T18:20:44.460897+0000","last_active":"2023-06-23T18:22:06.792659+0000","last_peered":"2023-06-23T18:22:06.792659+0000","last_clean":"2023-06-23T18:22:06.792659+0000","last_became_active":"2023-06-23T18:20:44.460476+0000","last_became_peered":"2023-06-23T18:20:44.460476+0000","last_unstale":"2023-06-23T18:22:06.792659+0000","last_undegraded":"2023-06-23T18:22:06.792659+0000","last_fullsized":"2023-06-23T18:22:06.792659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:43:49.396858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073866+0000","last_change":"2023-06-23T18:20:44.033013+0000","last_active":"2023-06-23T18:21:38.073866+0000","last_peered":"2023-06-23T18:21:38.073866+0000","last_clean":"2023-06-23T18:21:38.073866+0000","last_became_active":"2023-06-23T18:20:44.032580+0000","last_became_peered":"2023-06-23T18:20:44.032580+0000","last_unstale":"2023-06-23T18:21:38.073866+0000","last_undegraded":"2023-06-23T18:21:38.073866+0000","last_fullsized":"2023-06-23T18:21:38.073866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:54.441960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080378+0000","last_change":"2023-06-23T18:20:44.463585+0000","last_active":"2023-06-23T18:21:38.080378+0000","last_peered":"2023-06-23T18:21:38.080378+0000","last_clean":"2023-06-23T18:21:38.080378+0000","last_became_active":"2023-06-23T18:20:44.462512+0000","last_became_peered":"2023-06-23T18:20:44.462512+0000","last_unstale":"2023-06-23T18:21:38.080378+0000","last_undegraded":"2023-06-23T18:21:38.080378+0000","last_fullsized":"2023-06-23T18:21:38.080378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:01:51.999262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081156+0000","last_change":"2023-06-23T18:20:44.037134+0000","last_active":"2023-06-23T18:21:38.081156+0000","last_peered":"2023-06-23T18:21:38.081156+0000","last_clean":"2023-06-23T18:21:38.081156+0000","last_became_active":"2023-06-23T18:20:44.036536+0000","last_became_peered":"2023-06-23T18:20:44.036536+0000","last_unstale":"2023-06-23T18:21:38.081156+0000","last_undegraded":"2023-06-23T18:21:38.081156+0000","last_fullsized":"2023-06-23T18:21:38.081156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:45:16.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785436+0000","last_change":"2023-06-23T18:20:44.454453+0000","last_active":"2023-06-23T18:22:06.785436+0000","last_peered":"2023-06-23T18:22:06.785436+0000","last_clean":"2023-06-23T18:22:06.785436+0000","last_became_active":"2023-06-23T18:20:44.454133+0000","last_became_peered":"2023-06-23T18:20:44.454133+0000","last_unstale":"2023-06-23T18:22:06.785436+0000","last_undegraded":"2023-06-23T18:22:06.785436+0000","last_fullsized":"2023-06-23T18:22:06.785436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:12:10.199634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":234,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.148559+0000","last_change":"2023-06-23T18:21:38.108843+0000","last_active":"2023-06-23T18:22:07.148559+0000","last_peered":"2023-06-23T18:22:07.148559+0000","last_clean":"2023-06-23T18:22:07.148559+0000","last_became_active":"2023-06-23T18:21:38.102073+0000","last_became_peered":"2023-06-23T18:21:38.102073+0000","last_unstale":"2023-06-23T18:22:07.148559+0000","last_undegraded":"2023-06-23T18:22:07.148559+0000","last_fullsized":"2023-06-23T18:22:07.148559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:17:04.550195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58230,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":134,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57498,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.787027+0000","last_change":"2023-06-23T18:20:44.456558+0000","last_active":"2023-06-23T18:22:06.787027+0000","last_peered":"2023-06-23T18:22:06.787027+0000","last_clean":"2023-06-23T18:22:06.787027+0000","last_became_active":"2023-06-23T18:20:44.456199+0000","last_became_peered":"2023-06-23T18:20:44.456199+0000","last_unstale":"2023-06-23T18:22:06.787027+0000","last_undegraded":"2023-06-23T18:22:06.787027+0000","last_fullsized":"2023-06-23T18:22:06.787027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:53:48.531640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080152+0000","last_change":"2023-06-23T18:20:44.034101+0000","last_active":"2023-06-23T18:21:38.080152+0000","last_peered":"2023-06-23T18:21:38.080152+0000","last_clean":"2023-06-23T18:21:38.080152+0000","last_became_active":"2023-06-23T18:20:44.033287+0000","last_became_peered":"2023-06-23T18:20:44.033287+0000","last_unstale":"2023-06-23T18:21:38.080152+0000","last_undegraded":"2023-06-23T18:21:38.080152+0000","last_fullsized":"2023-06-23T18:21:38.080152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:44:53.381086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072861+0000","last_change":"2023-06-23T18:20:44.446298+0000","last_active":"2023-06-23T18:21:38.072861+0000","last_peered":"2023-06-23T18:21:38.072861+0000","last_clean":"2023-06-23T18:21:38.072861+0000","last_became_active":"2023-06-23T18:20:44.445922+0000","last_became_peered":"2023-06-23T18:20:44.445922+0000","last_unstale":"2023-06-23T18:21:38.072861+0000","last_undegraded":"2023-06-23T18:21:38.072861+0000","last_fullsized":"2023-06-23T18:21:38.072861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:51.263367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080693+0000","last_change":"2023-06-23T18:20:44.035791+0000","last_active":"2023-06-23T18:21:38.080693+0000","last_peered":"2023-06-23T18:21:38.080693+0000","last_clean":"2023-06-23T18:21:38.080693+0000","last_became_active":"2023-06-23T18:20:44.035416+0000","last_became_peered":"2023-06-23T18:20:44.035416+0000","last_unstale":"2023-06-23T18:21:38.080693+0000","last_undegraded":"2023-06-23T18:21:38.080693+0000","last_fullsized":"2023-06-23T18:21:38.080693+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:49:57.755758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790515+0000","last_change":"2023-06-23T18:20:44.455474+0000","last_active":"2023-06-23T18:22:06.790515+0000","last_peered":"2023-06-23T18:22:06.790515+0000","last_clean":"2023-06-23T18:22:06.790515+0000","last_became_active":"2023-06-23T18:20:44.455098+0000","last_became_peered":"2023-06-23T18:20:44.455098+0000","last_unstale":"2023-06-23T18:22:06.790515+0000","last_undegraded":"2023-06-23T18:22:06.790515+0000","last_fullsized":"2023-06-23T18:22:06.790515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:15:19.154271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":107,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.164451+0000","last_change":"2023-06-23T18:20:44.030305+0000","last_active":"2023-06-23T18:22:07.164451+0000","last_peered":"2023-06-23T18:22:07.164451+0000","last_clean":"2023-06-23T18:22:07.164451+0000","last_became_active":"2023-06-23T18:20:44.029898+0000","last_became_peered":"2023-06-23T18:20:44.029898+0000","last_unstale":"2023-06-23T18:22:07.164451+0000","last_undegraded":"2023-06-23T18:22:07.164451+0000","last_fullsized":"2023-06-23T18:22:07.164451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:49:33.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1554,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790958+0000","last_change":"2023-06-23T18:20:44.451321+0000","last_active":"2023-06-23T18:22:06.790958+0000","last_peered":"2023-06-23T18:22:06.790958+0000","last_clean":"2023-06-23T18:22:06.790958+0000","last_became_active":"2023-06-23T18:20:44.450549+0000","last_became_peered":"2023-06-23T18:20:44.450549+0000","last_unstale":"2023-06-23T18:22:06.790958+0000","last_undegraded":"2023-06-23T18:22:06.790958+0000","last_fullsized":"2023-06-23T18:22:06.790958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:31:37.630032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.097392+0000","last_change":"2023-06-23T18:21:38.097392+0000","last_active":"2023-06-23T18:21:38.097392+0000","last_peered":"2023-06-23T18:21:38.097392+0000","last_clean":"2023-06-23T18:21:38.097392+0000","last_became_active":"2023-06-23T18:21:38.082554+0000","last_became_peered":"2023-06-23T18:21:38.082554+0000","last_unstale":"2023-06-23T18:21:38.097392+0000","last_undegraded":"2023-06-23T18:21:38.097392+0000","last_fullsized":"2023-06-23T18:21:38.097392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:24:59.677445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789557+0000","last_change":"2023-06-23T18:21:38.085114+0000","last_active":"2023-06-23T18:22:06.789557+0000","last_peered":"2023-06-23T18:22:06.789557+0000","last_clean":"2023-06-23T18:22:06.789557+0000","last_became_active":"2023-06-23T18:21:38.084575+0000","last_became_peered":"2023-06-23T18:21:38.084575+0000","last_unstale":"2023-06-23T18:22:06.789557+0000","last_undegraded":"2023-06-23T18:22:06.789557+0000","last_fullsized":"2023-06-23T18:22:06.789557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:27:44.436006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076256+0000","last_change":"2023-06-23T18:20:44.038961+0000","last_active":"2023-06-23T18:21:38.076256+0000","last_peered":"2023-06-23T18:21:38.076256+0000","last_clean":"2023-06-23T18:21:38.076256+0000","last_became_active":"2023-06-23T18:20:44.038674+0000","last_became_peered":"2023-06-23T18:20:44.038674+0000","last_unstale":"2023-06-23T18:21:38.076256+0000","last_undegraded":"2023-06-23T18:21:38.076256+0000","last_fullsized":"2023-06-23T18:21:38.076256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:37:52.894959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.776481+0000","last_change":"2023-06-23T18:20:44.461077+0000","last_active":"2023-06-23T18:22:06.776481+0000","last_peered":"2023-06-23T18:22:06.776481+0000","last_clean":"2023-06-23T18:22:06.776481+0000","last_became_active":"2023-06-23T18:20:44.460767+0000","last_became_peered":"2023-06-23T18:20:44.460767+0000","last_unstale":"2023-06-23T18:22:06.776481+0000","last_undegraded":"2023-06-23T18:22:06.776481+0000","last_fullsized":"2023-06-23T18:22:06.776481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:33:02.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075336+0000","last_change":"2023-06-23T18:20:44.042114+0000","last_active":"2023-06-23T18:21:38.075336+0000","last_peered":"2023-06-23T18:21:38.075336+0000","last_clean":"2023-06-23T18:21:38.075336+0000","last_became_active":"2023-06-23T18:20:44.041822+0000","last_became_peered":"2023-06-23T18:20:44.041822+0000","last_unstale":"2023-06-23T18:21:38.075336+0000","last_undegraded":"2023-06-23T18:21:38.075336+0000","last_fullsized":"2023-06-23T18:21:38.075336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:09:25.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789488+0000","last_change":"2023-06-23T18:20:44.452903+0000","last_active":"2023-06-23T18:22:06.789488+0000","last_peered":"2023-06-23T18:22:06.789488+0000","last_clean":"2023-06-23T18:22:06.789488+0000","last_became_active":"2023-06-23T18:20:44.452539+0000","last_became_peered":"2023-06-23T18:20:44.452539+0000","last_unstale":"2023-06-23T18:22:06.789488+0000","last_undegraded":"2023-06-23T18:22:06.789488+0000","last_fullsized":"2023-06-23T18:22:06.789488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:00:37.883811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073621+0000","last_change":"2023-06-23T18:20:44.032905+0000","last_active":"2023-06-23T18:21:38.073621+0000","last_peered":"2023-06-23T18:21:38.073621+0000","last_clean":"2023-06-23T18:21:38.073621+0000","last_became_active":"2023-06-23T18:20:44.032606+0000","last_became_peered":"2023-06-23T18:20:44.032606+0000","last_unstale":"2023-06-23T18:21:38.073621+0000","last_undegraded":"2023-06-23T18:21:38.073621+0000","last_fullsized":"2023-06-23T18:21:38.073621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:44:01.287033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789982+0000","last_change":"2023-06-23T18:20:44.465580+0000","last_active":"2023-06-23T18:22:06.789982+0000","last_peered":"2023-06-23T18:22:06.789982+0000","last_clean":"2023-06-23T18:22:06.789982+0000","last_became_active":"2023-06-23T18:20:44.465264+0000","last_became_peered":"2023-06-23T18:20:44.465264+0000","last_unstale":"2023-06-23T18:22:06.789982+0000","last_undegraded":"2023-06-23T18:22:06.789982+0000","last_fullsized":"2023-06-23T18:22:06.789982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:40:26.704880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778940+0000","last_change":"2023-06-23T18:20:44.464324+0000","last_active":"2023-06-23T18:22:06.778940+0000","last_peered":"2023-06-23T18:22:06.778940+0000","last_clean":"2023-06-23T18:22:06.778940+0000","last_became_active":"2023-06-23T18:20:44.464021+0000","last_became_peered":"2023-06-23T18:20:44.464021+0000","last_unstale":"2023-06-23T18:22:06.778940+0000","last_undegraded":"2023-06-23T18:22:06.778940+0000","last_fullsized":"2023-06-23T18:22:06.778940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:09.888384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:05.160742+0000","last_change":"2023-06-23T18:20:44.034309+0000","last_active":"2023-06-23T18:22:05.160742+0000","last_peered":"2023-06-23T18:22:05.160742+0000","last_clean":"2023-06-23T18:22:05.160742+0000","last_became_active":"2023-06-23T18:20:44.034007+0000","last_became_peered":"2023-06-23T18:20:44.034007+0000","last_unstale":"2023-06-23T18:22:05.160742+0000","last_undegraded":"2023-06-23T18:22:05.160742+0000","last_fullsized":"2023-06-23T18:22:05.160742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:50:17.056971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080879+0000","last_change":"2023-06-23T18:20:44.468220+0000","last_active":"2023-06-23T18:21:38.080879+0000","last_peered":"2023-06-23T18:21:38.080879+0000","last_clean":"2023-06-23T18:21:38.080879+0000","last_became_active":"2023-06-23T18:20:44.467725+0000","last_became_peered":"2023-06-23T18:20:44.467725+0000","last_unstale":"2023-06-23T18:21:38.080879+0000","last_undegraded":"2023-06-23T18:21:38.080879+0000","last_fullsized":"2023-06-23T18:21:38.080879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:20:06.380008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100500+0000","last_change":"2023-06-23T18:20:44.030921+0000","last_active":"2023-06-23T18:21:38.100500+0000","last_peered":"2023-06-23T18:21:38.100500+0000","last_clean":"2023-06-23T18:21:38.100500+0000","last_became_active":"2023-06-23T18:20:44.030362+0000","last_became_peered":"2023-06-23T18:20:44.030362+0000","last_unstale":"2023-06-23T18:21:38.100500+0000","last_undegraded":"2023-06-23T18:21:38.100500+0000","last_fullsized":"2023-06-23T18:21:38.100500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:07:01.761788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073517+0000","last_change":"2023-06-23T18:19:44.952262+0000","last_active":"2023-06-23T18:21:38.073517+0000","last_peered":"2023-06-23T18:21:38.073517+0000","last_clean":"2023-06-23T18:21:38.073517+0000","last_became_active":"2023-06-23T18:19:44.951803+0000","last_became_peered":"2023-06-23T18:19:44.951803+0000","last_unstale":"2023-06-23T18:21:38.073517+0000","last_undegraded":"2023-06-23T18:21:38.073517+0000","last_fullsized":"2023-06-23T18:21:38.073517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_clean_scrub_stamp":"2023-06-23T18:16:49.458922+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:43:25.254131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072640+0000","last_change":"2023-06-23T18:20:44.456602+0000","last_active":"2023-06-23T18:21:38.072640+0000","last_peered":"2023-06-23T18:21:38.072640+0000","last_clean":"2023-06-23T18:21:38.072640+0000","last_became_active":"2023-06-23T18:20:44.456249+0000","last_became_peered":"2023-06-23T18:20:44.456249+0000","last_unstale":"2023-06-23T18:21:38.072640+0000","last_undegraded":"2023-06-23T18:21:38.072640+0000","last_fullsized":"2023-06-23T18:21:38.072640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:50:46.341121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073728+0000","last_change":"2023-06-23T18:20:44.039100+0000","last_active":"2023-06-23T18:21:38.073728+0000","last_peered":"2023-06-23T18:21:38.073728+0000","last_clean":"2023-06-23T18:21:38.073728+0000","last_became_active":"2023-06-23T18:20:44.038801+0000","last_became_peered":"2023-06-23T18:20:44.038801+0000","last_unstale":"2023-06-23T18:21:38.073728+0000","last_undegraded":"2023-06-23T18:21:38.073728+0000","last_fullsized":"2023-06-23T18:21:38.073728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:01:20.046240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076298+0000","last_change":"2023-06-23T18:20:44.038037+0000","last_active":"2023-06-23T18:21:38.076298+0000","last_peered":"2023-06-23T18:21:38.076298+0000","last_clean":"2023-06-23T18:21:38.076298+0000","last_became_active":"2023-06-23T18:20:44.037732+0000","last_became_peered":"2023-06-23T18:20:44.037732+0000","last_unstale":"2023-06-23T18:21:38.076298+0000","last_undegraded":"2023-06-23T18:21:38.076298+0000","last_fullsized":"2023-06-23T18:21:38.076298+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:17:33.256219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073019+0000","last_change":"2023-06-23T18:20:44.459927+0000","last_active":"2023-06-23T18:21:38.073019+0000","last_peered":"2023-06-23T18:21:38.073019+0000","last_clean":"2023-06-23T18:21:38.073019+0000","last_became_active":"2023-06-23T18:20:44.459594+0000","last_became_peered":"2023-06-23T18:20:44.459594+0000","last_unstale":"2023-06-23T18:21:38.073019+0000","last_undegraded":"2023-06-23T18:21:38.073019+0000","last_fullsized":"2023-06-23T18:21:38.073019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:16:49.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075638+0000","last_change":"2023-06-23T18:20:44.039429+0000","last_active":"2023-06-23T18:21:38.075638+0000","last_peered":"2023-06-23T18:21:38.075638+0000","last_clean":"2023-06-23T18:21:38.075638+0000","last_became_active":"2023-06-23T18:20:44.038944+0000","last_became_peered":"2023-06-23T18:20:44.038944+0000","last_unstale":"2023-06-23T18:21:38.075638+0000","last_undegraded":"2023-06-23T18:21:38.075638+0000","last_fullsized":"2023-06-23T18:21:38.075638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:03.297820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080916+0000","last_change":"2023-06-23T18:20:44.458227+0000","last_active":"2023-06-23T18:21:38.080916+0000","last_peered":"2023-06-23T18:21:38.080916+0000","last_clean":"2023-06-23T18:21:38.080916+0000","last_became_active":"2023-06-23T18:20:44.457904+0000","last_became_peered":"2023-06-23T18:20:44.457904+0000","last_unstale":"2023-06-23T18:21:38.080916+0000","last_undegraded":"2023-06-23T18:21:38.080916+0000","last_fullsized":"2023-06-23T18:21:38.080916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:49:55.045014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100759+0000","last_change":"2023-06-23T18:20:44.034089+0000","last_active":"2023-06-23T18:21:38.100759+0000","last_peered":"2023-06-23T18:21:38.100759+0000","last_clean":"2023-06-23T18:21:38.100759+0000","last_became_active":"2023-06-23T18:20:44.033745+0000","last_became_peered":"2023-06-23T18:20:44.033745+0000","last_unstale":"2023-06-23T18:21:38.100759+0000","last_undegraded":"2023-06-23T18:21:38.100759+0000","last_fullsized":"2023-06-23T18:21:38.100759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:37:54.500817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.082533+0000","last_change":"2023-06-23T18:21:38.082533+0000","last_active":"2023-06-23T18:21:38.082533+0000","last_peered":"2023-06-23T18:21:38.082533+0000","last_clean":"2023-06-23T18:21:38.082533+0000","last_became_active":"2023-06-23T18:21:38.082114+0000","last_became_peered":"2023-06-23T18:21:38.082114+0000","last_unstale":"2023-06-23T18:21:38.082533+0000","last_undegraded":"2023-06-23T18:21:38.082533+0000","last_fullsized":"2023-06-23T18:21:38.082533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:03:31.751873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.078831+0000","last_change":"2023-06-23T18:20:44.030647+0000","last_active":"2023-06-23T18:21:38.078831+0000","last_peered":"2023-06-23T18:21:38.078831+0000","last_clean":"2023-06-23T18:21:38.078831+0000","last_became_active":"2023-06-23T18:20:44.030316+0000","last_became_peered":"2023-06-23T18:20:44.030316+0000","last_unstale":"2023-06-23T18:21:38.078831+0000","last_undegraded":"2023-06-23T18:21:38.078831+0000","last_fullsized":"2023-06-23T18:21:38.078831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:35:45.816400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.743921+0000","last_change":"2023-06-23T18:20:44.458365+0000","last_active":"2023-06-23T18:22:06.743921+0000","last_peered":"2023-06-23T18:22:06.743921+0000","last_clean":"2023-06-23T18:22:06.743921+0000","last_became_active":"2023-06-23T18:20:44.458039+0000","last_became_peered":"2023-06-23T18:20:44.458039+0000","last_unstale":"2023-06-23T18:22:06.743921+0000","last_undegraded":"2023-06-23T18:22:06.743921+0000","last_fullsized":"2023-06-23T18:22:06.743921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:02:29.952453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080611+0000","last_change":"2023-06-23T18:20:44.028908+0000","last_active":"2023-06-23T18:21:38.080611+0000","last_peered":"2023-06-23T18:21:38.080611+0000","last_clean":"2023-06-23T18:21:38.080611+0000","last_became_active":"2023-06-23T18:20:44.028551+0000","last_became_peered":"2023-06-23T18:20:44.028551+0000","last_unstale":"2023-06-23T18:21:38.080611+0000","last_undegraded":"2023-06-23T18:21:38.080611+0000","last_fullsized":"2023-06-23T18:21:38.080611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:55:58.763919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780847+0000","last_change":"2023-06-23T18:20:44.465300+0000","last_active":"2023-06-23T18:22:06.780847+0000","last_peered":"2023-06-23T18:22:06.780847+0000","last_clean":"2023-06-23T18:22:06.780847+0000","last_became_active":"2023-06-23T18:20:44.465008+0000","last_became_peered":"2023-06-23T18:20:44.465008+0000","last_unstale":"2023-06-23T18:22:06.780847+0000","last_undegraded":"2023-06-23T18:22:06.780847+0000","last_fullsized":"2023-06-23T18:22:06.780847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:37:20.321477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100292+0000","last_change":"2023-06-23T18:21:38.100292+0000","last_active":"2023-06-23T18:21:38.100292+0000","last_peered":"2023-06-23T18:21:38.100292+0000","last_clean":"2023-06-23T18:21:38.100292+0000","last_became_active":"2023-06-23T18:21:38.083373+0000","last_became_peered":"2023-06-23T18:21:38.083373+0000","last_unstale":"2023-06-23T18:21:38.100292+0000","last_undegraded":"2023-06-23T18:21:38.100292+0000","last_fullsized":"2023-06-23T18:21:38.100292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:44:38.453889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081058+0000","last_change":"2023-06-23T18:20:44.465127+0000","last_active":"2023-06-23T18:21:38.081058+0000","last_peered":"2023-06-23T18:21:38.081058+0000","last_clean":"2023-06-23T18:21:38.081058+0000","last_became_active":"2023-06-23T18:20:44.464768+0000","last_became_peered":"2023-06-23T18:20:44.464768+0000","last_unstale":"2023-06-23T18:21:38.081058+0000","last_undegraded":"2023-06-23T18:21:38.081058+0000","last_fullsized":"2023-06-23T18:21:38.081058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:07:36.805439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075973+0000","last_change":"2023-06-23T18:20:44.046245+0000","last_active":"2023-06-23T18:21:38.075973+0000","last_peered":"2023-06-23T18:21:38.075973+0000","last_clean":"2023-06-23T18:21:38.075973+0000","last_became_active":"2023-06-23T18:20:44.045945+0000","last_became_peered":"2023-06-23T18:20:44.045945+0000","last_unstale":"2023-06-23T18:21:38.075973+0000","last_undegraded":"2023-06-23T18:21:38.075973+0000","last_fullsized":"2023-06-23T18:21:38.075973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:53:58.962783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789675+0000","last_change":"2023-06-23T18:20:44.453520+0000","last_active":"2023-06-23T18:22:06.789675+0000","last_peered":"2023-06-23T18:22:06.789675+0000","last_clean":"2023-06-23T18:22:06.789675+0000","last_became_active":"2023-06-23T18:20:44.453214+0000","last_became_peered":"2023-06-23T18:20:44.453214+0000","last_unstale":"2023-06-23T18:22:06.789675+0000","last_undegraded":"2023-06-23T18:22:06.789675+0000","last_fullsized":"2023-06-23T18:22:06.789675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:40:14.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080295+0000","last_change":"2023-06-23T18:20:44.042526+0000","last_active":"2023-06-23T18:21:38.080295+0000","last_peered":"2023-06-23T18:21:38.080295+0000","last_clean":"2023-06-23T18:21:38.080295+0000","last_became_active":"2023-06-23T18:20:44.042249+0000","last_became_peered":"2023-06-23T18:20:44.042249+0000","last_unstale":"2023-06-23T18:21:38.080295+0000","last_undegraded":"2023-06-23T18:21:38.080295+0000","last_fullsized":"2023-06-23T18:21:38.080295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:59:17.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790121+0000","last_change":"2023-06-23T18:20:44.454645+0000","last_active":"2023-06-23T18:22:06.790121+0000","last_peered":"2023-06-23T18:22:06.790121+0000","last_clean":"2023-06-23T18:22:06.790121+0000","last_became_active":"2023-06-23T18:20:44.453309+0000","last_became_peered":"2023-06-23T18:20:44.453309+0000","last_unstale":"2023-06-23T18:22:06.790121+0000","last_undegraded":"2023-06-23T18:22:06.790121+0000","last_fullsized":"2023-06-23T18:22:06.790121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:15:24.080996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080976+0000","last_change":"2023-06-23T18:20:44.037681+0000","last_active":"2023-06-23T18:21:38.080976+0000","last_peered":"2023-06-23T18:21:38.080976+0000","last_clean":"2023-06-23T18:21:38.080976+0000","last_became_active":"2023-06-23T18:20:44.037323+0000","last_became_peered":"2023-06-23T18:20:44.037323+0000","last_unstale":"2023-06-23T18:21:38.080976+0000","last_undegraded":"2023-06-23T18:21:38.080976+0000","last_fullsized":"2023-06-23T18:21:38.080976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:49:34.898906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.787192+0000","last_change":"2023-06-23T18:20:44.456434+0000","last_active":"2023-06-23T18:22:06.787192+0000","last_peered":"2023-06-23T18:22:06.787192+0000","last_clean":"2023-06-23T18:22:06.787192+0000","last_became_active":"2023-06-23T18:20:44.456110+0000","last_became_peered":"2023-06-23T18:20:44.456110+0000","last_unstale":"2023-06-23T18:22:06.787192+0000","last_undegraded":"2023-06-23T18:22:06.787192+0000","last_fullsized":"2023-06-23T18:22:06.787192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:38:39.438073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073300+0000","last_change":"2023-06-23T18:20:44.040722+0000","last_active":"2023-06-23T18:21:38.073300+0000","last_peered":"2023-06-23T18:21:38.073300+0000","last_clean":"2023-06-23T18:21:38.073300+0000","last_became_active":"2023-06-23T18:20:44.040390+0000","last_became_peered":"2023-06-23T18:20:44.040390+0000","last_unstale":"2023-06-23T18:21:38.073300+0000","last_undegraded":"2023-06-23T18:21:38.073300+0000","last_fullsized":"2023-06-23T18:21:38.073300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:49:31.045068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792445+0000","last_change":"2023-06-23T18:20:44.455559+0000","last_active":"2023-06-23T18:22:06.792445+0000","last_peered":"2023-06-23T18:22:06.792445+0000","last_clean":"2023-06-23T18:22:06.792445+0000","last_became_active":"2023-06-23T18:20:44.455256+0000","last_became_peered":"2023-06-23T18:20:44.455256+0000","last_unstale":"2023-06-23T18:22:06.792445+0000","last_undegraded":"2023-06-23T18:22:06.792445+0000","last_fullsized":"2023-06-23T18:22:06.792445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:43:47.289820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.779277+0000","last_change":"2023-06-23T18:20:44.460403+0000","last_active":"2023-06-23T18:22:06.779277+0000","last_peered":"2023-06-23T18:22:06.779277+0000","last_clean":"2023-06-23T18:22:06.779277+0000","last_became_active":"2023-06-23T18:20:44.459928+0000","last_became_peered":"2023-06-23T18:20:44.459928+0000","last_unstale":"2023-06-23T18:22:06.779277+0000","last_undegraded":"2023-06-23T18:22:06.779277+0000","last_fullsized":"2023-06-23T18:22:06.779277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:37:08.283159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780411+0000","last_change":"2023-06-23T18:20:44.458050+0000","last_active":"2023-06-23T18:22:06.780411+0000","last_peered":"2023-06-23T18:22:06.780411+0000","last_clean":"2023-06-23T18:22:06.780411+0000","last_became_active":"2023-06-23T18:20:44.457733+0000","last_became_peered":"2023-06-23T18:20:44.457733+0000","last_unstale":"2023-06-23T18:22:06.780411+0000","last_undegraded":"2023-06-23T18:22:06.780411+0000","last_fullsized":"2023-06-23T18:22:06.780411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:43:34.360833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.776703+0000","last_change":"2023-06-23T18:20:44.456233+0000","last_active":"2023-06-23T18:22:06.776703+0000","last_peered":"2023-06-23T18:22:06.776703+0000","last_clean":"2023-06-23T18:22:06.776703+0000","last_became_active":"2023-06-23T18:20:44.455677+0000","last_became_peered":"2023-06-23T18:20:44.455677+0000","last_unstale":"2023-06-23T18:22:06.776703+0000","last_undegraded":"2023-06-23T18:22:06.776703+0000","last_fullsized":"2023-06-23T18:22:06.776703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:30:25.029586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080153+0000","last_change":"2023-06-23T18:20:44.469668+0000","last_active":"2023-06-23T18:21:38.080153+0000","last_peered":"2023-06-23T18:21:38.080153+0000","last_clean":"2023-06-23T18:21:38.080153+0000","last_became_active":"2023-06-23T18:20:44.466563+0000","last_became_peered":"2023-06-23T18:20:44.466563+0000","last_unstale":"2023-06-23T18:21:38.080153+0000","last_undegraded":"2023-06-23T18:21:38.080153+0000","last_fullsized":"2023-06-23T18:21:38.080153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:05:04.024545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073109+0000","last_change":"2023-06-23T18:20:44.456021+0000","last_active":"2023-06-23T18:21:38.073109+0000","last_peered":"2023-06-23T18:21:38.073109+0000","last_clean":"2023-06-23T18:21:38.073109+0000","last_became_active":"2023-06-23T18:20:44.455705+0000","last_became_peered":"2023-06-23T18:20:44.455705+0000","last_unstale":"2023-06-23T18:21:38.073109+0000","last_undegraded":"2023-06-23T18:21:38.073109+0000","last_fullsized":"2023-06-23T18:21:38.073109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:50:09.933725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.775420+0000","last_change":"2023-06-23T18:20:44.460359+0000","last_active":"2023-06-23T18:22:06.775420+0000","last_peered":"2023-06-23T18:22:06.775420+0000","last_clean":"2023-06-23T18:22:06.775420+0000","last_became_active":"2023-06-23T18:20:44.460034+0000","last_became_peered":"2023-06-23T18:20:44.460034+0000","last_unstale":"2023-06-23T18:22:06.775420+0000","last_undegraded":"2023-06-23T18:22:06.775420+0000","last_fullsized":"2023-06-23T18:22:06.775420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:39.654726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.786166+0000","last_change":"2023-06-23T18:20:44.455743+0000","last_active":"2023-06-23T18:22:06.786166+0000","last_peered":"2023-06-23T18:22:06.786166+0000","last_clean":"2023-06-23T18:22:06.786166+0000","last_became_active":"2023-06-23T18:20:44.455266+0000","last_became_peered":"2023-06-23T18:20:44.455266+0000","last_unstale":"2023-06-23T18:22:06.786166+0000","last_undegraded":"2023-06-23T18:22:06.786166+0000","last_fullsized":"2023-06-23T18:22:06.786166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:51:16.923982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.783014+0000","last_change":"2023-06-23T18:21:38.101164+0000","last_active":"2023-06-23T18:22:06.783014+0000","last_peered":"2023-06-23T18:22:06.783014+0000","last_clean":"2023-06-23T18:22:06.783014+0000","last_became_active":"2023-06-23T18:21:38.100818+0000","last_became_peered":"2023-06-23T18:21:38.100818+0000","last_unstale":"2023-06-23T18:22:06.783014+0000","last_undegraded":"2023-06-23T18:22:06.783014+0000","last_fullsized":"2023-06-23T18:22:06.783014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:54:39.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.777099+0000","last_change":"2023-06-23T18:20:44.465192+0000","last_active":"2023-06-23T18:22:06.777099+0000","last_peered":"2023-06-23T18:22:06.777099+0000","last_clean":"2023-06-23T18:22:06.777099+0000","last_became_active":"2023-06-23T18:20:44.464905+0000","last_became_peered":"2023-06-23T18:20:44.464905+0000","last_unstale":"2023-06-23T18:22:06.777099+0000","last_undegraded":"2023-06-23T18:22:06.777099+0000","last_fullsized":"2023-06-23T18:22:06.777099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:32:18.229559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080712+0000","last_change":"2023-06-23T18:20:44.458725+0000","last_active":"2023-06-23T18:21:38.080712+0000","last_peered":"2023-06-23T18:21:38.080712+0000","last_clean":"2023-06-23T18:21:38.080712+0000","last_became_active":"2023-06-23T18:20:44.458414+0000","last_became_peered":"2023-06-23T18:20:44.458414+0000","last_unstale":"2023-06-23T18:21:38.080712+0000","last_undegraded":"2023-06-23T18:21:38.080712+0000","last_fullsized":"2023-06-23T18:21:38.080712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:34:31.245389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792579+0000","last_change":"2023-06-23T18:20:44.465665+0000","last_active":"2023-06-23T18:22:06.792579+0000","last_peered":"2023-06-23T18:22:06.792579+0000","last_clean":"2023-06-23T18:22:06.792579+0000","last_became_active":"2023-06-23T18:20:44.465337+0000","last_became_peered":"2023-06-23T18:20:44.465337+0000","last_unstale":"2023-06-23T18:22:06.792579+0000","last_undegraded":"2023-06-23T18:22:06.792579+0000","last_fullsized":"2023-06-23T18:22:06.792579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:05:22.890126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072919+0000","last_change":"2023-06-23T18:20:44.455107+0000","last_active":"2023-06-23T18:21:38.072919+0000","last_peered":"2023-06-23T18:21:38.072919+0000","last_clean":"2023-06-23T18:21:38.072919+0000","last_became_active":"2023-06-23T18:20:44.454798+0000","last_became_peered":"2023-06-23T18:20:44.454798+0000","last_unstale":"2023-06-23T18:21:38.072919+0000","last_undegraded":"2023-06-23T18:21:38.072919+0000","last_fullsized":"2023-06-23T18:21:38.072919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:19:46.878268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780928+0000","last_change":"2023-06-23T18:20:44.461044+0000","last_active":"2023-06-23T18:22:06.780928+0000","last_peered":"2023-06-23T18:22:06.780928+0000","last_clean":"2023-06-23T18:22:06.780928+0000","last_became_active":"2023-06-23T18:20:44.460737+0000","last_became_peered":"2023-06-23T18:20:44.460737+0000","last_unstale":"2023-06-23T18:22:06.780928+0000","last_undegraded":"2023-06-23T18:22:06.780928+0000","last_fullsized":"2023-06-23T18:22:06.780928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:22:32.348419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":358,"ondisk_log_size":358,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281641,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63241,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":492,"num_read_kb":311,"num_write":162,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":57610,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":188757,"data_compressed":8925,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":437933,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.76000000000000001,"15min":0.80000000000000004},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.624,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.54700000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.873,"15min":0.873},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.48599999999999999},"min":{"1min":0.35399999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.371,"5min":0.317,"15min":0.317},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.54700000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83899999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.873,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.873}]},{"osd":6,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6580000000000004,"5min":5.6580000000000004,"15min":5.6580000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95899999999999996,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70499999999999996,"15min":0.69399999999999995},"min":{"1min":0.61499999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0069999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.82099999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.87,"5min":1.369,"15min":1.369},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.66900000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95399999999999996,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.71499999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.63100000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98599999999999999,"5min":1.579,"15min":1.579},"last":0.89600000000000002}]}]},{"osd":10,"up_from":70,"seq":300647710753,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1088504,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":11.565,"5min":11.565,"15min":11.565},"last":2.0329999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.298,"5min":4.298,"15min":4.298},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76500000000000001,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82999999999999996,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.745},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80400000000000005,"5min":2.782,"15min":2.782},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71699999999999997,"15min":0.73499999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.73199999999999998,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.55100000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.58899999999999997,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90700000000000003,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.89200000000000002,"15min":0.85799999999999998},"min":{"1min":0.51700000000000002,"5min":0.501,"15min":0.501},"max":{"1min":7.2240000000000002,"5min":7.2240000000000002,"15min":7.2240000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.79600000000000004,"15min":0.80900000000000005},"min":{"1min":0.54700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89800000000000002,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84599999999999997,"15min":0.85299999999999998},"min":{"1min":0.61799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93799999999999994,"5min":1.782,"15min":1.782},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86899999999999999,"15min":0.88600000000000001},"min":{"1min":0.61899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94399999999999995,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.88100000000000001,"5min":1.645,"15min":1.645},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.90400000000000003,"15min":0.92600000000000005},"min":{"1min":0.64200000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91900000000000004,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.66300000000000003}]},{"osd":11,"last update":"Fri Jun 23 18:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.90800000000000003},"min":{"1min":0.51500000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.97699999999999998,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96299999999999997,"5min":1.754,"15min":1.754},"last":0.85699999999999998}]}]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":438057,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.247},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44600000000000001,"15min":0.45200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.32800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.67100000000000004,"15min":0.69499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.72999999999999998,"15min":0.753},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.57399999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.309},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.622},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.75,"15min":0.77500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83899999999999997,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.497}]},{"osd":5,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.70599999999999996,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.373},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.034,"5min":1.04,"15min":1.04},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0669999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":1.397,"15min":1.397},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.81599999999999995,"15min":0.83399999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0960000000000001,"5min":1.661,"15min":1.661},"last":0.66500000000000004}]},{"osd":8,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0349999999999999,"5min":1.079,"15min":1.079},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.93000000000000005,"15min":0.97299999999999998},"min":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.91900000000000004,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.434}]},{"osd":10,"last update":"Fri Jun 23 18:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0029999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":437534,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.42099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.501,"15min":0.49399999999999999},"min":{"1min":0.39400000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.68500000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.47399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45600000000000002,"15min":0.45100000000000001},"min":{"1min":0.35999999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63800000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.25700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.745,"5min":0.752,"15min":0.752},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.42899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76200000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.62}]},{"osd":4,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.52600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85699999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.86899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.61399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83199999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.89400000000000002}]},{"osd":10,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.56000000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.55300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":0.998,"15min":0.998},"last":0.92500000000000004}]},{"osd":11,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.94799999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":1029957,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61399999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63800000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.48999999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84599999999999997,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84999999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81299999999999994,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.73399999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.379}]},{"osd":5,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65600000000000003,"15min":0.63500000000000001},"min":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80200000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66500000000000004,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.877,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.65000000000000002},"min":{"1min":0.49299999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89400000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.752},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89200000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.71999999999999997},"min":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92100000000000004,"5min":1.024,"15min":1.024},"last":0.55300000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.93999999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.79800000000000004}]},{"osd":10,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94999999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.627,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93600000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":11,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73099999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495665,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":437452,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.58899999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.28699999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.379}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.89400000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70799999999999996,"5min":1.105,"15min":1.105},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65400000000000003,"5min":1.141,"15min":1.141},"last":0.432},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72199999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.872,"15min":0.872},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77200000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68300000000000005,"15min":0.69699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.82999999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91800000000000004,"5min":1.03,"15min":1.03},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97199999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58299999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69199999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96799999999999997,"5min":1.008,"15min":1.008},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71699999999999997,"15min":0.73599999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.47699999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.878,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":495469,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.48499999999999999},"min":{"1min":0.34799999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.84999999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.625},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52500000000000002,"15min":0.498},"min":{"1min":0.35699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73999999999999999,"5min":1.218,"15min":1.218},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.623,"15min":0.54900000000000004},"min":{"1min":0.436,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":1.25,"15min":1.25},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.91300000000000003,"5min":1.343,"15min":1.343},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.42799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.93200000000000005,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.49099999999999999,"15min":0.44},"min":{"1min":0.32300000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.42399999999999999},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77000000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.66500000000000004,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98399999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64200000000000002,"15min":0.60399999999999998},"min":{"1min":0.41799999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.96799999999999997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84599999999999997,"5min":1.169,"15min":1.169},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.60099999999999998},"min":{"1min":0.53200000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.093,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.754}]},{"osd":7,"last update":"Fri Jun 23 18:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.72799999999999998,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.80100000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.82299999999999995,"15min":0.80300000000000005},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0820000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98899999999999999,"5min":1.319,"15min":1.319},"last":0.77300000000000002}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.876,"15min":0.88900000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1180000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96399999999999997,"5min":1.288,"15min":1.288},"last":0.81499999999999995}]},{"osd":10,"last update":"Fri Jun 23 18:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.85599999999999998,"15min":0.86499999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.046,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.93400000000000005}]},{"osd":11,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":438717,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.48099999999999998},"min":{"1min":0.39200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51400000000000001,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.503}]},{"osd":2,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56100000000000005,"15min":0.50700000000000001},"min":{"1min":0.39200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.84999999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.372},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.55000000000000004},"min":{"1min":0.439,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.55400000000000005},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.64100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80800000000000005,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.443},{"interface":"front","average":{"1min":0.501,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82499999999999996,"5min":1.073,"15min":1.073},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Jun 23 18:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56399999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.876,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.76400000000000001,"15min":0.79000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0449999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73699999999999999,"15min":0.748},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.83899999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63800000000000001}]},{"osd":11,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69399999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":439024,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.36399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67300000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.38300000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56799999999999995,"15min":0.53000000000000003},"min":{"1min":0.52300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.47599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:22:00 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.34300000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.52200000000000002,"15min":0.48199999999999998},"min":{"1min":0.40300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0069999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59799999999999998}]},{"osd":8,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90500000000000003,"5min":1.095,"15min":1.095},"last":0.71299999999999997}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.621},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63300000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.71299999999999997,"15min":0.69199999999999995},"min":{"1min":0.58199999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79400000000000004}]},{"osd":11,"last update":"Fri Jun 23 18:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.745}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":437769,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.502,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":12.246,"5min":12.246,"15min":12.246},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58099999999999996,"15min":0.54200000000000004},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":8.3569999999999993,"5min":8.3569999999999993,"15min":8.3569999999999993},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.59099999999999997},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:21:26 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.70699999999999996,"15min":0.59799999999999998},"min":{"1min":0.377,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":12.183999999999999,"5min":12.183999999999999,"15min":12.183999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.59099999999999997,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.443}]},{"osd":7,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.69199999999999995,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3239999999999998,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.501},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.48299999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79600000000000004,"15min":0.75600000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":8.2650000000000006,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2649999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":17.315999999999999,"5min":17.315999999999999,"15min":17.315999999999999},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":497079,"data_compressed":14732,"data_compressed_allocated":20480,"data_compressed_original":69632,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,24,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.51700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40799999999999997,"15min":0.378},"min":{"1min":0.309,"5min":0.255,"15min":0.255},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":1.6459999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41699999999999998,"15min":0.38900000000000001},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60999999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":4.6230000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60599999999999998,"15min":0.55200000000000005},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81599999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.52800000000000002},"min":{"1min":0.40500000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":4.6520000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.629}]},{"osd":5,"last update":"Fri Jun 23 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63800000000000001,"15min":0.60699999999999998},"min":{"1min":0.55200000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.63400000000000001,"15min":0.60599999999999998},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":4.5309999999999997}]},{"osd":6,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":1.129,"15min":1.129},"last":1.6299999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":1.7150000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.497,"5min":0.378,"15min":0.378},"max":{"1min":0.87,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":7.2290000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.623,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97599999999999998,"5min":0.998,"15min":0.998},"last":4.694}]},{"osd":8,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.503,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94899999999999995,"5min":1.202,"15min":1.202},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.79600000000000004,"15min":0.77200000000000002},"min":{"1min":0.65200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":4.585}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":3.6360000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":3.8889999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77500000000000002,"15min":0.76000000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.022,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":11.475},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.754,"15min":0.74299999999999999},"min":{"1min":0.65800000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96599999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":4.718}]},{"osd":11,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":1.6830000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71999999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":1029641,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47999999999999998,"15min":0.46100000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.747,"5min":0.875,"15min":0.875},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.73199999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.372}]},{"osd":1,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54800000000000004,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81299999999999994,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.39600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78000000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.437,"15min":0.41699999999999998},"min":{"1min":0.34799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.44900000000000001,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.44900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85799999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":5.399},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":5.4690000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88800000000000001,"5min":0.998,"15min":0.998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.629},"min":{"1min":0.505,"5min":0.374,"15min":0.374},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53600000000000003}]},{"osd":9,"last update":"Fri Jun 23 18:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.60499999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.74299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":59840,"data_compressed":8925,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:22:11.728 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-06-23T18:22:11.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-06-23T18:22:11.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/821263591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b081058c0 msgr2=0x7f5b08107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/821263591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b081058c0 0x7f5b08107ca0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b0000b0a0 tx=0x7f5b0000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/821263591 shutdown_connections 2023-06-23T18:22:11.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/821263591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b081058c0 0x7f5b08107ca0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/821263591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b08100680 0x7f5b08102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/821263591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b08102fa0 0x7f5b08105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/821263591 >> 172.21.15.2:0/821263591 conn(0x7f5b080fa0e0 msgr2=0x7f5b080fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:11.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/821263591 shutdown_connections 2023-06-23T18:22:11.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/821263591 wait complete. 2023-06-23T18:22:11.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 Processor -- start 2023-06-23T18:22:11.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.881+0000 7f5b0f6aa700 1 -- start start 2023-06-23T18:22:11.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0f6aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b08100680 0x7f5b08197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0f6aa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b08102fa0 0x7f5b08197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0f6aa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b081058c0 0x7f5b0819f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0f6aa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0819f600 con 0x7f5b08100680 2023-06-23T18:22:11.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0f6aa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0819f740 con 0x7f5b081058c0 2023-06-23T18:22:11.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0f6aa700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0819f8d0 con 0x7f5b08102fa0 2023-06-23T18:22:11.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b08100680 0x7f5b08197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b08100680 0x7f5b08197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53908/0 (socket says 172.21.15.2:53908) 2023-06-23T18:22:11.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 -- 172.21.15.2:0/1039719446 learned_addr learned my addr 172.21.15.2:0/1039719446 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:11.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0cc45700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b08102fa0 0x7f5b08197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0dc47700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b081058c0 0x7f5b0819f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 -- 172.21.15.2:0/1039719446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b08102fa0 msgr2=0x7f5b08197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b08102fa0 0x7f5b08197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 -- 172.21.15.2:0/1039719446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b081058c0 msgr2=0x7f5b0819f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:11.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b081058c0 0x7f5b0819f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:11.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 -- 172.21.15.2:0/1039719446 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00009d00 con 0x7f5b08100680 2023-06-23T18:22:11.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0d446700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b08100680 0x7f5b08197a80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e9a0 tx=0x7f5b0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:11.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5afe7fc700 1 -- 172.21.15.2:0/1039719446 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04009790 con 0x7f5b08100680 2023-06-23T18:22:11.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0819fbb0 con 0x7f5b08100680 2023-06-23T18:22:11.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5afe7fc700 1 -- 172.21.15.2:0/1039719446 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b04010430 con 0x7f5b08100680 2023-06-23T18:22:11.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5afe7fc700 1 -- 172.21.15.2:0/1039719446 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400cea0 con 0x7f5b08100680 2023-06-23T18:22:11.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.882+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b081a00c0 con 0x7f5b08100680 2023-06-23T18:22:11.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.883+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0810b1c0 con 0x7f5b08100680 2023-06-23T18:22:11.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.885+0000 7f5afe7fc700 1 -- 172.21.15.2:0/1039719446 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5b040105a0 con 0x7f5b08100680 2023-06-23T18:22:11.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.885+0000 7f5afe7fc700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5af40a1150 0x7f5af40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:11.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.885+0000 7f5b0cc45700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5af40a1150 0x7f5af40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:11.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.885+0000 7f5afe7fc700 1 -- 172.21.15.2:0/1039719446 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5b04014070 con 0x7f5b08100680 2023-06-23T18:22:11.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.886+0000 7f5b0cc45700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5af40a1150 0x7f5af40a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5af8007be0 tx=0x7f5af80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:11.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:11.888+0000 7f5afe7fc700 1 -- 172.21.15.2:0/1039719446 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f5b04096430 con 0x7f5b08100680 2023-06-23T18:22:12.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.084+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 --> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b08106ca0 con 0x7f5af40a1150 2023-06-23T18:22:12.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.097+0000 7f5afe7fc700 1 -- 172.21.15.2:0/1039719446 <== mgr.14150 v2:172.21.15.2:6800/3622114004 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344209 (secure 0 0 0) 0x7f5b08106ca0 con 0x7f5af40a1150 2023-06-23T18:22:12.106 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:12.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5af40a1150 msgr2=0x7f5af40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5af40a1150 0x7f5af40a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5af8007be0 tx=0x7f5af80073d0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b08100680 msgr2=0x7f5b08197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b08100680 0x7f5b08197a80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e9a0 tx=0x7f5b0400ecb0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 shutdown_connections 2023-06-23T18:22:12.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5af40a1150 0x7f5af40a3600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b081058c0 0x7f5b0819f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b08100680 0x7f5b08197a80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 --2- 172.21.15.2:0/1039719446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b08102fa0 0x7f5b08197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 >> 172.21.15.2:0/1039719446 conn(0x7f5b080fa0e0 msgr2=0x7f5b08101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:12.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 shutdown_connections 2023-06-23T18:22:12.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.100+0000 7f5b0f6aa700 1 -- 172.21.15.2:0/1039719446 wait complete. 2023-06-23T18:22:12.111 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-06-23T18:22:12.117 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2023-06-23T18:22:10.757253+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653855,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":618,"num_read_kb":420,"num_write":345,"num_write_kb":3053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1238346,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":637,"ondisk_log_size":637,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334516,"kb_used_data":11092,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607308,"statfs":{"total":1151940427776,"available":1151597883392,"internally_reserved":0,"allocated":11358208,"data_stored":7207136,"data_compressed":186066,"data_compressed_allocated":245760,"data_compressed_original":2019328,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,24,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1839,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":33,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006963"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785637+0000","last_change":"2023-06-23T18:20:44.466650+0000","last_active":"2023-06-23T18:22:06.785637+0000","last_peered":"2023-06-23T18:22:06.785637+0000","last_clean":"2023-06-23T18:22:06.785637+0000","last_became_active":"2023-06-23T18:20:44.466425+0000","last_became_peered":"2023-06-23T18:20:44.466425+0000","last_unstale":"2023-06-23T18:22:06.785637+0000","last_undegraded":"2023-06-23T18:22:06.785637+0000","last_fullsized":"2023-06-23T18:22:06.785637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:30:41.271809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080017+0000","last_change":"2023-06-23T18:20:44.465762+0000","last_active":"2023-06-23T18:21:38.080017+0000","last_peered":"2023-06-23T18:21:38.080017+0000","last_clean":"2023-06-23T18:21:38.080017+0000","last_became_active":"2023-06-23T18:20:44.463592+0000","last_became_peered":"2023-06-23T18:20:44.463592+0000","last_unstale":"2023-06-23T18:21:38.080017+0000","last_undegraded":"2023-06-23T18:21:38.080017+0000","last_fullsized":"2023-06-23T18:21:38.080017+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:04:03.928905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079784+0000","last_change":"2023-06-23T18:20:44.464162+0000","last_active":"2023-06-23T18:21:38.079784+0000","last_peered":"2023-06-23T18:21:38.079784+0000","last_clean":"2023-06-23T18:21:38.079784+0000","last_became_active":"2023-06-23T18:20:44.463854+0000","last_became_peered":"2023-06-23T18:20:44.463854+0000","last_unstale":"2023-06-23T18:21:38.079784+0000","last_undegraded":"2023-06-23T18:21:38.079784+0000","last_fullsized":"2023-06-23T18:21:38.079784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:52:10.241558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.777090+0000","last_change":"2023-06-23T18:20:44.462923+0000","last_active":"2023-06-23T18:22:06.777090+0000","last_peered":"2023-06-23T18:22:06.777090+0000","last_clean":"2023-06-23T18:22:06.777090+0000","last_became_active":"2023-06-23T18:20:44.462511+0000","last_became_peered":"2023-06-23T18:20:44.462511+0000","last_unstale":"2023-06-23T18:22:06.777090+0000","last_undegraded":"2023-06-23T18:22:06.777090+0000","last_fullsized":"2023-06-23T18:22:06.777090+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:41:07.780308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.775980+0000","last_change":"2023-06-23T18:20:44.454727+0000","last_active":"2023-06-23T18:22:06.775980+0000","last_peered":"2023-06-23T18:22:06.775980+0000","last_clean":"2023-06-23T18:22:06.775980+0000","last_became_active":"2023-06-23T18:20:44.454418+0000","last_became_peered":"2023-06-23T18:20:44.454418+0000","last_unstale":"2023-06-23T18:22:06.775980+0000","last_undegraded":"2023-06-23T18:22:06.775980+0000","last_fullsized":"2023-06-23T18:22:06.775980+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:06:57.921477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790102+0000","last_change":"2023-06-23T18:20:44.460636+0000","last_active":"2023-06-23T18:22:06.790102+0000","last_peered":"2023-06-23T18:22:06.790102+0000","last_clean":"2023-06-23T18:22:06.790102+0000","last_became_active":"2023-06-23T18:20:44.460305+0000","last_became_peered":"2023-06-23T18:20:44.460305+0000","last_unstale":"2023-06-23T18:22:06.790102+0000","last_undegraded":"2023-06-23T18:22:06.790102+0000","last_fullsized":"2023-06-23T18:22:06.790102+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:08:27.871589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785720+0000","last_change":"2023-06-23T18:20:44.463552+0000","last_active":"2023-06-23T18:22:06.785720+0000","last_peered":"2023-06-23T18:22:06.785720+0000","last_clean":"2023-06-23T18:22:06.785720+0000","last_became_active":"2023-06-23T18:20:44.463187+0000","last_became_peered":"2023-06-23T18:20:44.463187+0000","last_unstale":"2023-06-23T18:22:06.785720+0000","last_undegraded":"2023-06-23T18:22:06.785720+0000","last_fullsized":"2023-06-23T18:22:06.785720+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:16:36.563197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790515+0000","last_change":"2023-06-23T18:21:38.083209+0000","last_active":"2023-06-23T18:22:06.790515+0000","last_peered":"2023-06-23T18:22:06.790515+0000","last_clean":"2023-06-23T18:22:06.790515+0000","last_became_active":"2023-06-23T18:21:38.082912+0000","last_became_peered":"2023-06-23T18:21:38.082912+0000","last_unstale":"2023-06-23T18:22:06.790515+0000","last_undegraded":"2023-06-23T18:22:06.790515+0000","last_fullsized":"2023-06-23T18:22:06.790515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:54:52.291671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778393+0000","last_change":"2023-06-23T18:20:44.461199+0000","last_active":"2023-06-23T18:22:06.778393+0000","last_peered":"2023-06-23T18:22:06.778393+0000","last_clean":"2023-06-23T18:22:06.778393+0000","last_became_active":"2023-06-23T18:20:44.460891+0000","last_became_peered":"2023-06-23T18:20:44.460891+0000","last_unstale":"2023-06-23T18:22:06.778393+0000","last_undegraded":"2023-06-23T18:22:06.778393+0000","last_fullsized":"2023-06-23T18:22:06.778393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:21:35.270311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.791756+0000","last_change":"2023-06-23T18:21:38.084799+0000","last_active":"2023-06-23T18:22:06.791756+0000","last_peered":"2023-06-23T18:22:06.791756+0000","last_clean":"2023-06-23T18:22:06.791756+0000","last_became_active":"2023-06-23T18:21:38.084336+0000","last_became_peered":"2023-06-23T18:21:38.084336+0000","last_unstale":"2023-06-23T18:22:06.791756+0000","last_undegraded":"2023-06-23T18:22:06.791756+0000","last_fullsized":"2023-06-23T18:22:06.791756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:09:38.488565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789987+0000","last_change":"2023-06-23T18:20:44.453337+0000","last_active":"2023-06-23T18:22:06.789987+0000","last_peered":"2023-06-23T18:22:06.789987+0000","last_clean":"2023-06-23T18:22:06.789987+0000","last_became_active":"2023-06-23T18:20:44.452925+0000","last_became_peered":"2023-06-23T18:20:44.452925+0000","last_unstale":"2023-06-23T18:22:06.789987+0000","last_undegraded":"2023-06-23T18:22:06.789987+0000","last_fullsized":"2023-06-23T18:22:06.789987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:39:25.750095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.774400+0000","last_change":"2023-06-23T18:20:44.456710+0000","last_active":"2023-06-23T18:22:06.774400+0000","last_peered":"2023-06-23T18:22:06.774400+0000","last_clean":"2023-06-23T18:22:06.774400+0000","last_became_active":"2023-06-23T18:20:44.456411+0000","last_became_peered":"2023-06-23T18:20:44.456411+0000","last_unstale":"2023-06-23T18:22:06.774400+0000","last_undegraded":"2023-06-23T18:22:06.774400+0000","last_fullsized":"2023-06-23T18:22:06.774400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:42:13.109612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072919+0000","last_change":"2023-06-23T18:20:44.453586+0000","last_active":"2023-06-23T18:21:38.072919+0000","last_peered":"2023-06-23T18:21:38.072919+0000","last_clean":"2023-06-23T18:21:38.072919+0000","last_became_active":"2023-06-23T18:20:44.453239+0000","last_became_peered":"2023-06-23T18:20:44.453239+0000","last_unstale":"2023-06-23T18:21:38.072919+0000","last_undegraded":"2023-06-23T18:21:38.072919+0000","last_fullsized":"2023-06-23T18:21:38.072919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:36:37.353139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079914+0000","last_change":"2023-06-23T18:20:44.465627+0000","last_active":"2023-06-23T18:21:38.079914+0000","last_peered":"2023-06-23T18:21:38.079914+0000","last_clean":"2023-06-23T18:21:38.079914+0000","last_became_active":"2023-06-23T18:20:44.464568+0000","last_became_peered":"2023-06-23T18:20:44.464568+0000","last_unstale":"2023-06-23T18:21:38.079914+0000","last_undegraded":"2023-06-23T18:21:38.079914+0000","last_fullsized":"2023-06-23T18:21:38.079914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:15:39.010793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.079452+0000","last_change":"2023-06-23T18:20:44.464169+0000","last_active":"2023-06-23T18:21:38.079452+0000","last_peered":"2023-06-23T18:21:38.079452+0000","last_clean":"2023-06-23T18:21:38.079452+0000","last_became_active":"2023-06-23T18:20:44.463867+0000","last_became_peered":"2023-06-23T18:20:44.463867+0000","last_unstale":"2023-06-23T18:21:38.079452+0000","last_undegraded":"2023-06-23T18:21:38.079452+0000","last_fullsized":"2023-06-23T18:21:38.079452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:12:31.642855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792291+0000","last_change":"2023-06-23T18:20:44.467101+0000","last_active":"2023-06-23T18:22:06.792291+0000","last_peered":"2023-06-23T18:22:06.792291+0000","last_clean":"2023-06-23T18:22:06.792291+0000","last_became_active":"2023-06-23T18:20:44.466788+0000","last_became_peered":"2023-06-23T18:20:44.466788+0000","last_unstale":"2023-06-23T18:22:06.792291+0000","last_undegraded":"2023-06-23T18:22:06.792291+0000","last_fullsized":"2023-06-23T18:22:06.792291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:41:04.135555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785439+0000","last_change":"2023-06-23T18:20:44.466055+0000","last_active":"2023-06-23T18:22:06.785439+0000","last_peered":"2023-06-23T18:22:06.785439+0000","last_clean":"2023-06-23T18:22:06.785439+0000","last_became_active":"2023-06-23T18:20:44.465608+0000","last_became_peered":"2023-06-23T18:20:44.465608+0000","last_unstale":"2023-06-23T18:22:06.785439+0000","last_undegraded":"2023-06-23T18:22:06.785439+0000","last_fullsized":"2023-06-23T18:22:06.785439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:16:31.985669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790712+0000","last_change":"2023-06-23T18:20:44.459414+0000","last_active":"2023-06-23T18:22:06.790712+0000","last_peered":"2023-06-23T18:22:06.790712+0000","last_clean":"2023-06-23T18:22:06.790712+0000","last_became_active":"2023-06-23T18:20:44.459014+0000","last_became_peered":"2023-06-23T18:20:44.459014+0000","last_unstale":"2023-06-23T18:22:06.790712+0000","last_undegraded":"2023-06-23T18:22:06.790712+0000","last_fullsized":"2023-06-23T18:22:06.790712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:23:16.590974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.774664+0000","last_change":"2023-06-23T18:20:44.446784+0000","last_active":"2023-06-23T18:22:06.774664+0000","last_peered":"2023-06-23T18:22:06.774664+0000","last_clean":"2023-06-23T18:22:06.774664+0000","last_became_active":"2023-06-23T18:20:44.446464+0000","last_became_peered":"2023-06-23T18:20:44.446464+0000","last_unstale":"2023-06-23T18:22:06.774664+0000","last_undegraded":"2023-06-23T18:22:06.774664+0000","last_fullsized":"2023-06-23T18:22:06.774664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:35:12.334055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"82'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075653+0000","last_change":"2023-06-23T18:20:44.039326+0000","last_active":"2023-06-23T18:21:38.075653+0000","last_peered":"2023-06-23T18:21:38.075653+0000","last_clean":"2023-06-23T18:21:38.075653+0000","last_became_active":"2023-06-23T18:20:44.038787+0000","last_became_peered":"2023-06-23T18:20:44.038787+0000","last_unstale":"2023-06-23T18:21:38.075653+0000","last_undegraded":"2023-06-23T18:21:38.075653+0000","last_fullsized":"2023-06-23T18:21:38.075653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:49:48.612922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081147+0000","last_change":"2023-06-23T18:20:44.465269+0000","last_active":"2023-06-23T18:21:38.081147+0000","last_peered":"2023-06-23T18:21:38.081147+0000","last_clean":"2023-06-23T18:21:38.081147+0000","last_became_active":"2023-06-23T18:20:44.464973+0000","last_became_peered":"2023-06-23T18:20:44.464973+0000","last_unstale":"2023-06-23T18:21:38.081147+0000","last_undegraded":"2023-06-23T18:21:38.081147+0000","last_fullsized":"2023-06-23T18:21:38.081147+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T06:09:55.905699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081281+0000","last_change":"2023-06-23T18:20:44.037175+0000","last_active":"2023-06-23T18:21:38.081281+0000","last_peered":"2023-06-23T18:21:38.081281+0000","last_clean":"2023-06-23T18:21:38.081281+0000","last_became_active":"2023-06-23T18:20:44.036788+0000","last_became_peered":"2023-06-23T18:20:44.036788+0000","last_unstale":"2023-06-23T18:21:38.081281+0000","last_undegraded":"2023-06-23T18:21:38.081281+0000","last_fullsized":"2023-06-23T18:21:38.081281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:19:02.349726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080329+0000","last_change":"2023-06-23T18:20:44.463344+0000","last_active":"2023-06-23T18:21:38.080329+0000","last_peered":"2023-06-23T18:21:38.080329+0000","last_clean":"2023-06-23T18:21:38.080329+0000","last_became_active":"2023-06-23T18:20:44.462649+0000","last_became_peered":"2023-06-23T18:20:44.462649+0000","last_unstale":"2023-06-23T18:21:38.080329+0000","last_undegraded":"2023-06-23T18:21:38.080329+0000","last_fullsized":"2023-06-23T18:21:38.080329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:36:25.801756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076445+0000","last_change":"2023-06-23T18:20:44.045525+0000","last_active":"2023-06-23T18:21:38.076445+0000","last_peered":"2023-06-23T18:21:38.076445+0000","last_clean":"2023-06-23T18:21:38.076445+0000","last_became_active":"2023-06-23T18:20:44.045212+0000","last_became_peered":"2023-06-23T18:20:44.045212+0000","last_unstale":"2023-06-23T18:21:38.076445+0000","last_undegraded":"2023-06-23T18:21:38.076445+0000","last_fullsized":"2023-06-23T18:21:38.076445+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:54:25.401039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790279+0000","last_change":"2023-06-23T18:20:44.461071+0000","last_active":"2023-06-23T18:22:06.790279+0000","last_peered":"2023-06-23T18:22:06.790279+0000","last_clean":"2023-06-23T18:22:06.790279+0000","last_became_active":"2023-06-23T18:20:44.460726+0000","last_became_peered":"2023-06-23T18:20:44.460726+0000","last_unstale":"2023-06-23T18:22:06.790279+0000","last_undegraded":"2023-06-23T18:22:06.790279+0000","last_fullsized":"2023-06-23T18:22:06.790279+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:04:57.655489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076111+0000","last_change":"2023-06-23T18:20:44.037162+0000","last_active":"2023-06-23T18:21:38.076111+0000","last_peered":"2023-06-23T18:21:38.076111+0000","last_clean":"2023-06-23T18:21:38.076111+0000","last_became_active":"2023-06-23T18:20:44.036809+0000","last_became_peered":"2023-06-23T18:20:44.036809+0000","last_unstale":"2023-06-23T18:21:38.076111+0000","last_undegraded":"2023-06-23T18:21:38.076111+0000","last_fullsized":"2023-06-23T18:21:38.076111+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:05:40.391806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.786602+0000","last_change":"2023-06-23T18:20:44.463525+0000","last_active":"2023-06-23T18:22:06.786602+0000","last_peered":"2023-06-23T18:22:06.786602+0000","last_clean":"2023-06-23T18:22:06.786602+0000","last_became_active":"2023-06-23T18:20:44.463245+0000","last_became_peered":"2023-06-23T18:20:44.463245+0000","last_unstale":"2023-06-23T18:22:06.786602+0000","last_undegraded":"2023-06-23T18:22:06.786602+0000","last_fullsized":"2023-06-23T18:22:06.786602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:29:11.041855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":108,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:08.167164+0000","last_change":"2023-06-23T18:20:44.044415+0000","last_active":"2023-06-23T18:22:08.167164+0000","last_peered":"2023-06-23T18:22:08.167164+0000","last_clean":"2023-06-23T18:22:08.167164+0000","last_became_active":"2023-06-23T18:20:44.044085+0000","last_became_peered":"2023-06-23T18:20:44.044085+0000","last_unstale":"2023-06-23T18:22:08.167164+0000","last_undegraded":"2023-06-23T18:22:08.167164+0000","last_fullsized":"2023-06-23T18:22:08.167164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:43:24.443528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1057,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778243+0000","last_change":"2023-06-23T18:21:38.077968+0000","last_active":"2023-06-23T18:22:06.778243+0000","last_peered":"2023-06-23T18:22:06.778243+0000","last_clean":"2023-06-23T18:22:06.778243+0000","last_became_active":"2023-06-23T18:21:38.077519+0000","last_became_peered":"2023-06-23T18:21:38.077519+0000","last_unstale":"2023-06-23T18:22:06.778243+0000","last_undegraded":"2023-06-23T18:22:06.778243+0000","last_fullsized":"2023-06-23T18:22:06.778243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:03:50.715602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076205+0000","last_change":"2023-06-23T18:20:44.039604+0000","last_active":"2023-06-23T18:21:38.076205+0000","last_peered":"2023-06-23T18:21:38.076205+0000","last_clean":"2023-06-23T18:21:38.076205+0000","last_became_active":"2023-06-23T18:20:44.039317+0000","last_became_peered":"2023-06-23T18:20:44.039317+0000","last_unstale":"2023-06-23T18:21:38.076205+0000","last_undegraded":"2023-06-23T18:21:38.076205+0000","last_fullsized":"2023-06-23T18:21:38.076205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:57:55.720236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780609+0000","last_change":"2023-06-23T18:20:44.465748+0000","last_active":"2023-06-23T18:22:06.780609+0000","last_peered":"2023-06-23T18:22:06.780609+0000","last_clean":"2023-06-23T18:22:06.780609+0000","last_became_active":"2023-06-23T18:20:44.465430+0000","last_became_peered":"2023-06-23T18:20:44.465430+0000","last_unstale":"2023-06-23T18:22:06.780609+0000","last_undegraded":"2023-06-23T18:22:06.780609+0000","last_fullsized":"2023-06-23T18:22:06.780609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:29:38.870085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072805+0000","last_change":"2023-06-23T18:20:44.034388+0000","last_active":"2023-06-23T18:21:38.072805+0000","last_peered":"2023-06-23T18:21:38.072805+0000","last_clean":"2023-06-23T18:21:38.072805+0000","last_became_active":"2023-06-23T18:20:44.034067+0000","last_became_peered":"2023-06-23T18:20:44.034067+0000","last_unstale":"2023-06-23T18:21:38.072805+0000","last_undegraded":"2023-06-23T18:21:38.072805+0000","last_fullsized":"2023-06-23T18:21:38.072805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:21:51.273582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080025+0000","last_change":"2023-06-23T18:20:44.453280+0000","last_active":"2023-06-23T18:21:38.080025+0000","last_peered":"2023-06-23T18:21:38.080025+0000","last_clean":"2023-06-23T18:21:38.080025+0000","last_became_active":"2023-06-23T18:20:44.452980+0000","last_became_peered":"2023-06-23T18:20:44.452980+0000","last_unstale":"2023-06-23T18:21:38.080025+0000","last_undegraded":"2023-06-23T18:21:38.080025+0000","last_fullsized":"2023-06-23T18:21:38.080025+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:01:27.547351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.082028+0000","last_change":"2023-06-23T18:20:44.036165+0000","last_active":"2023-06-23T18:21:38.082028+0000","last_peered":"2023-06-23T18:21:38.082028+0000","last_clean":"2023-06-23T18:21:38.082028+0000","last_became_active":"2023-06-23T18:20:44.035313+0000","last_became_peered":"2023-06-23T18:20:44.035313+0000","last_unstale":"2023-06-23T18:21:38.082028+0000","last_undegraded":"2023-06-23T18:21:38.082028+0000","last_fullsized":"2023-06-23T18:21:38.082028+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:15:36.463724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790545+0000","last_change":"2023-06-23T18:20:44.468358+0000","last_active":"2023-06-23T18:22:06.790545+0000","last_peered":"2023-06-23T18:22:06.790545+0000","last_clean":"2023-06-23T18:22:06.790545+0000","last_became_active":"2023-06-23T18:20:44.468032+0000","last_became_peered":"2023-06-23T18:20:44.468032+0000","last_unstale":"2023-06-23T18:22:06.790545+0000","last_undegraded":"2023-06-23T18:22:06.790545+0000","last_fullsized":"2023-06-23T18:22:06.790545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:23:56.870683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":101,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.101308+0000","last_change":"2023-06-23T18:20:44.037921+0000","last_active":"2023-06-23T18:22:07.101308+0000","last_peered":"2023-06-23T18:22:07.101308+0000","last_clean":"2023-06-23T18:22:07.101308+0000","last_became_active":"2023-06-23T18:20:44.037571+0000","last_became_peered":"2023-06-23T18:20:44.037571+0000","last_unstale":"2023-06-23T18:22:07.101308+0000","last_undegraded":"2023-06-23T18:22:07.101308+0000","last_fullsized":"2023-06-23T18:22:07.101308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:32:06.313509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792659+0000","last_change":"2023-06-23T18:20:44.460897+0000","last_active":"2023-06-23T18:22:06.792659+0000","last_peered":"2023-06-23T18:22:06.792659+0000","last_clean":"2023-06-23T18:22:06.792659+0000","last_became_active":"2023-06-23T18:20:44.460476+0000","last_became_peered":"2023-06-23T18:20:44.460476+0000","last_unstale":"2023-06-23T18:22:06.792659+0000","last_undegraded":"2023-06-23T18:22:06.792659+0000","last_fullsized":"2023-06-23T18:22:06.792659+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:43:49.396858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073866+0000","last_change":"2023-06-23T18:20:44.033013+0000","last_active":"2023-06-23T18:21:38.073866+0000","last_peered":"2023-06-23T18:21:38.073866+0000","last_clean":"2023-06-23T18:21:38.073866+0000","last_became_active":"2023-06-23T18:20:44.032580+0000","last_became_peered":"2023-06-23T18:20:44.032580+0000","last_unstale":"2023-06-23T18:21:38.073866+0000","last_undegraded":"2023-06-23T18:21:38.073866+0000","last_fullsized":"2023-06-23T18:21:38.073866+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:12:54.441960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080378+0000","last_change":"2023-06-23T18:20:44.463585+0000","last_active":"2023-06-23T18:21:38.080378+0000","last_peered":"2023-06-23T18:21:38.080378+0000","last_clean":"2023-06-23T18:21:38.080378+0000","last_became_active":"2023-06-23T18:20:44.462512+0000","last_became_peered":"2023-06-23T18:20:44.462512+0000","last_unstale":"2023-06-23T18:21:38.080378+0000","last_undegraded":"2023-06-23T18:21:38.080378+0000","last_fullsized":"2023-06-23T18:21:38.080378+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:01:51.999262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081156+0000","last_change":"2023-06-23T18:20:44.037134+0000","last_active":"2023-06-23T18:21:38.081156+0000","last_peered":"2023-06-23T18:21:38.081156+0000","last_clean":"2023-06-23T18:21:38.081156+0000","last_became_active":"2023-06-23T18:20:44.036536+0000","last_became_peered":"2023-06-23T18:20:44.036536+0000","last_unstale":"2023-06-23T18:21:38.081156+0000","last_undegraded":"2023-06-23T18:21:38.081156+0000","last_fullsized":"2023-06-23T18:21:38.081156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:45:16.097692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.785436+0000","last_change":"2023-06-23T18:20:44.454453+0000","last_active":"2023-06-23T18:22:06.785436+0000","last_peered":"2023-06-23T18:22:06.785436+0000","last_clean":"2023-06-23T18:22:06.785436+0000","last_became_active":"2023-06-23T18:20:44.454133+0000","last_became_peered":"2023-06-23T18:20:44.454133+0000","last_unstale":"2023-06-23T18:22:06.785436+0000","last_undegraded":"2023-06-23T18:22:06.785436+0000","last_fullsized":"2023-06-23T18:22:06.785436+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:12:10.199634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'33","reported_seq":234,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.148559+0000","last_change":"2023-06-23T18:21:38.108843+0000","last_active":"2023-06-23T18:22:07.148559+0000","last_peered":"2023-06-23T18:22:07.148559+0000","last_clean":"2023-06-23T18:22:07.148559+0000","last_became_active":"2023-06-23T18:21:38.102073+0000","last_became_peered":"2023-06-23T18:21:38.102073+0000","last_unstale":"2023-06-23T18:22:07.148559+0000","last_undegraded":"2023-06-23T18:22:07.148559+0000","last_fullsized":"2023-06-23T18:22:07.148559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:17:04.550195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58230,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":155,"num_read_kb":134,"num_write":34,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":57498,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.787027+0000","last_change":"2023-06-23T18:20:44.456558+0000","last_active":"2023-06-23T18:22:06.787027+0000","last_peered":"2023-06-23T18:22:06.787027+0000","last_clean":"2023-06-23T18:22:06.787027+0000","last_became_active":"2023-06-23T18:20:44.456199+0000","last_became_peered":"2023-06-23T18:20:44.456199+0000","last_unstale":"2023-06-23T18:22:06.787027+0000","last_undegraded":"2023-06-23T18:22:06.787027+0000","last_fullsized":"2023-06-23T18:22:06.787027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:53:48.531640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"82'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080152+0000","last_change":"2023-06-23T18:20:44.034101+0000","last_active":"2023-06-23T18:21:38.080152+0000","last_peered":"2023-06-23T18:21:38.080152+0000","last_clean":"2023-06-23T18:21:38.080152+0000","last_became_active":"2023-06-23T18:20:44.033287+0000","last_became_peered":"2023-06-23T18:20:44.033287+0000","last_unstale":"2023-06-23T18:21:38.080152+0000","last_undegraded":"2023-06-23T18:21:38.080152+0000","last_fullsized":"2023-06-23T18:21:38.080152+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:44:53.381086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072861+0000","last_change":"2023-06-23T18:20:44.446298+0000","last_active":"2023-06-23T18:21:38.072861+0000","last_peered":"2023-06-23T18:21:38.072861+0000","last_clean":"2023-06-23T18:21:38.072861+0000","last_became_active":"2023-06-23T18:20:44.445922+0000","last_became_peered":"2023-06-23T18:20:44.445922+0000","last_unstale":"2023-06-23T18:21:38.072861+0000","last_undegraded":"2023-06-23T18:21:38.072861+0000","last_fullsized":"2023-06-23T18:21:38.072861+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:25:51.263367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080693+0000","last_change":"2023-06-23T18:20:44.035791+0000","last_active":"2023-06-23T18:21:38.080693+0000","last_peered":"2023-06-23T18:21:38.080693+0000","last_clean":"2023-06-23T18:21:38.080693+0000","last_became_active":"2023-06-23T18:20:44.035416+0000","last_became_peered":"2023-06-23T18:20:44.035416+0000","last_unstale":"2023-06-23T18:21:38.080693+0000","last_undegraded":"2023-06-23T18:21:38.080693+0000","last_fullsized":"2023-06-23T18:21:38.080693+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:49:57.755758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790515+0000","last_change":"2023-06-23T18:20:44.455474+0000","last_active":"2023-06-23T18:22:06.790515+0000","last_peered":"2023-06-23T18:22:06.790515+0000","last_clean":"2023-06-23T18:22:06.790515+0000","last_became_active":"2023-06-23T18:20:44.455098+0000","last_became_peered":"2023-06-23T18:20:44.455098+0000","last_unstale":"2023-06-23T18:22:06.790515+0000","last_undegraded":"2023-06-23T18:22:06.790515+0000","last_fullsized":"2023-06-23T18:22:06.790515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:15:19.154271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":107,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:07.164451+0000","last_change":"2023-06-23T18:20:44.030305+0000","last_active":"2023-06-23T18:22:07.164451+0000","last_peered":"2023-06-23T18:22:07.164451+0000","last_clean":"2023-06-23T18:22:07.164451+0000","last_became_active":"2023-06-23T18:20:44.029898+0000","last_became_peered":"2023-06-23T18:20:44.029898+0000","last_unstale":"2023-06-23T18:22:07.164451+0000","last_undegraded":"2023-06-23T18:22:07.164451+0000","last_fullsized":"2023-06-23T18:22:07.164451+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:49:33.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1554,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790958+0000","last_change":"2023-06-23T18:20:44.451321+0000","last_active":"2023-06-23T18:22:06.790958+0000","last_peered":"2023-06-23T18:22:06.790958+0000","last_clean":"2023-06-23T18:22:06.790958+0000","last_became_active":"2023-06-23T18:20:44.450549+0000","last_became_peered":"2023-06-23T18:20:44.450549+0000","last_unstale":"2023-06-23T18:22:06.790958+0000","last_undegraded":"2023-06-23T18:22:06.790958+0000","last_fullsized":"2023-06-23T18:22:06.790958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:31:37.630032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"82'15","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.097392+0000","last_change":"2023-06-23T18:21:38.097392+0000","last_active":"2023-06-23T18:21:38.097392+0000","last_peered":"2023-06-23T18:21:38.097392+0000","last_clean":"2023-06-23T18:21:38.097392+0000","last_became_active":"2023-06-23T18:21:38.082554+0000","last_became_peered":"2023-06-23T18:21:38.082554+0000","last_unstale":"2023-06-23T18:21:38.097392+0000","last_undegraded":"2023-06-23T18:21:38.097392+0000","last_fullsized":"2023-06-23T18:21:38.097392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:24:59.677445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789557+0000","last_change":"2023-06-23T18:21:38.085114+0000","last_active":"2023-06-23T18:22:06.789557+0000","last_peered":"2023-06-23T18:22:06.789557+0000","last_clean":"2023-06-23T18:22:06.789557+0000","last_became_active":"2023-06-23T18:21:38.084575+0000","last_became_peered":"2023-06-23T18:21:38.084575+0000","last_unstale":"2023-06-23T18:22:06.789557+0000","last_undegraded":"2023-06-23T18:22:06.789557+0000","last_fullsized":"2023-06-23T18:22:06.789557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:27:44.436006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"82'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076256+0000","last_change":"2023-06-23T18:20:44.038961+0000","last_active":"2023-06-23T18:21:38.076256+0000","last_peered":"2023-06-23T18:21:38.076256+0000","last_clean":"2023-06-23T18:21:38.076256+0000","last_became_active":"2023-06-23T18:20:44.038674+0000","last_became_peered":"2023-06-23T18:20:44.038674+0000","last_unstale":"2023-06-23T18:21:38.076256+0000","last_undegraded":"2023-06-23T18:21:38.076256+0000","last_fullsized":"2023-06-23T18:21:38.076256+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:37:52.894959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.776481+0000","last_change":"2023-06-23T18:20:44.461077+0000","last_active":"2023-06-23T18:22:06.776481+0000","last_peered":"2023-06-23T18:22:06.776481+0000","last_clean":"2023-06-23T18:22:06.776481+0000","last_became_active":"2023-06-23T18:20:44.460767+0000","last_became_peered":"2023-06-23T18:20:44.460767+0000","last_unstale":"2023-06-23T18:22:06.776481+0000","last_undegraded":"2023-06-23T18:22:06.776481+0000","last_fullsized":"2023-06-23T18:22:06.776481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:33:02.755726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075336+0000","last_change":"2023-06-23T18:20:44.042114+0000","last_active":"2023-06-23T18:21:38.075336+0000","last_peered":"2023-06-23T18:21:38.075336+0000","last_clean":"2023-06-23T18:21:38.075336+0000","last_became_active":"2023-06-23T18:20:44.041822+0000","last_became_peered":"2023-06-23T18:20:44.041822+0000","last_unstale":"2023-06-23T18:21:38.075336+0000","last_undegraded":"2023-06-23T18:21:38.075336+0000","last_fullsized":"2023-06-23T18:21:38.075336+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:09:25.545580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789488+0000","last_change":"2023-06-23T18:20:44.452903+0000","last_active":"2023-06-23T18:22:06.789488+0000","last_peered":"2023-06-23T18:22:06.789488+0000","last_clean":"2023-06-23T18:22:06.789488+0000","last_became_active":"2023-06-23T18:20:44.452539+0000","last_became_peered":"2023-06-23T18:20:44.452539+0000","last_unstale":"2023-06-23T18:22:06.789488+0000","last_undegraded":"2023-06-23T18:22:06.789488+0000","last_fullsized":"2023-06-23T18:22:06.789488+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:00:37.883811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"82'1","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073621+0000","last_change":"2023-06-23T18:20:44.032905+0000","last_active":"2023-06-23T18:21:38.073621+0000","last_peered":"2023-06-23T18:21:38.073621+0000","last_clean":"2023-06-23T18:21:38.073621+0000","last_became_active":"2023-06-23T18:20:44.032606+0000","last_became_peered":"2023-06-23T18:20:44.032606+0000","last_unstale":"2023-06-23T18:21:38.073621+0000","last_undegraded":"2023-06-23T18:21:38.073621+0000","last_fullsized":"2023-06-23T18:21:38.073621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:44:01.287033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789982+0000","last_change":"2023-06-23T18:20:44.465580+0000","last_active":"2023-06-23T18:22:06.789982+0000","last_peered":"2023-06-23T18:22:06.789982+0000","last_clean":"2023-06-23T18:22:06.789982+0000","last_became_active":"2023-06-23T18:20:44.465264+0000","last_became_peered":"2023-06-23T18:20:44.465264+0000","last_unstale":"2023-06-23T18:22:06.789982+0000","last_undegraded":"2023-06-23T18:22:06.789982+0000","last_fullsized":"2023-06-23T18:22:06.789982+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:40:26.704880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.778940+0000","last_change":"2023-06-23T18:20:44.464324+0000","last_active":"2023-06-23T18:22:06.778940+0000","last_peered":"2023-06-23T18:22:06.778940+0000","last_clean":"2023-06-23T18:22:06.778940+0000","last_became_active":"2023-06-23T18:20:44.464021+0000","last_became_peered":"2023-06-23T18:20:44.464021+0000","last_unstale":"2023-06-23T18:22:06.778940+0000","last_undegraded":"2023-06-23T18:22:06.778940+0000","last_fullsized":"2023-06-23T18:22:06.778940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:23:09.888384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"82'4","reported_seq":100,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:05.160742+0000","last_change":"2023-06-23T18:20:44.034309+0000","last_active":"2023-06-23T18:22:05.160742+0000","last_peered":"2023-06-23T18:22:05.160742+0000","last_clean":"2023-06-23T18:22:05.160742+0000","last_became_active":"2023-06-23T18:20:44.034007+0000","last_became_peered":"2023-06-23T18:20:44.034007+0000","last_unstale":"2023-06-23T18:22:05.160742+0000","last_undegraded":"2023-06-23T18:22:05.160742+0000","last_fullsized":"2023-06-23T18:22:05.160742+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:50:17.056971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":78,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080879+0000","last_change":"2023-06-23T18:20:44.468220+0000","last_active":"2023-06-23T18:21:38.080879+0000","last_peered":"2023-06-23T18:21:38.080879+0000","last_clean":"2023-06-23T18:21:38.080879+0000","last_became_active":"2023-06-23T18:20:44.467725+0000","last_became_peered":"2023-06-23T18:20:44.467725+0000","last_unstale":"2023-06-23T18:21:38.080879+0000","last_undegraded":"2023-06-23T18:21:38.080879+0000","last_fullsized":"2023-06-23T18:21:38.080879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:20:06.380008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"82'3","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100500+0000","last_change":"2023-06-23T18:20:44.030921+0000","last_active":"2023-06-23T18:21:38.100500+0000","last_peered":"2023-06-23T18:21:38.100500+0000","last_clean":"2023-06-23T18:21:38.100500+0000","last_became_active":"2023-06-23T18:20:44.030362+0000","last_became_peered":"2023-06-23T18:20:44.030362+0000","last_unstale":"2023-06-23T18:21:38.100500+0000","last_undegraded":"2023-06-23T18:21:38.100500+0000","last_fullsized":"2023-06-23T18:21:38.100500+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:07:01.761788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073517+0000","last_change":"2023-06-23T18:19:44.952262+0000","last_active":"2023-06-23T18:21:38.073517+0000","last_peered":"2023-06-23T18:21:38.073517+0000","last_clean":"2023-06-23T18:21:38.073517+0000","last_became_active":"2023-06-23T18:19:44.951803+0000","last_became_peered":"2023-06-23T18:19:44.951803+0000","last_unstale":"2023-06-23T18:21:38.073517+0000","last_undegraded":"2023-06-23T18:21:38.073517+0000","last_fullsized":"2023-06-23T18:21:38.073517+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:16:49.458922+0000","last_clean_scrub_stamp":"2023-06-23T18:16:49.458922+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:43:25.254131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072640+0000","last_change":"2023-06-23T18:20:44.456602+0000","last_active":"2023-06-23T18:21:38.072640+0000","last_peered":"2023-06-23T18:21:38.072640+0000","last_clean":"2023-06-23T18:21:38.072640+0000","last_became_active":"2023-06-23T18:20:44.456249+0000","last_became_peered":"2023-06-23T18:20:44.456249+0000","last_unstale":"2023-06-23T18:21:38.072640+0000","last_undegraded":"2023-06-23T18:21:38.072640+0000","last_fullsized":"2023-06-23T18:21:38.072640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:50:46.341121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073728+0000","last_change":"2023-06-23T18:20:44.039100+0000","last_active":"2023-06-23T18:21:38.073728+0000","last_peered":"2023-06-23T18:21:38.073728+0000","last_clean":"2023-06-23T18:21:38.073728+0000","last_became_active":"2023-06-23T18:20:44.038801+0000","last_became_peered":"2023-06-23T18:20:44.038801+0000","last_unstale":"2023-06-23T18:21:38.073728+0000","last_undegraded":"2023-06-23T18:21:38.073728+0000","last_fullsized":"2023-06-23T18:21:38.073728+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:01:20.046240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"82'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.076298+0000","last_change":"2023-06-23T18:20:44.038037+0000","last_active":"2023-06-23T18:21:38.076298+0000","last_peered":"2023-06-23T18:21:38.076298+0000","last_clean":"2023-06-23T18:21:38.076298+0000","last_became_active":"2023-06-23T18:20:44.037732+0000","last_became_peered":"2023-06-23T18:20:44.037732+0000","last_unstale":"2023-06-23T18:21:38.076298+0000","last_undegraded":"2023-06-23T18:21:38.076298+0000","last_fullsized":"2023-06-23T18:21:38.076298+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:17:33.256219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073019+0000","last_change":"2023-06-23T18:20:44.459927+0000","last_active":"2023-06-23T18:21:38.073019+0000","last_peered":"2023-06-23T18:21:38.073019+0000","last_clean":"2023-06-23T18:21:38.073019+0000","last_became_active":"2023-06-23T18:20:44.459594+0000","last_became_peered":"2023-06-23T18:20:44.459594+0000","last_unstale":"2023-06-23T18:21:38.073019+0000","last_undegraded":"2023-06-23T18:21:38.073019+0000","last_fullsized":"2023-06-23T18:21:38.073019+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:16:49.640775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'6","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075638+0000","last_change":"2023-06-23T18:20:44.039429+0000","last_active":"2023-06-23T18:21:38.075638+0000","last_peered":"2023-06-23T18:21:38.075638+0000","last_clean":"2023-06-23T18:21:38.075638+0000","last_became_active":"2023-06-23T18:20:44.038944+0000","last_became_peered":"2023-06-23T18:20:44.038944+0000","last_unstale":"2023-06-23T18:21:38.075638+0000","last_undegraded":"2023-06-23T18:21:38.075638+0000","last_fullsized":"2023-06-23T18:21:38.075638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T19:43:03.297820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080916+0000","last_change":"2023-06-23T18:20:44.458227+0000","last_active":"2023-06-23T18:21:38.080916+0000","last_peered":"2023-06-23T18:21:38.080916+0000","last_clean":"2023-06-23T18:21:38.080916+0000","last_became_active":"2023-06-23T18:20:44.457904+0000","last_became_peered":"2023-06-23T18:20:44.457904+0000","last_unstale":"2023-06-23T18:21:38.080916+0000","last_undegraded":"2023-06-23T18:21:38.080916+0000","last_fullsized":"2023-06-23T18:21:38.080916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:49:55.045014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"82'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100759+0000","last_change":"2023-06-23T18:20:44.034089+0000","last_active":"2023-06-23T18:21:38.100759+0000","last_peered":"2023-06-23T18:21:38.100759+0000","last_clean":"2023-06-23T18:21:38.100759+0000","last_became_active":"2023-06-23T18:20:44.033745+0000","last_became_peered":"2023-06-23T18:20:44.033745+0000","last_unstale":"2023-06-23T18:21:38.100759+0000","last_undegraded":"2023-06-23T18:21:38.100759+0000","last_fullsized":"2023-06-23T18:21:38.100759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:37:54.500817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.082533+0000","last_change":"2023-06-23T18:21:38.082533+0000","last_active":"2023-06-23T18:21:38.082533+0000","last_peered":"2023-06-23T18:21:38.082533+0000","last_clean":"2023-06-23T18:21:38.082533+0000","last_became_active":"2023-06-23T18:21:38.082114+0000","last_became_peered":"2023-06-23T18:21:38.082114+0000","last_unstale":"2023-06-23T18:21:38.082533+0000","last_undegraded":"2023-06-23T18:21:38.082533+0000","last_fullsized":"2023-06-23T18:21:38.082533+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:03:31.751873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"82'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.078831+0000","last_change":"2023-06-23T18:20:44.030647+0000","last_active":"2023-06-23T18:21:38.078831+0000","last_peered":"2023-06-23T18:21:38.078831+0000","last_clean":"2023-06-23T18:21:38.078831+0000","last_became_active":"2023-06-23T18:20:44.030316+0000","last_became_peered":"2023-06-23T18:20:44.030316+0000","last_unstale":"2023-06-23T18:21:38.078831+0000","last_undegraded":"2023-06-23T18:21:38.078831+0000","last_fullsized":"2023-06-23T18:21:38.078831+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:35:45.816400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.743921+0000","last_change":"2023-06-23T18:20:44.458365+0000","last_active":"2023-06-23T18:22:06.743921+0000","last_peered":"2023-06-23T18:22:06.743921+0000","last_clean":"2023-06-23T18:22:06.743921+0000","last_became_active":"2023-06-23T18:20:44.458039+0000","last_became_peered":"2023-06-23T18:20:44.458039+0000","last_unstale":"2023-06-23T18:22:06.743921+0000","last_undegraded":"2023-06-23T18:22:06.743921+0000","last_fullsized":"2023-06-23T18:22:06.743921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:02:29.952453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"82'3","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080611+0000","last_change":"2023-06-23T18:20:44.028908+0000","last_active":"2023-06-23T18:21:38.080611+0000","last_peered":"2023-06-23T18:21:38.080611+0000","last_clean":"2023-06-23T18:21:38.080611+0000","last_became_active":"2023-06-23T18:20:44.028551+0000","last_became_peered":"2023-06-23T18:20:44.028551+0000","last_unstale":"2023-06-23T18:21:38.080611+0000","last_undegraded":"2023-06-23T18:21:38.080611+0000","last_fullsized":"2023-06-23T18:21:38.080611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:55:58.763919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780847+0000","last_change":"2023-06-23T18:20:44.465300+0000","last_active":"2023-06-23T18:22:06.780847+0000","last_peered":"2023-06-23T18:22:06.780847+0000","last_clean":"2023-06-23T18:22:06.780847+0000","last_became_active":"2023-06-23T18:20:44.465008+0000","last_became_peered":"2023-06-23T18:20:44.465008+0000","last_unstale":"2023-06-23T18:22:06.780847+0000","last_undegraded":"2023-06-23T18:22:06.780847+0000","last_fullsized":"2023-06-23T18:22:06.780847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:37:20.321477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"82'1","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.100292+0000","last_change":"2023-06-23T18:21:38.100292+0000","last_active":"2023-06-23T18:21:38.100292+0000","last_peered":"2023-06-23T18:21:38.100292+0000","last_clean":"2023-06-23T18:21:38.100292+0000","last_became_active":"2023-06-23T18:21:38.083373+0000","last_became_peered":"2023-06-23T18:21:38.083373+0000","last_unstale":"2023-06-23T18:21:38.100292+0000","last_undegraded":"2023-06-23T18:21:38.100292+0000","last_fullsized":"2023-06-23T18:21:38.100292+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:44:38.453889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.081058+0000","last_change":"2023-06-23T18:20:44.465127+0000","last_active":"2023-06-23T18:21:38.081058+0000","last_peered":"2023-06-23T18:21:38.081058+0000","last_clean":"2023-06-23T18:21:38.081058+0000","last_became_active":"2023-06-23T18:20:44.464768+0000","last_became_peered":"2023-06-23T18:20:44.464768+0000","last_unstale":"2023-06-23T18:21:38.081058+0000","last_undegraded":"2023-06-23T18:21:38.081058+0000","last_fullsized":"2023-06-23T18:21:38.081058+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:07:36.805439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"82'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.075973+0000","last_change":"2023-06-23T18:20:44.046245+0000","last_active":"2023-06-23T18:21:38.075973+0000","last_peered":"2023-06-23T18:21:38.075973+0000","last_clean":"2023-06-23T18:21:38.075973+0000","last_became_active":"2023-06-23T18:20:44.045945+0000","last_became_peered":"2023-06-23T18:20:44.045945+0000","last_unstale":"2023-06-23T18:21:38.075973+0000","last_undegraded":"2023-06-23T18:21:38.075973+0000","last_fullsized":"2023-06-23T18:21:38.075973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:53:58.962783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.789675+0000","last_change":"2023-06-23T18:20:44.453520+0000","last_active":"2023-06-23T18:22:06.789675+0000","last_peered":"2023-06-23T18:22:06.789675+0000","last_clean":"2023-06-23T18:22:06.789675+0000","last_became_active":"2023-06-23T18:20:44.453214+0000","last_became_peered":"2023-06-23T18:20:44.453214+0000","last_unstale":"2023-06-23T18:22:06.789675+0000","last_undegraded":"2023-06-23T18:22:06.789675+0000","last_fullsized":"2023-06-23T18:22:06.789675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:40:14.274731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"82'3","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080295+0000","last_change":"2023-06-23T18:20:44.042526+0000","last_active":"2023-06-23T18:21:38.080295+0000","last_peered":"2023-06-23T18:21:38.080295+0000","last_clean":"2023-06-23T18:21:38.080295+0000","last_became_active":"2023-06-23T18:20:44.042249+0000","last_became_peered":"2023-06-23T18:20:44.042249+0000","last_unstale":"2023-06-23T18:21:38.080295+0000","last_undegraded":"2023-06-23T18:21:38.080295+0000","last_fullsized":"2023-06-23T18:21:38.080295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:59:17.052247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.790121+0000","last_change":"2023-06-23T18:20:44.454645+0000","last_active":"2023-06-23T18:22:06.790121+0000","last_peered":"2023-06-23T18:22:06.790121+0000","last_clean":"2023-06-23T18:22:06.790121+0000","last_became_active":"2023-06-23T18:20:44.453309+0000","last_became_peered":"2023-06-23T18:20:44.453309+0000","last_unstale":"2023-06-23T18:22:06.790121+0000","last_undegraded":"2023-06-23T18:22:06.790121+0000","last_fullsized":"2023-06-23T18:22:06.790121+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T00:15:24.080996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080976+0000","last_change":"2023-06-23T18:20:44.037681+0000","last_active":"2023-06-23T18:21:38.080976+0000","last_peered":"2023-06-23T18:21:38.080976+0000","last_clean":"2023-06-23T18:21:38.080976+0000","last_became_active":"2023-06-23T18:20:44.037323+0000","last_became_peered":"2023-06-23T18:20:44.037323+0000","last_unstale":"2023-06-23T18:21:38.080976+0000","last_undegraded":"2023-06-23T18:21:38.080976+0000","last_fullsized":"2023-06-23T18:21:38.080976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:49:34.898906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.787192+0000","last_change":"2023-06-23T18:20:44.456434+0000","last_active":"2023-06-23T18:22:06.787192+0000","last_peered":"2023-06-23T18:22:06.787192+0000","last_clean":"2023-06-23T18:22:06.787192+0000","last_became_active":"2023-06-23T18:20:44.456110+0000","last_became_peered":"2023-06-23T18:20:44.456110+0000","last_unstale":"2023-06-23T18:22:06.787192+0000","last_undegraded":"2023-06-23T18:22:06.787192+0000","last_fullsized":"2023-06-23T18:22:06.787192+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:38:39.438073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073300+0000","last_change":"2023-06-23T18:20:44.040722+0000","last_active":"2023-06-23T18:21:38.073300+0000","last_peered":"2023-06-23T18:21:38.073300+0000","last_clean":"2023-06-23T18:21:38.073300+0000","last_became_active":"2023-06-23T18:20:44.040390+0000","last_became_peered":"2023-06-23T18:20:44.040390+0000","last_unstale":"2023-06-23T18:21:38.073300+0000","last_undegraded":"2023-06-23T18:21:38.073300+0000","last_fullsized":"2023-06-23T18:21:38.073300+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:43.014435+0000","last_clean_scrub_stamp":"2023-06-23T18:20:43.014435+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:49:31.045068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792445+0000","last_change":"2023-06-23T18:20:44.455559+0000","last_active":"2023-06-23T18:22:06.792445+0000","last_peered":"2023-06-23T18:22:06.792445+0000","last_clean":"2023-06-23T18:22:06.792445+0000","last_became_active":"2023-06-23T18:20:44.455256+0000","last_became_peered":"2023-06-23T18:20:44.455256+0000","last_unstale":"2023-06-23T18:22:06.792445+0000","last_undegraded":"2023-06-23T18:22:06.792445+0000","last_fullsized":"2023-06-23T18:22:06.792445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T02:43:47.289820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.779277+0000","last_change":"2023-06-23T18:20:44.460403+0000","last_active":"2023-06-23T18:22:06.779277+0000","last_peered":"2023-06-23T18:22:06.779277+0000","last_clean":"2023-06-23T18:22:06.779277+0000","last_became_active":"2023-06-23T18:20:44.459928+0000","last_became_peered":"2023-06-23T18:20:44.459928+0000","last_unstale":"2023-06-23T18:22:06.779277+0000","last_undegraded":"2023-06-23T18:22:06.779277+0000","last_fullsized":"2023-06-23T18:22:06.779277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T22:37:08.283159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780411+0000","last_change":"2023-06-23T18:20:44.458050+0000","last_active":"2023-06-23T18:22:06.780411+0000","last_peered":"2023-06-23T18:22:06.780411+0000","last_clean":"2023-06-23T18:22:06.780411+0000","last_became_active":"2023-06-23T18:20:44.457733+0000","last_became_peered":"2023-06-23T18:20:44.457733+0000","last_unstale":"2023-06-23T18:22:06.780411+0000","last_undegraded":"2023-06-23T18:22:06.780411+0000","last_fullsized":"2023-06-23T18:22:06.780411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T01:43:34.360833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.776703+0000","last_change":"2023-06-23T18:20:44.456233+0000","last_active":"2023-06-23T18:22:06.776703+0000","last_peered":"2023-06-23T18:22:06.776703+0000","last_clean":"2023-06-23T18:22:06.776703+0000","last_became_active":"2023-06-23T18:20:44.455677+0000","last_became_peered":"2023-06-23T18:20:44.455677+0000","last_unstale":"2023-06-23T18:22:06.776703+0000","last_undegraded":"2023-06-23T18:22:06.776703+0000","last_fullsized":"2023-06-23T18:22:06.776703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T20:30:25.029586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080153+0000","last_change":"2023-06-23T18:20:44.469668+0000","last_active":"2023-06-23T18:21:38.080153+0000","last_peered":"2023-06-23T18:21:38.080153+0000","last_clean":"2023-06-23T18:21:38.080153+0000","last_became_active":"2023-06-23T18:20:44.466563+0000","last_became_peered":"2023-06-23T18:20:44.466563+0000","last_unstale":"2023-06-23T18:21:38.080153+0000","last_undegraded":"2023-06-23T18:21:38.080153+0000","last_fullsized":"2023-06-23T18:21:38.080153+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T04:05:04.024545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.073109+0000","last_change":"2023-06-23T18:20:44.456021+0000","last_active":"2023-06-23T18:21:38.073109+0000","last_peered":"2023-06-23T18:21:38.073109+0000","last_clean":"2023-06-23T18:21:38.073109+0000","last_became_active":"2023-06-23T18:20:44.455705+0000","last_became_peered":"2023-06-23T18:20:44.455705+0000","last_unstale":"2023-06-23T18:21:38.073109+0000","last_undegraded":"2023-06-23T18:21:38.073109+0000","last_fullsized":"2023-06-23T18:21:38.073109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:50:09.933725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.775420+0000","last_change":"2023-06-23T18:20:44.460359+0000","last_active":"2023-06-23T18:22:06.775420+0000","last_peered":"2023-06-23T18:22:06.775420+0000","last_clean":"2023-06-23T18:22:06.775420+0000","last_became_active":"2023-06-23T18:20:44.460034+0000","last_became_peered":"2023-06-23T18:20:44.460034+0000","last_unstale":"2023-06-23T18:22:06.775420+0000","last_undegraded":"2023-06-23T18:22:06.775420+0000","last_fullsized":"2023-06-23T18:22:06.775420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:26:39.654726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.786166+0000","last_change":"2023-06-23T18:20:44.455743+0000","last_active":"2023-06-23T18:22:06.786166+0000","last_peered":"2023-06-23T18:22:06.786166+0000","last_clean":"2023-06-23T18:22:06.786166+0000","last_became_active":"2023-06-23T18:20:44.455266+0000","last_became_peered":"2023-06-23T18:20:44.455266+0000","last_unstale":"2023-06-23T18:22:06.786166+0000","last_undegraded":"2023-06-23T18:22:06.786166+0000","last_fullsized":"2023-06-23T18:22:06.786166+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:51:16.923982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.783014+0000","last_change":"2023-06-23T18:21:38.101164+0000","last_active":"2023-06-23T18:22:06.783014+0000","last_peered":"2023-06-23T18:22:06.783014+0000","last_clean":"2023-06-23T18:22:06.783014+0000","last_became_active":"2023-06-23T18:21:38.100818+0000","last_became_peered":"2023-06-23T18:21:38.100818+0000","last_unstale":"2023-06-23T18:22:06.783014+0000","last_undegraded":"2023-06-23T18:22:06.783014+0000","last_fullsized":"2023-06-23T18:22:06.783014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T23:54:39.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.777099+0000","last_change":"2023-06-23T18:20:44.465192+0000","last_active":"2023-06-23T18:22:06.777099+0000","last_peered":"2023-06-23T18:22:06.777099+0000","last_clean":"2023-06-23T18:22:06.777099+0000","last_became_active":"2023-06-23T18:20:44.464905+0000","last_became_peered":"2023-06-23T18:20:44.464905+0000","last_unstale":"2023-06-23T18:22:06.777099+0000","last_undegraded":"2023-06-23T18:22:06.777099+0000","last_fullsized":"2023-06-23T18:22:06.777099+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T18:32:18.229559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.080712+0000","last_change":"2023-06-23T18:20:44.458725+0000","last_active":"2023-06-23T18:21:38.080712+0000","last_peered":"2023-06-23T18:21:38.080712+0000","last_clean":"2023-06-23T18:21:38.080712+0000","last_became_active":"2023-06-23T18:20:44.458414+0000","last_became_peered":"2023-06-23T18:20:44.458414+0000","last_unstale":"2023-06-23T18:21:38.080712+0000","last_undegraded":"2023-06-23T18:21:38.080712+0000","last_fullsized":"2023-06-23T18:21:38.080712+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:34:31.245389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.792579+0000","last_change":"2023-06-23T18:20:44.465665+0000","last_active":"2023-06-23T18:22:06.792579+0000","last_peered":"2023-06-23T18:22:06.792579+0000","last_clean":"2023-06-23T18:22:06.792579+0000","last_became_active":"2023-06-23T18:20:44.465337+0000","last_became_peered":"2023-06-23T18:20:44.465337+0000","last_unstale":"2023-06-23T18:22:06.792579+0000","last_undegraded":"2023-06-23T18:22:06.792579+0000","last_fullsized":"2023-06-23T18:22:06.792579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-24T21:05:22.890126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:21:38.072919+0000","last_change":"2023-06-23T18:20:44.455107+0000","last_active":"2023-06-23T18:21:38.072919+0000","last_peered":"2023-06-23T18:21:38.072919+0000","last_clean":"2023-06-23T18:21:38.072919+0000","last_became_active":"2023-06-23T18:20:44.454798+0000","last_became_peered":"2023-06-23T18:20:44.454798+0000","last_unstale":"2023-06-23T18:21:38.072919+0000","last_undegraded":"2023-06-23T18:21:38.072919+0000","last_fullsized":"2023-06-23T18:21:38.072919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T03:19:46.878268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-06-23T18:22:06.780928+0000","last_change":"2023-06-23T18:20:44.461044+0000","last_active":"2023-06-23T18:22:06.780928+0000","last_peered":"2023-06-23T18:22:06.780928+0000","last_clean":"2023-06-23T18:22:06.780928+0000","last_became_active":"2023-06-23T18:20:44.460737+0000","last_became_peered":"2023-06-23T18:20:44.460737+0000","last_unstale":"2023-06-23T18:22:06.780928+0000","last_undegraded":"2023-06-23T18:22:06.780928+0000","last_fullsized":"2023-06-23T18:22:06.780928+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-06-23T18:20:44.016165+0000","last_clean_scrub_stamp":"2023-06-23T18:20:44.016165+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-06-25T05:22:32.348419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":358,"ondisk_log_size":358,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281641,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63241,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":492,"num_read_kb":311,"num_write":162,"num_write_kb":113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":57610,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":188757,"data_compressed":8925,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481820,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":437933,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.76000000000000001,"15min":0.80000000000000004},"min":{"1min":0.38900000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.624,"5min":5.7590000000000003,"15min":5.7590000000000003},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.497,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.39600000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65400000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.56000000000000005}]},{"osd":1,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.54700000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.80000000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.60299999999999998,"15min":0.59699999999999998},"min":{"1min":0.44500000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79400000000000004,"5min":0.873,"15min":0.873},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.48599999999999999},"min":{"1min":0.35399999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.502,"15min":0.49299999999999999},"min":{"1min":0.371,"5min":0.317,"15min":0.317},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":4,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.54700000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83899999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.53000000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.40999999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84299999999999997,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63200000000000001,"15min":0.624},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.873,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.873}]},{"osd":6,"last update":"Fri Jun 23 18:21:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":5.6580000000000004,"5min":5.6580000000000004,"15min":5.6580000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.54300000000000004}]},{"osd":7,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95899999999999996,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70499999999999996,"15min":0.69399999999999995},"min":{"1min":0.61499999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0069999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.82099999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.86199999999999999}]},{"osd":9,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69399999999999995,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.87,"5min":1.369,"15min":1.369},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.78000000000000003,"15min":0.77500000000000002},"min":{"1min":0.66900000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.95399999999999996,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.76000000000000001,"15min":0.751},"min":{"1min":0.71499999999999997,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.746,"15min":0.74099999999999999},"min":{"1min":0.63100000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.98599999999999999,"5min":1.579,"15min":1.579},"last":0.89600000000000002}]}]},{"osd":10,"up_from":70,"seq":300647710753,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":1056768,"data_stored":1088504,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":11.565,"5min":11.565,"15min":11.565},"last":2.0329999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":4.298,"5min":4.298,"15min":4.298},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.59899999999999998,"15min":0.61399999999999999},"min":{"1min":0.38500000000000001,"5min":0.316,"15min":0.316},"max":{"1min":0.71599999999999997,"5min":3.6970000000000001,"15min":3.6970000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.59099999999999997,"15min":0.60899999999999999},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.76500000000000001,"5min":2.9279999999999999,"15min":2.9279999999999999},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.73499999999999999},"min":{"1min":0.47899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.82999999999999996,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.72099999999999997,"15min":0.72899999999999998},"min":{"1min":0.53600000000000003,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.92000000000000004,"5min":2.6669999999999998,"15min":2.6669999999999998},"last":0.77300000000000002}]},{"osd":3,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.72499999999999998,"15min":0.745},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.80400000000000005,"5min":2.782,"15min":2.782},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71699999999999997,"15min":0.73499999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.73199999999999998,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.75600000000000001,"15min":0.77700000000000002},"min":{"1min":0.436,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82199999999999995,"5min":4.5499999999999998,"15min":4.5499999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.67000000000000004,"15min":0.68799999999999994},"min":{"1min":0.39800000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.84399999999999997,"5min":2.9780000000000002,"15min":2.9780000000000002},"last":0.55100000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.58899999999999997,"5min":0.437,"15min":0.437},"max":{"1min":0.89600000000000002,"5min":2.8559999999999999,"15min":2.8559999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.80400000000000005},"min":{"1min":0.58699999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90700000000000003,"5min":2.7210000000000001,"15min":2.7210000000000001},"last":0.72299999999999998}]},{"osd":6,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.89200000000000002,"15min":0.85799999999999998},"min":{"1min":0.51700000000000002,"5min":0.501,"15min":0.501},"max":{"1min":7.2240000000000002,"5min":7.2240000000000002,"15min":7.2240000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.79600000000000004,"15min":0.80900000000000005},"min":{"1min":0.54700000000000004,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.89800000000000002,"5min":1.7210000000000001,"15min":1.7210000000000001},"last":1.097}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":8,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.84599999999999997,"15min":0.85299999999999998},"min":{"1min":0.61799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93799999999999994,"5min":1.782,"15min":1.782},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86899999999999999,"15min":0.88600000000000001},"min":{"1min":0.61899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.94399999999999995,"5min":2.9089999999999998,"15min":2.9089999999999998},"last":0.69999999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.85599999999999998,"15min":0.874},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.88100000000000001,"5min":1.645,"15min":1.645},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.90400000000000003,"15min":0.92600000000000005},"min":{"1min":0.64200000000000002,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.91900000000000004,"5min":2.9430000000000001,"15min":2.9430000000000001},"last":0.66300000000000003}]},{"osd":11,"last update":"Fri Jun 23 18:22:04 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.90800000000000003},"min":{"1min":0.51500000000000001,"5min":0.497,"15min":0.497},"max":{"1min":0.97699999999999998,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.83799999999999997,"15min":0.84299999999999997},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.96299999999999997,"5min":1.754,"15min":1.754},"last":0.85699999999999998}]}]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":921600,"data_stored":438057,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.247},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.44600000000000001,"15min":0.45200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.32800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.67100000000000004,"15min":0.69499999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.72999999999999998,"15min":0.753},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":2.5409999999999999,"15min":2.5409999999999999},"last":0.57399999999999995}]},{"osd":2,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.56200000000000006,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.35999999999999999}]},{"osd":3,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.68700000000000006,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.309},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59799999999999998,"15min":0.60699999999999998},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.36199999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":0.622},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.75,"15min":0.77500000000000002},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.83899999999999997,"5min":2.5099999999999998,"15min":2.5099999999999998},"last":0.497}]},{"osd":5,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.70599999999999996,"15min":0.71999999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.373},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.499,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.034,"5min":1.04,"15min":1.04},"last":0.54700000000000004}]},{"osd":6,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.68400000000000005,"15min":0.70299999999999996},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0669999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.86199999999999999,"5min":1.397,"15min":1.397},"last":0.50900000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.74399999999999999,"15min":0.75700000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.81599999999999995,"15min":0.83399999999999996},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0960000000000001,"5min":1.661,"15min":1.661},"last":0.66500000000000004}]},{"osd":8,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.0349999999999999,"5min":1.079,"15min":1.079},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.93000000000000005,"15min":0.97299999999999998},"min":{"1min":0.55900000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.91900000000000004,"5min":5.2389999999999999,"15min":5.2389999999999999},"last":0.434}]},{"osd":10,"last update":"Fri Jun 23 18:21:48 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0029999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.69399999999999995,"15min":0.70399999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.65100000000000002}]}]},{"osd":8,"up_from":58,"seq":249108103209,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":437534,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.42099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.501,"15min":0.49399999999999999},"min":{"1min":0.39400000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.68500000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.60599999999999998,"15min":0.59299999999999997},"min":{"1min":0.42399999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.84799999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.47399999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.71099999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.45600000000000002,"15min":0.45100000000000001},"min":{"1min":0.35999999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63800000000000001,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.25700000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.745,"5min":0.752,"15min":0.752},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.42899999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76200000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.62}]},{"osd":4,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.65100000000000002,"15min":0.64300000000000002},"min":{"1min":0.52600000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85699999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.80300000000000005}]},{"osd":5,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.55700000000000005,"15min":0.54700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.33800000000000002,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.86899999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.66700000000000004,"15min":0.66400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.89200000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.48799999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.61399999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.83199999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68899999999999995,"15min":0.68400000000000005},"min":{"1min":0.51800000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.90900000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.89400000000000002}]},{"osd":10,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.70099999999999996,"15min":0.68100000000000005},"min":{"1min":0.56000000000000005,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97599999999999998,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.71599999999999997,"15min":0.69799999999999995},"min":{"1min":0.55300000000000005,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.96699999999999997,"5min":0.998,"15min":0.998},"last":0.92500000000000004}]},{"osd":11,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.94799999999999995}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":1029957,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.40899999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54300000000000004,"15min":0.53600000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38700000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61399999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.442,"15min":0.42599999999999999},"min":{"1min":0.30399999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63800000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.48999999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84599999999999997,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.47699999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.84999999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60699999999999998,"15min":0.59099999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.81299999999999994,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.64600000000000002}]},{"osd":4,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53400000000000003,"15min":0.51400000000000001},"min":{"1min":0.379,"5min":0.316,"15min":0.316},"max":{"1min":0.73399999999999999,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.379}]},{"osd":5,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.65600000000000003,"15min":0.63500000000000001},"min":{"1min":0.45300000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.80200000000000005,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.624},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66500000000000004,"15min":0.65000000000000002},"min":{"1min":0.51800000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.877,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.56599999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.67600000000000005,"15min":0.65000000000000002},"min":{"1min":0.49299999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.89400000000000002,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.752},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.66700000000000004,"15min":0.65400000000000003},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89200000000000002,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.77900000000000003}]},{"osd":8,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.72299999999999998,"15min":0.71299999999999997},"min":{"1min":0.52100000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.71999999999999997},"min":{"1min":0.57199999999999995,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.92100000000000004,"5min":1.024,"15min":1.024},"last":0.55300000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.93999999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78500000000000003,"15min":0.79000000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":0.95399999999999996,"5min":1.012,"15min":1.012},"last":0.79800000000000004}]},{"osd":10,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78800000000000003,"15min":0.79400000000000004},"min":{"1min":0.57299999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.94999999999999996,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78000000000000003,"15min":0.77600000000000002},"min":{"1min":0.627,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93600000000000005,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.71299999999999997}]},{"osd":11,"last update":"Fri Jun 23 18:21:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73099999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495665,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":909312,"data_stored":437452,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.41299999999999998,"15min":0.41199999999999998},"min":{"1min":0.30499999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.58899999999999997,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.42999999999999999,"15min":0.42899999999999999},"min":{"1min":0.28699999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.56699999999999995,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.379}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.33800000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.876,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60299999999999998,"15min":0.60199999999999998},"min":{"1min":0.41499999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.89400000000000002,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.59399999999999997,"15min":0.60699999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.70799999999999996,"5min":1.105,"15min":1.105},"last":0.41399999999999998}]},{"osd":3,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.51600000000000001,"15min":0.52500000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.65400000000000003,"5min":1.141,"15min":1.141},"last":0.432},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72199999999999998,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60399999999999998,"15min":0.59999999999999998},"min":{"1min":0.38200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.872,"15min":0.872},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69699999999999995,"15min":0.70299999999999996},"min":{"1min":0.41499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.93500000000000005,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.61399999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.77200000000000002,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.68300000000000005,"15min":0.69699999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.92800000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.73999999999999999,"15min":0.754},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.82999999999999996,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91800000000000004,"5min":1.03,"15min":1.03},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.73299999999999998,"15min":0.74099999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.98299999999999998,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79900000000000004,"15min":0.81299999999999994},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.97199999999999998,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.58299999999999996}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.69199999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96799999999999997,"5min":1.008,"15min":1.008},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.71699999999999997,"15min":0.73599999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.048,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.47699999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.745,"15min":0.75700000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.878,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.871,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.67100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574919,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":970752,"data_stored":495469,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52900000000000003,"15min":0.48499999999999999},"min":{"1min":0.34799999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.84999999999999998,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.625},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52500000000000002,"15min":0.498},"min":{"1min":0.35699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.73999999999999999,"5min":1.218,"15min":1.218},"last":0.66700000000000004}]},{"osd":2,"last update":"Fri Jun 23 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.623,"15min":0.54900000000000004},"min":{"1min":0.436,"5min":0.317,"15min":0.317},"max":{"1min":0.90700000000000003,"5min":1.25,"15min":1.25},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57899999999999996,"15min":0.54300000000000004},"min":{"1min":0.34200000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.91300000000000003,"5min":1.343,"15min":1.343},"last":0.64100000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.54700000000000004},"min":{"1min":0.42799999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.93200000000000005,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.624,"5min":0.59299999999999997,"15min":0.55100000000000005},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.82299999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.49099999999999999,"15min":0.44},"min":{"1min":0.32300000000000001,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78300000000000003,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.46200000000000002,"15min":0.42399999999999999},"min":{"1min":0.33800000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77000000000000002,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.66500000000000004,"15min":0.59499999999999997},"min":{"1min":0.38900000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.98399999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.64200000000000002,"15min":0.60399999999999998},"min":{"1min":0.41799999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.96799999999999997,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56799999999999995,"15min":0.54000000000000004},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84599999999999997,"5min":1.169,"15min":1.169},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.65300000000000002,"15min":0.60099999999999998},"min":{"1min":0.53200000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.093,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.754}]},{"osd":7,"last update":"Fri Jun 23 18:21:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.63700000000000001,"15min":0.60299999999999998},"min":{"1min":0.376,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92200000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.72799999999999998,"15min":0.68200000000000005},"min":{"1min":0.46899999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.80100000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.82299999999999995,"15min":0.80300000000000005},"min":{"1min":0.503,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0820000000000001,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.78100000000000003,"15min":0.76900000000000002},"min":{"1min":0.51900000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.98899999999999999,"5min":1.319,"15min":1.319},"last":0.77300000000000002}]},{"osd":9,"last update":"Fri Jun 23 18:21:21 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.876,"15min":0.88900000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.1180000000000001,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83299999999999996,"15min":0.84499999999999997},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96399999999999997,"5min":1.288,"15min":1.288},"last":0.81499999999999995}]},{"osd":10,"last update":"Fri Jun 23 18:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.85599999999999998,"15min":0.86499999999999999},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.046,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.90500000000000003,"15min":0.91100000000000003},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.93400000000000005}]},{"osd":11,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.82799999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":438717,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 23 18:21:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.52600000000000002,"15min":0.48099999999999998},"min":{"1min":0.39200000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51400000000000001,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.72699999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.503}]},{"osd":2,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56100000000000005,"15min":0.50700000000000001},"min":{"1min":0.39200000000000002,"5min":0.314,"15min":0.314},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.52200000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.84999999999999998,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.372},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Jun 23 18:21:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57299999999999995,"15min":0.55000000000000004},"min":{"1min":0.439,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.75,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56899999999999995,"15min":0.55400000000000005},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.64100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Jun 23 18:22:03 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.59399999999999997},"min":{"1min":0.41899999999999998,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.80800000000000005,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.55100000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.443},{"interface":"front","average":{"1min":0.501,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82499999999999996,"5min":1.073,"15min":1.073},"last":0.39500000000000002}]},{"osd":7,"last update":"Fri Jun 23 18:20:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.68600000000000005,"15min":0.67900000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.95899999999999996,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.56399999999999995}]},{"osd":8,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.71799999999999997,"15min":0.72599999999999998},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.86399999999999999,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.753,"15min":0.76700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.876,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90200000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.76400000000000001,"15min":0.79000000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.0449999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.61199999999999999}]},{"osd":10,"last update":"Fri Jun 23 18:21:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.77500000000000002,"15min":0.79100000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.94899999999999995,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.73699999999999999,"15min":0.748},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.83899999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63800000000000001}]},{"osd":11,"last update":"Fri Jun 23 18:21:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.69399999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":439024,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.378,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68700000000000006,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.36399999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.67300000000000004,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.38300000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.56799999999999995,"15min":0.53000000000000003},"min":{"1min":0.52300000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.80300000000000005,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.57799999999999996,"15min":0.54200000000000004},"min":{"1min":0.47599999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.55200000000000005,"15min":0.53800000000000003},"min":{"1min":0.33000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.73199999999999998,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.75800000000000001}]},{"osd":4,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Jun 23 18:22:00 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46700000000000003,"15min":0.44900000000000001},"min":{"1min":0.34300000000000003,"5min":0.252,"15min":0.252},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.52200000000000002,"15min":0.48199999999999998},"min":{"1min":0.40300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0069999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.624},"min":{"1min":0.39400000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:19:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.504},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.59799999999999998}]},{"osd":8,"last update":"Fri Jun 23 18:22:02 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.36099999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.89600000000000002,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.65900000000000003,"15min":0.64200000000000002},"min":{"1min":0.44700000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.90500000000000003,"5min":1.095,"15min":1.095},"last":0.71299999999999997}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.51300000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.621},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45300000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.63300000000000001}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.54700000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.71299999999999997,"15min":0.69199999999999995},"min":{"1min":0.58199999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.79400000000000004}]},{"osd":11,"last update":"Fri Jun 23 18:21:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.745}]}]},{"osd":3,"up_from":25,"seq":107374182461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":437769,"data_compressed":5807,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":0.502,"15min":0.38200000000000001},"min":{"1min":0.26400000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":12.246,"5min":12.246,"15min":12.246},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.309,"5min":0.245,"15min":0.245},"max":{"1min":0.54700000000000004,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.58099999999999996,"15min":0.54200000000000004},"min":{"1min":0.46600000000000003,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.61299999999999999,"15min":0.54200000000000004},"min":{"1min":0.35799999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":8.3569999999999993,"5min":8.3569999999999993,"15min":8.3569999999999993},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57399999999999995,"15min":0.55300000000000005},"min":{"1min":0.442,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.54900000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Jun 23 18:21:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.626,"15min":0.59099999999999997},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63400000000000001,"15min":0.623},"min":{"1min":0.42199999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.59099999999999997}]},{"osd":5,"last update":"Fri Jun 23 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.59699999999999998,"15min":0.58799999999999997},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Jun 23 18:21:26 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":0.70699999999999996,"15min":0.59799999999999998},"min":{"1min":0.377,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":12.183999999999999,"5min":12.183999999999999,"15min":12.183999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.59099999999999997,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":8.3859999999999992,"5min":8.3859999999999992,"15min":8.3859999999999992},"last":0.443}]},{"osd":7,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.69199999999999995,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":8.3239999999999998,"5min":8.3239999999999998,"15min":8.3239999999999998},"last":0.501},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.748,"15min":0.752},"min":{"1min":0.51900000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.68400000000000005}]},{"osd":8,"last update":"Fri Jun 23 18:19:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jun 23 18:21:20 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69399999999999995,"15min":0.70599999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.63800000000000001,"15min":0.628},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.48299999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":0.79600000000000004,"15min":0.75600000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":8.2650000000000006,"5min":8.2650000000000006,"15min":8.2650000000000006},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.79400000000000004,"15min":0.81000000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.2649999999999999,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.69399999999999995}]},{"osd":11,"last update":"Fri Jun 23 18:21:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":17.315999999999999,"5min":17.315999999999999,"15min":17.315999999999999},"last":0.72499999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":497079,"data_compressed":14732,"data_compressed_allocated":20480,"data_compressed_original":69632,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,24,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:20:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.51500000000000001,"15min":0.501},"min":{"1min":0.378,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.53400000000000003,"15min":0.51700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.50800000000000001}]},{"osd":1,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40799999999999997,"15min":0.378},"min":{"1min":0.309,"5min":0.255,"15min":0.255},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":1.6459999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.41699999999999998,"15min":0.38900000000000001},"min":{"1min":0.33700000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60999999999999999,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":4.6230000000000002}]},{"osd":2,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.38600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60599999999999998,"15min":0.55200000000000005},"min":{"1min":0.39300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.81599999999999995}]},{"osd":3,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.52800000000000002},"min":{"1min":0.40500000000000003,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.77100000000000002,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":4.6520000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73699999999999999,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.629}]},{"osd":5,"last update":"Fri Jun 23 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63800000000000001,"15min":0.60699999999999998},"min":{"1min":0.55200000000000005,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.63400000000000001,"15min":0.60599999999999998},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":4.5309999999999997}]},{"osd":6,"last update":"Fri Jun 23 18:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79900000000000004,"5min":1.129,"15min":1.129},"last":1.6299999999999999},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85599999999999998,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":1.7150000000000001}]},{"osd":7,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.63500000000000001,"15min":0.61699999999999999},"min":{"1min":0.497,"5min":0.378,"15min":0.378},"max":{"1min":0.87,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":7.2290000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.623,"15min":0.59799999999999998},"min":{"1min":0.42299999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97599999999999998,"5min":0.998,"15min":0.998},"last":4.694}]},{"osd":8,"last update":"Fri Jun 23 18:22:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.503,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94899999999999995,"5min":1.202,"15min":1.202},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.79600000000000004,"15min":0.77200000000000002},"min":{"1min":0.65200000000000002,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":4.585,"5min":4.585,"15min":4.585},"last":4.585}]},{"osd":9,"last update":"Fri Jun 23 18:21:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68300000000000005,"15min":0.67700000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":3.6360000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76500000000000001,"15min":0.76200000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.94399999999999995,"5min":1.024,"15min":1.024},"last":3.8889999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.77500000000000002,"15min":0.76000000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.022,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":11.475},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.754,"15min":0.74299999999999999},"min":{"1min":0.65800000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.96599999999999997,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":4.718}]},{"osd":11,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":1.6830000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71999999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":983040,"data_stored":1029641,"data_compressed":41626,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47999999999999998,"15min":0.46100000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.747,"5min":0.875,"15min":0.875},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.33600000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.73199999999999998,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.372}]},{"osd":1,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.54800000000000004,"15min":0.52200000000000002},"min":{"1min":0.35299999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81299999999999994,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.54600000000000004,"15min":0.52700000000000002},"min":{"1min":0.39600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78000000000000003,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57899999999999996}]},{"osd":2,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.437,"15min":0.41699999999999998},"min":{"1min":0.34799999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.44900000000000001,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.68500000000000005,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.38400000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.76600000000000001,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.54600000000000004,"15min":0.53400000000000003},"min":{"1min":0.44900000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60099999999999998,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.80000000000000004,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.621,"15min":0.59099999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85799999999999998,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Jun 23 18:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.624,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":5.399},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.67400000000000004,"15min":0.64400000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":5.4690000000000003}]},{"osd":7,"last update":"Fri Jun 23 18:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.68799999999999994,"15min":0.68500000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67300000000000004,"15min":0.67900000000000005},"min":{"1min":0.432,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84299999999999997,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.70299999999999996}]},{"osd":8,"last update":"Fri Jun 23 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.63500000000000001,"15min":0.63700000000000001},"min":{"1min":0.38400000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.88800000000000001,"5min":0.998,"15min":0.998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.624,"15min":0.629},"min":{"1min":0.505,"5min":0.374,"15min":0.374},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.53600000000000003}]},{"osd":9,"last update":"Fri Jun 23 18:21:24 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.67500000000000004,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":0.60499999999999998}]},{"osd":10,"last update":"Fri Jun 23 18:21:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.52100000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.68200000000000005,"15min":0.67900000000000005},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.84499999999999997,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.74299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34182,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35819,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":58230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":59840,"data_compressed":8925,"data_compressed_allocated":12288,"data_compressed_original":53248,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-06-23T18:22:12.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:22:12.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.278+0000 7f0a49e22700 1 -- 172.21.15.2:0/2457181093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440fea60 msgr2=0x7f0a440fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.278+0000 7f0a49e22700 1 --2- 172.21.15.2:0/2457181093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440fea60 0x7f0a440fee70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0a34009990 tx=0x7f0a34009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.278+0000 7f0a49e22700 1 -- 172.21.15.2:0/2457181093 shutdown_connections 2023-06-23T18:22:12.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.278+0000 7f0a49e22700 1 --2- 172.21.15.2:0/2457181093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a441005e0 0x7f0a44106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.278+0000 7f0a49e22700 1 --2- 172.21.15.2:0/2457181093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440fea60 0x7f0a440fee70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.278+0000 7f0a49e22700 1 --2- 172.21.15.2:0/2457181093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a440ffc50 0x7f0a441000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.278+0000 7f0a49e22700 1 -- 172.21.15.2:0/2457181093 >> 172.21.15.2:0/2457181093 conn(0x7f0a440fa0e0 msgr2=0x7f0a440fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 -- 172.21.15.2:0/2457181093 shutdown_connections 2023-06-23T18:22:12.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 -- 172.21.15.2:0/2457181093 wait complete. 2023-06-23T18:22:12.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 Processor -- start 2023-06-23T18:22:12.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 -- start start 2023-06-23T18:22:12.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a440fea60 0x7f0a44194690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:12.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440ffc50 0x7f0a44194bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:12.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a441005e0 0x7f0a4419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:12.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4419c210 con 0x7f0a440ffc50 2023-06-23T18:22:12.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4419c350 con 0x7f0a440fea60 2023-06-23T18:22:12.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.279+0000 7f0a49e22700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4419c4e0 con 0x7f0a441005e0 2023-06-23T18:22:12.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440ffc50 0x7f0a44194bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:12.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440ffc50 0x7f0a44194bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53924/0 (socket says 172.21.15.2:53924) 2023-06-23T18:22:12.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 -- 172.21.15.2:0/335330350 learned_addr learned my addr 172.21.15.2:0/335330350 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a437fe700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a440fea60 0x7f0a44194690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a43fff700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a441005e0 0x7f0a4419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 -- 172.21.15.2:0/335330350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a441005e0 msgr2=0x7f0a4419bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a441005e0 0x7f0a4419bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 -- 172.21.15.2:0/335330350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a440fea60 msgr2=0x7f0a44194690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a440fea60 0x7f0a44194690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 -- 172.21.15.2:0/335330350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340095f0 con 0x7f0a440ffc50 2023-06-23T18:22:12.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a43fff700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a441005e0 0x7f0a4419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a42ffd700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440ffc50 0x7f0a44194bd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800c9b0 tx=0x7f0a3800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a40ff9700 1 -- 172.21.15.2:0/335330350 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38007b00 con 0x7f0a440ffc50 2023-06-23T18:22:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a40ff9700 1 -- 172.21.15.2:0/335330350 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a38007c60 con 0x7f0a440ffc50 2023-06-23T18:22:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4419c7c0 con 0x7f0a440ffc50 2023-06-23T18:22:12.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a40ff9700 1 -- 172.21.15.2:0/335330350 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a38018660 con 0x7f0a440ffc50 2023-06-23T18:22:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.280+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4419cd50 con 0x7f0a440ffc50 2023-06-23T18:22:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.281+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a44105fa0 con 0x7f0a440ffc50 2023-06-23T18:22:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.282+0000 7f0a40ff9700 1 -- 172.21.15.2:0/335330350 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0a3800f450 con 0x7f0a440ffc50 2023-06-23T18:22:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.283+0000 7f0a40ff9700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0a2c0a1150 0x7f0a2c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.283+0000 7f0a437fe700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0a2c0a1150 0x7f0a2c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.283+0000 7f0a40ff9700 1 -- 172.21.15.2:0/335330350 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0a380ccc00 con 0x7f0a440ffc50 2023-06-23T18:22:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.284+0000 7f0a437fe700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0a2c0a1150 0x7f0a2c0a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400b5c0 tx=0x7f0a34005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.286+0000 7f0a40ff9700 1 -- 172.21.15.2:0/335330350 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f0a380963a0 con 0x7f0a440ffc50 2023-06-23T18:22:12.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:12 smithi017 ceph-mon[97472]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:12.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:12 smithi017 ceph-mon[97472]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:12.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.492+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a4402c810 con 0x7f0a440ffc50 2023-06-23T18:22:12.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.494+0000 7f0a40ff9700 1 -- 172.21.15.2:0/335330350 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f0a3800fe60 con 0x7f0a440ffc50 2023-06-23T18:22:12.499 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:12.499 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:22:12.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0a2c0a1150 msgr2=0x7f0a2c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0a2c0a1150 0x7f0a2c0a3600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400b5c0 tx=0x7f0a34005960 comp rx=0 tx=0).stop 2023-06-23T18:22:12.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440ffc50 msgr2=0x7f0a44194bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440ffc50 0x7f0a44194bd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800c9b0 tx=0x7f0a3800ccc0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 shutdown_connections 2023-06-23T18:22:12.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a441005e0 0x7f0a4419bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0a2c0a1150 0x7f0a2c0a3600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a440fea60 0x7f0a44194690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 --2- 172.21.15.2:0/335330350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a440ffc50 0x7f0a44194bd0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 >> 172.21.15.2:0/335330350 conn(0x7f0a440fa0e0 msgr2=0x7f0a441030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:12.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 shutdown_connections 2023-06-23T18:22:12.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.496+0000 7f0a49e22700 1 -- 172.21.15.2:0/335330350 wait complete. 2023-06-23T18:22:12.511 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:12 smithi002 ceph-mon[95947]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:12.512 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:12 smithi002 ceph-mon[95947]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:12.514 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:22:12.515 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:22:12.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:12 smithi082 ceph-mon[97889]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:12.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:12 smithi082 ceph-mon[97889]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.639+0000 7f245e05c700 1 -- 172.21.15.2:0/4082458661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2458101720 msgr2=0x7f2458107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.639+0000 7f245e05c700 1 --2- 172.21.15.2:0/4082458661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2458101720 0x7f2458107fe0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f245000b0a0 tx=0x7f245000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.639+0000 7f245e05c700 1 -- 172.21.15.2:0/4082458661 shutdown_connections 2023-06-23T18:22:12.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.639+0000 7f245e05c700 1 --2- 172.21.15.2:0/4082458661 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2458101720 0x7f2458107fe0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 --2- 172.21.15.2:0/4082458661 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24580ffba0 0x7f24580fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 --2- 172.21.15.2:0/4082458661 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2458100d90 0x7f24581011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 -- 172.21.15.2:0/4082458661 >> 172.21.15.2:0/4082458661 conn(0x7f24580fb1c0 msgr2=0x7f24580fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:12.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 -- 172.21.15.2:0/4082458661 shutdown_connections 2023-06-23T18:22:12.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 -- 172.21.15.2:0/4082458661 wait complete. 2023-06-23T18:22:12.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 Processor -- start 2023-06-23T18:22:12.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 -- start start 2023-06-23T18:22:12.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24580ffba0 0x7f2458074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:12.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2458100d90 0x7f24580728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:12.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.640+0000 7f245e05c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2458101720 0x7f2458072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:12.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f245e05c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24580735a0 con 0x7f24580ffba0 2023-06-23T18:22:12.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f245e05c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2458073710 con 0x7f2458100d90 2023-06-23T18:22:12.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f245e05c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245819f940 con 0x7f2458101720 2023-06-23T18:22:12.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24580ffba0 0x7f2458074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:12.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24580ffba0 0x7f2458074280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53936/0 (socket says 172.21.15.2:53936) 2023-06-23T18:22:12.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 -- 172.21.15.2:0/3443643963 learned_addr learned my addr 172.21.15.2:0/3443643963 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:12.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f2456ffd700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2458100d90 0x7f24580728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f2457fff700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2458101720 0x7f2458072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 -- 172.21.15.2:0/3443643963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2458101720 msgr2=0x7f2458072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2458101720 0x7f2458072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 -- 172.21.15.2:0/3443643963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2458100d90 msgr2=0x7f24580728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2458100d90 0x7f24580728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 -- 172.21.15.2:0/3443643963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2450009d00 con 0x7f24580ffba0 2023-06-23T18:22:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f2457fff700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2458101720 0x7f2458072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f24577fe700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24580ffba0 0x7f2458074280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f244800d980 tx=0x7f244800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f2454ff9700 1 -- 172.21.15.2:0/3443643963 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244800b950 con 0x7f24580ffba0 2023-06-23T18:22:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f2454ff9700 1 -- 172.21.15.2:0/3443643963 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2448007620 con 0x7f24580ffba0 2023-06-23T18:22:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245819fc20 con 0x7f24580ffba0 2023-06-23T18:22:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f2454ff9700 1 -- 172.21.15.2:0/3443643963 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244800f3c0 con 0x7f24580ffba0 2023-06-23T18:22:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.641+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24581a0130 con 0x7f24580ffba0 2023-06-23T18:22:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.642+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2458107140 con 0x7f24580ffba0 2023-06-23T18:22:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.643+0000 7f2454ff9700 1 -- 172.21.15.2:0/3443643963 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f244800bab0 con 0x7f24580ffba0 2023-06-23T18:22:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.644+0000 7f2454ff9700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f24400a1150 0x7f24400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.644+0000 7f2456ffd700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f24400a1150 0x7f24400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.644+0000 7f2454ff9700 1 -- 172.21.15.2:0/3443643963 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f24480cd080 con 0x7f24580ffba0 2023-06-23T18:22:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.645+0000 7f2456ffd700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f24400a1150 0x7f24400a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f244c005950 tx=0x7f244c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.647+0000 7f2454ff9700 1 -- 172.21.15.2:0/3443643963 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2448096790 con 0x7f24580ffba0 2023-06-23T18:22:12.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.847+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f245802c9a0 con 0x7f24580ffba0 2023-06-23T18:22:12.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.848+0000 7f2454ff9700 1 -- 172.21.15.2:0/3443643963 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f244800e350 con 0x7f24580ffba0 2023-06-23T18:22:12.852 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:22:12.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.849+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f24400a1150 msgr2=0x7f24400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.849+0000 7f245e05c700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f24400a1150 0x7f24400a3600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f244c005950 tx=0x7f244c00b410 comp rx=0 tx=0).stop 2023-06-23T18:22:12.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.849+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24580ffba0 msgr2=0x7f2458074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.849+0000 7f245e05c700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24580ffba0 0x7f2458074280 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f244800d980 tx=0x7f244800dc90 comp rx=0 tx=0).stop 2023-06-23T18:22:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.850+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 shutdown_connections 2023-06-23T18:22:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.850+0000 7f245e05c700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2458101720 0x7f2458072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.850+0000 7f245e05c700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24580ffba0 0x7f2458074280 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.850+0000 7f245e05c700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f24400a1150 0x7f24400a3600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.850+0000 7f245e05c700 1 --2- 172.21.15.2:0/3443643963 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2458100d90 0x7f24580728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.850+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 >> 172.21.15.2:0/3443643963 conn(0x7f24580fb1c0 msgr2=0x7f2458104280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.850+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 shutdown_connections 2023-06-23T18:22:12.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.850+0000 7f245e05c700 1 -- 172.21.15.2:0/3443643963 wait complete. 2023-06-23T18:22:12.866 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:22:12.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 -- 172.21.15.2:0/442394058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 msgr2=0x7f7858106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:12.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 --2- 172.21.15.2:0/442394058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 0x7f7858106e40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f785000b0a0 tx=0x7f785000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 -- 172.21.15.2:0/442394058 shutdown_connections 2023-06-23T18:22:12.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 --2- 172.21.15.2:0/442394058 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 0x7f7858106e40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 --2- 172.21.15.2:0/442394058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78580fea00 0x7f78580fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 --2- 172.21.15.2:0/442394058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78580ffbf0 0x7f7858100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:12.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 -- 172.21.15.2:0/442394058 >> 172.21.15.2:0/442394058 conn(0x7f78580fa020 msgr2=0x7f78580fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:12.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 -- 172.21.15.2:0/442394058 shutdown_connections 2023-06-23T18:22:12.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.991+0000 7f785e187700 1 -- 172.21.15.2:0/442394058 wait complete. 2023-06-23T18:22:12.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f785e187700 1 Processor -- start 2023-06-23T18:22:12.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f785e187700 1 -- start start 2023-06-23T18:22:12.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f785e187700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78580fea00 0x7f7858068f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f785e187700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78580ffbf0 0x7f785819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f785e187700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 0x7f785819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f785e187700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785819d2f0 con 0x7f7858100580 2023-06-23T18:22:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f785e187700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785819d460 con 0x7f78580ffbf0 2023-06-23T18:22:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f785e187700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785819d620 con 0x7f78580fea00 2023-06-23T18:22:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f7857fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 0x7f785819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f7857fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 0x7f785819cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53954/0 (socket says 172.21.15.2:53954) 2023-06-23T18:22:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.992+0000 7f7857fff700 1 -- 172.21.15.2:0/3144738072 learned_addr learned my addr 172.21.15.2:0/3144738072 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7856ffd700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78580ffbf0 0x7f785819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f78577fe700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78580fea00 0x7f7858068f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7857fff700 1 -- 172.21.15.2:0/3144738072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78580fea00 msgr2=0x7f7858068f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7857fff700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78580fea00 0x7f7858068f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7857fff700 1 -- 172.21.15.2:0/3144738072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78580ffbf0 msgr2=0x7f785819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7857fff700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78580ffbf0 0x7f785819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7857fff700 1 -- 172.21.15.2:0/3144738072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7850009d00 con 0x7f7858100580 2023-06-23T18:22:13.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7857fff700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 0x7f785819cb50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f785000b830 tx=0x7f7850004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7854ff9700 1 -- 172.21.15.2:0/3144738072 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785000e040 con 0x7f7858100580 2023-06-23T18:22:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7854ff9700 1 -- 172.21.15.2:0/3144738072 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7850009440 con 0x7f7858100580 2023-06-23T18:22:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785819d8a0 con 0x7f7858100580 2023-06-23T18:22:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f7854ff9700 1 -- 172.21.15.2:0/3144738072 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785001dc00 con 0x7f7858100580 2023-06-23T18:22:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.993+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785819dcc0 con 0x7f7858100580 2023-06-23T18:22:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.994+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7858105f80 con 0x7f7858100580 2023-06-23T18:22:13.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.995+0000 7f7854ff9700 1 -- 172.21.15.2:0/3144738072 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7850005450 con 0x7f7858100580 2023-06-23T18:22:13.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.996+0000 7f7854ff9700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78400a1150 0x7f78400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.996+0000 7f7854ff9700 1 -- 172.21.15.2:0/3144738072 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78500ce0f0 con 0x7f7858100580 2023-06-23T18:22:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.996+0000 7f78577fe700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78400a1150 0x7f78400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:12.996+0000 7f78577fe700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78400a1150 0x7f78400a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78580ffa50 tx=0x7f784800b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:13.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.000+0000 7f7854ff9700 1 -- 172.21.15.2:0/3144738072 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7850097890 con 0x7f7858100580 2023-06-23T18:22:13.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.204+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78580610e0 con 0x7f7858100580 2023-06-23T18:22:13.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.204+0000 7f7854ff9700 1 -- 172.21.15.2:0/3144738072 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f785009aeb0 con 0x7f7858100580 2023-06-23T18:22:13.209 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:22:13.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78400a1150 msgr2=0x7f78400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78400a1150 0x7f78400a3600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78580ffa50 tx=0x7f784800b5b0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 msgr2=0x7f785819cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 0x7f785819cb50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f785000b830 tx=0x7f7850004ce0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 shutdown_connections 2023-06-23T18:22:13.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7858100580 0x7f785819cb50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78580fea00 0x7f7858068f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f78400a1150 0x7f78400a3600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 --2- 172.21.15.2:0/3144738072 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78580ffbf0 0x7f785819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.206+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 >> 172.21.15.2:0/3144738072 conn(0x7f78580fa020 msgr2=0x7f78581030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:13.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.207+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 shutdown_connections 2023-06-23T18:22:13.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.207+0000 7f785e187700 1 -- 172.21.15.2:0/3144738072 wait complete. 2023-06-23T18:22:13.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:13 smithi002 ceph-mon[95947]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:13.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:13 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/335330350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:13.222 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:13 smithi002 ceph-mon[95947]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-06-23T18:22:13.223 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:13 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3443643963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:13.223 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:22:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 -- 172.21.15.2:0/4227180173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24100620 msgr2=0x7f9c24102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 --2- 172.21.15.2:0/4227180173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24100620 0x7f9c24102a00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009990 tx=0x7f9c0c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 -- 172.21.15.2:0/4227180173 shutdown_connections 2023-06-23T18:22:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 --2- 172.21.15.2:0/4227180173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c24105860 0x7f9c24107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 --2- 172.21.15.2:0/4227180173 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24100620 0x7f9c24102a00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 --2- 172.21.15.2:0/4227180173 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c24102f40 0x7f9c24105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 -- 172.21.15.2:0/4227180173 >> 172.21.15.2:0/4227180173 conn(0x7f9c240fa0c0 msgr2=0x7f9c240fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 -- 172.21.15.2:0/4227180173 shutdown_connections 2023-06-23T18:22:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.348+0000 7f9c2a051700 1 -- 172.21.15.2:0/4227180173 wait complete. 2023-06-23T18:22:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c2a051700 1 Processor -- start 2023-06-23T18:22:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c2a051700 1 -- start start 2023-06-23T18:22:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c2a051700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c24100620 0x7f9c24195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c2a051700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24102f40 0x7f9c24195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c2a051700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c24105860 0x7f9c2419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c2a051700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2419d310 con 0x7f9c24102f40 2023-06-23T18:22:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c2a051700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2419d450 con 0x7f9c24100620 2023-06-23T18:22:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c2a051700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2419d5e0 con 0x7f9c24105860 2023-06-23T18:22:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c22ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24102f40 0x7f9c24195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c22ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24102f40 0x7f9c24195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53964/0 (socket says 172.21.15.2:53964) 2023-06-23T18:22:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c22ffd700 1 -- 172.21.15.2:0/1135712698 learned_addr learned my addr 172.21.15.2:0/1135712698 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c237fe700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c24100620 0x7f9c24195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c23fff700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c24105860 0x7f9c2419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c22ffd700 1 -- 172.21.15.2:0/1135712698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c24105860 msgr2=0x7f9c2419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c22ffd700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c24105860 0x7f9c2419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c22ffd700 1 -- 172.21.15.2:0/1135712698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c24100620 msgr2=0x7f9c24195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c22ffd700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c24100620 0x7f9c24195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c22ffd700 1 -- 172.21.15.2:0/1135712698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0095f0 con 0x7f9c24102f40 2023-06-23T18:22:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.349+0000 7f9c23fff700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c24105860 0x7f9c2419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.350+0000 7f9c22ffd700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24102f40 0x7f9c24195cd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400e9a0 tx=0x7f9c1400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.350+0000 7f9c20ff9700 1 -- 172.21.15.2:0/1135712698 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14009790 con 0x7f9c24102f40 2023-06-23T18:22:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.350+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2419d8c0 con 0x7f9c24102f40 2023-06-23T18:22:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.350+0000 7f9c20ff9700 1 -- 172.21.15.2:0/1135712698 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9c1400cbd0 con 0x7f9c24102f40 2023-06-23T18:22:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.350+0000 7f9c20ff9700 1 -- 172.21.15.2:0/1135712698 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14018680 con 0x7f9c24102f40 2023-06-23T18:22:13.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.350+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2419ddd0 con 0x7f9c24102f40 2023-06-23T18:22:13.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.351+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c240fbc30 con 0x7f9c24102f40 2023-06-23T18:22:13.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.352+0000 7f9c20ff9700 1 -- 172.21.15.2:0/1135712698 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9c1400cd40 con 0x7f9c24102f40 2023-06-23T18:22:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.353+0000 7f9c20ff9700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c100a1150 0x7f9c100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.353+0000 7f9c20ff9700 1 -- 172.21.15.2:0/1135712698 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9c14014070 con 0x7f9c24102f40 2023-06-23T18:22:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.353+0000 7f9c237fe700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c100a1150 0x7f9c100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.353+0000 7f9c237fe700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c100a1150 0x7f9c100a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c006010 tx=0x7f9c0c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.356+0000 7f9c20ff9700 1 -- 172.21.15.2:0/1135712698 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9c140966c0 con 0x7f9c24102f40 2023-06-23T18:22:13.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:13 smithi017 ceph-mon[97472]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:13.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:13 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/335330350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:13.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:13 smithi017 ceph-mon[97472]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-06-23T18:22:13.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:13 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3443643963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:13.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.554+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c2402ca60 con 0x7f9c24102f40 2023-06-23T18:22:13.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.554+0000 7f9c20ff9700 1 -- 172.21.15.2:0/1135712698 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c14099ce0 con 0x7f9c24102f40 2023-06-23T18:22:13.559 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:22:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c100a1150 msgr2=0x7f9c100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c100a1150 0x7f9c100a3600 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c006010 tx=0x7f9c0c005960 comp rx=0 tx=0).stop 2023-06-23T18:22:13.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24102f40 msgr2=0x7f9c24195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24102f40 0x7f9c24195cd0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400e9a0 tx=0x7f9c1400ecb0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 shutdown_connections 2023-06-23T18:22:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c24105860 0x7f9c2419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9c100a1150 0x7f9c100a3600 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c24100620 0x7f9c24195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 --2- 172.21.15.2:0/1135712698 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9c24102f40 0x7f9c24195cd0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 >> 172.21.15.2:0/1135712698 conn(0x7f9c240fa0c0 msgr2=0x7f9c24101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:13.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 shutdown_connections 2023-06-23T18:22:13.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.557+0000 7f9c2a051700 1 -- 172.21.15.2:0/1135712698 wait complete. 2023-06-23T18:22:13.573 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:22:13.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:13 smithi082 ceph-mon[97889]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-06-23T18:22:13.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:13 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/335330350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:13.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:13 smithi082 ceph-mon[97889]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2023-06-23T18:22:13.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:13 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3443643963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:22:13.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 -- 172.21.15.2:0/3817924019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0ffc60 msgr2=0x7f9f9c1000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 --2- 172.21.15.2:0/3817924019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0ffc60 0x7f9f9c1000b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009990 tx=0x7f9f88009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 -- 172.21.15.2:0/3817924019 shutdown_connections 2023-06-23T18:22:13.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 --2- 172.21.15.2:0/3817924019 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f9c1005f0 0x7f9f9c106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 --2- 172.21.15.2:0/3817924019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f9c0fea70 0x7f9f9c0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 --2- 172.21.15.2:0/3817924019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0ffc60 0x7f9f9c1000b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 -- 172.21.15.2:0/3817924019 >> 172.21.15.2:0/3817924019 conn(0x7f9f9c0fa090 msgr2=0x7f9f9c0fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:13.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 -- 172.21.15.2:0/3817924019 shutdown_connections 2023-06-23T18:22:13.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 -- 172.21.15.2:0/3817924019 wait complete. 2023-06-23T18:22:13.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.695+0000 7f9fa1345700 1 Processor -- start 2023-06-23T18:22:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9fa1345700 1 -- start start 2023-06-23T18:22:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9fa1345700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0fea70 0x7f9f9c194760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9fa1345700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f9c0ffc60 0x7f9f9c194ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9fa1345700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f9c1005f0 0x7f9f9c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9fa1345700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19c2e0 con 0x7f9f9c0fea70 2023-06-23T18:22:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9fa1345700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19c420 con 0x7f9f9c1005f0 2023-06-23T18:22:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9fa1345700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c19c5b0 con 0x7f9f9c0ffc60 2023-06-23T18:22:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0fea70 0x7f9f9c194760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0fea70 0x7f9f9c194760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53976/0 (socket says 172.21.15.2:53976) 2023-06-23T18:22:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 -- 172.21.15.2:0/2632461048 learned_addr learned my addr 172.21.15.2:0/2632461048 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9a7fc700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f9c0ffc60 0x7f9f9c194ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9b7fe700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f9c1005f0 0x7f9f9c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 -- 172.21.15.2:0/2632461048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f9c0ffc60 msgr2=0x7f9f9c194ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f9c0ffc60 0x7f9f9c194ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 -- 172.21.15.2:0/2632461048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f9c1005f0 msgr2=0x7f9f9c19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f9c1005f0 0x7f9f9c19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 -- 172.21.15.2:0/2632461048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880095f0 con 0x7f9f9c0fea70 2023-06-23T18:22:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9b7fe700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f9c1005f0 0x7f9f9c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.696+0000 7f9f9affd700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0fea70 0x7f9f9c194760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00ea80 tx=0x7f9f8c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.697+0000 7f9f93fff700 1 -- 172.21.15.2:0/2632461048 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c013070 con 0x7f9f9c0fea70 2023-06-23T18:22:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.697+0000 7f9f93fff700 1 -- 172.21.15.2:0/2632461048 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9f8c00be50 con 0x7f9f9c0fea70 2023-06-23T18:22:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.697+0000 7f9f93fff700 1 -- 172.21.15.2:0/2632461048 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c003a00 con 0x7f9f9c0fea70 2023-06-23T18:22:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.697+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c0685c0 con 0x7f9f9c0fea70 2023-06-23T18:22:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.697+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c068ad0 con 0x7f9f9c0fea70 2023-06-23T18:22:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.698+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9c10a0f0 con 0x7f9f9c0fea70 2023-06-23T18:22:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.699+0000 7f9f93fff700 1 -- 172.21.15.2:0/2632461048 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9f8c00b9c0 con 0x7f9f9c0fea70 2023-06-23T18:22:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.699+0000 7f9f93fff700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f800a11d0 0x7f9f800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.700+0000 7f9f93fff700 1 -- 172.21.15.2:0/2632461048 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9f8c0ce790 con 0x7f9f9c0fea70 2023-06-23T18:22:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.700+0000 7f9f9a7fc700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f800a11d0 0x7f9f800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.700+0000 7f9f9a7fc700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f800a11d0 0x7f9f800a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b2e0 tx=0x7f9f88004000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:13.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.703+0000 7f9f93fff700 1 -- 172.21.15.2:0/2632461048 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f9f8c097ee0 con 0x7f9f9c0fea70 2023-06-23T18:22:13.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.947+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f9c069390 con 0x7f9f9c0fea70 2023-06-23T18:22:13.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.948+0000 7f9f93fff700 1 -- 172.21.15.2:0/2632461048 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7f9f8c09b500 con 0x7f9f9c0fea70 2023-06-23T18:22:13.952 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:13.953 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:22:13.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f800a11d0 msgr2=0x7f9f800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f800a11d0 0x7f9f800a3680 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800b2e0 tx=0x7f9f88004000 comp rx=0 tx=0).stop 2023-06-23T18:22:13.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0fea70 msgr2=0x7f9f9c194760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:13.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0fea70 0x7f9f9c194760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00ea80 tx=0x7f9f8c00ed90 comp rx=0 tx=0).stop 2023-06-23T18:22:13.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 shutdown_connections 2023-06-23T18:22:13.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f9f800a11d0 0x7f9f800a3680 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9f9c1005f0 0x7f9f9c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f9c0fea70 0x7f9f9c194760 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 --2- 172.21.15.2:0/2632461048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9f9c0ffc60 0x7f9f9c194ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 >> 172.21.15.2:0/2632461048 conn(0x7f9f9c0fa090 msgr2=0x7f9f9c0fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 shutdown_connections 2023-06-23T18:22:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:13.950+0000 7f9fa1345700 1 -- 172.21.15.2:0/2632461048 wait complete. 2023-06-23T18:22:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:22:13.967 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:22:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 -- 172.21.15.2:0/3900303278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc1005e0 msgr2=0x7f46dc106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/3900303278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc1005e0 0x7f46dc106ea0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f46d400b0a0 tx=0x7f46d400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 -- 172.21.15.2:0/3900303278 shutdown_connections 2023-06-23T18:22:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/3900303278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc1005e0 0x7f46dc106ea0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/3900303278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46dc0fea60 0x7f46dc0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/3900303278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46dc0ffc50 0x7f46dc1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 -- 172.21.15.2:0/3900303278 >> 172.21.15.2:0/3900303278 conn(0x7f46dc0fa0e0 msgr2=0x7f46dc0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 -- 172.21.15.2:0/3900303278 shutdown_connections 2023-06-23T18:22:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.090+0000 7f46e0cb3700 1 -- 172.21.15.2:0/3900303278 wait complete. 2023-06-23T18:22:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46e0cb3700 1 Processor -- start 2023-06-23T18:22:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46e0cb3700 1 -- start start 2023-06-23T18:22:14.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46e0cb3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc0fea60 0x7f46dc195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46e0cb3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46dc0ffc50 0x7f46dc195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46e0cb3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46dc1005e0 0x7f46dc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46e0cb3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc19d3a0 con 0x7f46dc0fea60 2023-06-23T18:22:14.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46e0cb3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc19d4e0 con 0x7f46dc1005e0 2023-06-23T18:22:14.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46e0cb3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc19d670 con 0x7f46dc0ffc50 2023-06-23T18:22:14.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46da59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc0fea60 0x7f46dc195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46da59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc0fea60 0x7f46dc195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:53990/0 (socket says 172.21.15.2:53990) 2023-06-23T18:22:14.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46da59c700 1 -- 172.21.15.2:0/2118904244 learned_addr learned my addr 172.21.15.2:0/2118904244 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46d9d9b700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46dc0ffc50 0x7f46dc195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46dad9d700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46dc1005e0 0x7f46dc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.091+0000 7f46da59c700 1 -- 172.21.15.2:0/2118904244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46dc0ffc50 msgr2=0x7f46dc195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46da59c700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46dc0ffc50 0x7f46dc195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46da59c700 1 -- 172.21.15.2:0/2118904244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46dc1005e0 msgr2=0x7f46dc19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46da59c700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46dc1005e0 0x7f46dc19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46da59c700 1 -- 172.21.15.2:0/2118904244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d4009d00 con 0x7f46dc0fea60 2023-06-23T18:22:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46da59c700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc0fea60 0x7f46dc195870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00d980 tx=0x7f46cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46cb7fe700 1 -- 172.21.15.2:0/2118904244 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc004020 con 0x7f46dc0fea60 2023-06-23T18:22:14.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46cb7fe700 1 -- 172.21.15.2:0/2118904244 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46cc00f690 con 0x7f46dc0fea60 2023-06-23T18:22:14.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc19d950 con 0x7f46dc0fea60 2023-06-23T18:22:14.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46cb7fe700 1 -- 172.21.15.2:0/2118904244 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0039e0 con 0x7f46dc0fea60 2023-06-23T18:22:14.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.092+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46dc19de60 con 0x7f46dc0fea60 2023-06-23T18:22:14.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.093+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46dc10a0f0 con 0x7f46dc0fea60 2023-06-23T18:22:14.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.094+0000 7f46cb7fe700 1 -- 172.21.15.2:0/2118904244 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f46cc00f800 con 0x7f46dc0fea60 2023-06-23T18:22:14.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.094+0000 7f46cb7fe700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46c40a1150 0x7f46c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.095+0000 7f46d9d9b700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46c40a1150 0x7f46c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.095+0000 7f46cb7fe700 1 -- 172.21.15.2:0/2118904244 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f46cc0cc540 con 0x7f46dc0fea60 2023-06-23T18:22:14.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.095+0000 7f46d9d9b700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46c40a1150 0x7f46c40a3600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f46d000ac50 tx=0x7f46d000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.099+0000 7f46cb7fe700 1 -- 172.21.15.2:0/2118904244 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f46cc095c50 con 0x7f46dc0fea60 2023-06-23T18:22:14.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:14 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3144738072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:14.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:14 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1135712698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:14.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:14 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2632461048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:14.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.300+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46dc10a310 con 0x7f46dc0fea60 2023-06-23T18:22:14.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.301+0000 7f46cb7fe700 1 -- 172.21.15.2:0/2118904244 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f46dc10a310 con 0x7f46dc0fea60 2023-06-23T18:22:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:14.307 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:22:14.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.303+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46c40a1150 msgr2=0x7f46c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.303+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46c40a1150 0x7f46c40a3600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f46d000ac50 tx=0x7f46d000a380 comp rx=0 tx=0).stop 2023-06-23T18:22:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.303+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc0fea60 msgr2=0x7f46dc195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.303+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc0fea60 0x7f46dc195870 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00d980 tx=0x7f46cc00dc90 comp rx=0 tx=0).stop 2023-06-23T18:22:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.304+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 shutdown_connections 2023-06-23T18:22:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.304+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46c40a1150 0x7f46c40a3600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.304+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46dc1005e0 0x7f46dc19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.304+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46dc0fea60 0x7f46dc195870 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.304+0000 7f46e0cb3700 1 --2- 172.21.15.2:0/2118904244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46dc0ffc50 0x7f46dc195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.304+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 >> 172.21.15.2:0/2118904244 conn(0x7f46dc0fa0e0 msgr2=0x7f46dc067860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.304+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 shutdown_connections 2023-06-23T18:22:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.304+0000 7f46e0cb3700 1 -- 172.21.15.2:0/2118904244 wait complete. 2023-06-23T18:22:14.323 DEBUG:tasks.fwd_scrub:Yielding 2023-06-23T18:22:14.323 INFO:teuthology.run_tasks:Running task exec... 2023-06-23T18:22:14.337 INFO:teuthology.task.exec:Executing custom commands... 2023-06-23T18:22:14.337 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi002.front.sepia.ceph.com 2023-06-23T18:22:14.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-06-23T18:22:14.338 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-06-23T18:22:14.339 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:22:14.339 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:22:14.380 INFO:teuthology.orchestra.run.smithi002.stdout:total 0 2023-06-23T18:22:14.393 INFO:teuthology.orchestra.run.smithi002.stdout:Filesystem Size Used Avail Use% Mounted on 2023-06-23T18:22:14.393 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-fuse 340G 0 340G 0% /tmp/cephfs 2023-06-23T18:22:14.395 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-06-23T18:22:14.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:14 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3144738072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:14.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:14 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1135712698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:14.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:14 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2632461048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:14.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 -- 172.21.15.2:0/241399537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0105850 msgr2=0x7fcdc0107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/241399537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0105850 0x7fcdc0107cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcdb800b0a0 tx=0x7fcdb800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 -- 172.21.15.2:0/241399537 shutdown_connections 2023-06-23T18:22:14.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/241399537 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0105850 0x7fcdc0107cb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/241399537 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc0100610 0x7fcdc01029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/241399537 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc0102f30 0x7fcdc0105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 -- 172.21.15.2:0/241399537 >> 172.21.15.2:0/241399537 conn(0x7fcdc00fa090 msgr2=0x7fcdc00fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 -- 172.21.15.2:0/241399537 shutdown_connections 2023-06-23T18:22:14.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.468+0000 7fcdc7f58700 1 -- 172.21.15.2:0/241399537 wait complete. 2023-06-23T18:22:14.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc7f58700 1 Processor -- start 2023-06-23T18:22:14.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc7f58700 1 -- start start 2023-06-23T18:22:14.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc7f58700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0100610 0x7fcdc0195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc7f58700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc0102f30 0x7fcdc0195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc7f58700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc0105850 0x7fcdc019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc7f58700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc019d3e0 con 0x7fcdc0100610 2023-06-23T18:22:14.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc7f58700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc019d520 con 0x7fcdc0105850 2023-06-23T18:22:14.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc7f58700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc019d6b0 con 0x7fcdc0102f30 2023-06-23T18:22:14.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0100610 0x7fcdc0195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0100610 0x7fcdc0195860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54004/0 (socket says 172.21.15.2:54004) 2023-06-23T18:22:14.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 -- 172.21.15.2:0/904962930 learned_addr learned my addr 172.21.15.2:0/904962930 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc0102f30 0x7fcdc0195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc64f5700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc0105850 0x7fcdc019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 -- 172.21.15.2:0/904962930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc0102f30 msgr2=0x7fcdc0195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc0102f30 0x7fcdc0195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 -- 172.21.15.2:0/904962930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc0105850 msgr2=0x7fcdc019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc0105850 0x7fcdc019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 -- 172.21.15.2:0/904962930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8009d00 con 0x7fcdc0100610 2023-06-23T18:22:14.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc64f5700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc0105850 0x7fcdc019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:14.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.469+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0100610 0x7fcdc0195860 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00d950 tx=0x7fcdbc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.470+0000 7fcdb6ffd700 1 -- 172.21.15.2:0/904962930 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc009b40 con 0x7fcdc0100610 2023-06-23T18:22:14.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.470+0000 7fcdb6ffd700 1 -- 172.21.15.2:0/904962930 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdbc009ca0 con 0x7fcdc0100610 2023-06-23T18:22:14.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.470+0000 7fcdc7f58700 1 -- 172.21.15.2:0/904962930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc019d990 con 0x7fcdc0100610 2023-06-23T18:22:14.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.470+0000 7fcdb6ffd700 1 -- 172.21.15.2:0/904962930 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc00f620 con 0x7fcdc0100610 2023-06-23T18:22:14.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.470+0000 7fcdc7f58700 1 -- 172.21.15.2:0/904962930 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc019def0 con 0x7fcdc0100610 2023-06-23T18:22:14.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.472+0000 7fcdb6ffd700 1 -- 172.21.15.2:0/904962930 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcdbc010460 con 0x7fcdc0100610 2023-06-23T18:22:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.473+0000 7fcdb6ffd700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcdac0a11d0 0x7fcdac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.473+0000 7fcdb6ffd700 1 -- 172.21.15.2:0/904962930 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcdbc0ccc40 con 0x7fcdc0100610 2023-06-23T18:22:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.473+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcdac0a11d0 0x7fcdac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.473+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcdac0a11d0 0x7fcdac0a3680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009bf0 tx=0x7fcdb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.473+0000 7fcdd40ddb80 1 Processor -- start 2023-06-23T18:22:14.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdd40ddb80 1 -- start start 2023-06-23T18:22:14.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd7c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd7caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd7d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fc80d30 con 0x559c7fd7c650 2023-06-23T18:22:14.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fcd3ac0 con 0x559c7fd7bd00 2023-06-23T18:22:14.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fcccf50 con 0x559c7fd7d080 2023-06-23T18:22:14.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc54f3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd7caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc54f3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd7caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54014/0 (socket says 172.21.15.2:54014) 2023-06-23T18:22:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc54f3700 1 -- 172.21.15.2:0/3802616185 learned_addr learned my addr 172.21.15.2:0/3802616185 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd7c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc64f5700 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd7d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc54f3700 1 -- 172.21.15.2:0/3802616185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 msgr2=0x559c7fd7d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd7d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc54f3700 1 -- 172.21.15.2:0/3802616185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 msgr2=0x559c7fd7c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd7c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.474+0000 7fcdc54f3700 1 -- 172.21.15.2:0/3802616185 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7fb4e6b0 con 0x559c7fd7c650 2023-06-23T18:22:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.475+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd7caf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0011e40 tx=0x7fcdb0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974e7d13e6b3ca87 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.475+0000 7fcdabfff700 1 -- 172.21.15.2:0/3802616185 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0012dc0 con 0x559c7fd7c650 2023-06-23T18:22:14.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.475+0000 7fcdabfff700 1 -- 172.21.15.2:0/3802616185 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdb0016070 con 0x559c7fd7c650 2023-06-23T18:22:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.475+0000 7fcdabfff700 1 -- 172.21.15.2:0/3802616185 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0003d60 con 0x559c7fd7c650 2023-06-23T18:22:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.475+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/3802616185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 msgr2=0x559c7fd7caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.475+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd7caf0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0011e40 tx=0x7fcdb0014000 comp rx=0 tx=0).stop 2023-06-23T18:22:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/3802616185 shutdown_connections 2023-06-23T18:22:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd7d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd7c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/3802616185 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd7caf0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/3802616185 >> 172.21.15.2:0/3802616185 conn(0x559c7fd71470 msgr2=0x559c7fd765a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/3802616185 shutdown_connections 2023-06-23T18:22:14.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/3802616185 wait complete. 2023-06-23T18:22:14.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 -- start start 2023-06-23T18:22:14.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 Processor -- start 2023-06-23T18:22:14.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.476+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7bd00 0x559c7fd855b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fd85af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd8cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fc6cb40 con 0x559c7fd7bd00 2023-06-23T18:22:14.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fc6c840 con 0x559c7fd7c650 2023-06-23T18:22:14.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fb509a0 con 0x559c7fd7d080 2023-06-23T18:22:14.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdd40ddb80 10 client.0 _ll_register_callbacks cb 0x559c7fc722e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:14.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc5cf4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7bd00 0x559c7fd855b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc54f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fd85af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc5cf4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7bd00 0x559c7fd855b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54018/0 (socket says 172.21.15.2:54018) 2023-06-23T18:22:14.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc5cf4700 1 -- 172.21.15.2:0/2972454773 learned_addr learned my addr 172.21.15.2:0/2972454773 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc64f5700 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd8cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc54f3700 1 -- 172.21.15.2:0/2972454773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 msgr2=0x559c7fd8cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd8cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc54f3700 1 -- 172.21.15.2:0/2972454773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7bd00 msgr2=0x559c7fd855b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7bd00 0x559c7fd855b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc54f3700 1 -- 172.21.15.2:0/2972454773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb0011b20 con 0x559c7fd7c650 2023-06-23T18:22:14.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7bd00 0x559c7fd855b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:14.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fd85af0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0011e40 tx=0x7fcdb0028680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed8602fe3c489d32 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.477+0000 7fcda97fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:22:14.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcda97fa700 1 -- 172.21.15.2:0/2972454773 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb002d070 con 0x559c7fd7c650 2023-06-23T18:22:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcda97fa700 1 -- 172.21.15.2:0/2972454773 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcdb0018070 con 0x559c7fd7c650 2023-06-23T18:22:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2972454773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c7fd6ec30 con 0x559c7fd7c650 2023-06-23T18:22:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcda97fa700 1 -- 172.21.15.2:0/2972454773 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb0011e70 con 0x559c7fd7c650 2023-06-23T18:22:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 10 client.25004 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2972454773 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559c7fafae60 con 0x559c7fd7c650 2023-06-23T18:22:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcda97fa700 1 -- 172.21.15.2:0/2972454773 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559c7fd6ec30 con 0x559c7fd7c650 2023-06-23T18:22:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcda97fa700 1 -- 172.21.15.2:0/2972454773 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcdb0025070 con 0x559c7fd7c650 2023-06-23T18:22:14.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 10 client.25004 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:14.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 10 client.25004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:14.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 20 client.25004 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:14.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559c7fd911f0 0x559c7fd93600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 4 client.25004 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:14.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2972454773 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c7fcc5f30 con 0x559c7fd911f0 2023-06-23T18:22:14.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.478+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559c7fd911f0 0x559c7fd93600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.479+0000 7fcda97fa700 10 client.25004 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:14.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.479+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559c7fd911f0 0x559c7fd93600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c4c610f03f5151 server_cookie=836630bdc67d619a in_seq=0 out_seq=0 2023-06-23T18:22:14.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcda97fa700 1 -- 172.21.15.2:0/2972454773 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559c7fcc5f30 con 0x559c7fd911f0 2023-06-23T18:22:14.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcda97fa700 10 client.25004 handle_command_reply: tid=0 2023-06-23T18:22:14.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 client.25004 shutdown 2023-06-23T18:22:14.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2972454773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 msgr2=0x559c7fd85af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fd85af0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0011e40 tx=0x7fcdb0028680 comp rx=0 tx=0).stop 2023-06-23T18:22:14.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2972454773 shutdown_connections 2023-06-23T18:22:14.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559c7fd911f0 0x559c7fd93600 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd8cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7bd00 0x559c7fd855b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2972454773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fd85af0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcda97fa700 0 client.25004 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:14.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2972454773 >> 172.21.15.2:0/2972454773 conn(0x559c7fd71470 msgr2=0x559c7fd75cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2972454773 shutdown_connections 2023-06-23T18:22:14.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2972454773 wait complete. 2023-06-23T18:22:14.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.480+0000 7fcdd40ddb80 20 client.25004 trim_cache size 0 max 16384 2023-06-23T18:22:14.498 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-06-23T18:22:14.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.499+0000 7fcdd40ddb80 1 Processor -- start 2023-06-23T18:22:14.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.499+0000 7fcdd40ddb80 1 -- start start 2023-06-23T18:22:14.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd83390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd75440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fd778c0 con 0x559c7fd7c650 2023-06-23T18:22:14.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fd864e0 con 0x559c7fd7bd00 2023-06-23T18:22:14.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fd85120 con 0x559c7fd7d080 2023-06-23T18:22:14.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd838d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54028/0 (socket says 172.21.15.2:54028) 2023-06-23T18:22:14.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 -- 172.21.15.2:0/2423971151 learned_addr learned my addr 172.21.15.2:0/2423971151 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc64f5700 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd75440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd83390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 -- 172.21.15.2:0/2423971151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 msgr2=0x559c7fd75440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd75440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 -- 172.21.15.2:0/2423971151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 msgr2=0x559c7fd83390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd83390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 -- 172.21.15.2:0/2423971151 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c7fd845e0 con 0x559c7fd7c650 2023-06-23T18:22:14.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd83390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:14.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.500+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd838d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0028e30 tx=0x7fcdb0003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75a5ea1fe8fae37 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdb4ff9700 1 -- 172.21.15.2:0/2423971151 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb002d030 con 0x559c7fd7c650 2023-06-23T18:22:14.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdb4ff9700 1 -- 172.21.15.2:0/2423971151 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdb00056f0 con 0x559c7fd7c650 2023-06-23T18:22:14.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdb4ff9700 1 -- 172.21.15.2:0/2423971151 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0028950 con 0x559c7fd7c650 2023-06-23T18:22:14.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2423971151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 msgr2=0x559c7fd838d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd838d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0028e30 tx=0x7fcdb0003800 comp rx=0 tx=0).stop 2023-06-23T18:22:14.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2423971151 shutdown_connections 2023-06-23T18:22:14.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7d080 0x559c7fd75440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7bd00 0x559c7fd83390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/2423971151 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7c650 0x559c7fd838d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2423971151 >> 172.21.15.2:0/2423971151 conn(0x559c7fd71470 msgr2=0x559c7fd85570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.501+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2423971151 shutdown_connections 2023-06-23T18:22:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/2423971151 wait complete. 2023-06-23T18:22:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 -- start start 2023-06-23T18:22:14.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 Processor -- start 2023-06-23T18:22:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7bd00 0x559c7fde1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fde1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7d080 0x559c7fde89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fde8ef0 con 0x559c7fd7d080 2023-06-23T18:22:14.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fde9030 con 0x559c7fd7c650 2023-06-23T18:22:14.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c7fde91c0 con 0x559c7fd7bd00 2023-06-23T18:22:14.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.502+0000 7fcdd40ddb80 10 client.0 _ll_register_callbacks cb 0x559c7fc722e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc64f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7d080 0x559c7fde89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc54f3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fde1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc64f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7d080 0x559c7fde89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54036/0 (socket says 172.21.15.2:54036) 2023-06-23T18:22:14.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc64f5700 1 -- 172.21.15.2:0/1489403455 learned_addr learned my addr 172.21.15.2:0/1489403455 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7bd00 0x559c7fde1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc54f3700 1 -- 172.21.15.2:0/1489403455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7bd00 msgr2=0x559c7fde1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7bd00 0x559c7fde1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc54f3700 1 -- 172.21.15.2:0/1489403455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7d080 msgr2=0x559c7fde89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7d080 0x559c7fde89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc54f3700 1 -- 172.21.15.2:0/1489403455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c7fafae60 con 0x559c7fd7c650 2023-06-23T18:22:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc64f5700 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7d080 0x559c7fde89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdc54f3700 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fde1970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0011e40 tx=0x7fcdb00092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e45757acb345472 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcda97fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:22:14.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcda97fa700 1 -- 172.21.15.2:0/1489403455 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0028a70 con 0x559c7fd7c650 2023-06-23T18:22:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcda97fa700 1 -- 172.21.15.2:0/1489403455 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcdb002e070 con 0x559c7fd7c650 2023-06-23T18:22:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.503+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/1489403455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdb0016070 con 0x559c7fd7c650 2023-06-23T18:22:14.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcda97fa700 1 -- 172.21.15.2:0/1489403455 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb0011980 con 0x559c7fd7c650 2023-06-23T18:22:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdd40ddb80 10 client.25007 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/1489403455 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559c7fae3ce0 con 0x559c7fd7c650 2023-06-23T18:22:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcda97fa700 1 -- 172.21.15.2:0/1489403455 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcdb0016070 con 0x559c7fd7c650 2023-06-23T18:22:14.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcda97fa700 1 -- 172.21.15.2:0/1489403455 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcdb0025070 con 0x559c7fd7c650 2023-06-23T18:22:14.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdd40ddb80 10 client.25007 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:14.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdd40ddb80 10 client.25007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:14.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdd40ddb80 20 client.25007 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:14.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559c7fdea640 0x559c7fdeca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdd40ddb80 4 client.25007 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:22:14.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/1489403455 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559c7fc880d0 con 0x559c7fdea640 2023-06-23T18:22:14.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.504+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559c7fdea640 0x559c7fdeca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.505+0000 7fcdc5cf4700 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559c7fdea640 0x559c7fdeca20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7efa88c0eb562b server_cookie=25b10c67fe877806 in_seq=0 out_seq=0 2023-06-23T18:22:14.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.505+0000 7fcda97fa700 10 client.25007 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:14.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcda97fa700 1 -- 172.21.15.2:0/1489403455 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559c7fc880d0 con 0x559c7fdea640 2023-06-23T18:22:14.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcda97fa700 10 client.25007 handle_command_reply: tid=0 2023-06-23T18:22:14.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 client.25007 shutdown 2023-06-23T18:22:14.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/1489403455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 msgr2=0x559c7fde1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fde1970 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0011e40 tx=0x7fcdb00092d0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/1489403455 shutdown_connections 2023-06-23T18:22:14.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559c7fdea640 0x559c7fdeca20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559c7fd7d080 0x559c7fde89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559c7fd7bd00 0x559c7fde1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 --2- 172.21.15.2:0/1489403455 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559c7fd7c650 0x559c7fde1970 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/1489403455 >> 172.21.15.2:0/1489403455 conn(0x559c7fd71470 msgr2=0x559c7fd85550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcda97fa700 0 client.25007 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:14.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/1489403455 shutdown_connections 2023-06-23T18:22:14.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.506+0000 7fcdd40ddb80 1 -- 172.21.15.2:0/1489403455 wait complete. 2023-06-23T18:22:14.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdd40ddb80 20 client.25007 trim_cache size 0 max 16384 2023-06-23T18:22:14.532 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:14.533 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:22:14.533 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "31aedc34-1172-43af-86d1-ff97b0e8e410", 2023-06-23T18:22:14.533 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:22:14.533 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:14.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 -- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcdac0a11d0 msgr2=0x7fcdac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcdac0a11d0 0x7fcdac0a3680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009bf0 tx=0x7fcdb0009260 comp rx=0 tx=0).stop 2023-06-23T18:22:14.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 -- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0100610 msgr2=0x7fcdc0195860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0100610 0x7fcdc0195860 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc00d950 tx=0x7fcdbc00dd10 comp rx=0 tx=0).stop 2023-06-23T18:22:14.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 -- 172.21.15.2:0/904962930 shutdown_connections 2023-06-23T18:22:14.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcdac0a11d0 0x7fcdac0a3680 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdc0105850 0x7fcdc019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcdc0100610 0x7fcdc0195860 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 --2- 172.21.15.2:0/904962930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc0102f30 0x7fcdc0195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 -- 172.21.15.2:0/904962930 >> 172.21.15.2:0/904962930 conn(0x7fcdc00fa090 msgr2=0x7fcdc01012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 -- 172.21.15.2:0/904962930 shutdown_connections 2023-06-23T18:22:14.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.507+0000 7fcdc7f58700 1 -- 172.21.15.2:0/904962930 wait complete. 2023-06-23T18:22:14.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:22:14.549 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-06-23T18:22:14.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:14 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3144738072' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:22:14.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:14 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1135712698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:22:14.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:14 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2632461048' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:14.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 -- 172.21.15.2:0/888434575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c102f20 msgr2=0x7f700c105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 --2- 172.21.15.2:0/888434575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c102f20 0x7f700c105300 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7000009990 tx=0x7f7000009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 -- 172.21.15.2:0/888434575 shutdown_connections 2023-06-23T18:22:14.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 --2- 172.21.15.2:0/888434575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c105840 0x7f700c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 --2- 172.21.15.2:0/888434575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f700c100600 0x7f700c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 --2- 172.21.15.2:0/888434575 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c102f20 0x7f700c105300 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 -- 172.21.15.2:0/888434575 >> 172.21.15.2:0/888434575 conn(0x7f700c0fa0e0 msgr2=0x7f700c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 -- 172.21.15.2:0/888434575 shutdown_connections 2023-06-23T18:22:14.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.715+0000 7f7011a8b700 1 -- 172.21.15.2:0/888434575 wait complete. 2023-06-23T18:22:14.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f7011a8b700 1 Processor -- start 2023-06-23T18:22:14.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f7011a8b700 1 -- start start 2023-06-23T18:22:14.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f7011a8b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c100600 0x7f700c0fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f7011a8b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c102f20 0x7f700c0fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f7011a8b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f700c105840 0x7f700c0feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f7011a8b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0ff290 con 0x7f700c100600 2023-06-23T18:22:14.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f7011a8b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0ff400 con 0x7f700c105840 2023-06-23T18:22:14.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f7011a8b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c19f7e0 con 0x7f700c102f20 2023-06-23T18:22:14.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c100600 0x7f700c0fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c100600 0x7f700c0fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54046/0 (socket says 172.21.15.2:54046) 2023-06-23T18:22:14.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.716+0000 7f700b7fe700 1 -- 172.21.15.2:0/3776243651 learned_addr learned my addr 172.21.15.2:0/3776243651 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f700affd700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c102f20 0x7f700c0fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f700bfff700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f700c105840 0x7f700c0feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f700b7fe700 1 -- 172.21.15.2:0/3776243651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c102f20 msgr2=0x7f700c0fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f700b7fe700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c102f20 0x7f700c0fe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f700b7fe700 1 -- 172.21.15.2:0/3776243651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f700c105840 msgr2=0x7f700c0feaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f700b7fe700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f700c105840 0x7f700c0feaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f700b7fe700 1 -- 172.21.15.2:0/3776243651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70000095f0 con 0x7f700c100600 2023-06-23T18:22:14.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f700b7fe700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c100600 0x7f700c0fff70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d980 tx=0x7f6ffc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f7008ff9700 1 -- 172.21.15.2:0/3776243651 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc004220 con 0x7f700c100600 2023-06-23T18:22:14.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f7008ff9700 1 -- 172.21.15.2:0/3776243651 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ffc007640 con 0x7f700c100600 2023-06-23T18:22:14.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f7011a8b700 1 -- 172.21.15.2:0/3776243651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c19fac0 con 0x7f700c100600 2023-06-23T18:22:14.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f7008ff9700 1 -- 172.21.15.2:0/3776243651 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc005660 con 0x7f700c100600 2023-06-23T18:22:14.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.717+0000 7f7011a8b700 1 -- 172.21.15.2:0/3776243651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1a0020 con 0x7f700c100600 2023-06-23T18:22:14.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.719+0000 7f7008ff9700 1 -- 172.21.15.2:0/3776243651 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6ffc004750 con 0x7f700c100600 2023-06-23T18:22:14.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.720+0000 7f7008ff9700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6ff40a11d0 0x7f6ff40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.720+0000 7f7008ff9700 1 -- 172.21.15.2:0/3776243651 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6ffc0cc600 con 0x7f700c100600 2023-06-23T18:22:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.720+0000 7f700affd700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6ff40a11d0 0x7f6ff40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.720+0000 7f700affd700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6ff40a11d0 0x7f6ff40a3680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7000006010 tx=0x7f7000005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.722+0000 7f701dc10b80 1 Processor -- start 2023-06-23T18:22:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.722+0000 7f701dc10b80 1 -- start start 2023-06-23T18:22:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.722+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54f1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.722+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 0x5579d54f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.722+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f2000 0x5579d54f2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.722+0000 7f701dc10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d53f5cb0 con 0x5579d54f2000 2023-06-23T18:22:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.722+0000 7f701dc10b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d5448a40 con 0x5579d54f15d0 2023-06-23T18:22:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.722+0000 7f701dc10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d5441ed0 con 0x5579d54f0c80 2023-06-23T18:22:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f2000 0x5579d54f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f2000 0x5579d54f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54058/0 (socket says 172.21.15.2:54058) 2023-06-23T18:22:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 -- 172.21.15.2:0/2131857859 learned_addr learned my addr 172.21.15.2:0/2131857859 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700affd700 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 0x5579d54f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700b7fe700 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54f1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 -- 172.21.15.2:0/2131857859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 msgr2=0x5579d54f1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54f1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 -- 172.21.15.2:0/2131857859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 msgr2=0x5579d54f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 0x5579d54f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 -- 172.21.15.2:0/2131857859 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579d52c3630 con 0x5579d54f2000 2023-06-23T18:22:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f700bfff700 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f2000 0x5579d54f2470 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7004011600 tx=0x7f7004009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b9a51baf28d333 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f6ff1ffb700 1 -- 172.21.15.2:0/2131857859 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400e040 con 0x5579d54f2000 2023-06-23T18:22:14.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f6ff1ffb700 1 -- 172.21.15.2:0/2131857859 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f700400f040 con 0x5579d54f2000 2023-06-23T18:22:14.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.723+0000 7f6ff1ffb700 1 -- 172.21.15.2:0/2131857859 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70040046f0 con 0x5579d54f2000 2023-06-23T18:22:14.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 -- 172.21.15.2:0/2131857859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f2000 msgr2=0x5579d54f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f2000 0x5579d54f2470 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7004011600 tx=0x7f7004009920 comp rx=0 tx=0).stop 2023-06-23T18:22:14.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 -- 172.21.15.2:0/2131857859 shutdown_connections 2023-06-23T18:22:14.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f2000 0x5579d54f2470 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54f1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 --2- 172.21.15.2:0/2131857859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 0x5579d54f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 -- 172.21.15.2:0/2131857859 >> 172.21.15.2:0/2131857859 conn(0x5579d54e63f0 msgr2=0x5579d54eb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 -- 172.21.15.2:0/2131857859 shutdown_connections 2023-06-23T18:22:14.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 -- 172.21.15.2:0/2131857859 wait complete. 2023-06-23T18:22:14.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.724+0000 7f701dc10b80 1 -- start start 2023-06-23T18:22:14.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f701dc10b80 1 Processor -- start 2023-06-23T18:22:14.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f0c80 0x5579d54fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 0x5579d54faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d5501a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f701dc10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d53e1ac0 con 0x5579d54f0c80 2023-06-23T18:22:14.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f701dc10b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d53e17c0 con 0x5579d54f15d0 2023-06-23T18:22:14.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f701dc10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d52c5920 con 0x5579d54f2000 2023-06-23T18:22:14.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f701dc10b80 10 client.0 _ll_register_callbacks cb 0x5579d53e7260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:14.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f0c80 0x5579d54fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f0c80 0x5579d54fa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54060/0 (socket says 172.21.15.2:54060) 2023-06-23T18:22:14.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700b7fe700 1 -- 172.21.15.2:0/1959574689 learned_addr learned my addr 172.21.15.2:0/1959574689 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700affd700 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 0x5579d54faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700bfff700 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d5501a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700b7fe700 1 -- 172.21.15.2:0/1959574689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 msgr2=0x5579d5501a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700b7fe700 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d5501a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700b7fe700 1 -- 172.21.15.2:0/1959574689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 msgr2=0x5579d54faa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700b7fe700 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 0x5579d54faa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.725+0000 7f700b7fe700 1 -- 172.21.15.2:0/1959574689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70040112b0 con 0x5579d54f0c80 2023-06-23T18:22:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f700b7fe700 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f0c80 0x5579d54fa500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc007e80 tx=0x7f6ffc022350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2baf346b49e5c4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f6feaffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f6feaffd700 1 -- 172.21.15.2:0/1959574689 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc099100 con 0x5579d54f0c80 2023-06-23T18:22:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f6feaffd700 1 -- 172.21.15.2:0/1959574689 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6ffc09da40 con 0x5579d54f0c80 2023-06-23T18:22:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 1 -- 172.21.15.2:0/1959574689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5579d54e3bb0 con 0x5579d54f0c80 2023-06-23T18:22:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f6feaffd700 1 -- 172.21.15.2:0/1959574689 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ffc0b2220 con 0x5579d54f0c80 2023-06-23T18:22:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 10 client.15396 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 1 -- 172.21.15.2:0/1959574689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5579d526fde0 con 0x5579d54f0c80 2023-06-23T18:22:14.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f6feaffd700 1 -- 172.21.15.2:0/1959574689 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5579d54e3bb0 con 0x5579d54f0c80 2023-06-23T18:22:14.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f6feaffd700 1 -- 172.21.15.2:0/1959574689 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ffc0cc0b0 con 0x5579d54f0c80 2023-06-23T18:22:14.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 10 client.15396 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:14.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 10 client.15396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:14.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 20 client.15396 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:14.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5579d5506260 0x5579d5508670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 4 client.15396 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:14.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.726+0000 7f701dc10b80 1 -- 172.21.15.2:0/1959574689 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579d543aeb0 con 0x5579d5506260 2023-06-23T18:22:14.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.727+0000 7f700affd700 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5579d5506260 0x5579d5508670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.727+0000 7f6feaffd700 10 client.15396 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:14.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.727+0000 7f700affd700 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5579d5506260 0x5579d5508670 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=406cadf33c6ee9ec server_cookie=42f3f0532e7f3c44 in_seq=0 out_seq=0 2023-06-23T18:22:14.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f6feaffd700 1 -- 172.21.15.2:0/1959574689 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5579d543aeb0 con 0x5579d5506260 2023-06-23T18:22:14.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f6feaffd700 10 client.15396 handle_command_reply: tid=0 2023-06-23T18:22:14.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 client.15396 shutdown 2023-06-23T18:22:14.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 -- 172.21.15.2:0/1959574689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f0c80 msgr2=0x5579d54fa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f0c80 0x5579d54fa500 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc007e80 tx=0x7f6ffc022350 comp rx=0 tx=0).stop 2023-06-23T18:22:14.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 -- 172.21.15.2:0/1959574689 shutdown_connections 2023-06-23T18:22:14.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5579d5506260 0x5579d5508670 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d5501a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f0c80 0x5579d54fa500 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1959574689 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f15d0 0x5579d54faa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 -- 172.21.15.2:0/1959574689 >> 172.21.15.2:0/1959574689 conn(0x5579d54e63f0 msgr2=0x5579d54eac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f6feaffd700 0 client.15396 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:14.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 -- 172.21.15.2:0/1959574689 shutdown_connections 2023-06-23T18:22:14.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 1 -- 172.21.15.2:0/1959574689 wait complete. 2023-06-23T18:22:14.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.728+0000 7f701dc10b80 20 client.15396 trim_cache size 0 max 16384 2023-06-23T18:22:14.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f701dc10b80 1 Processor -- start 2023-06-23T18:22:14.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f701dc10b80 1 -- start start 2023-06-23T18:22:14.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f0c80 0x5579d550cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d5501a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d54f8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f701dc10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d54fa0d0 con 0x5579d54f15d0 2023-06-23T18:22:14.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f701dc10b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d54f59b0 con 0x5579d54f0c80 2023-06-23T18:22:14.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f701dc10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d5505ff0 con 0x5579d54f2000 2023-06-23T18:22:14.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f700affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d5501a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f700affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d5501a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54068/0 (socket says 172.21.15.2:54068) 2023-06-23T18:22:14.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f700affd700 1 -- 172.21.15.2:0/1580261051 learned_addr learned my addr 172.21.15.2:0/1580261051 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.745+0000 7f700bfff700 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d54f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f700b7fe700 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f0c80 0x5579d550cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f700bfff700 1 -- 172.21.15.2:0/1580261051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f0c80 msgr2=0x5579d550cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f700bfff700 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f0c80 0x5579d550cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f700bfff700 1 -- 172.21.15.2:0/1580261051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 msgr2=0x5579d5501a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f700bfff700 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d5501a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f700bfff700 1 -- 172.21.15.2:0/1580261051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579d5258c60 con 0x5579d54f2000 2023-06-23T18:22:14.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f700affd700 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d5501a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:14.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f700bfff700 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d54f8310 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f70040115d0 tx=0x7f7004006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92eaf151b168d84d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f6ff27fc700 1 -- 172.21.15.2:0/1580261051 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400e040 con 0x5579d54f2000 2023-06-23T18:22:14.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f6ff27fc700 1 -- 172.21.15.2:0/1580261051 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7004003e90 con 0x5579d54f2000 2023-06-23T18:22:14.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.746+0000 7f6ff27fc700 1 -- 172.21.15.2:0/1580261051 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70040059f0 con 0x5579d54f2000 2023-06-23T18:22:14.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 -- 172.21.15.2:0/1580261051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 msgr2=0x5579d54f8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d54f8310 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f70040115d0 tx=0x7f7004006000 comp rx=0 tx=0).stop 2023-06-23T18:22:14.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 -- 172.21.15.2:0/1580261051 shutdown_connections 2023-06-23T18:22:14.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f2000 0x5579d54f8310 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f0c80 0x5579d550cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 --2- 172.21.15.2:0/1580261051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d5501a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 -- 172.21.15.2:0/1580261051 >> 172.21.15.2:0/1580261051 conn(0x5579d54e63f0 msgr2=0x5579d54e78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 -- 172.21.15.2:0/1580261051 shutdown_connections 2023-06-23T18:22:14.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 -- 172.21.15.2:0/1580261051 wait complete. 2023-06-23T18:22:14.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.747+0000 7f701dc10b80 1 -- start start 2023-06-23T18:22:14.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f701dc10b80 1 Processor -- start 2023-06-23T18:22:14.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d54fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f701dc10b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f2000 0x5579d5500c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f701dc10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d55011a0 con 0x5579d54f15d0 2023-06-23T18:22:14.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f701dc10b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d5501310 con 0x5579d54f2000 2023-06-23T18:22:14.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f701dc10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579d55014d0 con 0x5579d54f0c80 2023-06-23T18:22:14.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f701dc10b80 10 client.0 _ll_register_callbacks cb 0x5579d53e7260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:14.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d54fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:54412/0 (socket says 172.21.15.2:54412) 2023-06-23T18:22:14.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 -- 172.21.15.2:0/31444153 learned_addr learned my addr 172.21.15.2:0/31444153 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:14.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700bfff700 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f2000 0x5579d5500c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 -- 172.21.15.2:0/31444153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f2000 msgr2=0x5579d5500c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f2000 0x5579d5500c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 -- 172.21.15.2:0/31444153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 msgr2=0x5579d54fdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:14.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d54fdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 -- 172.21.15.2:0/31444153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c108890 con 0x5579d54f0c80 2023-06-23T18:22:14.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f700b7fe700 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54fd700 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0cc430 tx=0x7f6ffc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3c3c879c638657 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:14.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.748+0000 7f6feaffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:22:14.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f6feaffd700 1 -- 172.21.15.2:0/31444153 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc014070 con 0x5579d54f0c80 2023-06-23T18:22:14.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f6feaffd700 1 -- 172.21.15.2:0/31444153 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6ffc09da40 con 0x5579d54f0c80 2023-06-23T18:22:14.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 1 -- 172.21.15.2:0/31444153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f700400f040 con 0x5579d54f0c80 2023-06-23T18:22:14.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f6feaffd700 1 -- 172.21.15.2:0/31444153 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ffc0b2220 con 0x5579d54f0c80 2023-06-23T18:22:14.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 10 client.25180 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:14.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 1 -- 172.21.15.2:0/31444153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5579d52c3630 con 0x5579d54f0c80 2023-06-23T18:22:14.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f6feaffd700 1 -- 172.21.15.2:0/31444153 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f700400f040 con 0x5579d54f0c80 2023-06-23T18:22:14.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f6feaffd700 1 -- 172.21.15.2:0/31444153 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ffc098f00 con 0x5579d54f0c80 2023-06-23T18:22:14.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 10 client.25180 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:14.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 10 client.25180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:14.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 20 client.25180 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:14.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5579d555a9e0 0x5579d555cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:14.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 4 client.25180 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:22:14.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f701dc10b80 1 -- 172.21.15.2:0/31444153 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5579d53fd050 con 0x5579d555a9e0 2023-06-23T18:22:14.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.749+0000 7f700affd700 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5579d555a9e0 0x5579d555cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f6feaffd700 10 client.25180 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f700affd700 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5579d555a9e0 0x5579d555cdf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69b3ed18a057c740 server_cookie=599db77d9c7de70 in_seq=0 out_seq=0 2023-06-23T18:22:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f6feaffd700 1 -- 172.21.15.2:0/31444153 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5579d53fd050 con 0x5579d555a9e0 2023-06-23T18:22:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f6feaffd700 10 client.25180 handle_command_reply: tid=0 2023-06-23T18:22:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 client.25180 shutdown 2023-06-23T18:22:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 -- 172.21.15.2:0/31444153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 msgr2=0x5579d54fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54fd700 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0cc430 tx=0x7f6ffc000bc0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 -- 172.21.15.2:0/31444153 shutdown_connections 2023-06-23T18:22:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5579d555a9e0 0x5579d555cdf0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5579d54f2000 0x5579d5500c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5579d54f0c80 0x5579d54fd700 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 --2- 172.21.15.2:0/31444153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5579d54f15d0 0x5579d54fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f6feaffd700 0 client.25180 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 -- 172.21.15.2:0/31444153 >> 172.21.15.2:0/31444153 conn(0x5579d54e63f0 msgr2=0x5579d550cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 -- 172.21.15.2:0/31444153 shutdown_connections 2023-06-23T18:22:14.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 1 -- 172.21.15.2:0/31444153 wait complete. 2023-06-23T18:22:14.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.750+0000 7f701dc10b80 20 client.25180 trim_cache size 0 max 16384 2023-06-23T18:22:14.765 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:22:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:22:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "31aedc34-1172-43af-86d1-ff97b0e8e410": { 2023-06-23T18:22:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:22:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "31aedc34-1172-43af-86d1-ff97b0e8e410", 2023-06-23T18:22:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:22:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:22:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:22:14.766 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:14.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 -- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6ff40a11d0 msgr2=0x7f6ff40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6ff40a11d0 0x7f6ff40a3680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7000006010 tx=0x7f7000005960 comp rx=0 tx=0).stop 2023-06-23T18:22:14.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 -- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c100600 msgr2=0x7f700c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c100600 0x7f700c0fff70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc00d980 tx=0x7f6ffc00dc90 comp rx=0 tx=0).stop 2023-06-23T18:22:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 -- 172.21.15.2:0/3776243651 shutdown_connections 2023-06-23T18:22:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6ff40a11d0 0x7f6ff40a3680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f700c105840 0x7f700c0feaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f700c100600 0x7f700c0fff70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 --2- 172.21.15.2:0/3776243651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f700c102f20 0x7f700c0fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:14.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 -- 172.21.15.2:0/3776243651 >> 172.21.15.2:0/3776243651 conn(0x7f700c0fa0e0 msgr2=0x7f700c101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:14.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 -- 172.21.15.2:0/3776243651 shutdown_connections 2023-06-23T18:22:14.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:14.751+0000 7f7011a8b700 1 -- 172.21.15.2:0/3776243651 wait complete. 2023-06-23T18:22:14.770 INFO:tasks.cephfs.filesystem:scrub status for tag:31aedc34-1172-43af-86d1-ff97b0e8e410 - {'path': '/', 'tag': '31aedc34-1172-43af-86d1-ff97b0e8e410', 'options': 'recursive,force'} 2023-06-23T18:22:15.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:15 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2118904244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:15.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:15 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:22:15.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:15 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:15.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:15 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:22:15.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:15 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:15.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:15 smithi017 ceph-mon[97472]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-06-23T18:22:15.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:15 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2118904244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:15.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:15 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:22:15.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:15 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:15.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:15 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:22:15.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:15 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:15.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:15 smithi002 ceph-mon[95947]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-06-23T18:22:15.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:15 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2118904244' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:22:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:15 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:22:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:15 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:15 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:22:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:15 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:15 smithi082 ceph-mon[97889]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-06-23T18:22:17.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-06-23T18:22:17.081 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:16 smithi002 ceph-mon[95947]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-06-23T18:22:17.083 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-06-23T18:22:17.086 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-06-23T18:22:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:16 smithi082 ceph-mon[97889]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-06-23T18:22:17.169 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-06-23T18:22:17.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:16 smithi017 ceph-mon[97472]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2023-06-23T18:22:17.255 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/: 2023-06-23T18:22:17.255 INFO:teuthology.orchestra.run.smithi002.stdout:total 2 2023-06-23T18:22:17.256 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:17.256 INFO:teuthology.orchestra.run.smithi002.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 .. 2023-06-23T18:22:17.256 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 data 2023-06-23T18:22:17.256 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:17.256 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data: 2023-06-23T18:22:17.256 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:17.256 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:17.257 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 .. 2023-06-23T18:22:17.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-06-23T18:22:17.305 INFO:teuthology.orchestra.run.smithi002.stdout:[Service] 2023-06-23T18:22:17.305 INFO:teuthology.orchestra.run.smithi002.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-06-23T18:22:17.305 INFO:teuthology.orchestra.run.smithi002.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-06-23T18:22:17.309 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-06-23T18:22:17.397 INFO:teuthology.orchestra.run.smithi002.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-06-23T18:22:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:18 smithi082 ceph-mon[97889]: pgmap v305: 97 pgs: 97 active+clean; 754 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-06-23T18:22:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:18 smithi017 ceph-mon[97472]: pgmap v305: 97 pgs: 97 active+clean; 754 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-06-23T18:22:19.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:18 smithi002 ceph-mon[95947]: pgmap v305: 97 pgs: 97 active+clean; 754 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 11 KiB/s wr, 9 op/s 2023-06-23T18:22:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:20 smithi082 ceph-mon[97889]: pgmap v306: 97 pgs: 97 active+clean; 1021 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-06-23T18:22:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:20 smithi017 ceph-mon[97472]: pgmap v306: 97 pgs: 97 active+clean; 1021 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-06-23T18:22:21.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:20 smithi002 ceph-mon[95947]: pgmap v306: 97 pgs: 97 active+clean; 1021 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 36 KiB/s wr, 10 op/s 2023-06-23T18:22:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:22 smithi082 ceph-mon[97889]: pgmap v307: 97 pgs: 97 active+clean; 25 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.4 MiB/s wr, 46 op/s 2023-06-23T18:22:23.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:22 smithi017 ceph-mon[97472]: pgmap v307: 97 pgs: 97 active+clean; 25 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.4 MiB/s wr, 46 op/s 2023-06-23T18:22:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:22 smithi002 ceph-mon[95947]: pgmap v307: 97 pgs: 97 active+clean; 25 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 2.4 MiB/s wr, 46 op/s 2023-06-23T18:22:25.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:25 smithi082 ceph-mon[97889]: pgmap v308: 97 pgs: 97 active+clean; 27 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.8 MiB/s wr, 63 op/s 2023-06-23T18:22:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:25 smithi017 ceph-mon[97472]: pgmap v308: 97 pgs: 97 active+clean; 27 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.8 MiB/s wr, 63 op/s 2023-06-23T18:22:25.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:25 smithi002 ceph-mon[95947]: pgmap v308: 97 pgs: 97 active+clean; 27 MiB data, 340 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 2.8 MiB/s wr, 63 op/s 2023-06-23T18:22:26.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:26 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:22:26.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:26 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:22:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:26 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:22:27.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:27 smithi082 ceph-mon[97889]: pgmap v309: 97 pgs: 97 active+clean; 30 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 3.2 MiB/s wr, 78 op/s 2023-06-23T18:22:27.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:27 smithi017 ceph-mon[97472]: pgmap v309: 97 pgs: 97 active+clean; 30 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 3.2 MiB/s wr, 78 op/s 2023-06-23T18:22:27.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:27 smithi002 ceph-mon[95947]: pgmap v309: 97 pgs: 97 active+clean; 30 MiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 3.2 MiB/s wr, 78 op/s 2023-06-23T18:22:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:28 smithi082 ceph-mon[97889]: pgmap v310: 97 pgs: 97 active+clean; 33 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 3.8 MiB/s wr, 89 op/s 2023-06-23T18:22:29.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:28 smithi017 ceph-mon[97472]: pgmap v310: 97 pgs: 97 active+clean; 33 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 3.8 MiB/s wr, 89 op/s 2023-06-23T18:22:29.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:28 smithi002 ceph-mon[95947]: pgmap v310: 97 pgs: 97 active+clean; 33 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 3.8 MiB/s wr, 89 op/s 2023-06-23T18:22:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:30 smithi082 ceph-mon[97889]: pgmap v311: 97 pgs: 97 active+clean; 35 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 3.9 MiB/s wr, 92 op/s 2023-06-23T18:22:31.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:30 smithi017 ceph-mon[97472]: pgmap v311: 97 pgs: 97 active+clean; 35 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 3.9 MiB/s wr, 92 op/s 2023-06-23T18:22:31.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:30 smithi002 ceph-mon[95947]: pgmap v311: 97 pgs: 97 active+clean; 35 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 3.9 MiB/s wr, 92 op/s 2023-06-23T18:22:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:32 smithi082 ceph-mon[97889]: pgmap v312: 97 pgs: 97 active+clean; 41 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 4.4 MiB/s wr, 103 op/s 2023-06-23T18:22:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:32 smithi017 ceph-mon[97472]: pgmap v312: 97 pgs: 97 active+clean; 41 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 4.4 MiB/s wr, 103 op/s 2023-06-23T18:22:33.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:32 smithi002 ceph-mon[95947]: pgmap v312: 97 pgs: 97 active+clean; 41 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 4.4 MiB/s wr, 103 op/s 2023-06-23T18:22:35.170 INFO:teuthology.orchestra.run.smithi002.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-06-23T18:22:35.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-06-23T18:22:35.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:35 smithi082 ceph-mon[97889]: pgmap v313: 97 pgs: 97 active+clean; 45 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 2.4 MiB/s wr, 112 op/s 2023-06-23T18:22:35.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:35 smithi017 ceph-mon[97472]: pgmap v313: 97 pgs: 97 active+clean; 45 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 2.4 MiB/s wr, 112 op/s 2023-06-23T18:22:35.509 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/: 2023-06-23T18:22:35.510 INFO:teuthology.orchestra.run.smithi002.stdout:total 2 2023-06-23T18:22:35.510 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 . 2023-06-23T18:22:35.510 INFO:teuthology.orchestra.run.smithi002.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:35.510 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 data 2023-06-23T18:22:35.510 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:35.510 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data: 2023-06-23T18:22:35.510 INFO:teuthology.orchestra.run.smithi002.stdout:total 40 2023-06-23T18:22:35.511 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 . 2023-06-23T18:22:35.511 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:35.511 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 Jun 23 18:22 base 2023-06-23T18:22:35.511 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 565760 Jun 23 18:22 global 2023-06-23T18:22:35.511 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 log 2023-06-23T18:22:35.511 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_commit_ts 2023-06-23T18:22:35.511 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_dynshmem 2023-06-23T18:22:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jun 23 18:22 pg_hba.conf 2023-06-23T18:22:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jun 23 18:22 pg_ident.conf 2023-06-23T18:22:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:22 pg_logical 2023-06-23T18:22:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 pg_multixact 2023-06-23T18:22:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 pg_notify 2023-06-23T18:22:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_replslot 2023-06-23T18:22:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_serial 2023-06-23T18:22:35.514 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_snapshots 2023-06-23T18:22:35.515 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_stat 2023-06-23T18:22:35.515 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_stat_tmp 2023-06-23T18:22:35.515 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 pg_subtrans 2023-06-23T18:22:35.516 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_tblspc 2023-06-23T18:22:35.516 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_twophase 2023-06-23T18:22:35.516 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 23 18:22 PG_VERSION 2023-06-23T18:22:35.516 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:22 pg_wal 2023-06-23T18:22:35.517 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 pg_xact 2023-06-23T18:22:35.517 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jun 23 18:22 postgresql.auto.conf 2023-06-23T18:22:35.517 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jun 23 18:22 postgresql.conf 2023-06-23T18:22:35.518 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:35.518 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base: 2023-06-23T18:22:35.518 INFO:teuthology.orchestra.run.smithi002.stdout:total 3 2023-06-23T18:22:35.518 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 Jun 23 18:22 . 2023-06-23T18:22:35.519 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:35.519 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 1 2023-06-23T18:22:35.519 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 12346 2023-06-23T18:22:35.519 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 12347 2023-06-23T18:22:35.519 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:35.520 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/1: 2023-06-23T18:22:35.520 INFO:teuthology.orchestra.run.smithi002.stdout:total 7202 2023-06-23T18:22:35.520 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 . 2023-06-23T18:22:35.521 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 Jun 23 18:22 .. 2023-06-23T18:22:35.521 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 112 2023-06-23T18:22:35.521 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 113 2023-06-23T18:22:35.521 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 23 18:22 12182 2023-06-23T18:22:35.522 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12182_fsm 2023-06-23T18:22:35.522 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12182_vm 2023-06-23T18:22:35.522 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12184 2023-06-23T18:22:35.522 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12186 2023-06-23T18:22:35.522 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187 2023-06-23T18:22:35.523 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12187_fsm 2023-06-23T18:22:35.523 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187_vm 2023-06-23T18:22:35.523 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12189 2023-06-23T18:22:35.523 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12191 2023-06-23T18:22:35.523 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192 2023-06-23T18:22:35.523 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12192_fsm 2023-06-23T18:22:35.523 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192_vm 2023-06-23T18:22:35.524 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12194 2023-06-23T18:22:35.524 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12196 2023-06-23T18:22:35.524 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197 2023-06-23T18:22:35.524 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12197_fsm 2023-06-23T18:22:35.524 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197_vm 2023-06-23T18:22:35.524 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12199 2023-06-23T18:22:35.524 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12201 2023-06-23T18:22:35.525 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202 2023-06-23T18:22:35.525 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12202_fsm 2023-06-23T18:22:35.525 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202_vm 2023-06-23T18:22:35.525 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12204 2023-06-23T18:22:35.525 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12206 2023-06-23T18:22:35.525 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207 2023-06-23T18:22:35.525 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12207_fsm 2023-06-23T18:22:35.526 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207_vm 2023-06-23T18:22:35.526 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12209 2023-06-23T18:22:35.526 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12211 2023-06-23T18:22:35.526 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12212 2023-06-23T18:22:35.526 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12214 2023-06-23T18:22:35.526 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12216 2023-06-23T18:22:35.527 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jun 23 18:22 1247 2023-06-23T18:22:35.527 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1247_fsm 2023-06-23T18:22:35.527 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1247_vm 2023-06-23T18:22:35.527 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jun 23 18:22 1249 2023-06-23T18:22:35.527 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1249_fsm 2023-06-23T18:22:35.528 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1249_vm 2023-06-23T18:22:35.528 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jun 23 18:22 1255 2023-06-23T18:22:35.528 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1255_fsm 2023-06-23T18:22:35.528 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1255_vm 2023-06-23T18:22:35.528 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 23 18:22 1259 2023-06-23T18:22:35.528 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1259_fsm 2023-06-23T18:22:35.529 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1259_vm 2023-06-23T18:22:35.529 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417 2023-06-23T18:22:35.529 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417_vm 2023-06-23T18:22:35.529 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418 2023-06-23T18:22:35.529 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418_vm 2023-06-23T18:22:35.529 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 174 2023-06-23T18:22:35.529 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 175 2023-06-23T18:22:35.529 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2187 2023-06-23T18:22:35.530 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224 2023-06-23T18:22:35.530 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224_vm 2023-06-23T18:22:35.530 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328 2023-06-23T18:22:35.530 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328_vm 2023-06-23T18:22:35.530 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336 2023-06-23T18:22:35.530 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336_vm 2023-06-23T18:22:35.530 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2337 2023-06-23T18:22:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2600 2023-06-23T18:22:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2600_fsm 2023-06-23T18:22:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2600_vm 2023-06-23T18:22:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601 2023-06-23T18:22:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2601_fsm 2023-06-23T18:22:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601_vm 2023-06-23T18:22:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2602 2023-06-23T18:22:35.531 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2602_fsm 2023-06-23T18:22:35.532 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2602_vm 2023-06-23T18:22:35.532 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2603 2023-06-23T18:22:35.532 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2603_fsm 2023-06-23T18:22:35.532 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2603_vm 2023-06-23T18:22:35.532 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604 2023-06-23T18:22:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604_vm 2023-06-23T18:22:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2605 2023-06-23T18:22:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2605_fsm 2023-06-23T18:22:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2605_vm 2023-06-23T18:22:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606 2023-06-23T18:22:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2606_fsm 2023-06-23T18:22:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606_vm 2023-06-23T18:22:35.533 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607 2023-06-23T18:22:35.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607_fsm 2023-06-23T18:22:35.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2607_vm 2023-06-23T18:22:35.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jun 23 18:22 2608 2023-06-23T18:22:35.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2608_fsm 2023-06-23T18:22:35.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2608_vm 2023-06-23T18:22:35.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jun 23 18:22 2609 2023-06-23T18:22:35.534 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2609_fsm 2023-06-23T18:22:35.535 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2609_vm 2023-06-23T18:22:35.535 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610 2023-06-23T18:22:35.535 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610_fsm 2023-06-23T18:22:35.535 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2610_vm 2023-06-23T18:22:35.535 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611 2023-06-23T18:22:35.535 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611_vm 2023-06-23T18:22:35.536 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612 2023-06-23T18:22:35.536 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2612_fsm 2023-06-23T18:22:35.536 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612_vm 2023-06-23T18:22:35.536 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613 2023-06-23T18:22:35.536 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613_vm 2023-06-23T18:22:35.536 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615 2023-06-23T18:22:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2615_fsm 2023-06-23T18:22:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615_vm 2023-06-23T18:22:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616 2023-06-23T18:22:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616_fsm 2023-06-23T18:22:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2616_vm 2023-06-23T18:22:35.537 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 23 18:22 2617 2023-06-23T18:22:35.538 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2617_fsm 2023-06-23T18:22:35.538 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2617_vm 2023-06-23T18:22:35.538 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 23 18:22 2618 2023-06-23T18:22:35.538 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2618_fsm 2023-06-23T18:22:35.539 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2618_vm 2023-06-23T18:22:35.539 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jun 23 18:22 2619 2023-06-23T18:22:35.539 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2619_fsm 2023-06-23T18:22:35.539 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2619_vm 2023-06-23T18:22:35.540 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620 2023-06-23T18:22:35.540 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620_vm 2023-06-23T18:22:35.540 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2650 2023-06-23T18:22:35.541 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2651 2023-06-23T18:22:35.541 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2652 2023-06-23T18:22:35.541 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2653 2023-06-23T18:22:35.541 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2654 2023-06-23T18:22:35.542 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2655 2023-06-23T18:22:35.542 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2656 2023-06-23T18:22:35.542 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2657 2023-06-23T18:22:35.543 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 23 18:22 2658 2023-06-23T18:22:35.543 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2659 2023-06-23T18:22:35.543 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2660 2023-06-23T18:22:35.544 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2661 2023-06-23T18:22:35.544 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2662 2023-06-23T18:22:35.544 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2663 2023-06-23T18:22:35.544 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2664 2023-06-23T18:22:35.544 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2665 2023-06-23T18:22:35.545 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2666 2023-06-23T18:22:35.545 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2667 2023-06-23T18:22:35.545 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2668 2023-06-23T18:22:35.546 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2669 2023-06-23T18:22:35.546 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2670 2023-06-23T18:22:35.546 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jun 23 18:22 2673 2023-06-23T18:22:35.546 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jun 23 18:22 2674 2023-06-23T18:22:35.547 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:22 2675 2023-06-23T18:22:35.547 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2678 2023-06-23T18:22:35.548 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2679 2023-06-23T18:22:35.548 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2680 2023-06-23T18:22:35.548 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2681 2023-06-23T18:22:35.548 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2682 2023-06-23T18:22:35.549 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2683 2023-06-23T18:22:35.549 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2684 2023-06-23T18:22:35.549 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2685 2023-06-23T18:22:35.549 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2686 2023-06-23T18:22:35.550 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2687 2023-06-23T18:22:35.550 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2688 2023-06-23T18:22:35.550 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2689 2023-06-23T18:22:35.550 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2690 2023-06-23T18:22:35.551 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jun 23 18:22 2691 2023-06-23T18:22:35.551 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2692 2023-06-23T18:22:35.551 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2693 2023-06-23T18:22:35.552 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2696 2023-06-23T18:22:35.552 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2699 2023-06-23T18:22:35.552 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2701 2023-06-23T18:22:35.552 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2702 2023-06-23T18:22:35.552 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2703 2023-06-23T18:22:35.552 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2704 2023-06-23T18:22:35.552 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2753 2023-06-23T18:22:35.553 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2753_fsm 2023-06-23T18:22:35.553 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2753_vm 2023-06-23T18:22:35.553 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2754 2023-06-23T18:22:35.553 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2755 2023-06-23T18:22:35.553 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2756 2023-06-23T18:22:35.553 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2757 2023-06-23T18:22:35.553 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830 2023-06-23T18:22:35.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830_vm 2023-06-23T18:22:35.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2831 2023-06-23T18:22:35.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832 2023-06-23T18:22:35.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832_vm 2023-06-23T18:22:35.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2833 2023-06-23T18:22:35.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834 2023-06-23T18:22:35.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834_vm 2023-06-23T18:22:35.554 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2835 2023-06-23T18:22:35.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836 2023-06-23T18:22:35.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836_vm 2023-06-23T18:22:35.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2837 2023-06-23T18:22:35.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jun 23 18:22 2838 2023-06-23T18:22:35.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2838_fsm 2023-06-23T18:22:35.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2838_vm 2023-06-23T18:22:35.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2839 2023-06-23T18:22:35.555 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840 2023-06-23T18:22:35.556 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840_fsm 2023-06-23T18:22:35.556 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2840_vm 2023-06-23T18:22:35.556 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2841 2023-06-23T18:22:35.556 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995 2023-06-23T18:22:35.556 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995_vm 2023-06-23T18:22:35.556 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2996 2023-06-23T18:22:35.556 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079 2023-06-23T18:22:35.557 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3079_fsm 2023-06-23T18:22:35.557 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079_vm 2023-06-23T18:22:35.557 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3080 2023-06-23T18:22:35.557 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3081 2023-06-23T18:22:35.557 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3085 2023-06-23T18:22:35.557 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118 2023-06-23T18:22:35.558 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118_vm 2023-06-23T18:22:35.558 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3119 2023-06-23T18:22:35.558 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3164 2023-06-23T18:22:35.558 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256 2023-06-23T18:22:35.558 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256_vm 2023-06-23T18:22:35.558 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3257 2023-06-23T18:22:35.558 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3258 2023-06-23T18:22:35.558 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350 2023-06-23T18:22:35.559 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350_vm 2023-06-23T18:22:35.559 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3351 2023-06-23T18:22:35.559 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3379 2023-06-23T18:22:35.559 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3380 2023-06-23T18:22:35.559 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381 2023-06-23T18:22:35.559 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381_vm 2023-06-23T18:22:35.559 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3394 2023-06-23T18:22:35.560 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3394_fsm 2023-06-23T18:22:35.560 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3394_vm 2023-06-23T18:22:35.560 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3395 2023-06-23T18:22:35.560 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439 2023-06-23T18:22:35.561 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439_vm 2023-06-23T18:22:35.561 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3440 2023-06-23T18:22:35.561 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 3455 2023-06-23T18:22:35.562 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3456 2023-06-23T18:22:35.562 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3456_fsm 2023-06-23T18:22:35.562 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3456_vm 2023-06-23T18:22:35.562 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466 2023-06-23T18:22:35.563 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466_vm 2023-06-23T18:22:35.563 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3467 2023-06-23T18:22:35.563 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3468 2023-06-23T18:22:35.563 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501 2023-06-23T18:22:35.564 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501_vm 2023-06-23T18:22:35.564 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3502 2023-06-23T18:22:35.564 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3503 2023-06-23T18:22:35.565 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3534 2023-06-23T18:22:35.565 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541 2023-06-23T18:22:35.565 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3541_fsm 2023-06-23T18:22:35.565 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541_vm 2023-06-23T18:22:35.565 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3542 2023-06-23T18:22:35.566 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3574 2023-06-23T18:22:35.566 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3575 2023-06-23T18:22:35.566 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576 2023-06-23T18:22:35.566 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576_vm 2023-06-23T18:22:35.566 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596 2023-06-23T18:22:35.566 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596_vm 2023-06-23T18:22:35.566 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3597 2023-06-23T18:22:35.566 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598 2023-06-23T18:22:35.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598_vm 2023-06-23T18:22:35.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3599 2023-06-23T18:22:35.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600 2023-06-23T18:22:35.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3600_fsm 2023-06-23T18:22:35.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600_vm 2023-06-23T18:22:35.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601 2023-06-23T18:22:35.567 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3601_fsm 2023-06-23T18:22:35.568 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601_vm 2023-06-23T18:22:35.568 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602 2023-06-23T18:22:35.568 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3602_fsm 2023-06-23T18:22:35.568 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:35 smithi002 ceph-mon[95947]: pgmap v313: 97 pgs: 97 active+clean; 45 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 2.4 MiB/s wr, 112 op/s 2023-06-23T18:22:35.783 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602_vm 2023-06-23T18:22:35.784 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3603 2023-06-23T18:22:35.784 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3603_fsm 2023-06-23T18:22:35.784 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3603_vm 2023-06-23T18:22:35.784 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3604 2023-06-23T18:22:35.784 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3605 2023-06-23T18:22:35.784 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3606 2023-06-23T18:22:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3607 2023-06-23T18:22:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3608 2023-06-23T18:22:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 3609 2023-06-23T18:22:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3712 2023-06-23T18:22:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764 2023-06-23T18:22:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3764_fsm 2023-06-23T18:22:35.785 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764_vm 2023-06-23T18:22:35.786 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3766 2023-06-23T18:22:35.786 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3767 2023-06-23T18:22:35.786 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3997 2023-06-23T18:22:35.786 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 5002 2023-06-23T18:22:35.786 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 548 2023-06-23T18:22:35.786 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 549 2023-06-23T18:22:35.786 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102 2023-06-23T18:22:35.787 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102_vm 2023-06-23T18:22:35.787 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104 2023-06-23T18:22:35.787 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104_vm 2023-06-23T18:22:35.787 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106 2023-06-23T18:22:35.787 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106_vm 2023-06-23T18:22:35.787 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6110 2023-06-23T18:22:35.787 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6111 2023-06-23T18:22:35.787 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6112 2023-06-23T18:22:35.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6113 2023-06-23T18:22:35.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6117 2023-06-23T18:22:35.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826 2023-06-23T18:22:35.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826_vm 2023-06-23T18:22:35.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 827 2023-06-23T18:22:35.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 828 2023-06-23T18:22:35.788 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 23 18:22 pg_filenode.map 2023-06-23T18:22:35.789 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 23 18:22 PG_VERSION 2023-06-23T18:22:35.789 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:35.789 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-06-23T18:22:35.789 INFO:teuthology.orchestra.run.smithi002.stdout:total 7202 2023-06-23T18:22:35.789 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 . 2023-06-23T18:22:35.789 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 Jun 23 18:22 .. 2023-06-23T18:22:35.789 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 112 2023-06-23T18:22:35.790 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 113 2023-06-23T18:22:35.790 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 23 18:22 12182 2023-06-23T18:22:35.790 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12182_fsm 2023-06-23T18:22:35.790 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12182_vm 2023-06-23T18:22:35.790 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12184 2023-06-23T18:22:35.791 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12186 2023-06-23T18:22:35.791 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187 2023-06-23T18:22:35.791 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12187_fsm 2023-06-23T18:22:35.791 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187_vm 2023-06-23T18:22:35.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12189 2023-06-23T18:22:35.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12191 2023-06-23T18:22:35.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192 2023-06-23T18:22:35.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12192_fsm 2023-06-23T18:22:35.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192_vm 2023-06-23T18:22:35.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12194 2023-06-23T18:22:35.792 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12196 2023-06-23T18:22:35.793 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197 2023-06-23T18:22:35.793 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12197_fsm 2023-06-23T18:22:35.793 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197_vm 2023-06-23T18:22:35.793 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12199 2023-06-23T18:22:35.793 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12201 2023-06-23T18:22:35.793 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202 2023-06-23T18:22:35.793 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12202_fsm 2023-06-23T18:22:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202_vm 2023-06-23T18:22:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12204 2023-06-23T18:22:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12206 2023-06-23T18:22:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207 2023-06-23T18:22:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12207_fsm 2023-06-23T18:22:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207_vm 2023-06-23T18:22:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12209 2023-06-23T18:22:35.794 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12211 2023-06-23T18:22:35.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12212 2023-06-23T18:22:35.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12214 2023-06-23T18:22:35.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12216 2023-06-23T18:22:35.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jun 23 18:22 1247 2023-06-23T18:22:35.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1247_fsm 2023-06-23T18:22:35.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1247_vm 2023-06-23T18:22:35.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jun 23 18:22 1249 2023-06-23T18:22:35.795 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1249_fsm 2023-06-23T18:22:35.796 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1249_vm 2023-06-23T18:22:35.796 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jun 23 18:22 1255 2023-06-23T18:22:35.796 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1255_fsm 2023-06-23T18:22:35.796 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1255_vm 2023-06-23T18:22:35.796 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 23 18:22 1259 2023-06-23T18:22:35.796 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1259_fsm 2023-06-23T18:22:35.796 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1259_vm 2023-06-23T18:22:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417 2023-06-23T18:22:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417_vm 2023-06-23T18:22:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418 2023-06-23T18:22:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418_vm 2023-06-23T18:22:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 174 2023-06-23T18:22:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 175 2023-06-23T18:22:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2187 2023-06-23T18:22:35.797 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224 2023-06-23T18:22:35.798 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224_vm 2023-06-23T18:22:35.798 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328 2023-06-23T18:22:35.798 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328_vm 2023-06-23T18:22:35.798 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336 2023-06-23T18:22:35.798 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336_vm 2023-06-23T18:22:35.798 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2337 2023-06-23T18:22:35.798 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2600 2023-06-23T18:22:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2600_fsm 2023-06-23T18:22:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2600_vm 2023-06-23T18:22:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601 2023-06-23T18:22:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2601_fsm 2023-06-23T18:22:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601_vm 2023-06-23T18:22:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2602 2023-06-23T18:22:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2602_fsm 2023-06-23T18:22:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2602_vm 2023-06-23T18:22:35.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2603 2023-06-23T18:22:35.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2603_fsm 2023-06-23T18:22:35.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2603_vm 2023-06-23T18:22:35.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604 2023-06-23T18:22:35.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604_vm 2023-06-23T18:22:35.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2605 2023-06-23T18:22:35.800 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2605_fsm 2023-06-23T18:22:35.801 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2605_vm 2023-06-23T18:22:35.801 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606 2023-06-23T18:22:35.801 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2606_fsm 2023-06-23T18:22:35.801 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606_vm 2023-06-23T18:22:35.801 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607 2023-06-23T18:22:35.802 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607_fsm 2023-06-23T18:22:35.802 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2607_vm 2023-06-23T18:22:35.802 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jun 23 18:22 2608 2023-06-23T18:22:35.802 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2608_fsm 2023-06-23T18:22:35.802 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2608_vm 2023-06-23T18:22:35.802 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jun 23 18:22 2609 2023-06-23T18:22:35.802 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2609_fsm 2023-06-23T18:22:35.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2609_vm 2023-06-23T18:22:35.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610 2023-06-23T18:22:35.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610_fsm 2023-06-23T18:22:35.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2610_vm 2023-06-23T18:22:35.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611 2023-06-23T18:22:35.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611_vm 2023-06-23T18:22:35.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612 2023-06-23T18:22:35.803 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2612_fsm 2023-06-23T18:22:35.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612_vm 2023-06-23T18:22:35.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613 2023-06-23T18:22:35.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613_vm 2023-06-23T18:22:35.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615 2023-06-23T18:22:35.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2615_fsm 2023-06-23T18:22:35.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615_vm 2023-06-23T18:22:35.804 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616 2023-06-23T18:22:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616_fsm 2023-06-23T18:22:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2616_vm 2023-06-23T18:22:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 23 18:22 2617 2023-06-23T18:22:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2617_fsm 2023-06-23T18:22:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2617_vm 2023-06-23T18:22:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 23 18:22 2618 2023-06-23T18:22:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2618_fsm 2023-06-23T18:22:35.805 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2618_vm 2023-06-23T18:22:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jun 23 18:22 2619 2023-06-23T18:22:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2619_fsm 2023-06-23T18:22:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2619_vm 2023-06-23T18:22:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620 2023-06-23T18:22:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620_vm 2023-06-23T18:22:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2650 2023-06-23T18:22:35.806 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2651 2023-06-23T18:22:35.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2652 2023-06-23T18:22:35.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2653 2023-06-23T18:22:35.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2654 2023-06-23T18:22:35.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2655 2023-06-23T18:22:35.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2656 2023-06-23T18:22:35.807 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2657 2023-06-23T18:22:35.808 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 23 18:22 2658 2023-06-23T18:22:35.808 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2659 2023-06-23T18:22:35.808 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2660 2023-06-23T18:22:35.808 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2661 2023-06-23T18:22:35.808 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2662 2023-06-23T18:22:35.808 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2663 2023-06-23T18:22:35.808 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2664 2023-06-23T18:22:35.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2665 2023-06-23T18:22:35.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2666 2023-06-23T18:22:35.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2667 2023-06-23T18:22:35.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2668 2023-06-23T18:22:35.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2669 2023-06-23T18:22:35.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2670 2023-06-23T18:22:35.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jun 23 18:22 2673 2023-06-23T18:22:35.809 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jun 23 18:22 2674 2023-06-23T18:22:35.810 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:22 2675 2023-06-23T18:22:35.810 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2678 2023-06-23T18:22:35.810 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2679 2023-06-23T18:22:35.810 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2680 2023-06-23T18:22:35.810 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2681 2023-06-23T18:22:35.810 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2682 2023-06-23T18:22:35.810 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2683 2023-06-23T18:22:35.811 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2684 2023-06-23T18:22:35.811 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2685 2023-06-23T18:22:35.811 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2686 2023-06-23T18:22:35.811 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2687 2023-06-23T18:22:35.811 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2688 2023-06-23T18:22:35.811 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2689 2023-06-23T18:22:35.811 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2690 2023-06-23T18:22:35.811 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jun 23 18:22 2691 2023-06-23T18:22:35.812 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2692 2023-06-23T18:22:35.812 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2693 2023-06-23T18:22:35.812 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2696 2023-06-23T18:22:35.812 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2699 2023-06-23T18:22:35.813 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2701 2023-06-23T18:22:35.813 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2702 2023-06-23T18:22:35.813 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2703 2023-06-23T18:22:35.813 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2704 2023-06-23T18:22:35.813 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2753 2023-06-23T18:22:35.813 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2753_fsm 2023-06-23T18:22:35.813 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2753_vm 2023-06-23T18:22:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2754 2023-06-23T18:22:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2755 2023-06-23T18:22:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2756 2023-06-23T18:22:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2757 2023-06-23T18:22:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830 2023-06-23T18:22:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830_vm 2023-06-23T18:22:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2831 2023-06-23T18:22:35.814 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832 2023-06-23T18:22:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832_vm 2023-06-23T18:22:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2833 2023-06-23T18:22:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834 2023-06-23T18:22:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834_vm 2023-06-23T18:22:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2835 2023-06-23T18:22:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836 2023-06-23T18:22:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836_vm 2023-06-23T18:22:35.815 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2837 2023-06-23T18:22:35.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jun 23 18:22 2838 2023-06-23T18:22:35.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2838_fsm 2023-06-23T18:22:35.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2838_vm 2023-06-23T18:22:35.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2839 2023-06-23T18:22:35.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840 2023-06-23T18:22:35.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840_fsm 2023-06-23T18:22:35.816 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2840_vm 2023-06-23T18:22:35.817 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2841 2023-06-23T18:22:35.817 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995 2023-06-23T18:22:35.817 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995_vm 2023-06-23T18:22:35.817 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2996 2023-06-23T18:22:35.817 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079 2023-06-23T18:22:35.817 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3079_fsm 2023-06-23T18:22:35.817 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079_vm 2023-06-23T18:22:35.817 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3080 2023-06-23T18:22:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3081 2023-06-23T18:22:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3085 2023-06-23T18:22:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118 2023-06-23T18:22:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118_vm 2023-06-23T18:22:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3119 2023-06-23T18:22:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3164 2023-06-23T18:22:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256 2023-06-23T18:22:35.818 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256_vm 2023-06-23T18:22:35.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3257 2023-06-23T18:22:35.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3258 2023-06-23T18:22:35.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350 2023-06-23T18:22:35.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350_vm 2023-06-23T18:22:35.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3351 2023-06-23T18:22:35.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3379 2023-06-23T18:22:35.819 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3380 2023-06-23T18:22:35.820 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381 2023-06-23T18:22:35.820 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381_vm 2023-06-23T18:22:35.820 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3394 2023-06-23T18:22:35.820 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3394_fsm 2023-06-23T18:22:35.820 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3394_vm 2023-06-23T18:22:35.820 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3395 2023-06-23T18:22:35.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439 2023-06-23T18:22:35.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439_vm 2023-06-23T18:22:35.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3440 2023-06-23T18:22:35.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 3455 2023-06-23T18:22:35.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3456 2023-06-23T18:22:35.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3456_fsm 2023-06-23T18:22:35.821 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3456_vm 2023-06-23T18:22:35.822 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466 2023-06-23T18:22:35.822 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466_vm 2023-06-23T18:22:35.822 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3467 2023-06-23T18:22:35.822 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3468 2023-06-23T18:22:35.822 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501 2023-06-23T18:22:35.823 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501_vm 2023-06-23T18:22:35.823 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3502 2023-06-23T18:22:35.823 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3503 2023-06-23T18:22:35.823 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3534 2023-06-23T18:22:35.823 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541 2023-06-23T18:22:35.823 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3541_fsm 2023-06-23T18:22:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541_vm 2023-06-23T18:22:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3542 2023-06-23T18:22:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3574 2023-06-23T18:22:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3575 2023-06-23T18:22:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576 2023-06-23T18:22:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576_vm 2023-06-23T18:22:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596 2023-06-23T18:22:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596_vm 2023-06-23T18:22:35.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3597 2023-06-23T18:22:35.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598 2023-06-23T18:22:35.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598_vm 2023-06-23T18:22:35.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3599 2023-06-23T18:22:35.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600 2023-06-23T18:22:35.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3600_fsm 2023-06-23T18:22:35.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600_vm 2023-06-23T18:22:35.825 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601 2023-06-23T18:22:35.826 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3601_fsm 2023-06-23T18:22:35.826 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601_vm 2023-06-23T18:22:35.826 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602 2023-06-23T18:22:36.053 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3602_fsm 2023-06-23T18:22:36.054 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602_vm 2023-06-23T18:22:36.054 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3603 2023-06-23T18:22:36.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3603_fsm 2023-06-23T18:22:36.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3603_vm 2023-06-23T18:22:36.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3604 2023-06-23T18:22:36.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3605 2023-06-23T18:22:36.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3606 2023-06-23T18:22:36.056 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3607 2023-06-23T18:22:36.056 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3608 2023-06-23T18:22:36.056 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 3609 2023-06-23T18:22:36.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3712 2023-06-23T18:22:36.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764 2023-06-23T18:22:36.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3764_fsm 2023-06-23T18:22:36.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764_vm 2023-06-23T18:22:36.058 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3766 2023-06-23T18:22:36.058 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3767 2023-06-23T18:22:36.058 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3997 2023-06-23T18:22:36.058 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 5002 2023-06-23T18:22:36.059 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 548 2023-06-23T18:22:36.059 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 549 2023-06-23T18:22:36.059 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102 2023-06-23T18:22:36.060 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102_vm 2023-06-23T18:22:36.060 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104 2023-06-23T18:22:36.060 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104_vm 2023-06-23T18:22:36.061 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106 2023-06-23T18:22:36.061 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106_vm 2023-06-23T18:22:36.061 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6110 2023-06-23T18:22:36.062 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6111 2023-06-23T18:22:36.062 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6112 2023-06-23T18:22:36.062 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6113 2023-06-23T18:22:36.062 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6117 2023-06-23T18:22:36.062 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826 2023-06-23T18:22:36.063 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826_vm 2023-06-23T18:22:36.063 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 827 2023-06-23T18:22:36.063 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 828 2023-06-23T18:22:36.064 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 23 18:22 pg_filenode.map 2023-06-23T18:22:36.064 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 23 18:22 PG_VERSION 2023-06-23T18:22:36.064 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.064 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-06-23T18:22:36.064 INFO:teuthology.orchestra.run.smithi002.stdout:total 7202 2023-06-23T18:22:36.064 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 . 2023-06-23T18:22:36.064 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22119945 Jun 23 18:22 .. 2023-06-23T18:22:36.064 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 112 2023-06-23T18:22:36.065 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 113 2023-06-23T18:22:36.065 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 23 18:22 12182 2023-06-23T18:22:36.065 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12182_fsm 2023-06-23T18:22:36.065 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12182_vm 2023-06-23T18:22:36.065 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12184 2023-06-23T18:22:36.065 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12186 2023-06-23T18:22:36.065 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187 2023-06-23T18:22:36.065 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12187_fsm 2023-06-23T18:22:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187_vm 2023-06-23T18:22:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12189 2023-06-23T18:22:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12191 2023-06-23T18:22:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192 2023-06-23T18:22:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12192_fsm 2023-06-23T18:22:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192_vm 2023-06-23T18:22:36.066 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12194 2023-06-23T18:22:36.067 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12196 2023-06-23T18:22:36.067 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197 2023-06-23T18:22:36.067 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12197_fsm 2023-06-23T18:22:36.067 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197_vm 2023-06-23T18:22:36.067 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12199 2023-06-23T18:22:36.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12201 2023-06-23T18:22:36.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202 2023-06-23T18:22:36.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12202_fsm 2023-06-23T18:22:36.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202_vm 2023-06-23T18:22:36.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12204 2023-06-23T18:22:36.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12206 2023-06-23T18:22:36.068 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207 2023-06-23T18:22:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12207_fsm 2023-06-23T18:22:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207_vm 2023-06-23T18:22:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12209 2023-06-23T18:22:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12211 2023-06-23T18:22:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12212 2023-06-23T18:22:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12214 2023-06-23T18:22:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12216 2023-06-23T18:22:36.069 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jun 23 18:22 1247 2023-06-23T18:22:36.070 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1247_fsm 2023-06-23T18:22:36.070 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1247_vm 2023-06-23T18:22:36.070 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jun 23 18:22 1249 2023-06-23T18:22:36.070 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1249_fsm 2023-06-23T18:22:36.070 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1249_vm 2023-06-23T18:22:36.070 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jun 23 18:22 1255 2023-06-23T18:22:36.070 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1255_fsm 2023-06-23T18:22:36.070 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1255_vm 2023-06-23T18:22:36.071 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 23 18:22 1259 2023-06-23T18:22:36.071 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1259_fsm 2023-06-23T18:22:36.071 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1259_vm 2023-06-23T18:22:36.071 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417 2023-06-23T18:22:36.071 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417_vm 2023-06-23T18:22:36.071 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418 2023-06-23T18:22:36.071 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418_vm 2023-06-23T18:22:36.071 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 174 2023-06-23T18:22:36.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 175 2023-06-23T18:22:36.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2187 2023-06-23T18:22:36.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224 2023-06-23T18:22:36.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224_vm 2023-06-23T18:22:36.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328 2023-06-23T18:22:36.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328_vm 2023-06-23T18:22:36.072 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336 2023-06-23T18:22:36.073 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336_vm 2023-06-23T18:22:36.073 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2337 2023-06-23T18:22:36.073 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2600 2023-06-23T18:22:36.073 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2600_fsm 2023-06-23T18:22:36.073 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2600_vm 2023-06-23T18:22:36.073 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601 2023-06-23T18:22:36.073 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2601_fsm 2023-06-23T18:22:36.073 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601_vm 2023-06-23T18:22:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2602 2023-06-23T18:22:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2602_fsm 2023-06-23T18:22:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2602_vm 2023-06-23T18:22:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2603 2023-06-23T18:22:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2603_fsm 2023-06-23T18:22:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2603_vm 2023-06-23T18:22:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604 2023-06-23T18:22:36.074 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604_vm 2023-06-23T18:22:36.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2605 2023-06-23T18:22:36.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2605_fsm 2023-06-23T18:22:36.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2605_vm 2023-06-23T18:22:36.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606 2023-06-23T18:22:36.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2606_fsm 2023-06-23T18:22:36.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606_vm 2023-06-23T18:22:36.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607 2023-06-23T18:22:36.075 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607_fsm 2023-06-23T18:22:36.076 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2607_vm 2023-06-23T18:22:36.076 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jun 23 18:22 2608 2023-06-23T18:22:36.076 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2608_fsm 2023-06-23T18:22:36.076 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2608_vm 2023-06-23T18:22:36.076 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jun 23 18:22 2609 2023-06-23T18:22:36.076 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2609_fsm 2023-06-23T18:22:36.076 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2609_vm 2023-06-23T18:22:36.076 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610 2023-06-23T18:22:36.077 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610_fsm 2023-06-23T18:22:36.077 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2610_vm 2023-06-23T18:22:36.077 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611 2023-06-23T18:22:36.077 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611_vm 2023-06-23T18:22:36.078 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612 2023-06-23T18:22:36.078 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2612_fsm 2023-06-23T18:22:36.078 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612_vm 2023-06-23T18:22:36.078 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613 2023-06-23T18:22:36.078 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613_vm 2023-06-23T18:22:36.078 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615 2023-06-23T18:22:36.078 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2615_fsm 2023-06-23T18:22:36.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615_vm 2023-06-23T18:22:36.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616 2023-06-23T18:22:36.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616_fsm 2023-06-23T18:22:36.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2616_vm 2023-06-23T18:22:36.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 23 18:22 2617 2023-06-23T18:22:36.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2617_fsm 2023-06-23T18:22:36.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2617_vm 2023-06-23T18:22:36.079 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 23 18:22 2618 2023-06-23T18:22:36.080 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2618_fsm 2023-06-23T18:22:36.080 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2618_vm 2023-06-23T18:22:36.080 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jun 23 18:22 2619 2023-06-23T18:22:36.080 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2619_fsm 2023-06-23T18:22:36.080 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2619_vm 2023-06-23T18:22:36.080 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620 2023-06-23T18:22:36.080 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620_vm 2023-06-23T18:22:36.080 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2650 2023-06-23T18:22:36.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2651 2023-06-23T18:22:36.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2652 2023-06-23T18:22:36.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2653 2023-06-23T18:22:36.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2654 2023-06-23T18:22:36.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2655 2023-06-23T18:22:36.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2656 2023-06-23T18:22:36.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2657 2023-06-23T18:22:36.081 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 23 18:22 2658 2023-06-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2659 2023-06-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2660 2023-06-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2661 2023-06-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2662 2023-06-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2663 2023-06-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2664 2023-06-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2665 2023-06-23T18:22:36.082 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2666 2023-06-23T18:22:36.083 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2667 2023-06-23T18:22:36.083 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2668 2023-06-23T18:22:36.083 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2669 2023-06-23T18:22:36.083 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2670 2023-06-23T18:22:36.083 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jun 23 18:22 2673 2023-06-23T18:22:36.083 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jun 23 18:22 2674 2023-06-23T18:22:36.083 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:22 2675 2023-06-23T18:22:36.083 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2678 2023-06-23T18:22:36.084 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2679 2023-06-23T18:22:36.084 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2680 2023-06-23T18:22:36.084 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2681 2023-06-23T18:22:36.084 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2682 2023-06-23T18:22:36.084 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2683 2023-06-23T18:22:36.084 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2684 2023-06-23T18:22:36.084 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2685 2023-06-23T18:22:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2686 2023-06-23T18:22:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2687 2023-06-23T18:22:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2688 2023-06-23T18:22:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2689 2023-06-23T18:22:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2690 2023-06-23T18:22:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jun 23 18:22 2691 2023-06-23T18:22:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2692 2023-06-23T18:22:36.085 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2693 2023-06-23T18:22:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2696 2023-06-23T18:22:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2699 2023-06-23T18:22:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2701 2023-06-23T18:22:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2702 2023-06-23T18:22:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2703 2023-06-23T18:22:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2704 2023-06-23T18:22:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2753 2023-06-23T18:22:36.086 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2753_fsm 2023-06-23T18:22:36.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2753_vm 2023-06-23T18:22:36.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2754 2023-06-23T18:22:36.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2755 2023-06-23T18:22:36.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2756 2023-06-23T18:22:36.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2757 2023-06-23T18:22:36.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830 2023-06-23T18:22:36.087 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830_vm 2023-06-23T18:22:36.088 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2831 2023-06-23T18:22:36.088 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832 2023-06-23T18:22:36.088 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832_vm 2023-06-23T18:22:36.088 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2833 2023-06-23T18:22:36.088 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834 2023-06-23T18:22:36.089 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834_vm 2023-06-23T18:22:36.089 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2835 2023-06-23T18:22:36.089 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836 2023-06-23T18:22:36.089 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836_vm 2023-06-23T18:22:36.089 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2837 2023-06-23T18:22:36.089 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jun 23 18:22 2838 2023-06-23T18:22:36.089 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2838_fsm 2023-06-23T18:22:36.089 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2838_vm 2023-06-23T18:22:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2839 2023-06-23T18:22:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840 2023-06-23T18:22:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840_fsm 2023-06-23T18:22:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2840_vm 2023-06-23T18:22:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2841 2023-06-23T18:22:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995 2023-06-23T18:22:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995_vm 2023-06-23T18:22:36.090 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2996 2023-06-23T18:22:36.091 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079 2023-06-23T18:22:36.091 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3079_fsm 2023-06-23T18:22:36.091 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079_vm 2023-06-23T18:22:36.091 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3080 2023-06-23T18:22:36.091 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3081 2023-06-23T18:22:36.091 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3085 2023-06-23T18:22:36.091 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118 2023-06-23T18:22:36.091 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118_vm 2023-06-23T18:22:36.092 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3119 2023-06-23T18:22:36.092 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3164 2023-06-23T18:22:36.092 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256 2023-06-23T18:22:36.092 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256_vm 2023-06-23T18:22:36.092 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3257 2023-06-23T18:22:36.092 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3258 2023-06-23T18:22:36.092 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350 2023-06-23T18:22:36.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350_vm 2023-06-23T18:22:36.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3351 2023-06-23T18:22:36.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3379 2023-06-23T18:22:36.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3380 2023-06-23T18:22:36.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381 2023-06-23T18:22:36.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381_vm 2023-06-23T18:22:36.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3394 2023-06-23T18:22:36.093 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3394_fsm 2023-06-23T18:22:36.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3394_vm 2023-06-23T18:22:36.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3395 2023-06-23T18:22:36.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439 2023-06-23T18:22:36.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439_vm 2023-06-23T18:22:36.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3440 2023-06-23T18:22:36.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 3455 2023-06-23T18:22:36.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3456 2023-06-23T18:22:36.094 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3456_fsm 2023-06-23T18:22:36.095 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3456_vm 2023-06-23T18:22:36.095 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466 2023-06-23T18:22:36.095 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466_vm 2023-06-23T18:22:36.095 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3467 2023-06-23T18:22:36.095 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3468 2023-06-23T18:22:36.095 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501 2023-06-23T18:22:36.095 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501_vm 2023-06-23T18:22:36.095 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3502 2023-06-23T18:22:36.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3503 2023-06-23T18:22:36.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3534 2023-06-23T18:22:36.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541 2023-06-23T18:22:36.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3541_fsm 2023-06-23T18:22:36.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541_vm 2023-06-23T18:22:36.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3542 2023-06-23T18:22:36.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3574 2023-06-23T18:22:36.096 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3575 2023-06-23T18:22:36.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576 2023-06-23T18:22:36.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576_vm 2023-06-23T18:22:36.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596 2023-06-23T18:22:36.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596_vm 2023-06-23T18:22:36.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3597 2023-06-23T18:22:36.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598 2023-06-23T18:22:36.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598_vm 2023-06-23T18:22:36.097 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3599 2023-06-23T18:22:36.098 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600 2023-06-23T18:22:36.098 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3600_fsm 2023-06-23T18:22:36.098 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600_vm 2023-06-23T18:22:36.098 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601 2023-06-23T18:22:36.099 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3601_fsm 2023-06-23T18:22:36.099 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601_vm 2023-06-23T18:22:36.099 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602 2023-06-23T18:22:36.100 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3602_fsm 2023-06-23T18:22:36.100 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602_vm 2023-06-23T18:22:36.100 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3603 2023-06-23T18:22:36.100 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3603_fsm 2023-06-23T18:22:36.100 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3603_vm 2023-06-23T18:22:36.100 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3604 2023-06-23T18:22:36.100 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3605 2023-06-23T18:22:36.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3606 2023-06-23T18:22:36.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3607 2023-06-23T18:22:36.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3608 2023-06-23T18:22:36.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 3609 2023-06-23T18:22:36.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3712 2023-06-23T18:22:36.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764 2023-06-23T18:22:36.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3764_fsm 2023-06-23T18:22:36.101 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764_vm 2023-06-23T18:22:36.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3766 2023-06-23T18:22:36.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3767 2023-06-23T18:22:36.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3997 2023-06-23T18:22:36.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 5002 2023-06-23T18:22:36.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 548 2023-06-23T18:22:36.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 549 2023-06-23T18:22:36.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102 2023-06-23T18:22:36.102 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102_vm 2023-06-23T18:22:36.103 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104 2023-06-23T18:22:36.103 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104_vm 2023-06-23T18:22:36.103 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106 2023-06-23T18:22:36.103 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106_vm 2023-06-23T18:22:36.103 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6110 2023-06-23T18:22:36.104 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6111 2023-06-23T18:22:36.104 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6112 2023-06-23T18:22:36.104 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6113 2023-06-23T18:22:36.104 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6117 2023-06-23T18:22:36.104 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826 2023-06-23T18:22:36.104 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826_vm 2023-06-23T18:22:36.104 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 827 2023-06-23T18:22:36.105 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 828 2023-06-23T18:22:36.105 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 23 18:22 pg_filenode.map 2023-06-23T18:22:36.105 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 23 18:22 PG_VERSION 2023-06-23T18:22:36.105 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.105 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/global: 2023-06-23T18:22:36.105 INFO:teuthology.orchestra.run.smithi002.stdout:total 554 2023-06-23T18:22:36.105 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 565760 Jun 23 18:22 . 2023-06-23T18:22:36.106 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.106 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1136 2023-06-23T18:22:36.106 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1136_fsm 2023-06-23T18:22:36.106 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1136_vm 2023-06-23T18:22:36.106 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 1137 2023-06-23T18:22:36.106 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1213 2023-06-23T18:22:36.106 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1213_fsm 2023-06-23T18:22:36.106 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1213_vm 2023-06-23T18:22:36.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1214 2023-06-23T18:22:36.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1214_fsm 2023-06-23T18:22:36.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1214_vm 2023-06-23T18:22:36.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 1232 2023-06-23T18:22:36.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 1233 2023-06-23T18:22:36.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1260 2023-06-23T18:22:36.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1260_fsm 2023-06-23T18:22:36.107 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1260_vm 2023-06-23T18:22:36.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1261 2023-06-23T18:22:36.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1261_fsm 2023-06-23T18:22:36.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1261_vm 2023-06-23T18:22:36.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1262 2023-06-23T18:22:36.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1262_fsm 2023-06-23T18:22:36.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1262_vm 2023-06-23T18:22:36.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2396 2023-06-23T18:22:36.108 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2396_fsm 2023-06-23T18:22:36.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2396_vm 2023-06-23T18:22:36.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2397 2023-06-23T18:22:36.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2671 2023-06-23T18:22:36.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2672 2023-06-23T18:22:36.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2676 2023-06-23T18:22:36.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2677 2023-06-23T18:22:36.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2694 2023-06-23T18:22:36.109 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2695 2023-06-23T18:22:36.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2697 2023-06-23T18:22:36.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2698 2023-06-23T18:22:36.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2846 2023-06-23T18:22:36.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2846_vm 2023-06-23T18:22:36.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2847 2023-06-23T18:22:36.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2964 2023-06-23T18:22:36.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2964_vm 2023-06-23T18:22:36.110 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2965 2023-06-23T18:22:36.111 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2966 2023-06-23T18:22:36.111 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2966_vm 2023-06-23T18:22:36.111 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2967 2023-06-23T18:22:36.111 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3592 2023-06-23T18:22:36.140 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3592_vm 2023-06-23T18:22:36.141 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3593 2023-06-23T18:22:36.141 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 4060 2023-06-23T18:22:36.141 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 4060_vm 2023-06-23T18:22:36.141 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 4061 2023-06-23T18:22:36.141 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6000 2023-06-23T18:22:36.142 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6000_vm 2023-06-23T18:22:36.142 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6001 2023-06-23T18:22:36.142 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6002 2023-06-23T18:22:36.142 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6100 2023-06-23T18:22:36.142 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6100_vm 2023-06-23T18:22:36.142 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6114 2023-06-23T18:22:36.143 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6115 2023-06-23T18:22:36.143 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 pg_control 2023-06-23T18:22:36.143 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 23 18:22 pg_filenode.map 2023-06-23T18:22:36.143 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.143 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/log: 2023-06-23T18:22:36.143 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.143 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.144 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.144 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.144 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-06-23T18:22:36.144 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.144 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.144 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.144 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.145 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-06-23T18:22:36.145 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.145 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.145 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.145 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.145 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-06-23T18:22:36.145 INFO:teuthology.orchestra.run.smithi002.stdout:total 3 2023-06-23T18:22:36.145 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:22 . 2023-06-23T18:22:36.146 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.146 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 mappings 2023-06-23T18:22:36.146 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:22 replorigin_checkpoint 2023-06-23T18:22:36.147 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 snapshots 2023-06-23T18:22:36.147 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.147 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-06-23T18:22:36.147 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.147 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.147 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:22 .. 2023-06-23T18:22:36.148 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.148 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-06-23T18:22:36.148 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.148 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.148 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:22 .. 2023-06-23T18:22:36.148 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.148 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-06-23T18:22:36.149 INFO:teuthology.orchestra.run.smithi002.stdout:total 2 2023-06-23T18:22:36.149 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 . 2023-06-23T18:22:36.149 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.149 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 members 2023-06-23T18:22:36.149 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 offsets 2023-06-23T18:22:36.149 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.149 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-06-23T18:22:36.149 INFO:teuthology.orchestra.run.smithi002.stdout:total 9 2023-06-23T18:22:36.150 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 . 2023-06-23T18:22:36.150 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 .. 2023-06-23T18:22:36.150 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 0000 2023-06-23T18:22:36.150 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.150 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-06-23T18:22:36.150 INFO:teuthology.orchestra.run.smithi002.stdout:total 9 2023-06-23T18:22:36.150 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 . 2023-06-23T18:22:36.189 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 .. 2023-06-23T18:22:36.190 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 0000 2023-06-23T18:22:36.190 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.190 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-06-23T18:22:36.190 INFO:teuthology.orchestra.run.smithi002.stdout:total 9 2023-06-23T18:22:36.190 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 . 2023-06-23T18:22:36.190 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.191 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 0000 2023-06-23T18:22:36.191 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.191 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-06-23T18:22:36.191 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.191 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.192 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.192 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.192 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-06-23T18:22:36.192 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.192 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.193 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.193 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.193 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-06-23T18:22:36.193 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.193 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.193 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.193 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-06-23T18:22:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.194 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-06-23T18:22:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.194 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.195 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.195 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.195 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-06-23T18:22:36.195 INFO:teuthology.orchestra.run.smithi002.stdout:total 9 2023-06-23T18:22:36.195 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 . 2023-06-23T18:22:36.195 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.195 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 0000 2023-06-23T18:22:36.195 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.196 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-06-23T18:22:36.196 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.196 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.196 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.196 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.196 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-06-23T18:22:36.196 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.197 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.197 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.197 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.197 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-06-23T18:22:36.197 INFO:teuthology.orchestra.run.smithi002.stdout:total 16386 2023-06-23T18:22:36.197 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:22 . 2023-06-23T18:22:36.197 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.197 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:22 000000010000000000000001 2023-06-23T18:22:36.198 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 archive_status 2023-06-23T18:22:36.198 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.198 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-06-23T18:22:36.198 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:22:36.198 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:22:36.198 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:22 .. 2023-06-23T18:22:36.198 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:36.198 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-06-23T18:22:36.199 INFO:teuthology.orchestra.run.smithi002.stdout:total 9 2023-06-23T18:22:36.199 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 . 2023-06-23T18:22:36.199 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39532874 Jun 23 18:22 .. 2023-06-23T18:22:36.200 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 0000 2023-06-23T18:22:36.201 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-06-23T18:22:36.390 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-06-23T18:22:36.483 INFO:teuthology.orchestra.run.smithi002.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-06-23T18:22:36.531 INFO:teuthology.orchestra.run.smithi002.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-06-23T18:22:36.536 INFO:teuthology.orchestra.run.smithi002.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-06-23T18:22:36.542 INFO:teuthology.orchestra.run.smithi002.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-06-23T18:22:36.756 INFO:teuthology.orchestra.run.smithi002.stderr:creating tables... 2023-06-23T18:22:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:36 smithi082 ceph-mon[97889]: pgmap v314: 97 pgs: 97 active+clean; 45 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2023-06-23T18:22:37.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:36 smithi017 ceph-mon[97472]: pgmap v314: 97 pgs: 97 active+clean; 45 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2023-06-23T18:22:37.282 INFO:teuthology.orchestra.run.smithi002.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.52 s, remaining 261.36 s) 2023-06-23T18:22:37.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:36 smithi002 ceph-mon[95947]: pgmap v314: 97 pgs: 97 active+clean; 45 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2023-06-23T18:22:38.542 INFO:teuthology.orchestra.run.smithi002.stderr:200000 of 50000000 tuples (0%) done (elapsed 1.78 s, remaining 444.06 s) 2023-06-23T18:22:39.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:38 smithi082 ceph-mon[97889]: pgmap v315: 97 pgs: 97 active+clean; 54 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 2.5 MiB/s wr, 94 op/s 2023-06-23T18:22:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:38 smithi017 ceph-mon[97472]: pgmap v315: 97 pgs: 97 active+clean; 54 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 2.5 MiB/s wr, 94 op/s 2023-06-23T18:22:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:38 smithi002 ceph-mon[95947]: pgmap v315: 97 pgs: 97 active+clean; 54 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 2.5 MiB/s wr, 94 op/s 2023-06-23T18:22:39.803 INFO:teuthology.orchestra.run.smithi002.stderr:300000 of 50000000 tuples (0%) done (elapsed 3.04 s, remaining 504.35 s) 2023-06-23T18:22:40.643 INFO:teuthology.orchestra.run.smithi002.stderr:400000 of 50000000 tuples (0%) done (elapsed 3.88 s, remaining 481.60 s) 2023-06-23T18:22:41.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:40 smithi082 ceph-mon[97889]: pgmap v316: 97 pgs: 97 active+clean; 71 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 4.1 MiB/s wr, 101 op/s 2023-06-23T18:22:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:40 smithi017 ceph-mon[97472]: pgmap v316: 97 pgs: 97 active+clean; 71 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 4.1 MiB/s wr, 101 op/s 2023-06-23T18:22:41.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:40 smithi002 ceph-mon[95947]: pgmap v316: 97 pgs: 97 active+clean; 71 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 4.1 MiB/s wr, 101 op/s 2023-06-23T18:22:41.920 INFO:teuthology.orchestra.run.smithi002.stderr:500000 of 50000000 tuples (1%) done (elapsed 5.16 s, remaining 510.91 s) 2023-06-23T18:22:42.794 INFO:teuthology.orchestra.run.smithi002.stderr:600000 of 50000000 tuples (1%) done (elapsed 6.04 s, remaining 496.91 s) 2023-06-23T18:22:43.104 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:42 smithi082 ceph-mon[97889]: pgmap v317: 97 pgs: 97 active+clean; 135 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 15 MiB/s wr, 102 op/s 2023-06-23T18:22:43.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:42 smithi017 ceph-mon[97472]: pgmap v317: 97 pgs: 97 active+clean; 135 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 15 MiB/s wr, 102 op/s 2023-06-23T18:22:43.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:42 smithi002 ceph-mon[95947]: pgmap v317: 97 pgs: 97 active+clean; 135 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 15 MiB/s wr, 102 op/s 2023-06-23T18:22:43.996 INFO:teuthology.orchestra.run.smithi002.stderr:700000 of 50000000 tuples (1%) done (elapsed 7.24 s, remaining 509.65 s) 2023-06-23T18:22:44.771 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:22:44.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 -- 172.21.15.2:0/1764848357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 msgr2=0x7f06980fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 --2- 172.21.15.2:0/1764848357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 0x7f06980fee90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0688009990 tx=0x7f0688009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 -- 172.21.15.2:0/1764848357 shutdown_connections 2023-06-23T18:22:44.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 --2- 172.21.15.2:0/1764848357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0698100600 0x7f0698106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 --2- 172.21.15.2:0/1764848357 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 0x7f06980fee90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 --2- 172.21.15.2:0/1764848357 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06980ffc70 0x7f06981000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 -- 172.21.15.2:0/1764848357 >> 172.21.15.2:0/1764848357 conn(0x7f06980fa0e0 msgr2=0x7f06980fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:44.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 -- 172.21.15.2:0/1764848357 shutdown_connections 2023-06-23T18:22:44.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.910+0000 7f069d195700 1 -- 172.21.15.2:0/1764848357 wait complete. 2023-06-23T18:22:44.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.911+0000 7f069d195700 1 Processor -- start 2023-06-23T18:22:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.911+0000 7f069d195700 1 -- start start 2023-06-23T18:22:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.911+0000 7f069d195700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 0x7f0698109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.911+0000 7f069d195700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06980ffc70 0x7f06981079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.911+0000 7f069d195700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0698100600 0x7f0698107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.911+0000 7f069d195700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698108690 con 0x7f06980fea80 2023-06-23T18:22:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.911+0000 7f069d195700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698108800 con 0x7f06980ffc70 2023-06-23T18:22:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.911+0000 7f069d195700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069819f830 con 0x7f0698100600 2023-06-23T18:22:44.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f0696d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 0x7f0698109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06980ffc70 0x7f06981079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069759e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0698100600 0x7f0698107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f0696d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 0x7f0698109370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54112/0 (socket says 172.21.15.2:54112) 2023-06-23T18:22:44.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f0696d9d700 1 -- 172.21.15.2:0/1418959925 learned_addr learned my addr 172.21.15.2:0/1418959925 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:44.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069659c700 1 -- 172.21.15.2:0/1418959925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0698100600 msgr2=0x7f0698107ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069659c700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0698100600 0x7f0698107ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069659c700 1 -- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 msgr2=0x7f0698109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069659c700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 0x7f0698109370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069659c700 1 -- 172.21.15.2:0/1418959925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06880095f0 con 0x7f06980ffc70 2023-06-23T18:22:44.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069659c700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06980ffc70 0x7f06981079b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e9f0 tx=0x7f068c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:44.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f067ffff700 1 -- 172.21.15.2:0/1418959925 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00c9c0 con 0x7f06980ffc70 2023-06-23T18:22:44.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f069d195700 1 -- 172.21.15.2:0/1418959925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069819fb10 con 0x7f06980ffc70 2023-06-23T18:22:44.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f067ffff700 1 -- 172.21.15.2:0/1418959925 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f068c010430 con 0x7f06980ffc70 2023-06-23T18:22:44.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.912+0000 7f067ffff700 1 -- 172.21.15.2:0/1418959925 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c018680 con 0x7f06980ffc70 2023-06-23T18:22:44.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.913+0000 7f069d195700 1 -- 172.21.15.2:0/1418959925 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06981a0070 con 0x7f06980ffc70 2023-06-23T18:22:44.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.915+0000 7f067ffff700 1 -- 172.21.15.2:0/1418959925 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f068c0105a0 con 0x7f06980ffc70 2023-06-23T18:22:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f067ffff700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06800a1150 0x7f06800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f06a931ab80 1 Processor -- start 2023-06-23T18:22:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f06a931ab80 1 -- start start 2023-06-23T18:22:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f0696d9d700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06800a1150 0x7f06800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f067ffff700 1 -- 172.21.15.2:0/1418959925 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f068c014070 con 0x7f06980ffc70 2023-06-23T18:22:44.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd4fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 0x56076fd50470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f06a931ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fc53cb0 con 0x56076fd4ec80 2023-06-23T18:22:44.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f06a931ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fca6a40 con 0x56076fd50000 2023-06-23T18:22:44.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f06a931ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fc9fed0 con 0x56076fd4f5d0 2023-06-23T18:22:44.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f069659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd4fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f069659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd4fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:54460/0 (socket says 172.21.15.2:54460) 2023-06-23T18:22:44.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.916+0000 7f0696d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f069659c700 1 -- 172.21.15.2:0/3755345370 learned_addr learned my addr 172.21.15.2:0/3755345370 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:44.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f069759e700 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 0x56076fd50470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f0696d9d700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06800a1150 0x7f06800a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f068800b2e0 tx=0x7f06880042b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:44.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f0696d9d700 1 -- 172.21.15.2:0/3755345370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 msgr2=0x56076fd4fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:44.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f0696d9d700 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd4fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f0696d9d700 1 -- 172.21.15.2:0/3755345370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 msgr2=0x56076fd50470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:44.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f0696d9d700 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 0x56076fd50470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f0696d9d700 1 -- 172.21.15.2:0/3755345370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076fb21630 con 0x56076fd4ec80 2023-06-23T18:22:44.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f069659c700 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd4fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:44.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f0696d9d700 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd4f090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0688027000 tx=0x7f0688028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d75c500e81f1d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:44.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f067d7fa700 1 -- 172.21.15.2:0/3755345370 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688003ae0 con 0x56076fd4ec80 2023-06-23T18:22:44.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f067d7fa700 1 -- 172.21.15.2:0/3755345370 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f068801e040 con 0x56076fd4ec80 2023-06-23T18:22:44.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.917+0000 7f067d7fa700 1 -- 172.21.15.2:0/3755345370 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688004740 con 0x56076fd4ec80 2023-06-23T18:22:44.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 -- 172.21.15.2:0/3755345370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 msgr2=0x56076fd4f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd4f090 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0688027000 tx=0x7f0688028000 comp rx=0 tx=0).stop 2023-06-23T18:22:44.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 -- 172.21.15.2:0/3755345370 shutdown_connections 2023-06-23T18:22:44.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 0x56076fd50470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd4f090 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 --2- 172.21.15.2:0/3755345370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd4fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 -- 172.21.15.2:0/3755345370 >> 172.21.15.2:0/3755345370 conn(0x56076fd443f0 msgr2=0x56076fd49520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:44.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 -- 172.21.15.2:0/3755345370 shutdown_connections 2023-06-23T18:22:44.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 -- 172.21.15.2:0/3755345370 wait complete. 2023-06-23T18:22:44.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.918+0000 7f06a931ab80 1 -- start start 2023-06-23T18:22:44.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f06a931ab80 1 Processor -- start 2023-06-23T18:22:44.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd58320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4f5d0 0x56076fd58860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd50000 0x56076fd5f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f06a931ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fc3fac0 con 0x56076fd4ec80 2023-06-23T18:22:44.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f06a931ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fc3f7c0 con 0x56076fd4f5d0 2023-06-23T18:22:44.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f06a931ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fb23920 con 0x56076fd50000 2023-06-23T18:22:44.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f06a931ab80 10 client.0 _ll_register_callbacks cb 0x56076fc45260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:44.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f0696d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd58320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f0696d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd58320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54126/0 (socket says 172.21.15.2:54126) 2023-06-23T18:22:44.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f0696d9d700 1 -- 172.21.15.2:0/2020390681 learned_addr learned my addr 172.21.15.2:0/2020390681 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:44.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f069759e700 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd50000 0x56076fd5f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f069659c700 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4f5d0 0x56076fd58860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f0696d9d700 1 -- 172.21.15.2:0/2020390681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd50000 msgr2=0x56076fd5f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:44.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f0696d9d700 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd50000 0x56076fd5f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f0696d9d700 1 -- 172.21.15.2:0/2020390681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4f5d0 msgr2=0x56076fd58860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:44.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f0696d9d700 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4f5d0 0x56076fd58860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f0696d9d700 1 -- 172.21.15.2:0/2020390681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06981a0070 con 0x56076fd4ec80 2023-06-23T18:22:44.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.919+0000 7f069759e700 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd50000 0x56076fd5f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:44.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f0696d9d700 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd58320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0688003df0 tx=0x7f068800b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bdf8ad752b640d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:44.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f067aff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:44.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f067aff5700 1 -- 172.21.15.2:0/2020390681 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068800bae0 con 0x56076fd4ec80 2023-06-23T18:22:44.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f067aff5700 1 -- 172.21.15.2:0/2020390681 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0688004500 con 0x56076fd4ec80 2023-06-23T18:22:44.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 1 -- 172.21.15.2:0/2020390681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56076fd41bb0 con 0x56076fd4ec80 2023-06-23T18:22:44.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f067aff5700 1 -- 172.21.15.2:0/2020390681 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f068800bc40 con 0x56076fd4ec80 2023-06-23T18:22:44.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 10 client.15414 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:44.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 1 -- 172.21.15.2:0/2020390681 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f06980677a0 con 0x56076fd4ec80 2023-06-23T18:22:44.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f067aff5700 1 -- 172.21.15.2:0/2020390681 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56076fd41bb0 con 0x56076fd4ec80 2023-06-23T18:22:44.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f067aff5700 1 -- 172.21.15.2:0/2020390681 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0688021050 con 0x56076fd4ec80 2023-06-23T18:22:44.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 10 client.15414 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:44.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 10 client.15414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:44.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 20 client.15414 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:44.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56076fd63bd0 0x56076fd65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 4 client.15414 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:44.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.920+0000 7f06a931ab80 1 -- 172.21.15.2:0/2020390681 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56076fc98eb0 con 0x56076fd63bd0 2023-06-23T18:22:44.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.921+0000 7f069659c700 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56076fd63bd0 0x56076fd65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.921+0000 7f069659c700 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56076fd63bd0 0x56076fd65fe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=637269684b045e50 server_cookie=45e316d7e8562f13 in_seq=0 out_seq=0 2023-06-23T18:22:44.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.921+0000 7f067aff5700 10 client.15414 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:44.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f067aff5700 1 -- 172.21.15.2:0/2020390681 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56076fc98eb0 con 0x56076fd63bd0 2023-06-23T18:22:44.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f067aff5700 10 client.15414 handle_command_reply: tid=0 2023-06-23T18:22:44.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 client.15414 shutdown 2023-06-23T18:22:44.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 -- 172.21.15.2:0/2020390681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 msgr2=0x56076fd58320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd58320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0688003df0 tx=0x7f068800b6b0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 -- 172.21.15.2:0/2020390681 shutdown_connections 2023-06-23T18:22:44.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56076fd63bd0 0x56076fd65fe0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd50000 0x56076fd5f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd58320 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2020390681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4f5d0 0x56076fd58860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 -- 172.21.15.2:0/2020390681 >> 172.21.15.2:0/2020390681 conn(0x56076fd443f0 msgr2=0x56076fd48c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:44.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f067aff5700 0 client.15414 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:44.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 -- 172.21.15.2:0/2020390681 shutdown_connections 2023-06-23T18:22:44.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 1 -- 172.21.15.2:0/2020390681 wait complete. 2023-06-23T18:22:44.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.922+0000 7f06a931ab80 20 client.15414 trim_cache size 0 max 16384 2023-06-23T18:22:44.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.942+0000 7f06a931ab80 1 Processor -- start 2023-06-23T18:22:44.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.942+0000 7f06a931ab80 1 -- start start 2023-06-23T18:22:44.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4ec80 0x56076fd580d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd50000 0x56076fd58610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f06a931ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fd4a430 con 0x56076fd50000 2023-06-23T18:22:44.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f06a931ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fd4ad50 con 0x56076fd4ec80 2023-06-23T18:22:44.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f06a931ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fd577a0 con 0x56076fd4f5d0 2023-06-23T18:22:44.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd50000 0x56076fd58610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd50000 0x56076fd58610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54138/0 (socket says 172.21.15.2:54138) 2023-06-23T18:22:44.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 -- 172.21.15.2:0/2550373801 learned_addr learned my addr 172.21.15.2:0/2550373801 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069659c700 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f0696d9d700 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4ec80 0x56076fd580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 -- 172.21.15.2:0/2550373801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 msgr2=0x56076fd5f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 -- 172.21.15.2:0/2550373801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4ec80 msgr2=0x56076fd580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4ec80 0x56076fd580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 -- 172.21.15.2:0/2550373801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56076facdde0 con 0x56076fd50000 2023-06-23T18:22:44.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069659c700 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:44.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.943+0000 7f069759e700 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd50000 0x56076fd58610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0690011b90 tx=0x7f0690009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b14a81bc6225b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:44.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f067dffb700 1 -- 172.21.15.2:0/2550373801 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000e040 con 0x56076fd50000 2023-06-23T18:22:44.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f067dffb700 1 -- 172.21.15.2:0/2550373801 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f069000f040 con 0x56076fd50000 2023-06-23T18:22:44.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f067dffb700 1 -- 172.21.15.2:0/2550373801 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0690007d10 con 0x56076fd50000 2023-06-23T18:22:44.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 -- 172.21.15.2:0/2550373801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd50000 msgr2=0x56076fd58610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd50000 0x56076fd58610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0690011b90 tx=0x7f0690009920 comp rx=0 tx=0).stop 2023-06-23T18:22:44.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 -- 172.21.15.2:0/2550373801 shutdown_connections 2023-06-23T18:22:44.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd50000 0x56076fd58610 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd4ec80 0x56076fd580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 --2- 172.21.15.2:0/2550373801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 -- 172.21.15.2:0/2550373801 >> 172.21.15.2:0/2550373801 conn(0x56076fd443f0 msgr2=0x56076fd60130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:44.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 -- 172.21.15.2:0/2550373801 shutdown_connections 2023-06-23T18:22:44.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.944+0000 7f06a931ab80 1 -- 172.21.15.2:0/2550373801 wait complete. 2023-06-23T18:22:44.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 1 -- start start 2023-06-23T18:22:44.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 1 Processor -- start 2023-06-23T18:22:44.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd5b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 0x56076fd5ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fd5f320 con 0x56076fd4ec80 2023-06-23T18:22:44.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fd5f490 con 0x56076fd50000 2023-06-23T18:22:44.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56076fd5f650 con 0x56076fd4f5d0 2023-06-23T18:22:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f06a931ab80 10 client.0 _ll_register_callbacks cb 0x56076fc45260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f0696d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd5b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f0696d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd5b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54140/0 (socket says 172.21.15.2:54140) 2023-06-23T18:22:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f0696d9d700 1 -- 172.21.15.2:0/94278935 learned_addr learned my addr 172.21.15.2:0/94278935 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f069759e700 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 0x56076fd5ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.945+0000 7f069659c700 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f0696d9d700 1 -- 172.21.15.2:0/94278935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 msgr2=0x56076fd5bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:44.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f0696d9d700 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f0696d9d700 1 -- 172.21.15.2:0/94278935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 msgr2=0x56076fd5ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:44.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f0696d9d700 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 0x56076fd5ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f0696d9d700 1 -- 172.21.15.2:0/94278935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0690011840 con 0x56076fd4ec80 2023-06-23T18:22:44.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f0696d9d700 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd5b880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f068803a930 tx=0x7f0688005a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd242c26af39543 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:44.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f067aff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:44.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f067aff5700 1 -- 172.21.15.2:0/94278935 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068800bce0 con 0x56076fd4ec80 2023-06-23T18:22:44.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f067aff5700 1 -- 172.21.15.2:0/94278935 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f068800be40 con 0x56076fd4ec80 2023-06-23T18:22:44.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f06a931ab80 1 -- 172.21.15.2:0/94278935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f068801e040 con 0x56076fd4ec80 2023-06-23T18:22:44.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f067aff5700 1 -- 172.21.15.2:0/94278935 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0688004500 con 0x56076fd4ec80 2023-06-23T18:22:44.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f06a931ab80 10 client.15420 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:44.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f06a931ab80 1 -- 172.21.15.2:0/94278935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x56076fab6c60 con 0x56076fd4ec80 2023-06-23T18:22:44.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.946+0000 7f067aff5700 1 -- 172.21.15.2:0/94278935 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f068801e040 con 0x56076fd4ec80 2023-06-23T18:22:44.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f067aff5700 1 -- 172.21.15.2:0/94278935 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0688021070 con 0x56076fd4ec80 2023-06-23T18:22:44.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f06a931ab80 10 client.15420 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:44.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f06a931ab80 10 client.15420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:44.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f06a931ab80 20 client.15420 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:44.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f06a931ab80 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56076fdb8710 0x56076fdbab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:44.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f06a931ab80 4 client.15420 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:22:44.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f06a931ab80 1 -- 172.21.15.2:0/94278935 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56076fc5b050 con 0x56076fdb8710 2023-06-23T18:22:44.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f069659c700 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56076fdb8710 0x56076fdbab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:44.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f069659c700 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56076fdb8710 0x56076fdbab20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c800a76ca5e04881 server_cookie=dd845e3044797f3b in_seq=0 out_seq=0 2023-06-23T18:22:44.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.947+0000 7f067aff5700 10 client.15420 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:44.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f067aff5700 1 -- 172.21.15.2:0/94278935 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56076fc5b050 con 0x56076fdb8710 2023-06-23T18:22:44.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f067aff5700 10 client.15420 handle_command_reply: tid=0 2023-06-23T18:22:44.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 client.15420 shutdown 2023-06-23T18:22:44.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 -- 172.21.15.2:0/94278935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 msgr2=0x56076fd5b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd5b880 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f068803a930 tx=0x7f0688005a30 comp rx=0 tx=0).stop 2023-06-23T18:22:44.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 -- 172.21.15.2:0/94278935 shutdown_connections 2023-06-23T18:22:44.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56076fdb8710 0x56076fdbab20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56076fd50000 0x56076fd5ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56076fd4ec80 0x56076fd5b880 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f067aff5700 0 client.15420 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:44.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 --2- 172.21.15.2:0/94278935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56076fd4f5d0 0x56076fd5bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 -- 172.21.15.2:0/94278935 >> 172.21.15.2:0/94278935 conn(0x56076fd443f0 msgr2=0x56076fd580d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:44.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 -- 172.21.15.2:0/94278935 shutdown_connections 2023-06-23T18:22:44.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 1 -- 172.21.15.2:0/94278935 wait complete. 2023-06-23T18:22:44.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.948+0000 7f06a931ab80 20 client.15420 trim_cache size 0 max 16384 2023-06-23T18:22:44.964 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:22:44.964 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:22:44.964 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:44.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 -- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06800a1150 msgr2=0x7f06800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06800a1150 0x7f06800a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f068800b2e0 tx=0x7f06880042b0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 -- 172.21.15.2:0/1418959925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06980ffc70 msgr2=0x7f06981079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:44.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06980ffc70 0x7f06981079b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f068c00e9f0 tx=0x7f068c00ed00 comp rx=0 tx=0).stop 2023-06-23T18:22:44.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 -- 172.21.15.2:0/1418959925 shutdown_connections 2023-06-23T18:22:44.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0698100600 0x7f0698107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06980fea80 0x7f0698109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06800a1150 0x7f06800a3600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 --2- 172.21.15.2:0/1418959925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06980ffc70 0x7f06981079b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:44.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 -- 172.21.15.2:0/1418959925 >> 172.21.15.2:0/1418959925 conn(0x7f06980fa0e0 msgr2=0x7f06980fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:44.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 -- 172.21.15.2:0/1418959925 shutdown_connections 2023-06-23T18:22:44.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:44.949+0000 7f069d195700 1 -- 172.21.15.2:0/1418959925 wait complete. 2023-06-23T18:22:44.973 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:22:44.973 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:22:45.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.118+0000 7fdda841e700 1 -- 172.21.15.2:0/580955274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00fea60 msgr2=0x7fdda00fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.118+0000 7fdda841e700 1 --2- 172.21.15.2:0/580955274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00fea60 0x7fdda00fee70 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009990 tx=0x7fdd9c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 -- 172.21.15.2:0/580955274 shutdown_connections 2023-06-23T18:22:45.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 --2- 172.21.15.2:0/580955274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdda01005e0 0x7fdda0106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 --2- 172.21.15.2:0/580955274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00fea60 0x7fdda00fee70 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 --2- 172.21.15.2:0/580955274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda00ffc50 0x7fdda01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 -- 172.21.15.2:0/580955274 >> 172.21.15.2:0/580955274 conn(0x7fdda00fa0e0 msgr2=0x7fdda00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 -- 172.21.15.2:0/580955274 shutdown_connections 2023-06-23T18:22:45.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 -- 172.21.15.2:0/580955274 wait complete. 2023-06-23T18:22:45.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 Processor -- start 2023-06-23T18:22:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.119+0000 7fdda841e700 1 -- start start 2023-06-23T18:22:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda841e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdda00fea60 0x7fdda0068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda841e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00ffc50 0x7fdda019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda841e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda01005e0 0x7fdda019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda841e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019d320 con 0x7fdda00ffc50 2023-06-23T18:22:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda841e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019d490 con 0x7fdda00fea60 2023-06-23T18:22:45.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda841e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda019d650 con 0x7fdda01005e0 2023-06-23T18:22:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00ffc50 0x7fdda019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00ffc50 0x7fdda019a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54156/0 (socket says 172.21.15.2:54156) 2023-06-23T18:22:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 -- 172.21.15.2:0/1556620367 learned_addr learned my addr 172.21.15.2:0/1556620367 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda61ba700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdda00fea60 0x7fdda0068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda69bb700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda01005e0 0x7fdda019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 -- 172.21.15.2:0/1556620367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda01005e0 msgr2=0x7fdda019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda01005e0 0x7fdda019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 -- 172.21.15.2:0/1556620367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdda00fea60 msgr2=0x7fdda0068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdda00fea60 0x7fdda0068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 -- 172.21.15.2:0/1556620367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c0095f0 con 0x7fdda00ffc50 2023-06-23T18:22:45.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdda59b9700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00ffc50 0x7fdda019a500 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000e9a0 tx=0x7fdd9000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.120+0000 7fdd977fe700 1 -- 172.21.15.2:0/1556620367 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90009790 con 0x7fdda00ffc50 2023-06-23T18:22:45.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.121+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda019d930 con 0x7fdda00ffc50 2023-06-23T18:22:45.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.121+0000 7fdd977fe700 1 -- 172.21.15.2:0/1556620367 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd9000cbd0 con 0x7fdda00ffc50 2023-06-23T18:22:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.121+0000 7fdd977fe700 1 -- 172.21.15.2:0/1556620367 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90018680 con 0x7fdda00ffc50 2023-06-23T18:22:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.121+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda019de40 con 0x7fdda00ffc50 2023-06-23T18:22:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.122+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda010a0e0 con 0x7fdda00ffc50 2023-06-23T18:22:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.123+0000 7fdd977fe700 1 -- 172.21.15.2:0/1556620367 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdd9000cd40 con 0x7fdda00ffc50 2023-06-23T18:22:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.124+0000 7fdd977fe700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd8c0a1150 0x7fdd8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.124+0000 7fdd977fe700 1 -- 172.21.15.2:0/1556620367 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdd90014070 con 0x7fdda00ffc50 2023-06-23T18:22:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.124+0000 7fdda61ba700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd8c0a1150 0x7fdd8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.124+0000 7fdda61ba700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd8c0a1150 0x7fdd8c0a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c006010 tx=0x7fdd9c00b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.128+0000 7fdd977fe700 1 -- 172.21.15.2:0/1556620367 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fdd90051770 con 0x7fdda00ffc50 2023-06-23T18:22:45.267 INFO:teuthology.orchestra.run.smithi002.stderr:800000 of 50000000 tuples (1%) done (elapsed 8.51 s, remaining 523.24 s) 2023-06-23T18:22:45.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:45 smithi082 ceph-mon[97889]: pgmap v318: 97 pgs: 97 active+clean; 156 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 18 MiB/s wr, 93 op/s 2023-06-23T18:22:45.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:45 smithi017 ceph-mon[97472]: pgmap v318: 97 pgs: 97 active+clean; 156 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 18 MiB/s wr, 93 op/s 2023-06-23T18:22:45.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.437+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdda01059e0 con 0x7fdda00ffc50 2023-06-23T18:22:45.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.438+0000 7fdd977fe700 1 -- 172.21.15.2:0/1556620367 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+10777 (secure 0 0 0) 0x7fdd90096300 con 0x7fdda00ffc50 2023-06-23T18:22:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:22:45.444 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:20:49.041282+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:22:45.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd8c0a1150 msgr2=0x7fdd8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd8c0a1150 0x7fdd8c0a3600 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c006010 tx=0x7fdd9c00b2a0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00ffc50 msgr2=0x7fdda019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00ffc50 0x7fdda019a500 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000e9a0 tx=0x7fdd9000ecb0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 shutdown_connections 2023-06-23T18:22:45.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdda01005e0 0x7fdda019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd8c0a1150 0x7fdd8c0a3600 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdda00fea60 0x7fdda0068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 --2- 172.21.15.2:0/1556620367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdda00ffc50 0x7fdda019a500 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.440+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 >> 172.21.15.2:0/1556620367 conn(0x7fdda00fa0e0 msgr2=0x7fdda00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.441+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 shutdown_connections 2023-06-23T18:22:45.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.441+0000 7fdda841e700 1 -- 172.21.15.2:0/1556620367 wait complete. 2023-06-23T18:22:45.448 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 22 2023-06-23T18:22:45.458 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:45 smithi002 ceph-mon[95947]: pgmap v318: 97 pgs: 97 active+clean; 156 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 18 MiB/s wr, 93 op/s 2023-06-23T18:22:45.460 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:22:45.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.609+0000 7f388f336700 1 -- 172.21.15.2:0/3878836499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 msgr2=0x7f38880fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.609+0000 7f388f336700 1 --2- 172.21.15.2:0/3878836499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 0x7f38880fee90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f388400faf0 tx=0x7f388400fe00 comp rx=0 tx=0).stop 2023-06-23T18:22:45.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.610+0000 7f388f336700 1 -- 172.21.15.2:0/3878836499 shutdown_connections 2023-06-23T18:22:45.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.610+0000 7f388f336700 1 --2- 172.21.15.2:0/3878836499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3888100600 0x7f3888106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.610+0000 7f388f336700 1 --2- 172.21.15.2:0/3878836499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 0x7f38880fee90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.610+0000 7f388f336700 1 --2- 172.21.15.2:0/3878836499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38880ffc70 0x7f38881000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.610+0000 7f388f336700 1 -- 172.21.15.2:0/3878836499 >> 172.21.15.2:0/3878836499 conn(0x7f38880fa0e0 msgr2=0x7f38880fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.610+0000 7f388f336700 1 -- 172.21.15.2:0/3878836499 shutdown_connections 2023-06-23T18:22:45.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.610+0000 7f388f336700 1 -- 172.21.15.2:0/3878836499 wait complete. 2023-06-23T18:22:45.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.610+0000 7f388f336700 1 Processor -- start 2023-06-23T18:22:45.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388f336700 1 -- start start 2023-06-23T18:22:45.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388f336700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 0x7f388806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388f336700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880ffc70 0x7f388806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388f336700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3888100600 0x7f38881a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388f336700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3888072390 con 0x7f38880fea80 2023-06-23T18:22:45.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388f336700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3888072500 con 0x7f38880ffc70 2023-06-23T18:22:45.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388f336700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38881a3cc0 con 0x7f3888100600 2023-06-23T18:22:45.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d0d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 0x7f388806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d0d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 0x7f388806ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54168/0 (socket says 172.21.15.2:54168) 2023-06-23T18:22:45.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d0d2700 1 -- 172.21.15.2:0/123564136 learned_addr learned my addr 172.21.15.2:0/123564136 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388c8d1700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880ffc70 0x7f388806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d8d3700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3888100600 0x7f38881a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d0d2700 1 -- 172.21.15.2:0/123564136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3888100600 msgr2=0x7f38881a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d0d2700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3888100600 0x7f38881a3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d0d2700 1 -- 172.21.15.2:0/123564136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880ffc70 msgr2=0x7f388806f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d0d2700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880ffc70 0x7f388806f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.611+0000 7f388d0d2700 1 -- 172.21.15.2:0/123564136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388400f750 con 0x7f38880fea80 2023-06-23T18:22:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.612+0000 7f388d0d2700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 0x7f388806ed70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3884009cd0 tx=0x7f3884009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.612+0000 7f387e7fc700 1 -- 172.21.15.2:0/123564136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884003db0 con 0x7f38880fea80 2023-06-23T18:22:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.612+0000 7f388f336700 1 -- 172.21.15.2:0/123564136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38881a3ec0 con 0x7f38880fea80 2023-06-23T18:22:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.612+0000 7f387e7fc700 1 -- 172.21.15.2:0/123564136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f388400e660 con 0x7f38880fea80 2023-06-23T18:22:45.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.612+0000 7f387e7fc700 1 -- 172.21.15.2:0/123564136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38840216c0 con 0x7f38880fea80 2023-06-23T18:22:45.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.612+0000 7f388f336700 1 -- 172.21.15.2:0/123564136 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38881a4280 con 0x7f38880fea80 2023-06-23T18:22:45.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.614+0000 7f387e7fc700 1 -- 172.21.15.2:0/123564136 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f388400e7d0 con 0x7f38880fea80 2023-06-23T18:22:45.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.615+0000 7f387e7fc700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38740a1150 0x7f38740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.615+0000 7f388c8d1700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38740a1150 0x7f38740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.615+0000 7f387e7fc700 1 -- 172.21.15.2:0/123564136 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f38840d21c0 con 0x7f38880fea80 2023-06-23T18:22:45.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f388c8d1700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38740a1150 0x7f38740a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f387800bce0 tx=0x7f387800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f389b4bbb80 1 Processor -- start 2023-06-23T18:22:45.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f389b4bbb80 1 -- start start 2023-06-23T18:22:45.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557579158c80 0x557579159090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 0x557579159a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55757915a000 0x55757915a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55757905dcb0 con 0x557579158c80 2023-06-23T18:22:45.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575790b0a40 con 0x55757915a000 2023-06-23T18:22:45.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.616+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575790a9ed0 con 0x5575791595d0 2023-06-23T18:22:45.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557579158c80 0x557579159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557579158c80 0x557579159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54176/0 (socket says 172.21.15.2:54176) 2023-06-23T18:22:45.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 -- 172.21.15.2:0/1557526511 learned_addr learned my addr 172.21.15.2:0/1557526511 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d8d3700 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55757915a000 0x55757915a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388c8d1700 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 0x557579159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 -- 172.21.15.2:0/1557526511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 msgr2=0x557579159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 0x557579159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 -- 172.21.15.2:0/1557526511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55757915a000 msgr2=0x55757915a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55757915a000 0x55757915a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 -- 172.21.15.2:0/1557526511 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557578f2b630 con 0x557579158c80 2023-06-23T18:22:45.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d8d3700 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55757915a000 0x55757915a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:45.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557579158c80 0x557579159090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3884012040 tx=0x7f388409efc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e843ff2def400dae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f38737fe700 1 -- 172.21.15.2:0/1557526511 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38840bf0d0 con 0x557579158c80 2023-06-23T18:22:45.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f38737fe700 1 -- 172.21.15.2:0/1557526511 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38840a7a20 con 0x557579158c80 2023-06-23T18:22:45.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.617+0000 7f38737fe700 1 -- 172.21.15.2:0/1557526511 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388409fb20 con 0x557579158c80 2023-06-23T18:22:45.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1557526511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557579158c80 msgr2=0x557579159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557579158c80 0x557579159090 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3884012040 tx=0x7f388409efc0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1557526511 shutdown_connections 2023-06-23T18:22:45.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55757915a000 0x55757915a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557579158c80 0x557579159090 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1557526511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 0x557579159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1557526511 >> 172.21.15.2:0/1557526511 conn(0x55757914e3f0 msgr2=0x557579153520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1557526511 shutdown_connections 2023-06-23T18:22:45.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1557526511 wait complete. 2023-06-23T18:22:45.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.618+0000 7f389b4bbb80 1 -- start start 2023-06-23T18:22:45.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f389b4bbb80 1 Processor -- start 2023-06-23T18:22:45.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 0x5575791621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575791595d0 0x5575791626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55757915a000 0x557579169720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557579049ac0 con 0x5575791595d0 2023-06-23T18:22:45.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575790497c0 con 0x557579158c80 2023-06-23T18:22:45.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557578f2d920 con 0x55757915a000 2023-06-23T18:22:45.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f389b4bbb80 10 client.0 _ll_register_callbacks cb 0x55757904f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:45.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388c8d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575791595d0 0x5575791626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388c8d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575791595d0 0x5575791626e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54184/0 (socket says 172.21.15.2:54184) 2023-06-23T18:22:45.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388c8d1700 1 -- 172.21.15.2:0/1321578068 learned_addr learned my addr 172.21.15.2:0/1321578068 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 0x5575791621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388d8d3700 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55757915a000 0x557579169720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388c8d1700 1 -- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55757915a000 msgr2=0x557579169720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388c8d1700 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55757915a000 0x557579169720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388c8d1700 1 -- 172.21.15.2:0/1321578068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 msgr2=0x5575791621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.619+0000 7f388c8d1700 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 0x5575791621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f388c8d1700 1 -- 172.21.15.2:0/1321578068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3880009900 con 0x5575791595d0 2023-06-23T18:22:45.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f388d8d3700 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55757915a000 0x557579169720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:45.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f388c8d1700 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575791595d0 0x5575791626e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f38780045f0 tx=0x7f3878004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae065fe610ed2a2e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f3870ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:45.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f3870ff9700 1 -- 172.21.15.2:0/1321578068 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878010040 con 0x5575791595d0 2023-06-23T18:22:45.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f3870ff9700 1 -- 172.21.15.2:0/1321578068 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f387801d040 con 0x5575791595d0 2023-06-23T18:22:45.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1321578068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55757914bbb0 con 0x5575791595d0 2023-06-23T18:22:45.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f3870ff9700 1 -- 172.21.15.2:0/1321578068 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3878003ea0 con 0x5575791595d0 2023-06-23T18:22:45.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f389b4bbb80 10 client.15438 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:45.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1321578068 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f38881a4280 con 0x5575791595d0 2023-06-23T18:22:45.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.620+0000 7f3870ff9700 1 -- 172.21.15.2:0/1321578068 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55757914bbb0 con 0x5575791595d0 2023-06-23T18:22:45.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f3870ff9700 1 -- 172.21.15.2:0/1321578068 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3878018800 con 0x5575791595d0 2023-06-23T18:22:45.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f389b4bbb80 10 client.15438 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:45.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f389b4bbb80 10 client.15438 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:22:45.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f389b4bbb80 20 client.15438 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:45.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55757916dfa0 0x5575791703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f389b4bbb80 4 client.15438 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:45.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1321578068 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575790a2eb0 con 0x55757916dfa0 2023-06-23T18:22:45.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55757916dfa0 0x5575791703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f3870ff9700 10 client.15438 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:22:45.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.621+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55757916dfa0 0x5575791703b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb2ecfd2afe42b40 server_cookie=b9a39449de2d8700 in_seq=0 out_seq=0 2023-06-23T18:22:45.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f3870ff9700 1 -- 172.21.15.2:0/1321578068 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5575790a2eb0 con 0x55757916dfa0 2023-06-23T18:22:45.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f3870ff9700 10 client.15438 handle_command_reply: tid=0 2023-06-23T18:22:45.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 client.15438 shutdown 2023-06-23T18:22:45.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1321578068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575791595d0 msgr2=0x5575791626e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575791595d0 0x5575791626e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f38780045f0 tx=0x7f3878004750 comp rx=0 tx=0).stop 2023-06-23T18:22:45.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1321578068 shutdown_connections 2023-06-23T18:22:45.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55757916dfa0 0x5575791703b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55757915a000 0x557579169720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f3870ff9700 0 client.15438 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:22:45.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 0x5575791621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1321578068 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5575791595d0 0x5575791626e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1321578068 >> 172.21.15.2:0/1321578068 conn(0x55757914e3f0 msgr2=0x557579152c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1321578068 shutdown_connections 2023-06-23T18:22:45.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.622+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1321578068 wait complete. 2023-06-23T18:22:45.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.623+0000 7f389b4bbb80 20 client.15438 trim_cache size 0 max 16384 2023-06-23T18:22:45.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f389b4bbb80 1 Processor -- start 2023-06-23T18:22:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f389b4bbb80 1 -- start start 2023-06-23T18:22:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x5575791621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 0x5575791696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 0x5575791626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575791630e0 con 0x55757914e3f0 2023-06-23T18:22:45.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557579161680 con 0x557579158c80 2023-06-23T18:22:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557579161d10 con 0x5575791595d0 2023-06-23T18:22:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x5575791621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x5575791621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54190/0 (socket says 172.21.15.2:54190) 2023-06-23T18:22:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 -- 172.21.15.2:0/2397168347 learned_addr learned my addr 172.21.15.2:0/2397168347 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d8d3700 1 --2- 172.21.15.2:0/2397168347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 0x5575791626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 -- 172.21.15.2:0/2397168347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 msgr2=0x5575791626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 --2- 172.21.15.2:0/2397168347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 0x5575791626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 -- 172.21.15.2:0/2397168347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 msgr2=0x5575791696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 --2- 172.21.15.2:0/2397168347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 0x5575791696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 -- 172.21.15.2:0/2397168347 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557578ed7de0 con 0x55757914e3f0 2023-06-23T18:22:45.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.638+0000 7f388d0d2700 1 --2- 172.21.15.2:0/2397168347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x5575791621a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f388409f360 tx=0x7f3884005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771a024abcb2cd18 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f3873fff700 1 -- 172.21.15.2:0/2397168347 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388409efc0 con 0x55757914e3f0 2023-06-23T18:22:45.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f3873fff700 1 -- 172.21.15.2:0/2397168347 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f388409bd70 con 0x55757914e3f0 2023-06-23T18:22:45.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f3873fff700 1 -- 172.21.15.2:0/2397168347 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388409eb50 con 0x55757914e3f0 2023-06-23T18:22:45.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f389b4bbb80 1 -- 172.21.15.2:0/2397168347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 msgr2=0x5575791621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/2397168347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x5575791621a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f388409f360 tx=0x7f3884005fd0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f389b4bbb80 1 -- 172.21.15.2:0/2397168347 shutdown_connections 2023-06-23T18:22:45.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/2397168347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5575791595d0 0x5575791626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/2397168347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x5575791621a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/2397168347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557579158c80 0x5575791696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.639+0000 7f389b4bbb80 1 -- 172.21.15.2:0/2397168347 >> 172.21.15.2:0/2397168347 conn(0x557579161250 msgr2=0x557579174d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.640+0000 7f389b4bbb80 1 -- 172.21.15.2:0/2397168347 shutdown_connections 2023-06-23T18:22:45.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.640+0000 7f389b4bbb80 1 -- 172.21.15.2:0/2397168347 wait complete. 2023-06-23T18:22:45.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.640+0000 7f389b4bbb80 1 -- start start 2023-06-23T18:22:45.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.640+0000 7f389b4bbb80 1 Processor -- start 2023-06-23T18:22:45.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x557579165700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557579158c80 0x557579165c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f389b4bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5575791595d0 0x557579168c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575791691a0 con 0x55757914e3f0 2023-06-23T18:22:45.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557579169310 con 0x5575791595d0 2023-06-23T18:22:45.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f389b4bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575791694d0 con 0x557579158c80 2023-06-23T18:22:45.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f389b4bbb80 10 client.0 _ll_register_callbacks cb 0x55757904f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:45.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x557579165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x557579165700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54196/0 (socket says 172.21.15.2:54196) 2023-06-23T18:22:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 -- 172.21.15.2:0/1887663198 learned_addr learned my addr 172.21.15.2:0/1887663198 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388c8d1700 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557579158c80 0x557579165c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d8d3700 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5575791595d0 0x557579168c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 -- 172.21.15.2:0/1887663198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557579158c80 msgr2=0x557579165c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557579158c80 0x557579165c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 -- 172.21.15.2:0/1887663198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5575791595d0 msgr2=0x557579168c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5575791595d0 0x557579168c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 -- 172.21.15.2:0/1887663198 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557578f2b630 con 0x55757914e3f0 2023-06-23T18:22:45.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d8d3700 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5575791595d0 0x557579168c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:45.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f388d0d2700 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x557579165700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38840a74f0 tx=0x7f388409b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542cdcd4c9fa3183 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f3870ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:45.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.641+0000 7f3870ff9700 1 -- 172.21.15.2:0/1887663198 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388409bee0 con 0x55757914e3f0 2023-06-23T18:22:45.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f3870ff9700 1 -- 172.21.15.2:0/1887663198 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f388409fad0 con 0x55757914e3f0 2023-06-23T18:22:45.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1887663198 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38840a7a20 con 0x55757914e3f0 2023-06-23T18:22:45.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f3870ff9700 1 -- 172.21.15.2:0/1887663198 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f388409eb50 con 0x55757914e3f0 2023-06-23T18:22:45.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 10 client.15444 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:45.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1887663198 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x557578ec0c60 con 0x55757914e3f0 2023-06-23T18:22:45.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f3870ff9700 1 -- 172.21.15.2:0/1887663198 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f38840a7a20 con 0x55757914e3f0 2023-06-23T18:22:45.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f3870ff9700 1 -- 172.21.15.2:0/1887663198 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f388408a0b0 con 0x55757914e3f0 2023-06-23T18:22:45.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 10 client.15444 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:45.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 10 client.15444 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:22:45.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 20 client.15444 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:45.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55757915a000 0x5575791c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 4 client.15444 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:22:45.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1887663198 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557579065050 con 0x55757915a000 2023-06-23T18:22:45.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.642+0000 7f388c8d1700 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55757915a000 0x5575791c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f3870ff9700 10 client.15444 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:22:45.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f388c8d1700 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55757915a000 0x5575791c4bf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=603ee98783b182a3 server_cookie=13542fe5aad89e51 in_seq=0 out_seq=0 2023-06-23T18:22:45.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f3870ff9700 1 -- 172.21.15.2:0/1887663198 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557579065050 con 0x55757915a000 2023-06-23T18:22:45.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f3870ff9700 10 client.15444 handle_command_reply: tid=0 2023-06-23T18:22:45.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 client.15444 shutdown 2023-06-23T18:22:45.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1887663198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 msgr2=0x557579165700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x557579165700 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38840a74f0 tx=0x7f388409b9a0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1887663198 shutdown_connections 2023-06-23T18:22:45.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55757915a000 0x5575791c4bf0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5575791595d0 0x557579168c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f3870ff9700 0 client.15444 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:22:45.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55757914e3f0 0x557579165700 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 --2- 172.21.15.2:0/1887663198 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557579158c80 0x557579165c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1887663198 >> 172.21.15.2:0/1887663198 conn(0x557579161250 msgr2=0x55757915ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1887663198 shutdown_connections 2023-06-23T18:22:45.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.643+0000 7f389b4bbb80 1 -- 172.21.15.2:0/1887663198 wait complete. 2023-06-23T18:22:45.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f389b4bbb80 20 client.15444 trim_cache size 0 max 16384 2023-06-23T18:22:45.680 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:22:45.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 -- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38740a1150 msgr2=0x7f38740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38740a1150 0x7f38740a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f387800bce0 tx=0x7f387800d040 comp rx=0 tx=0).stop 2023-06-23T18:22:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 -- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 msgr2=0x7f388806ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 0x7f388806ed70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3884009cd0 tx=0x7f3884009d00 comp rx=0 tx=0).stop 2023-06-23T18:22:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 -- 172.21.15.2:0/123564136 shutdown_connections 2023-06-23T18:22:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3888100600 0x7f38881a3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38880fea80 0x7f388806ed70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38740a1150 0x7f38740a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 --2- 172.21.15.2:0/123564136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880ffc70 0x7f388806f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 -- 172.21.15.2:0/123564136 >> 172.21.15.2:0/123564136 conn(0x7f38880fa0e0 msgr2=0x7f38880fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 -- 172.21.15.2:0/123564136 shutdown_connections 2023-06-23T18:22:45.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.644+0000 7f388f336700 1 -- 172.21.15.2:0/123564136 wait complete. 2023-06-23T18:22:45.683 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:22:45.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 -- 172.21.15.2:0/1552519548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81241006a0 msgr2=0x7f8124106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1552519548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81241006a0 0x7f8124106f40 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b0a0 tx=0x7f811c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 -- 172.21.15.2:0/1552519548 shutdown_connections 2023-06-23T18:22:45.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1552519548 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81241006a0 0x7f8124106f40 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1552519548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81240fea60 0x7f81240fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1552519548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240ffcf0 0x7f8124100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 -- 172.21.15.2:0/1552519548 >> 172.21.15.2:0/1552519548 conn(0x7f81240fa080 msgr2=0x7f81240fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 -- 172.21.15.2:0/1552519548 shutdown_connections 2023-06-23T18:22:45.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.824+0000 7f812c6c5700 1 -- 172.21.15.2:0/1552519548 wait complete. 2023-06-23T18:22:45.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812c6c5700 1 Processor -- start 2023-06-23T18:22:45.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812c6c5700 1 -- start start 2023-06-23T18:22:45.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812c6c5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240fea60 0x7f81241958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812c6c5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81240ffcf0 0x7f8124195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812c6c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81241006a0 0x7f812419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812c6c5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812419d450 con 0x7f81240ffcf0 2023-06-23T18:22:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812c6c5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812419d590 con 0x7f81240fea60 2023-06-23T18:22:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812c6c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812419d720 con 0x7f81241006a0 2023-06-23T18:22:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f8129c60700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81240ffcf0 0x7f8124195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f8129c60700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81240ffcf0 0x7f8124195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54212/0 (socket says 172.21.15.2:54212) 2023-06-23T18:22:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f8129c60700 1 -- 172.21.15.2:0/1556011101 learned_addr learned my addr 172.21.15.2:0/1556011101 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812a461700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240fea60 0x7f81241958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f812ac62700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81241006a0 0x7f812419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f8129c60700 1 -- 172.21.15.2:0/1556011101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81241006a0 msgr2=0x7f812419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.825+0000 7f8129c60700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81241006a0 0x7f812419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f8129c60700 1 -- 172.21.15.2:0/1556011101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240fea60 msgr2=0x7f81241958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f8129c60700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240fea60 0x7f81241958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f8129c60700 1 -- 172.21.15.2:0/1556011101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c009d00 con 0x7f81240ffcf0 2023-06-23T18:22:45.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f8129c60700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81240ffcf0 0x7f8124195e10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f811400d950 tx=0x7f811400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f811b7fe700 1 -- 172.21.15.2:0/1556011101 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114009940 con 0x7f81240ffcf0 2023-06-23T18:22:45.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f812c6c5700 1 -- 172.21.15.2:0/1556011101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812419da00 con 0x7f81240ffcf0 2023-06-23T18:22:45.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f811b7fe700 1 -- 172.21.15.2:0/1556011101 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8114010460 con 0x7f81240ffcf0 2023-06-23T18:22:45.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f811b7fe700 1 -- 172.21.15.2:0/1556011101 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400f620 con 0x7f81240ffcf0 2023-06-23T18:22:45.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.826+0000 7f812c6c5700 1 -- 172.21.15.2:0/1556011101 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812419df10 con 0x7f81240ffcf0 2023-06-23T18:22:45.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.828+0000 7f811b7fe700 1 -- 172.21.15.2:0/1556011101 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f81140105d0 con 0x7f81240ffcf0 2023-06-23T18:22:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.829+0000 7f811b7fe700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f81100a1150 0x7f81100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.829+0000 7f811b7fe700 1 -- 172.21.15.2:0/1556011101 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f81140cc760 con 0x7f81240ffcf0 2023-06-23T18:22:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.829+0000 7f812a461700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f81100a1150 0x7f81100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f812a461700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f81100a1150 0x7f81100a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8120009c60 tx=0x7f8120009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f813884ab80 1 Processor -- start 2023-06-23T18:22:45.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f813884ab80 1 -- start start 2023-06-23T18:22:45.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f813884ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ff9c80 0x563f57ffa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f813884ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f57ffaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f813884ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f57ffb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f813884ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f57efecb0 con 0x563f57ffb000 2023-06-23T18:22:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f813884ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f57f51a40 con 0x563f57ff9c80 2023-06-23T18:22:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f813884ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f57f4aed0 con 0x563f57ffa5d0 2023-06-23T18:22:45.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f812ac62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f57ffb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f8129c60700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f57ffaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f812ac62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f57ffb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54222/0 (socket says 172.21.15.2:54222) 2023-06-23T18:22:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f812ac62700 1 -- 172.21.15.2:0/1047002099 learned_addr learned my addr 172.21.15.2:0/1047002099 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.830+0000 7f812a461700 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ff9c80 0x563f57ffa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8129c60700 1 -- 172.21.15.2:0/1047002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ff9c80 msgr2=0x563f57ffa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8129c60700 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ff9c80 0x563f57ffa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8129c60700 1 -- 172.21.15.2:0/1047002099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 msgr2=0x563f57ffb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8129c60700 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f57ffb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8129c60700 1 -- 172.21.15.2:0/1047002099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f57dcc630 con 0x563f57ffa5d0 2023-06-23T18:22:45.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f812ac62700 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f57ffb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:45.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f812a461700 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ff9c80 0x563f57ffa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:45.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8129c60700 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f57ffaa70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81140ca260 tx=0x7f81140994d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afd4d19b3ac9a95a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8118ff9700 1 -- 172.21.15.2:0/1047002099 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114099b70 con 0x563f57ffa5d0 2023-06-23T18:22:45.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8118ff9700 1 -- 172.21.15.2:0/1047002099 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f811409dba0 con 0x563f57ffa5d0 2023-06-23T18:22:45.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.831+0000 7f8118ff9700 1 -- 172.21.15.2:0/1047002099 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81140a1b30 con 0x563f57ffa5d0 2023-06-23T18:22:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 -- 172.21.15.2:0/1047002099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 msgr2=0x563f57ffaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f57ffaa70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81140ca260 tx=0x7f81140994d0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 -- 172.21.15.2:0/1047002099 shutdown_connections 2023-06-23T18:22:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f57ffb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ff9c80 0x563f57ffa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 --2- 172.21.15.2:0/1047002099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f57ffaa70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 -- 172.21.15.2:0/1047002099 >> 172.21.15.2:0/1047002099 conn(0x563f57fef3f0 msgr2=0x563f57ff4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 -- 172.21.15.2:0/1047002099 shutdown_connections 2023-06-23T18:22:45.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 -- 172.21.15.2:0/1047002099 wait complete. 2023-06-23T18:22:45.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.832+0000 7f813884ab80 1 -- start start 2023-06-23T18:22:45.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f813884ab80 1 Processor -- start 2023-06-23T18:22:45.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f813884ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f58003350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f813884ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f58003890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f813884ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffb000 0x563f5800a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f813884ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f57eeaac0 con 0x563f57ff9c80 2023-06-23T18:22:45.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f813884ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f57eea7c0 con 0x563f57ffb000 2023-06-23T18:22:45.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f813884ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f57dce920 con 0x563f57ffa5d0 2023-06-23T18:22:45.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f813884ab80 10 client.0 _ll_register_callbacks cb 0x563f57ef0260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:45.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f812a461700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f58003350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f812a461700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f58003350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54224/0 (socket says 172.21.15.2:54224) 2023-06-23T18:22:45.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f812a461700 1 -- 172.21.15.2:0/3812420359 learned_addr learned my addr 172.21.15.2:0/3812420359 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f8129c60700 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f58003890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f812ac62700 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffb000 0x563f5800a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f8129c60700 1 -- 172.21.15.2:0/3812420359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffb000 msgr2=0x563f5800a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f8129c60700 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffb000 0x563f5800a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f8129c60700 1 -- 172.21.15.2:0/3812420359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 msgr2=0x563f58003350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f8129c60700 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f58003350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f8129c60700 1 -- 172.21.15.2:0/3812420359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8124068950 con 0x563f57ffa5d0 2023-06-23T18:22:45.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.833+0000 7f812a461700 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f58003350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:45.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f8129c60700 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f58003890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x563f58001e00 tx=0x7f81140a1fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cff22915072cebb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f810dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:22:45.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f810dffb700 1 -- 172.21.15.2:0/3812420359 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81140a1b30 con 0x563f57ffa5d0 2023-06-23T18:22:45.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f813884ab80 1 -- 172.21.15.2:0/3812420359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f57fecbb0 con 0x563f57ffa5d0 2023-06-23T18:22:45.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f810dffb700 1 -- 172.21.15.2:0/3812420359 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f81140a1c90 con 0x563f57ffa5d0 2023-06-23T18:22:45.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f810dffb700 1 -- 172.21.15.2:0/3812420359 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8114095960 con 0x563f57ffa5d0 2023-06-23T18:22:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f813884ab80 10 client.25234 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f813884ab80 1 -- 172.21.15.2:0/3812420359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x7f812419df10 con 0x563f57ffa5d0 2023-06-23T18:22:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f810dffb700 1 -- 172.21.15.2:0/3812420359 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563f57fecbb0 con 0x563f57ffa5d0 2023-06-23T18:22:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.834+0000 7f810dffb700 1 -- 172.21.15.2:0/3812420359 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8114009aa0 con 0x563f57ffa5d0 2023-06-23T18:22:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f813884ab80 10 client.25234 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f813884ab80 10 client.25234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:22:45.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f813884ab80 20 client.25234 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f813884ab80 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x563f5800ec20 0x563f58011030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f813884ab80 4 client.25234 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f813884ab80 1 -- 172.21.15.2:0/3812420359 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f57f43eb0 con 0x563f5800ec20 2023-06-23T18:22:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f812a461700 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x563f5800ec20 0x563f58011030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f812a461700 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x563f5800ec20 0x563f58011030 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e31d05b7005406ff server_cookie=d14f3ce70d1536f6 in_seq=0 out_seq=0 2023-06-23T18:22:45.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.835+0000 7f810dffb700 10 client.25234 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:22:45.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f810dffb700 1 -- 172.21.15.2:0/3812420359 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563f57f43eb0 con 0x563f5800ec20 2023-06-23T18:22:45.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f810dffb700 10 client.25234 handle_command_reply: tid=0 2023-06-23T18:22:45.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 client.25234 shutdown 2023-06-23T18:22:45.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 -- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 msgr2=0x563f58003890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f58003890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x563f58001e00 tx=0x7f81140a1fc0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 -- 172.21.15.2:0/3812420359 shutdown_connections 2023-06-23T18:22:45.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x563f5800ec20 0x563f58011030 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffb000 0x563f5800a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f810dffb700 0 client.25234 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:22:45.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f58003350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 --2- 172.21.15.2:0/3812420359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffa5d0 0x563f58003890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 -- 172.21.15.2:0/3812420359 >> 172.21.15.2:0/3812420359 conn(0x563f57fef3f0 msgr2=0x563f57ff3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 -- 172.21.15.2:0/3812420359 shutdown_connections 2023-06-23T18:22:45.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 1 -- 172.21.15.2:0/3812420359 wait complete. 2023-06-23T18:22:45.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.836+0000 7f813884ab80 20 client.25234 trim_cache size 0 max 16384 2023-06-23T18:22:45.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.852+0000 7f813884ab80 1 Processor -- start 2023-06-23T18:22:45.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.852+0000 7f813884ab80 1 -- start start 2023-06-23T18:22:45.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.852+0000 7f813884ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f5800b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.852+0000 7f813884ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f813884ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffb000 0x563f58001110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f813884ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f58001e30 con 0x563f57ff9c80 2023-06-23T18:22:45.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f813884ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f57ff5810 con 0x563f57ffa5d0 2023-06-23T18:22:45.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f813884ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f57ff5d50 con 0x563f57ffb000 2023-06-23T18:22:45.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f5800b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f5800b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54232/0 (socket says 172.21.15.2:54232) 2023-06-23T18:22:45.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 -- 172.21.15.2:0/2134737506 learned_addr learned my addr 172.21.15.2:0/2134737506 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812ac62700 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffb000 0x563f58001110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f8129c60700 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 -- 172.21.15.2:0/2134737506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffb000 msgr2=0x563f58001110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffb000 0x563f58001110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 -- 172.21.15.2:0/2134737506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 msgr2=0x563f5800a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 -- 172.21.15.2:0/2134737506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f57d78de0 con 0x563f57ff9c80 2023-06-23T18:22:45.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f812a461700 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f5800b1c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8120011ab0 tx=0x7f8120013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2acbb5089c865d27 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.853+0000 7f81197fa700 1 -- 172.21.15.2:0/2134737506 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812002c250 con 0x563f57ff9c80 2023-06-23T18:22:45.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.854+0000 7f81197fa700 1 -- 172.21.15.2:0/2134737506 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8120024040 con 0x563f57ff9c80 2023-06-23T18:22:45.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.854+0000 7f81197fa700 1 -- 172.21.15.2:0/2134737506 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120003680 con 0x563f57ff9c80 2023-06-23T18:22:45.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.854+0000 7f813884ab80 1 -- 172.21.15.2:0/2134737506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 msgr2=0x563f5800b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f5800b1c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8120011ab0 tx=0x7f8120013040 comp rx=0 tx=0).stop 2023-06-23T18:22:45.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 -- 172.21.15.2:0/2134737506 shutdown_connections 2023-06-23T18:22:45.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ffb000 0x563f58001110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ff9c80 0x563f5800b1c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 --2- 172.21.15.2:0/2134737506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 -- 172.21.15.2:0/2134737506 >> 172.21.15.2:0/2134737506 conn(0x563f57fef3f0 msgr2=0x563f58002060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 -- 172.21.15.2:0/2134737506 shutdown_connections 2023-06-23T18:22:45.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 -- 172.21.15.2:0/2134737506 wait complete. 2023-06-23T18:22:45.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 -- start start 2023-06-23T18:22:45.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 Processor -- start 2023-06-23T18:22:45.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ff9c80 0x563f5800a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f58006db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f580074c0 con 0x563f57ffb000 2023-06-23T18:22:45.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f58007630 con 0x563f57ffa5d0 2023-06-23T18:22:45.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.855+0000 7f813884ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f580077f0 con 0x563f57ff9c80 2023-06-23T18:22:45.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f813884ab80 10 client.0 _ll_register_callbacks cb 0x563f57ef0260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:45.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f812ac62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f58006db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f812ac62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f58006db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54240/0 (socket says 172.21.15.2:54240) 2023-06-23T18:22:45.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f812ac62700 1 -- 172.21.15.2:0/3723123384 learned_addr learned my addr 172.21.15.2:0/3723123384 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:45.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f8129c60700 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f812a461700 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ff9c80 0x563f5800a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f8129c60700 1 -- 172.21.15.2:0/3723123384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ff9c80 msgr2=0x563f5800a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f8129c60700 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ff9c80 0x563f5800a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f8129c60700 1 -- 172.21.15.2:0/3723123384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 msgr2=0x563f58006db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:45.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f8129c60700 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f58006db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f8129c60700 1 -- 172.21.15.2:0/3723123384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8120017070 con 0x563f57ffa5d0 2023-06-23T18:22:45.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f812ac62700 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f58006db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:45.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f8129c60700 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8114009700 tx=0x7f8114009cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa5c4381153a593b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:45.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.856+0000 7f810dffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:22:45.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f810dffb700 1 -- 172.21.15.2:0/3723123384 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81140994d0 con 0x563f57ffa5d0 2023-06-23T18:22:45.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f810dffb700 1 -- 172.21.15.2:0/3723123384 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8114095960 con 0x563f57ffa5d0 2023-06-23T18:22:45.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 1 -- 172.21.15.2:0/3723123384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f811409dba0 con 0x563f57ffa5d0 2023-06-23T18:22:45.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f810dffb700 1 -- 172.21.15.2:0/3723123384 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81140a2030 con 0x563f57ffa5d0 2023-06-23T18:22:45.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 10 client.25082 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:45.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 1 -- 172.21.15.2:0/3723123384 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x563f57d61c60 con 0x563f57ffa5d0 2023-06-23T18:22:45.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f810dffb700 1 -- 172.21.15.2:0/3723123384 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f811409dba0 con 0x563f57ffa5d0 2023-06-23T18:22:45.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f810dffb700 1 -- 172.21.15.2:0/3723123384 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f811405fa30 con 0x563f57ffa5d0 2023-06-23T18:22:45.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 10 client.25082 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:45.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 10 client.25082 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:22:45.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 20 client.25082 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:45.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x563f58063ca0 0x563f58066080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:45.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 4 client.25082 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:22:45.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.857+0000 7f813884ab80 1 -- 172.21.15.2:0/3723123384 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f57f06050 con 0x563f58063ca0 2023-06-23T18:22:45.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.858+0000 7f812a461700 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x563f58063ca0 0x563f58066080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.858+0000 7f810dffb700 10 client.25082 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:22:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.858+0000 7f812a461700 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x563f58063ca0 0x563f58066080 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be6f87d61ec2d809 server_cookie=13f4fbac219a19d7 in_seq=0 out_seq=0 2023-06-23T18:22:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f810dffb700 1 -- 172.21.15.2:0/3723123384 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f57f06050 con 0x563f58063ca0 2023-06-23T18:22:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f810dffb700 10 client.25082 handle_command_reply: tid=0 2023-06-23T18:22:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 client.25082 shutdown 2023-06-23T18:22:45.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 -- 172.21.15.2:0/3723123384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 msgr2=0x563f5800a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8114009700 tx=0x7f8114009cb0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 -- 172.21.15.2:0/3723123384 shutdown_connections 2023-06-23T18:22:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x563f58063ca0 0x563f58066080 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563f57ffb000 0x563f58006db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563f57ff9c80 0x563f5800a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 --2- 172.21.15.2:0/3723123384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563f57ffa5d0 0x563f5800a850 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 -- 172.21.15.2:0/3723123384 >> 172.21.15.2:0/3723123384 conn(0x563f57fef3f0 msgr2=0x563f580032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f810dffb700 0 client.25082 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:22:45.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 -- 172.21.15.2:0/3723123384 shutdown_connections 2023-06-23T18:22:45.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 1 -- 172.21.15.2:0/3723123384 wait complete. 2023-06-23T18:22:45.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.859+0000 7f813884ab80 20 client.25082 trim_cache size 0 max 16384 2023-06-23T18:22:45.894 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:22:45.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 -- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f81100a1150 msgr2=0x7f81100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f81100a1150 0x7f81100a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8120009c60 tx=0x7f8120009260 comp rx=0 tx=0).stop 2023-06-23T18:22:45.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 -- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81240ffcf0 msgr2=0x7f8124195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:45.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81240ffcf0 0x7f8124195e10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f811400d950 tx=0x7f811400dc60 comp rx=0 tx=0).stop 2023-06-23T18:22:45.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 -- 172.21.15.2:0/1556011101 shutdown_connections 2023-06-23T18:22:45.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81241006a0 0x7f812419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f81100a1150 0x7f81100a3600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240fea60 0x7f81241958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 --2- 172.21.15.2:0/1556011101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f81240ffcf0 0x7f8124195e10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:45.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 -- 172.21.15.2:0/1556011101 >> 172.21.15.2:0/1556011101 conn(0x7f81240fa080 msgr2=0x7f81241031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:45.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 -- 172.21.15.2:0/1556011101 shutdown_connections 2023-06-23T18:22:45.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:45.860+0000 7f812c6c5700 1 -- 172.21.15.2:0/1556011101 wait complete. 2023-06-23T18:22:45.897 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:22:46.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.042+0000 7f0b61141700 1 -- 172.21.15.2:0/2463551952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c0fea80 msgr2=0x7f0b5c0fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.042+0000 7f0b61141700 1 --2- 172.21.15.2:0/2463551952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c0fea80 0x7f0b5c0fee90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c009990 tx=0x7f0b4c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.043+0000 7f0b61141700 1 -- 172.21.15.2:0/2463551952 shutdown_connections 2023-06-23T18:22:46.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.043+0000 7f0b61141700 1 --2- 172.21.15.2:0/2463551952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b5c100600 0x7f0b5c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.043+0000 7f0b61141700 1 --2- 172.21.15.2:0/2463551952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c0fea80 0x7f0b5c0fee90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.043+0000 7f0b61141700 1 --2- 172.21.15.2:0/2463551952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b5c0ffc70 0x7f0b5c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.043+0000 7f0b61141700 1 -- 172.21.15.2:0/2463551952 >> 172.21.15.2:0/2463551952 conn(0x7f0b5c0fa0e0 msgr2=0x7f0b5c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.043+0000 7f0b61141700 1 -- 172.21.15.2:0/2463551952 shutdown_connections 2023-06-23T18:22:46.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.043+0000 7f0b61141700 1 -- 172.21.15.2:0/2463551952 wait complete. 2023-06-23T18:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.043+0000 7f0b61141700 1 Processor -- start 2023-06-23T18:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b61141700 1 -- start start 2023-06-23T18:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b61141700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b5c0fea80 0x7f0b5c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b61141700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b5c0ffc70 0x7f0b5c1a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b61141700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c100600 0x7f0b5c1a3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b61141700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1a3a10 con 0x7f0b5c100600 2023-06-23T18:22:46.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b61141700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1a3b80 con 0x7f0b5c0fea80 2023-06-23T18:22:46.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b61141700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c1a3d40 con 0x7f0b5c0ffc70 2023-06-23T18:22:46.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b5b59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c100600 0x7f0b5c1a3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b5b59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c100600 0x7f0b5c1a3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54256/0 (socket says 172.21.15.2:54256) 2023-06-23T18:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b5a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b5c0ffc70 0x7f0b5c1a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b5b59e700 1 -- 172.21.15.2:0/1839724603 learned_addr learned my addr 172.21.15.2:0/1839724603 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.044+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b5c0fea80 0x7f0b5c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b5b59e700 1 -- 172.21.15.2:0/1839724603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b5c0ffc70 msgr2=0x7f0b5c1a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b5b59e700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b5c0ffc70 0x7f0b5c1a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b5b59e700 1 -- 172.21.15.2:0/1839724603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b5c0fea80 msgr2=0x7f0b5c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b5b59e700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b5c0fea80 0x7f0b5c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b5b59e700 1 -- 172.21.15.2:0/1839724603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0095f0 con 0x7f0b5c100600 2023-06-23T18:22:46.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b5c0ffc70 0x7f0b5c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:46.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b5b59e700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c100600 0x7f0b5c1a3240 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0b54007d70 tx=0x7f0b5400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:46.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b53fff700 1 -- 172.21.15.2:0/1839724603 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5400ac90 con 0x7f0b5c100600 2023-06-23T18:22:46.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b61141700 1 -- 172.21.15.2:0/1839724603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c1a3fc0 con 0x7f0b5c100600 2023-06-23T18:22:46.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b53fff700 1 -- 172.21.15.2:0/1839724603 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b5400adf0 con 0x7f0b5c100600 2023-06-23T18:22:46.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b53fff700 1 -- 172.21.15.2:0/1839724603 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b540115e0 con 0x7f0b5c100600 2023-06-23T18:22:46.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.045+0000 7f0b61141700 1 -- 172.21.15.2:0/1839724603 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c1a44d0 con 0x7f0b5c100600 2023-06-23T18:22:46.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.048+0000 7f0b53fff700 1 -- 172.21.15.2:0/1839724603 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0b54012420 con 0x7f0b5c100600 2023-06-23T18:22:46.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.049+0000 7f0b53fff700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0b480a1150 0x7f0b480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.049+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0b480a1150 0x7f0b480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.049+0000 7f0b53fff700 1 -- 172.21.15.2:0/1839724603 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0b540ccae0 con 0x7f0b5c100600 2023-06-23T18:22:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.049+0000 7f0b6d2c6b80 1 Processor -- start 2023-06-23T18:22:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b6d2c6b80 1 -- start start 2023-06-23T18:22:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0b480a1150 0x7f0b480a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c1072a0 tx=0x7f0b4c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e0a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0a5d0 0x55ff29e0aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 0x55ff29e0b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29d0ecb0 con 0x55ff29e09c80 2023-06-23T18:22:46.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29d61a40 con 0x55ff29e0a5d0 2023-06-23T18:22:46.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29d5aed0 con 0x55ff29e0b000 2023-06-23T18:22:46.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e0a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e0a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54260/0 (socket says 172.21.15.2:54260) 2023-06-23T18:22:46.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/3943304490 learned_addr learned my addr 172.21.15.2:0/3943304490 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:46.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/3943304490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0a5d0 0x55ff29e0aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/3943304490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 msgr2=0x55ff29e0b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/3943304490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 0x55ff29e0b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/3943304490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0a5d0 msgr2=0x55ff29e0aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/3943304490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0a5d0 0x55ff29e0aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.050+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/3943304490 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff29bdc630 con 0x55ff29e09c80 2023-06-23T18:22:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.051+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/3943304490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e0a090 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0ffad0 tx=0x7f0b4c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8341e958ab7a1c8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.051+0000 7f0b517fa700 1 -- 172.21.15.2:0/3943304490 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c02b070 con 0x55ff29e09c80 2023-06-23T18:22:46.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.051+0000 7f0b517fa700 1 -- 172.21.15.2:0/3943304490 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b4c01e040 con 0x55ff29e09c80 2023-06-23T18:22:46.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.051+0000 7f0b517fa700 1 -- 172.21.15.2:0/3943304490 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c003820 con 0x55ff29e09c80 2023-06-23T18:22:46.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.051+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3943304490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 msgr2=0x55ff29e0a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3943304490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e0a090 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c0ffad0 tx=0x7f0b4c020000 comp rx=0 tx=0).stop 2023-06-23T18:22:46.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3943304490 shutdown_connections 2023-06-23T18:22:46.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3943304490 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 0x55ff29e0b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3943304490 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e0a090 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3943304490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0a5d0 0x55ff29e0aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3943304490 >> 172.21.15.2:0/3943304490 conn(0x55ff29dff3f0 msgr2=0x55ff29e04520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:46.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3943304490 shutdown_connections 2023-06-23T18:22:46.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3943304490 wait complete. 2023-06-23T18:22:46.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 -- start start 2023-06-23T18:22:46.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 Processor -- start 2023-06-23T18:22:46.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e09c80 0x55ff29e13530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e0a5d0 0x55ff29e13a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 0x55ff29e1aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29cfaac0 con 0x55ff29e0a5d0 2023-06-23T18:22:46.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29cfa7c0 con 0x55ff29e09c80 2023-06-23T18:22:46.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.052+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29bde920 con 0x55ff29e0b000 2023-06-23T18:22:46.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b6d2c6b80 10 client.0 _ll_register_callbacks cb 0x55ff29d00260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:46.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e0a5d0 0x55ff29e13a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e0a5d0 0x55ff29e13a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54268/0 (socket says 172.21.15.2:54268) 2023-06-23T18:22:46.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 -- 172.21.15.2:0/838235736 learned_addr learned my addr 172.21.15.2:0/838235736 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:46.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5b59e700 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 0x55ff29e1aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e09c80 0x55ff29e13530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 -- 172.21.15.2:0/838235736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 msgr2=0x55ff29e1aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:46.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 0x55ff29e1aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 -- 172.21.15.2:0/838235736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e09c80 msgr2=0x55ff29e13530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:46.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e09c80 0x55ff29e13530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 -- 172.21.15.2:0/838235736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c1077f0 con 0x55ff29e0a5d0 2023-06-23T18:22:46.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5b59e700 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 0x55ff29e1aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:22:46.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e0a5d0 0x55ff29e13a70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0b44011a00 tx=0x7f0b44011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a639ee11b24320 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:46.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b427fc700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:46.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b427fc700 1 -- 172.21.15.2:0/838235736 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400f940 con 0x55ff29e0a5d0 2023-06-23T18:22:46.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b427fc700 1 -- 172.21.15.2:0/838235736 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0b4400faa0 con 0x55ff29e0a5d0 2023-06-23T18:22:46.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.053+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/838235736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ff29dfcbb0 con 0x55ff29e0a5d0 2023-06-23T18:22:46.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b427fc700 1 -- 172.21.15.2:0/838235736 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b4400fc60 con 0x55ff29e0a5d0 2023-06-23T18:22:46.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b6d2c6b80 10 client.15474 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:46.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/838235736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ff29b88de0 con 0x55ff29e0a5d0 2023-06-23T18:22:46.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b427fc700 1 -- 172.21.15.2:0/838235736 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ff29dfcbb0 con 0x55ff29e0a5d0 2023-06-23T18:22:46.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b427fc700 1 -- 172.21.15.2:0/838235736 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0b4401b020 con 0x55ff29e0a5d0 2023-06-23T18:22:46.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b6d2c6b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:46.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b6d2c6b80 10 client.15474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:46.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b6d2c6b80 20 client.15474 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:46.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ff29e1f3c0 0x55ff29e217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b6d2c6b80 4 client.15474 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:46.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/838235736 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ff29d53eb0 con 0x55ff29e1f3c0 2023-06-23T18:22:46.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.054+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ff29e1f3c0 0x55ff29e217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.055+0000 7f0b427fc700 10 client.15474 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:46.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.055+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ff29e1f3c0 0x55ff29e217d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e369a21b4a806835 server_cookie=d0d1f1f1cb822ca4 in_seq=0 out_seq=0 2023-06-23T18:22:46.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b427fc700 1 -- 172.21.15.2:0/838235736 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ff29d53eb0 con 0x55ff29e1f3c0 2023-06-23T18:22:46.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b427fc700 10 client.15474 handle_command_reply: tid=0 2023-06-23T18:22:46.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 client.15474 shutdown 2023-06-23T18:22:46.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/838235736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e0a5d0 msgr2=0x55ff29e13a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e0a5d0 0x55ff29e13a70 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0b44011a00 tx=0x7f0b44011dc0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/838235736 shutdown_connections 2023-06-23T18:22:46.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ff29e1f3c0 0x55ff29e217d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0b000 0x55ff29e1aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b427fc700 0 client.15474 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:46.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e09c80 0x55ff29e13530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/838235736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e0a5d0 0x55ff29e13a70 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/838235736 >> 172.21.15.2:0/838235736 conn(0x55ff29dff3f0 msgr2=0x55ff29e03c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:46.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/838235736 shutdown_connections 2023-06-23T18:22:46.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/838235736 wait complete. 2023-06-23T18:22:46.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.056+0000 7f0b6d2c6b80 20 client.15474 trim_cache size 0 max 16384 2023-06-23T18:22:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b6d2c6b80 1 Processor -- start 2023-06-23T18:22:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b6d2c6b80 1 -- start start 2023-06-23T18:22:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e1aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e13530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29e129e0 con 0x55ff29e09c80 2023-06-23T18:22:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29e130a0 con 0x55ff29e0b000 2023-06-23T18:22:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29dfef70 con 0x55ff29e0a5d0 2023-06-23T18:22:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b5ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b5ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e12130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54274/0 (socket says 172.21.15.2:54274) 2023-06-23T18:22:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/2236799599 learned_addr learned my addr 172.21.15.2:0/2236799599 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e1aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.071+0000 7f0b5b59e700 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e13530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/2236799599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 msgr2=0x55ff29e1aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:46.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e1aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/2236799599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 msgr2=0x55ff29e13530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:46.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e13530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/2236799599 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff29b71c60 con 0x55ff29e09c80 2023-06-23T18:22:46.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e12130 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00a010 tx=0x7f0b4c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab249f7d3635cddf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:46.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b51ffb700 1 -- 172.21.15.2:0/2236799599 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c02b040 con 0x55ff29e09c80 2023-06-23T18:22:46.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b51ffb700 1 -- 172.21.15.2:0/2236799599 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b4c02c040 con 0x55ff29e09c80 2023-06-23T18:22:46.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.072+0000 7f0b51ffb700 1 -- 172.21.15.2:0/2236799599 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0047f0 con 0x55ff29e09c80 2023-06-23T18:22:46.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/2236799599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 msgr2=0x55ff29e12130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e12130 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00a010 tx=0x7f0b4c0092d0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/2236799599 shutdown_connections 2023-06-23T18:22:46.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e13530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e12130 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/2236799599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e1aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/2236799599 >> 172.21.15.2:0/2236799599 conn(0x55ff29dff3f0 msgr2=0x55ff29e26130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:46.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/2236799599 shutdown_connections 2023-06-23T18:22:46.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/2236799599 wait complete. 2023-06-23T18:22:46.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 -- start start 2023-06-23T18:22:46.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.073+0000 7f0b6d2c6b80 1 Processor -- start 2023-06-23T18:22:46.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e18400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e18940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b6d2c6b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e19950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29e19e90 con 0x55ff29e09c80 2023-06-23T18:22:46.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29e1a000 con 0x55ff29e0b000 2023-06-23T18:22:46.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b6d2c6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff29e1a1c0 con 0x55ff29e0a5d0 2023-06-23T18:22:46.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b6d2c6b80 10 client.0 _ll_register_callbacks cb 0x55ff29d00260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:46.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e18400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e18400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54280/0 (socket says 172.21.15.2:54280) 2023-06-23T18:22:46.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/3052218858 learned_addr learned my addr 172.21.15.2:0/3052218858 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:46.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5b59e700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e19950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e18940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/3052218858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 msgr2=0x55ff29e18940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:46.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e18940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 msgr2=0x55ff29e19950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:46.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e19950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 -- 172.21.15.2:0/3052218858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff29bdc630 con 0x55ff29e09c80 2023-06-23T18:22:46.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e18940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:46.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5b59e700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e19950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:22:46.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b5ad9d700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e18400 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c011040 tx=0x7f0b4c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9085a306a90fd32a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:46.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.074+0000 7f0b427fc700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:46.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.075+0000 7f0b427fc700 1 -- 172.21.15.2:0/3052218858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c02b040 con 0x55ff29e09c80 2023-06-23T18:22:46.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.075+0000 7f0b427fc700 1 -- 172.21.15.2:0/3052218858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0b4c03a0e0 con 0x55ff29e09c80 2023-06-23T18:22:46.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.075+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3052218858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b4c01e040 con 0x55ff29e09c80 2023-06-23T18:22:46.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.075+0000 7f0b427fc700 1 -- 172.21.15.2:0/3052218858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b4c0055e0 con 0x55ff29e09c80 2023-06-23T18:22:46.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.075+0000 7f0b6d2c6b80 10 client.15480 fetch_fsmap learned FSMap version 22 2023-06-23T18:22:46.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.075+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3052218858 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55ff29baf250 con 0x55ff29e09c80 2023-06-23T18:22:46.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.075+0000 7f0b427fc700 1 -- 172.21.15.2:0/3052218858 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0b4c01e040 con 0x55ff29e09c80 2023-06-23T18:22:46.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b427fc700 1 -- 172.21.15.2:0/3052218858 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 22) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0b4c043070 con 0x55ff29e09c80 2023-06-23T18:22:46.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b6d2c6b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 22 2023-06-23T18:22:46.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b6d2c6b80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:46.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b6d2c6b80 20 client.15480 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:46.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ff29e73970 0x55ff29e75d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:46.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b6d2c6b80 4 client.15480 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:22:46.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3052218858 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ff29d16050 con 0x55ff29e73970 2023-06-23T18:22:46.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ff29e73970 0x55ff29e75d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:46.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b427fc700 10 client.15480 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:46.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.076+0000 7f0b5a59c700 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ff29e73970 0x55ff29e75d80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28c04e1e2f911f2 server_cookie=3a1c14b72ab8a273 in_seq=0 out_seq=0 2023-06-23T18:22:46.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b427fc700 1 -- 172.21.15.2:0/3052218858 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ff29d16050 con 0x55ff29e73970 2023-06-23T18:22:46.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b427fc700 10 client.15480 handle_command_reply: tid=0 2023-06-23T18:22:46.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 client.15480 shutdown 2023-06-23T18:22:46.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3052218858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 msgr2=0x55ff29e18400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e18400 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c011040 tx=0x7f0b4c004ee0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3052218858 shutdown_connections 2023-06-23T18:22:46.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ff29e73970 0x55ff29e75d80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ff29e0b000 0x55ff29e19950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ff29e09c80 0x55ff29e18400 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 --2- 172.21.15.2:0/3052218858 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ff29e0a5d0 0x55ff29e18940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3052218858 >> 172.21.15.2:0/3052218858 conn(0x55ff29dff3f0 msgr2=0x55ff29e12130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:46.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b427fc700 0 client.15480 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:46.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3052218858 shutdown_connections 2023-06-23T18:22:46.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 1 -- 172.21.15.2:0/3052218858 wait complete. 2023-06-23T18:22:46.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.077+0000 7f0b6d2c6b80 20 client.15480 trim_cache size 0 max 16384 2023-06-23T18:22:46.097 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:22:46.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 -- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0b480a1150 msgr2=0x7f0b480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0b480a1150 0x7f0b480a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c1072a0 tx=0x7f0b4c005fb0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 -- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c100600 msgr2=0x7f0b5c1a3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:46.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c100600 0x7f0b5c1a3240 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0b54007d70 tx=0x7f0b5400a340 comp rx=0 tx=0).stop 2023-06-23T18:22:46.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 -- 172.21.15.2:0/1839724603 shutdown_connections 2023-06-23T18:22:46.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5c100600 0x7f0b5c1a3240 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0b480a1150 0x7f0b480a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0b5c0fea80 0x7f0b5c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 --2- 172.21.15.2:0/1839724603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b5c0ffc70 0x7f0b5c1a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:46.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 -- 172.21.15.2:0/1839724603 >> 172.21.15.2:0/1839724603 conn(0x7f0b5c0fa0e0 msgr2=0x7f0b5c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:46.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 -- 172.21.15.2:0/1839724603 shutdown_connections 2023-06-23T18:22:46.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:46.078+0000 7f0b61141700 1 -- 172.21.15.2:0/1839724603 wait complete. 2023-06-23T18:22:46.210 INFO:teuthology.orchestra.run.smithi002.stderr:900000 of 50000000 tuples (1%) done (elapsed 9.45 s, remaining 515.63 s) 2023-06-23T18:22:46.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:46 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1556620367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:46.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:46 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1556620367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:46.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:46 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1556620367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:22:47.101 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:22:47.102 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:22:47.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 -- 172.21.15.2:0/857837142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59300fe3f0 msgr2=0x7f5930100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 --2- 172.21.15.2:0/857837142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59300fe3f0 0x7f5930100810 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f592c009990 tx=0x7f592c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 -- 172.21.15.2:0/857837142 shutdown_connections 2023-06-23T18:22:47.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 --2- 172.21.15.2:0/857837142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5930103710 0x7f5930103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 --2- 172.21.15.2:0/857837142 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59300fe3f0 0x7f5930100810 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 --2- 172.21.15.2:0/857837142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5930100d50 0x7f59301031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 -- 172.21.15.2:0/857837142 >> 172.21.15.2:0/857837142 conn(0x7f59300fa0e0 msgr2=0x7f59300fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 -- 172.21.15.2:0/857837142 shutdown_connections 2023-06-23T18:22:47.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.235+0000 7f59384c3700 1 -- 172.21.15.2:0/857837142 wait complete. 2023-06-23T18:22:47.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f59384c3700 1 Processor -- start 2023-06-23T18:22:47.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f59384c3700 1 -- start start 2023-06-23T18:22:47.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f59384c3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59300fe3f0 0x7f593019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f59384c3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5930100d50 0x7f593019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f59384c3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5930103710 0x7f59301a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f59384c3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59301a3980 con 0x7f5930103710 2023-06-23T18:22:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f59384c3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59301a3ac0 con 0x7f59300fe3f0 2023-06-23T18:22:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f59384c3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59301a3c50 con 0x7f5930100d50 2023-06-23T18:22:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f5936a60700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5930103710 0x7f59301a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f593625f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59300fe3f0 0x7f593019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f5936a60700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5930103710 0x7f59301a33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54298/0 (socket says 172.21.15.2:54298) 2023-06-23T18:22:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f593625f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59300fe3f0 0x7f593019be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:44886/0 (socket says 172.21.15.2:44886) 2023-06-23T18:22:47.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f5936a60700 1 -- 172.21.15.2:0/3351289593 learned_addr learned my addr 172.21.15.2:0/3351289593 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f5935a5e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5930100d50 0x7f593019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.236+0000 7f5936a60700 1 -- 172.21.15.2:0/3351289593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5930100d50 msgr2=0x7f593019c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f5936a60700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5930100d50 0x7f593019c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f5936a60700 1 -- 172.21.15.2:0/3351289593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59300fe3f0 msgr2=0x7f593019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f5936a60700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59300fe3f0 0x7f593019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f5936a60700 1 -- 172.21.15.2:0/3351289593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c0095f0 con 0x7f5930103710 2023-06-23T18:22:47.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f5936a60700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5930103710 0x7f59301a33b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f592800ef30 tx=0x7f592800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f59277fe700 1 -- 172.21.15.2:0/3351289593 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928018070 con 0x7f5930103710 2023-06-23T18:22:47.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f59277fe700 1 -- 172.21.15.2:0/3351289593 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f592800f040 con 0x7f5930103710 2023-06-23T18:22:47.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f59384c3700 1 -- 172.21.15.2:0/3351289593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59301a3ed0 con 0x7f5930103710 2023-06-23T18:22:47.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f59277fe700 1 -- 172.21.15.2:0/3351289593 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592800cbc0 con 0x7f5930103710 2023-06-23T18:22:47.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.237+0000 7f59384c3700 1 -- 172.21.15.2:0/3351289593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59301a4350 con 0x7f5930103710 2023-06-23T18:22:47.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.239+0000 7f59277fe700 1 -- 172.21.15.2:0/3351289593 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f592800cd80 con 0x7f5930103710 2023-06-23T18:22:47.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.240+0000 7f59277fe700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f591c0a11d0 0x7f591c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.240+0000 7f593625f700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f591c0a11d0 0x7f591c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.240+0000 7f59277fe700 1 -- 172.21.15.2:0/3351289593 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f59280cc340 con 0x7f5930103710 2023-06-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.240+0000 7f5944648b80 1 Processor -- start 2023-06-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.240+0000 7f593625f700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f591c0a11d0 0x7f591c0a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f592c006010 tx=0x7f592c00b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5944648b80 1 -- start start 2023-06-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5944648b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c37495110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5944648b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c37495af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5944648b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5944648b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37399d30 con 0x560c37495650 2023-06-23T18:22:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5944648b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c373ecac0 con 0x560c37494d00 2023-06-23T18:22:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5944648b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c373e5f50 con 0x560c37496080 2023-06-23T18:22:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c37495af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f593625f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c37495110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c37495af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54304/0 (socket says 172.21.15.2:54304) 2023-06-23T18:22:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f593625f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c37495110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:44894/0 (socket says 172.21.15.2:44894) 2023-06-23T18:22:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 -- 172.21.15.2:0/1582510210 learned_addr learned my addr 172.21.15.2:0/1582510210 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5936a60700 1 --2- 172.21.15.2:0/1582510210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 -- 172.21.15.2:0/1582510210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 msgr2=0x560c374964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 --2- 172.21.15.2:0/1582510210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 -- 172.21.15.2:0/1582510210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 msgr2=0x560c37495110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 --2- 172.21.15.2:0/1582510210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c37495110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 -- 172.21.15.2:0/1582510210 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c372676b0 con 0x560c37495650 2023-06-23T18:22:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.241+0000 7f5935a5e700 1 --2- 172.21.15.2:0/1582510210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c37495af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5920009d80 tx=0x7f5920011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad44e72858374a21 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5924ff9700 1 -- 172.21.15.2:0/1582510210 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920004230 con 0x560c37495650 2023-06-23T18:22:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5924ff9700 1 -- 172.21.15.2:0/1582510210 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5920004390 con 0x560c37495650 2023-06-23T18:22:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5924ff9700 1 -- 172.21.15.2:0/1582510210 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200197e0 con 0x560c37495650 2023-06-23T18:22:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5944648b80 1 -- 172.21.15.2:0/1582510210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 msgr2=0x560c37495af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5944648b80 1 --2- 172.21.15.2:0/1582510210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c37495af0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5920009d80 tx=0x7f5920011720 comp rx=0 tx=0).stop 2023-06-23T18:22:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5944648b80 1 -- 172.21.15.2:0/1582510210 shutdown_connections 2023-06-23T18:22:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5944648b80 1 --2- 172.21.15.2:0/1582510210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5944648b80 1 --2- 172.21.15.2:0/1582510210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c37495110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5944648b80 1 --2- 172.21.15.2:0/1582510210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c37495af0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.242+0000 7f5944648b80 1 -- 172.21.15.2:0/1582510210 >> 172.21.15.2:0/1582510210 conn(0x560c3748a470 msgr2=0x560c3748f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 -- 172.21.15.2:0/1582510210 shutdown_connections 2023-06-23T18:22:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 -- 172.21.15.2:0/1582510210 wait complete. 2023-06-23T18:22:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 -- start start 2023-06-23T18:22:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 Processor -- start 2023-06-23T18:22:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37494d00 0x560c3749e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c3749eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37496080 0x560c374a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37385b40 con 0x560c37494d00 2023-06-23T18:22:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37385840 con 0x560c37496080 2023-06-23T18:22:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c372699a0 con 0x560c37495650 2023-06-23T18:22:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f593625f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37494d00 0x560c3749e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5944648b80 10 client.0 _ll_register_callbacks cb 0x560c3738b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f593625f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37494d00 0x560c3749e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54308/0 (socket says 172.21.15.2:54308) 2023-06-23T18:22:47.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f593625f700 1 -- 172.21.15.2:0/3897129716 learned_addr learned my addr 172.21.15.2:0/3897129716 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5935a5e700 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c3749eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.243+0000 7f5936a60700 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37496080 0x560c374a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5935a5e700 1 -- 172.21.15.2:0/3897129716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37496080 msgr2=0x560c374a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5935a5e700 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37496080 0x560c374a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5935a5e700 1 -- 172.21.15.2:0/3897129716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37494d00 msgr2=0x560c3749e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5935a5e700 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37494d00 0x560c3749e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5935a5e700 1 -- 172.21.15.2:0/3897129716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59200099e0 con 0x560c37495650 2023-06-23T18:22:47.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5935a5e700 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c3749eaf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x560c37495a60 tx=0x7f5920003910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9abd9b11c16940c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5919ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:22:47.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5919ffb700 1 -- 172.21.15.2:0/3897129716 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5920009ed0 con 0x560c37495650 2023-06-23T18:22:47.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5919ffb700 1 -- 172.21.15.2:0/3897129716 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5920004500 con 0x560c37495650 2023-06-23T18:22:47.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5944648b80 1 -- 172.21.15.2:0/3897129716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c37487c30 con 0x560c37495650 2023-06-23T18:22:47.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5919ffb700 1 -- 172.21.15.2:0/3897129716 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5920005250 con 0x560c37495650 2023-06-23T18:22:47.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5944648b80 10 client.25261 fetch_fsmap learned FSMap version 23 2023-06-23T18:22:47.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.244+0000 7f5944648b80 1 -- 172.21.15.2:0/3897129716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560c37213e60 con 0x560c37495650 2023-06-23T18:22:47.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5919ffb700 1 -- 172.21.15.2:0/3897129716 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c37487c30 con 0x560c37495650 2023-06-23T18:22:47.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5919ffb700 1 -- 172.21.15.2:0/3897129716 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 23) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f5920021040 con 0x560c37495650 2023-06-23T18:22:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5944648b80 10 client.25261 fetch_fsmap finished waiting for FSMap version 23 2023-06-23T18:22:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5944648b80 10 client.25261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5944648b80 20 client.25261 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5944648b80 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560c374aa380 0x560c374ac790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5944648b80 4 client.25261 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5944648b80 1 -- 172.21.15.2:0/3897129716 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c373def30 con 0x560c374aa380 2023-06-23T18:22:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f593625f700 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560c374aa380 0x560c374ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f5919ffb700 10 client.25261 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.245+0000 7f593625f700 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560c374aa380 0x560c374ac790 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a2d9fc64a30175 server_cookie=cb857886a1450ba1 in_seq=0 out_seq=0 2023-06-23T18:22:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5919ffb700 1 -- 172.21.15.2:0/3897129716 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c373def30 con 0x560c374aa380 2023-06-23T18:22:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5919ffb700 10 client.25261 handle_command_reply: tid=0 2023-06-23T18:22:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 client.25261 shutdown 2023-06-23T18:22:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 -- 172.21.15.2:0/3897129716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 msgr2=0x560c3749eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c3749eaf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x560c37495a60 tx=0x7f5920003910 comp rx=0 tx=0).stop 2023-06-23T18:22:47.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 -- 172.21.15.2:0/3897129716 shutdown_connections 2023-06-23T18:22:47.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560c374aa380 0x560c374ac790 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37496080 0x560c374a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37494d00 0x560c3749e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 --2- 172.21.15.2:0/3897129716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c3749eaf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5919ffb700 0 client.25261 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:47.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 -- 172.21.15.2:0/3897129716 >> 172.21.15.2:0/3897129716 conn(0x560c3748a470 msgr2=0x560c3748ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 -- 172.21.15.2:0/3897129716 shutdown_connections 2023-06-23T18:22:47.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.246+0000 7f5944648b80 1 -- 172.21.15.2:0/3897129716 wait complete. 2023-06-23T18:22:47.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.247+0000 7f5944648b80 20 client.25261 trim_cache size 0 max 16384 2023-06-23T18:22:47.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.264+0000 7f5944648b80 1 Processor -- start 2023-06-23T18:22:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.264+0000 7f5944648b80 1 -- start start 2023-06-23T18:22:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.264+0000 7f5944648b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c3748e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.264+0000 7f5944648b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c374a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.264+0000 7f5944648b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37496080 0x560c3748e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.264+0000 7f5944648b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c37490890 con 0x560c37496080 2023-06-23T18:22:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.264+0000 7f5944648b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3749da60 con 0x560c37494d00 2023-06-23T18:22:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.264+0000 7f5944648b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3749e120 con 0x560c37495650 2023-06-23T18:22:47.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c3748e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f5936a60700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37496080 0x560c3748e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f5935a5e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c374a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c3748e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:44908/0 (socket says 172.21.15.2:44908) 2023-06-23T18:22:47.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 -- 172.21.15.2:0/869260768 learned_addr learned my addr 172.21.15.2:0/869260768 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 -- 172.21.15.2:0/869260768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 msgr2=0x560c374a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 --2- 172.21.15.2:0/869260768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c374a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 -- 172.21.15.2:0/869260768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37496080 msgr2=0x560c3748e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 --2- 172.21.15.2:0/869260768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37496080 0x560c3748e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 -- 172.21.15.2:0/869260768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c37496720 con 0x560c37494d00 2023-06-23T18:22:47.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f5935a5e700 1 --2- 172.21.15.2:0/869260768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c374a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:47.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f593625f700 1 --2- 172.21.15.2:0/869260768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c3748e440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f592c003f60 tx=0x7f592c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b59c882b18f024e8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f59257fa700 1 -- 172.21.15.2:0/869260768 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c005a00 con 0x560c37494d00 2023-06-23T18:22:47.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f59257fa700 1 -- 172.21.15.2:0/869260768 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f592c01e040 con 0x560c37494d00 2023-06-23T18:22:47.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.265+0000 7f59257fa700 1 -- 172.21.15.2:0/869260768 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c00b710 con 0x560c37494d00 2023-06-23T18:22:47.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 -- 172.21.15.2:0/869260768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 msgr2=0x560c3748e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 --2- 172.21.15.2:0/869260768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c3748e440 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f592c003f60 tx=0x7f592c020000 comp rx=0 tx=0).stop 2023-06-23T18:22:47.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 -- 172.21.15.2:0/869260768 shutdown_connections 2023-06-23T18:22:47.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 --2- 172.21.15.2:0/869260768 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37496080 0x560c3748e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 --2- 172.21.15.2:0/869260768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c3748e440 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 --2- 172.21.15.2:0/869260768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37495650 0x560c374a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 -- 172.21.15.2:0/869260768 >> 172.21.15.2:0/869260768 conn(0x560c3748a470 msgr2=0x560c37498ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 -- 172.21.15.2:0/869260768 shutdown_connections 2023-06-23T18:22:47.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 -- 172.21.15.2:0/869260768 wait complete. 2023-06-23T18:22:47.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 -- start start 2023-06-23T18:22:47.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.266+0000 7f5944648b80 1 Processor -- start 2023-06-23T18:22:47.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5944648b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c374a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5944648b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c374a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5944648b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5944648b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c374a4b40 con 0x560c37495650 2023-06-23T18:22:47.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5944648b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c374a4cb0 con 0x560c37494d00 2023-06-23T18:22:47.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5944648b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c374a4e70 con 0x560c37496080 2023-06-23T18:22:47.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5944648b80 10 client.0 _ll_register_callbacks cb 0x560c3738b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:47.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5935a5e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c374a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f593625f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c374a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5936a60700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5935a5e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c374a35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54324/0 (socket says 172.21.15.2:54324) 2023-06-23T18:22:47.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f593625f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c374a30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:44914/0 (socket says 172.21.15.2:44914) 2023-06-23T18:22:47.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5935a5e700 1 -- 172.21.15.2:0/209739403 learned_addr learned my addr 172.21.15.2:0/209739403 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f593625f700 1 -- 172.21.15.2:0/209739403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 msgr2=0x560c374a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f593625f700 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f593625f700 1 -- 172.21.15.2:0/209739403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 msgr2=0x560c374a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f593625f700 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c374a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f593625f700 1 -- 172.21.15.2:0/209739403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c37213e60 con 0x560c37494d00 2023-06-23T18:22:47.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5936a60700 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:47.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5935a5e700 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c374a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:47.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f593625f700 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c374a30b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f592c009990 tx=0x7f592c005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b20624209a4dc9d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.267+0000 7f5919ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:22:47.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5919ffb700 1 -- 172.21.15.2:0/209739403 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c036d50 con 0x560c37494d00 2023-06-23T18:22:47.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5919ffb700 1 -- 172.21.15.2:0/209739403 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f592c003a60 con 0x560c37494d00 2023-06-23T18:22:47.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 1 -- 172.21.15.2:0/209739403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5920004390 con 0x560c37494d00 2023-06-23T18:22:47.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5919ffb700 1 -- 172.21.15.2:0/209739403 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f592c003c20 con 0x560c37494d00 2023-06-23T18:22:47.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 10 client.25109 fetch_fsmap learned FSMap version 23 2023-06-23T18:22:47.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 1 -- 172.21.15.2:0/209739403 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f59301046b0 con 0x560c37494d00 2023-06-23T18:22:47.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5919ffb700 1 -- 172.21.15.2:0/209739403 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5920004390 con 0x560c37494d00 2023-06-23T18:22:47.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5919ffb700 1 -- 172.21.15.2:0/209739403 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f592c004500 con 0x560c37494d00 2023-06-23T18:22:47.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 10 client.25109 fetch_fsmap finished waiting for FSMap version 23 2023-06-23T18:22:47.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 10 client.25109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:47.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 20 client.25109 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:47.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560c37500600 0x560c37502a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 4 client.25109 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:22:47.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5944648b80 1 -- 172.21.15.2:0/209739403 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560c373a10d0 con 0x560c37500600 2023-06-23T18:22:47.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.268+0000 7f5935a5e700 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560c37500600 0x560c37502a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.269+0000 7f5919ffb700 10 client.25109 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:47.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.269+0000 7f5935a5e700 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560c37500600 0x560c37502a10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1416295325c87ac server_cookie=c6a18e49968619fb in_seq=0 out_seq=0 2023-06-23T18:22:47.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.270+0000 7f5919ffb700 1 -- 172.21.15.2:0/209739403 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560c373a10d0 con 0x560c37500600 2023-06-23T18:22:47.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5919ffb700 10 client.25109 handle_command_reply: tid=0 2023-06-23T18:22:47.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 client.25109 shutdown 2023-06-23T18:22:47.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 -- 172.21.15.2:0/209739403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 msgr2=0x560c374a30b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c374a30b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f592c009990 tx=0x7f592c005a00 comp rx=0 tx=0).stop 2023-06-23T18:22:47.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 -- 172.21.15.2:0/209739403 shutdown_connections 2023-06-23T18:22:47.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560c37500600 0x560c37502a10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c37496080 0x560c374a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560c37494d00 0x560c374a30b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5919ffb700 0 client.25109 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:47.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 --2- 172.21.15.2:0/209739403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560c37495650 0x560c374a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 -- 172.21.15.2:0/209739403 >> 172.21.15.2:0/209739403 conn(0x560c3748a470 msgr2=0x560c3748e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 -- 172.21.15.2:0/209739403 shutdown_connections 2023-06-23T18:22:47.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 1 -- 172.21.15.2:0/209739403 wait complete. 2023-06-23T18:22:47.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f5944648b80 20 client.25109 trim_cache size 0 max 16384 2023-06-23T18:22:47.288 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:47.288 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:22:47.288 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "53442e4e-9202-469e-99be-ad350e61d3a9", 2023-06-23T18:22:47.288 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:22:47.288 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:47.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f59384c3700 1 -- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f591c0a11d0 msgr2=0x7f591c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f59384c3700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f591c0a11d0 0x7f591c0a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f592c006010 tx=0x7f592c00b260 comp rx=0 tx=0).stop 2023-06-23T18:22:47.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f59384c3700 1 -- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5930103710 msgr2=0x7f59301a33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.271+0000 7f59384c3700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5930103710 0x7f59301a33b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f592800ef30 tx=0x7f592800c3a0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.272+0000 7f59384c3700 1 -- 172.21.15.2:0/3351289593 shutdown_connections 2023-06-23T18:22:47.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.272+0000 7f59384c3700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5930103710 0x7f59301a33b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.272+0000 7f59384c3700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f591c0a11d0 0x7f591c0a3680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.272+0000 7f59384c3700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f59300fe3f0 0x7f593019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.272+0000 7f59384c3700 1 --2- 172.21.15.2:0/3351289593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5930100d50 0x7f593019c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.272+0000 7f59384c3700 1 -- 172.21.15.2:0/3351289593 >> 172.21.15.2:0/3351289593 conn(0x7f59300fa0e0 msgr2=0x7f59301025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.272+0000 7f59384c3700 1 -- 172.21.15.2:0/3351289593 shutdown_connections 2023-06-23T18:22:47.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.272+0000 7f59384c3700 1 -- 172.21.15.2:0/3351289593 wait complete. 2023-06-23T18:22:47.291 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:22:47.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:47 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:22:47.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:47 smithi082 ceph-mon[97889]: pgmap v319: 97 pgs: 97 active+clean; 192 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 24 MiB/s wr, 50 op/s 2023-06-23T18:22:47.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:47 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:22:47.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:47 smithi017 ceph-mon[97472]: pgmap v319: 97 pgs: 97 active+clean; 192 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 24 MiB/s wr, 50 op/s 2023-06-23T18:22:47.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 -- 172.21.15.2:0/2132129523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580ffc70 msgr2=0x7fa6581000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 --2- 172.21.15.2:0/2132129523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580ffc70 0x7fa6581000c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa6400098f0 tx=0x7fa640009c00 comp rx=0 tx=0).stop 2023-06-23T18:22:47.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 -- 172.21.15.2:0/2132129523 shutdown_connections 2023-06-23T18:22:47.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 --2- 172.21.15.2:0/2132129523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa658100600 0x7fa658106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 --2- 172.21.15.2:0/2132129523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6580fea80 0x7fa6580fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 --2- 172.21.15.2:0/2132129523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580ffc70 0x7fa6581000c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 -- 172.21.15.2:0/2132129523 >> 172.21.15.2:0/2132129523 conn(0x7fa6580fa0e0 msgr2=0x7fa6580fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 -- 172.21.15.2:0/2132129523 shutdown_connections 2023-06-23T18:22:47.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.442+0000 7fa65dca6700 1 -- 172.21.15.2:0/2132129523 wait complete. 2023-06-23T18:22:47.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.443+0000 7fa65dca6700 1 Processor -- start 2023-06-23T18:22:47.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.443+0000 7fa65dca6700 1 -- start start 2023-06-23T18:22:47.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa65dca6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580fea80 0x7fa658068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa65dca6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6580ffc70 0x7fa65819c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa65dca6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa658100600 0x7fa65819ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa65dca6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65819f550 con 0x7fa6580fea80 2023-06-23T18:22:47.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa65dca6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65819f6c0 con 0x7fa6580ffc70 2023-06-23T18:22:47.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa65dca6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65819f880 con 0x7fa658100600 2023-06-23T18:22:47.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580fea80 0x7fa658068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580fea80 0x7fa658068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54336/0 (socket says 172.21.15.2:54336) 2023-06-23T18:22:47.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa657fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa658100600 0x7fa65819ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa6577fe700 1 -- 172.21.15.2:0/1453302824 learned_addr learned my addr 172.21.15.2:0/1453302824 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa656ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6580ffc70 0x7fa65819c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa657fff700 1 -- 172.21.15.2:0/1453302824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6580ffc70 msgr2=0x7fa65819c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa657fff700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6580ffc70 0x7fa65819c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa657fff700 1 -- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580fea80 msgr2=0x7fa658068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa657fff700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580fea80 0x7fa658068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa657fff700 1 -- 172.21.15.2:0/1453302824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa640009520 con 0x7fa658100600 2023-06-23T18:22:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.444+0000 7fa657fff700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa658100600 0x7fa65819ee70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa65000efc0 tx=0x7fa65000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.445+0000 7fa654ff9700 1 -- 172.21.15.2:0/1453302824 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650018070 con 0x7fa658100600 2023-06-23T18:22:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.445+0000 7fa65dca6700 1 -- 172.21.15.2:0/1453302824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65819fb00 con 0x7fa658100600 2023-06-23T18:22:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.445+0000 7fa654ff9700 1 -- 172.21.15.2:0/1453302824 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa65000f040 con 0x7fa658100600 2023-06-23T18:22:47.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.445+0000 7fa654ff9700 1 -- 172.21.15.2:0/1453302824 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65000cba0 con 0x7fa658100600 2023-06-23T18:22:47.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.445+0000 7fa65dca6700 1 -- 172.21.15.2:0/1453302824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65819ffe0 con 0x7fa658100600 2023-06-23T18:22:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.447+0000 7fa654ff9700 1 -- 172.21.15.2:0/1453302824 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa650007500 con 0x7fa658100600 2023-06-23T18:22:47.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.448+0000 7fa654ff9700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6440a11d0 0x7fa6440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.448+0000 7fa654ff9700 1 -- 172.21.15.2:0/1453302824 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa6500cc450 con 0x7fa658100600 2023-06-23T18:22:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.448+0000 7fa6577fe700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6440a11d0 0x7fa6440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.449+0000 7fa6577fe700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6440a11d0 0x7fa6440a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa648009cf0 tx=0x7fa648009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.449+0000 7fa669e2bb80 1 Processor -- start 2023-06-23T18:22:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.449+0000 7fa669e2bb80 1 -- start start 2023-06-23T18:22:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.449+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417346090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.449+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e417346a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e417347470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41724acb0 con 0x55e417345c80 2023-06-23T18:22:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41729da40 con 0x55e4173465d0 2023-06-23T18:22:47.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e417296ed0 con 0x55e417347000 2023-06-23T18:22:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417346090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417346090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54344/0 (socket says 172.21.15.2:54344) 2023-06-23T18:22:47.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa6577fe700 1 -- 172.21.15.2:0/2438782457 learned_addr learned my addr 172.21.15.2:0/2438782457 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa657fff700 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e417347470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa656ffd700 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e417346a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa656ffd700 1 -- 172.21.15.2:0/2438782457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 msgr2=0x55e417347470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa656ffd700 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e417347470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa656ffd700 1 -- 172.21.15.2:0/2438782457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 msgr2=0x55e417346090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa656ffd700 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417346090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa656ffd700 1 -- 172.21.15.2:0/2438782457 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e417118630 con 0x55e4173465d0 2023-06-23T18:22:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.450+0000 7fa656ffd700 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e417346a70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa640009230 tx=0x7fa640005b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54b5a529703ae36 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.451+0000 7fa64dffb700 1 -- 172.21.15.2:0/2438782457 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64000f660 con 0x55e4173465d0 2023-06-23T18:22:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.451+0000 7fa64dffb700 1 -- 172.21.15.2:0/2438782457 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa64001d040 con 0x55e4173465d0 2023-06-23T18:22:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.451+0000 7fa64dffb700 1 -- 172.21.15.2:0/2438782457 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640005040 con 0x55e4173465d0 2023-06-23T18:22:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.451+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2438782457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 msgr2=0x55e417346a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.451+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e417346a70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa640009230 tx=0x7fa640005b60 comp rx=0 tx=0).stop 2023-06-23T18:22:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2438782457 shutdown_connections 2023-06-23T18:22:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e417347470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417346090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2438782457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e417346a70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2438782457 >> 172.21.15.2:0/2438782457 conn(0x55e41733b3f0 msgr2=0x55e417340520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2438782457 shutdown_connections 2023-06-23T18:22:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2438782457 wait complete. 2023-06-23T18:22:47.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 -- start start 2023-06-23T18:22:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.452+0000 7fa669e2bb80 1 Processor -- start 2023-06-23T18:22:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e41734f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e41734f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e417356a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e417236ac0 con 0x55e417345c80 2023-06-23T18:22:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4172367c0 con 0x55e4173465d0 2023-06-23T18:22:47.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41711a920 con 0x55e417347000 2023-06-23T18:22:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa669e2bb80 10 client.0 _ll_register_callbacks cb 0x55e41723c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e41734f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e41734f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54350/0 (socket says 172.21.15.2:54350) 2023-06-23T18:22:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 -- 172.21.15.2:0/2576159834 learned_addr learned my addr 172.21.15.2:0/2576159834 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa656ffd700 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e41734f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa657fff700 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e417356a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 -- 172.21.15.2:0/2576159834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 msgr2=0x55e417356a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e417356a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 -- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 msgr2=0x55e41734f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e41734f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 -- 172.21.15.2:0/2576159834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6581077f0 con 0x55e417345c80 2023-06-23T18:22:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa656ffd700 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e41734f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa6577fe700 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e41734f4a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa648003930 tx=0x7fa6480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de0707dbc72c7c9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.453+0000 7fa636ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:22:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.454+0000 7fa636ffd700 1 -- 172.21.15.2:0/2576159834 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648022070 con 0x55e417345c80 2023-06-23T18:22:47.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.454+0000 7fa636ffd700 1 -- 172.21.15.2:0/2576159834 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa64800e6e0 con 0x55e417345c80 2023-06-23T18:22:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.454+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2576159834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e417338bb0 con 0x55e417345c80 2023-06-23T18:22:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.454+0000 7fa636ffd700 1 -- 172.21.15.2:0/2576159834 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa64800e8a0 con 0x55e417345c80 2023-06-23T18:22:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.454+0000 7fa669e2bb80 10 client.15510 fetch_fsmap learned FSMap version 23 2023-06-23T18:22:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.454+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2576159834 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e4170c4de0 con 0x55e417345c80 2023-06-23T18:22:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.454+0000 7fa636ffd700 1 -- 172.21.15.2:0/2576159834 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e417338bb0 con 0x55e417345c80 2023-06-23T18:22:47.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.454+0000 7fa636ffd700 1 -- 172.21.15.2:0/2576159834 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 23) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa648027a50 con 0x55e417345c80 2023-06-23T18:22:47.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa669e2bb80 10 client.15510 fetch_fsmap finished waiting for FSMap version 23 2023-06-23T18:22:47.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa669e2bb80 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:47.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa669e2bb80 20 client.15510 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:47.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e41735b250 0x55e41735d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa669e2bb80 4 client.15510 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:22:47.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2576159834 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e41728feb0 con 0x55e41735b250 2023-06-23T18:22:47.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa656ffd700 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e41735b250 0x55e41735d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa636ffd700 10 client.15510 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.455+0000 7fa656ffd700 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e41735b250 0x55e41735d660 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a225df36402ddb1f server_cookie=6ac21076aa871b81 in_seq=0 out_seq=0 2023-06-23T18:22:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.456+0000 7fa636ffd700 1 -- 172.21.15.2:0/2576159834 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e41728feb0 con 0x55e41735b250 2023-06-23T18:22:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.456+0000 7fa636ffd700 10 client.15510 handle_command_reply: tid=0 2023-06-23T18:22:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.456+0000 7fa669e2bb80 1 client.15510 shutdown 2023-06-23T18:22:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2576159834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 msgr2=0x55e41734f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e41734f4a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa648003930 tx=0x7fa6480047c0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2576159834 shutdown_connections 2023-06-23T18:22:47.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e41735b250 0x55e41735d660 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e417356a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e41734f4a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa636ffd700 0 client.15510 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/2576159834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e41734f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2576159834 >> 172.21.15.2:0/2576159834 conn(0x55e41733b3f0 msgr2=0x55e41733fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2576159834 shutdown_connections 2023-06-23T18:22:47.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 1 -- 172.21.15.2:0/2576159834 wait complete. 2023-06-23T18:22:47.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.457+0000 7fa669e2bb80 20 client.15510 trim_cache size 0 max 16384 2023-06-23T18:22:47.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.474+0000 7fa669e2bb80 1 Processor -- start 2023-06-23T18:22:47.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.474+0000 7fa669e2bb80 1 -- start start 2023-06-23T18:22:47.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.474+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417357310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.474+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e4173569e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.474+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e41734f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.474+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41733af70 con 0x55e417345c80 2023-06-23T18:22:47.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.474+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4173411f0 con 0x55e4173465d0 2023-06-23T18:22:47.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.474+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41734aad0 con 0x55e417347000 2023-06-23T18:22:47.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417357310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417357310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54358/0 (socket says 172.21.15.2:54358) 2023-06-23T18:22:47.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 -- 172.21.15.2:0/3901975221 learned_addr learned my addr 172.21.15.2:0/3901975221 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa656ffd700 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e4173569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa657fff700 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e41734f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 -- 172.21.15.2:0/3901975221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 msgr2=0x55e41734f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e41734f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 -- 172.21.15.2:0/3901975221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 msgr2=0x55e4173569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e4173569e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 -- 172.21.15.2:0/3901975221 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4170adc60 con 0x55e417345c80 2023-06-23T18:22:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa656ffd700 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e4173569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:22:47.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa6577fe700 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417357310 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa6480099b0 tx=0x7fa648009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0aacebb19642d4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa64e7fc700 1 -- 172.21.15.2:0/3901975221 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648022030 con 0x55e417345c80 2023-06-23T18:22:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa64e7fc700 1 -- 172.21.15.2:0/3901975221 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa648003f20 con 0x55e417345c80 2023-06-23T18:22:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.475+0000 7fa64e7fc700 1 -- 172.21.15.2:0/3901975221 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa648027b20 con 0x55e417345c80 2023-06-23T18:22:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3901975221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 msgr2=0x55e417357310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417357310 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa6480099b0 tx=0x7fa648009a00 comp rx=0 tx=0).stop 2023-06-23T18:22:47.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3901975221 shutdown_connections 2023-06-23T18:22:47.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417347000 0x55e41734f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e417345c80 0x55e417357310 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3901975221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e4173465d0 0x55e4173569e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3901975221 >> 172.21.15.2:0/3901975221 conn(0x55e41733b3f0 msgr2=0x55e41734e190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3901975221 shutdown_connections 2023-06-23T18:22:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.476+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3901975221 wait complete. 2023-06-23T18:22:47.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 1 -- start start 2023-06-23T18:22:47.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 1 Processor -- start 2023-06-23T18:22:47.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417345c80 0x55e417352c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4173465d0 0x55e417353160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e417347000 0x55e417356180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4173566c0 con 0x55e4173465d0 2023-06-23T18:22:47.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e417356830 con 0x55e417347000 2023-06-23T18:22:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4173569f0 con 0x55e417345c80 2023-06-23T18:22:47.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa669e2bb80 10 client.0 _ll_register_callbacks cb 0x55e41723c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:22:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa657fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e417347000 0x55e417356180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa657fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e417347000 0x55e417356180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:44960/0 (socket says 172.21.15.2:44960) 2023-06-23T18:22:47.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.477+0000 7fa657fff700 1 -- 172.21.15.2:0/3890895737 learned_addr learned my addr 172.21.15.2:0/3890895737 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:22:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa657fff700 1 -- 172.21.15.2:0/3890895737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417345c80 msgr2=0x55e417352c20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:22:47.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa657fff700 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417345c80 0x55e417352c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa657fff700 1 -- 172.21.15.2:0/3890895737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4173465d0 msgr2=0x55e417353160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:22:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa657fff700 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4173465d0 0x55e417353160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa657fff700 1 -- 172.21.15.2:0/3890895737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e417118630 con 0x55e417347000 2023-06-23T18:22:47.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa657fff700 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e417347000 0x55e417356180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa650095db0 tx=0x7fa650004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f665ee256257d638 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:22:47.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa636ffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:22:47.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa636ffd700 1 -- 172.21.15.2:0/3890895737 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650099d10 con 0x55e417347000 2023-06-23T18:22:47.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa636ffd700 1 -- 172.21.15.2:0/3890895737 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa65009d8e0 con 0x55e417347000 2023-06-23T18:22:47.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3890895737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa64001d040 con 0x55e417347000 2023-06-23T18:22:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa636ffd700 1 -- 172.21.15.2:0/3890895737 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6500a1c70 con 0x55e417347000 2023-06-23T18:22:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa669e2bb80 10 client.25127 fetch_fsmap learned FSMap version 23 2023-06-23T18:22:47.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.478+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3890895737 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55e4170c4de0 con 0x55e417347000 2023-06-23T18:22:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa636ffd700 1 -- 172.21.15.2:0/3890895737 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa650063080 con 0x55e417347000 2023-06-23T18:22:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa636ffd700 1 -- 172.21.15.2:0/3890895737 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 23) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa6500a1870 con 0x55e417347000 2023-06-23T18:22:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa669e2bb80 10 client.25127 fetch_fsmap finished waiting for FSMap version 23 2023-06-23T18:22:47.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa669e2bb80 10 client.25127 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:22:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa669e2bb80 20 client.25127 populate_metadata read hostname 'smithi002' 2023-06-23T18:22:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e4173b0460 0x55e4173b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:22:47.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa669e2bb80 4 client.25127 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:22:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3890895737 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e417252050 con 0x55e4173b0460 2023-06-23T18:22:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa6577fe700 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e4173b0460 0x55e4173b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:22:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa636ffd700 10 client.25127 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:47.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa6577fe700 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e4173b0460 0x55e4173b2870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e03bfb3d85cee6 server_cookie=98bb0084d7549da7 in_seq=0 out_seq=0 2023-06-23T18:22:47.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa636ffd700 1 -- 172.21.15.2:0/3890895737 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e417252050 con 0x55e4173b0460 2023-06-23T18:22:47.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa636ffd700 10 client.25127 handle_command_reply: tid=0 2023-06-23T18:22:47.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.479+0000 7fa669e2bb80 1 client.25127 shutdown 2023-06-23T18:22:47.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3890895737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e417347000 msgr2=0x55e417356180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e417347000 0x55e417356180 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa650095db0 tx=0x7fa650004b80 comp rx=0 tx=0).stop 2023-06-23T18:22:47.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3890895737 shutdown_connections 2023-06-23T18:22:47.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e4173b0460 0x55e4173b2870 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e417347000 0x55e417356180 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa636ffd700 0 client.25127 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:22:47.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e417345c80 0x55e417352c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 --2- 172.21.15.2:0/3890895737 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e4173465d0 0x55e417353160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3890895737 >> 172.21.15.2:0/3890895737 conn(0x55e41733b3f0 msgr2=0x55e417357310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3890895737 shutdown_connections 2023-06-23T18:22:47.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 1 -- 172.21.15.2:0/3890895737 wait complete. 2023-06-23T18:22:47.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.480+0000 7fa669e2bb80 20 client.25127 trim_cache size 0 max 16384 2023-06-23T18:22:47.509 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:22:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:22:47.510 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:22:47.511 INFO:teuthology.orchestra.run.smithi002.stdout: "53442e4e-9202-469e-99be-ad350e61d3a9": { 2023-06-23T18:22:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:22:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "53442e4e-9202-469e-99be-ad350e61d3a9", 2023-06-23T18:22:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:22:47.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:22:47.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:22:47.513 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:22:47.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 -- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6440a11d0 msgr2=0x7fa6440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6440a11d0 0x7fa6440a3680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa648009cf0 tx=0x7fa648009310 comp rx=0 tx=0).stop 2023-06-23T18:22:47.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 -- 172.21.15.2:0/1453302824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa658100600 msgr2=0x7fa65819ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:22:47.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa658100600 0x7fa65819ee70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa65000efc0 tx=0x7fa65000c3a0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 -- 172.21.15.2:0/1453302824 shutdown_connections 2023-06-23T18:22:47.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa658100600 0x7fa65819ee70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6580fea80 0x7fa658068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa6440a11d0 0x7fa6440a3680 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 --2- 172.21.15.2:0/1453302824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa6580ffc70 0x7fa65819c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:22:47.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 -- 172.21.15.2:0/1453302824 >> 172.21.15.2:0/1453302824 conn(0x7fa6580fa0e0 msgr2=0x7fa6580fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:22:47.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 -- 172.21.15.2:0/1453302824 shutdown_connections 2023-06-23T18:22:47.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:22:47.481+0000 7fa65dca6700 1 -- 172.21.15.2:0/1453302824 wait complete. 2023-06-23T18:22:47.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:47 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:22:47.519 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:47 smithi002 ceph-mon[95947]: pgmap v319: 97 pgs: 97 active+clean; 192 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 24 MiB/s wr, 50 op/s 2023-06-23T18:22:47.521 INFO:tasks.cephfs.filesystem:scrub status for tag:53442e4e-9202-469e-99be-ad350e61d3a9 - {'path': '/', 'tag': '53442e4e-9202-469e-99be-ad350e61d3a9', 'options': 'recursive,force'} 2023-06-23T18:22:47.640 INFO:teuthology.orchestra.run.smithi002.stderr:1000000 of 50000000 tuples (2%) done (elapsed 10.88 s, remaining 533.19 s) 2023-06-23T18:22:48.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:22:48.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:22:48.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:22:48.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:22:48.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:22:48.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:22:48.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:48 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:22:48.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:22:48.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:22:48.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:22:48.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:22:48.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:22:48.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.443 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:48 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:22:48.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:22:48.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:22:48.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:22:48.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:22:48.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:22:48.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:48 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:22:48.887 INFO:teuthology.orchestra.run.smithi002.stderr:1100000 of 50000000 tuples (2%) done (elapsed 12.13 s, remaining 539.15 s) 2023-06-23T18:22:49.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:49 smithi082 ceph-mon[97889]: pgmap v320: 97 pgs: 97 active+clean; 248 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 32 MiB/s wr, 65 op/s 2023-06-23T18:22:49.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:49 smithi017 ceph-mon[97472]: pgmap v320: 97 pgs: 97 active+clean; 248 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 32 MiB/s wr, 65 op/s 2023-06-23T18:22:49.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:49 smithi002 ceph-mon[95947]: pgmap v320: 97 pgs: 97 active+clean; 248 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 32 MiB/s wr, 65 op/s 2023-06-23T18:22:49.808 INFO:teuthology.orchestra.run.smithi002.stderr:1200000 of 50000000 tuples (2%) done (elapsed 13.05 s, remaining 530.67 s) 2023-06-23T18:22:51.047 INFO:teuthology.orchestra.run.smithi002.stderr:1300000 of 50000000 tuples (2%) done (elapsed 14.29 s, remaining 535.27 s) 2023-06-23T18:22:51.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:51 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:22:51.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:51 smithi082 ceph-mon[97889]: pgmap v321: 97 pgs: 97 active+clean; 272 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 35 MiB/s wr, 58 op/s 2023-06-23T18:22:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:51 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:22:51.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:51 smithi017 ceph-mon[97472]: pgmap v321: 97 pgs: 97 active+clean; 272 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 35 MiB/s wr, 58 op/s 2023-06-23T18:22:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:51 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:22:52.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:51 smithi002 ceph-mon[95947]: pgmap v321: 97 pgs: 97 active+clean; 272 MiB data, 463 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 35 MiB/s wr, 58 op/s 2023-06-23T18:22:52.306 INFO:teuthology.orchestra.run.smithi002.stderr:1400000 of 50000000 tuples (2%) done (elapsed 15.55 s, remaining 539.70 s) 2023-06-23T18:22:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:52 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:22:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:52 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:22:53.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:52 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:22:53.197 INFO:teuthology.orchestra.run.smithi002.stderr:1500000 of 50000000 tuples (3%) done (elapsed 16.44 s, remaining 531.49 s) 2023-06-23T18:22:53.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:53 smithi017 ceph-mon[97472]: pgmap v322: 97 pgs: 97 active+clean; 320 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:22:54.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:53 smithi002 ceph-mon[95947]: pgmap v322: 97 pgs: 97 active+clean; 320 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:22:54.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:53 smithi082 ceph-mon[97889]: pgmap v322: 97 pgs: 97 active+clean; 320 MiB data, 482 MiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:22:54.501 INFO:teuthology.orchestra.run.smithi002.stderr:1600000 of 50000000 tuples (3%) done (elapsed 17.74 s, remaining 536.68 s) 2023-06-23T18:22:55.443 INFO:teuthology.orchestra.run.smithi002.stderr:1700000 of 50000000 tuples (3%) done (elapsed 18.68 s, remaining 530.87 s) 2023-06-23T18:22:56.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: pgmap v323: 97 pgs: 97 active+clean; 364 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 38 MiB/s wr, 49 op/s 2023-06-23T18:22:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:22:56.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:22:56.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:55 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:22:56.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: pgmap v323: 97 pgs: 97 active+clean; 364 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 38 MiB/s wr, 49 op/s 2023-06-23T18:22:56.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:22:56.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:22:56.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:55 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:22:56.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: pgmap v323: 97 pgs: 97 active+clean; 364 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 38 MiB/s wr, 49 op/s 2023-06-23T18:22:56.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:22:56.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:22:56.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:22:56.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:55 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:22:56.660 INFO:teuthology.orchestra.run.smithi002.stderr:1800000 of 50000000 tuples (3%) done (elapsed 19.90 s, remaining 532.92 s) 2023-06-23T18:22:57.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:56 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:22:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:56 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:22:57.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:56 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:22:57.926 INFO:teuthology.orchestra.run.smithi002.stderr:1900000 of 50000000 tuples (3%) done (elapsed 21.17 s, remaining 535.87 s) 2023-06-23T18:22:58.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:57 smithi002 ceph-mon[95947]: pgmap v324: 97 pgs: 97 active+clean; 408 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 40 MiB/s wr, 46 op/s 2023-06-23T18:22:58.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:57 smithi082 ceph-mon[97889]: pgmap v324: 97 pgs: 97 active+clean; 408 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 40 MiB/s wr, 46 op/s 2023-06-23T18:22:58.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:57 smithi017 ceph-mon[97472]: pgmap v324: 97 pgs: 97 active+clean; 408 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 40 MiB/s wr, 46 op/s 2023-06-23T18:22:58.757 INFO:teuthology.orchestra.run.smithi002.stderr:2000000 of 50000000 tuples (4%) done (elapsed 22.00 s, remaining 527.96 s) 2023-06-23T18:22:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:22:58 smithi082 ceph-mon[97889]: pgmap v325: 97 pgs: 97 active+clean; 461 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 42 MiB/s wr, 51 op/s 2023-06-23T18:22:59.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:22:58 smithi017 ceph-mon[97472]: pgmap v325: 97 pgs: 97 active+clean; 461 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 42 MiB/s wr, 51 op/s 2023-06-23T18:22:59.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:22:58 smithi002 ceph-mon[95947]: pgmap v325: 97 pgs: 97 active+clean; 461 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 42 MiB/s wr, 51 op/s 2023-06-23T18:23:00.027 INFO:teuthology.orchestra.run.smithi002.stderr:2100000 of 50000000 tuples (4%) done (elapsed 23.27 s, remaining 530.74 s) 2023-06-23T18:23:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:00 smithi082 ceph-mon[97889]: pgmap v326: 97 pgs: 97 active+clean; 477 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 MiB/s wr, 38 op/s 2023-06-23T18:23:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:00 smithi017 ceph-mon[97472]: pgmap v326: 97 pgs: 97 active+clean; 477 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 MiB/s wr, 38 op/s 2023-06-23T18:23:01.268 INFO:teuthology.orchestra.run.smithi002.stderr:2200000 of 50000000 tuples (4%) done (elapsed 24.51 s, remaining 532.52 s) 2023-06-23T18:23:01.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:00 smithi002 ceph-mon[95947]: pgmap v326: 97 pgs: 97 active+clean; 477 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 37 MiB/s wr, 38 op/s 2023-06-23T18:23:02.161 INFO:teuthology.orchestra.run.smithi002.stderr:2300000 of 50000000 tuples (4%) done (elapsed 25.40 s, remaining 526.83 s) 2023-06-23T18:23:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:02 smithi082 ceph-mon[97889]: pgmap v327: 97 pgs: 97 active+clean; 537 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 44 MiB/s wr, 36 op/s 2023-06-23T18:23:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:02 smithi017 ceph-mon[97472]: pgmap v327: 97 pgs: 97 active+clean; 537 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 44 MiB/s wr, 36 op/s 2023-06-23T18:23:03.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:02 smithi002 ceph-mon[95947]: pgmap v327: 97 pgs: 97 active+clean; 537 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 44 MiB/s wr, 36 op/s 2023-06-23T18:23:03.424 INFO:teuthology.orchestra.run.smithi002.stderr:2400000 of 50000000 tuples (4%) done (elapsed 26.67 s, remaining 528.86 s) 2023-06-23T18:23:04.674 INFO:teuthology.orchestra.run.smithi002.stderr:2500000 of 50000000 tuples (5%) done (elapsed 27.91 s, remaining 530.38 s) 2023-06-23T18:23:05.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:05 smithi082 ceph-mon[97889]: pgmap v328: 97 pgs: 97 active+clean; 565 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 MiB/s wr, 34 op/s 2023-06-23T18:23:05.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:05 smithi017 ceph-mon[97472]: pgmap v328: 97 pgs: 97 active+clean; 565 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 MiB/s wr, 34 op/s 2023-06-23T18:23:05.545 INFO:teuthology.orchestra.run.smithi002.stderr:2600000 of 50000000 tuples (5%) done (elapsed 28.79 s, remaining 524.80 s) 2023-06-23T18:23:05.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:05 smithi002 ceph-mon[95947]: pgmap v328: 97 pgs: 97 active+clean; 565 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 40 MiB/s wr, 34 op/s 2023-06-23T18:23:06.761 INFO:teuthology.orchestra.run.smithi002.stderr:2700000 of 50000000 tuples (5%) done (elapsed 30.00 s, remaining 525.60 s) 2023-06-23T18:23:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:06 smithi082 ceph-mon[97889]: pgmap v329: 97 pgs: 97 active+clean; 613 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 29 op/s 2023-06-23T18:23:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:06 smithi017 ceph-mon[97472]: pgmap v329: 97 pgs: 97 active+clean; 613 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 29 op/s 2023-06-23T18:23:07.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:06 smithi002 ceph-mon[95947]: pgmap v329: 97 pgs: 97 active+clean; 613 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 40 MiB/s wr, 29 op/s 2023-06-23T18:23:08.056 INFO:teuthology.orchestra.run.smithi002.stderr:2800000 of 50000000 tuples (5%) done (elapsed 31.30 s, remaining 527.58 s) 2023-06-23T18:23:08.907 INFO:teuthology.orchestra.run.smithi002.stderr:2900000 of 50000000 tuples (5%) done (elapsed 32.15 s, remaining 522.13 s) 2023-06-23T18:23:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:08 smithi082 ceph-mon[97889]: pgmap v330: 97 pgs: 97 active+clean; 653 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:08 smithi017 ceph-mon[97472]: pgmap v330: 97 pgs: 97 active+clean; 653 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:09.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:08 smithi002 ceph-mon[95947]: pgmap v330: 97 pgs: 97 active+clean; 653 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:10.183 INFO:teuthology.orchestra.run.smithi002.stderr:3000000 of 50000000 tuples (6%) done (elapsed 33.42 s, remaining 523.64 s) 2023-06-23T18:23:11.076 INFO:teuthology.orchestra.run.smithi002.stderr:3100000 of 50000000 tuples (6%) done (elapsed 34.32 s, remaining 519.19 s) 2023-06-23T18:23:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:10 smithi082 ceph-mon[97889]: pgmap v331: 97 pgs: 97 active+clean; 669 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-06-23T18:23:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:10 smithi017 ceph-mon[97472]: pgmap v331: 97 pgs: 97 active+clean; 669 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-06-23T18:23:11.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:10 smithi002 ceph-mon[95947]: pgmap v331: 97 pgs: 97 active+clean; 669 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-06-23T18:23:12.330 INFO:teuthology.orchestra.run.smithi002.stderr:3200000 of 50000000 tuples (6%) done (elapsed 35.57 s, remaining 520.23 s) 2023-06-23T18:23:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:12 smithi082 ceph-mon[97889]: pgmap v332: 97 pgs: 97 active+clean; 749 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 47 MiB/s wr, 32 op/s 2023-06-23T18:23:13.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:12 smithi017 ceph-mon[97472]: pgmap v332: 97 pgs: 97 active+clean; 749 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 47 MiB/s wr, 32 op/s 2023-06-23T18:23:13.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:12 smithi002 ceph-mon[95947]: pgmap v332: 97 pgs: 97 active+clean; 749 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 47 MiB/s wr, 32 op/s 2023-06-23T18:23:13.549 INFO:teuthology.orchestra.run.smithi002.stderr:3300000 of 50000000 tuples (6%) done (elapsed 36.79 s, remaining 520.64 s) 2023-06-23T18:23:14.400 INFO:teuthology.orchestra.run.smithi002.stderr:3400000 of 50000000 tuples (6%) done (elapsed 37.64 s, remaining 515.91 s) 2023-06-23T18:23:15.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:15 smithi082 ceph-mon[97889]: pgmap v333: 97 pgs: 97 active+clean; 778 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:15.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:15 smithi017 ceph-mon[97472]: pgmap v333: 97 pgs: 97 active+clean; 778 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:15.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:15 smithi002 ceph-mon[95947]: pgmap v333: 97 pgs: 97 active+clean; 778 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:15.648 INFO:teuthology.orchestra.run.smithi002.stderr:3500000 of 50000000 tuples (7%) done (elapsed 38.89 s, remaining 516.67 s) 2023-06-23T18:23:16.899 INFO:teuthology.orchestra.run.smithi002.stderr:3600000 of 50000000 tuples (7%) done (elapsed 40.14 s, remaining 517.36 s) 2023-06-23T18:23:17.522 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:23:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 -- 172.21.15.2:0/2956394259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 msgr2=0x7fdd1c0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 --2- 172.21.15.2:0/2956394259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 0x7fdd1c0feea0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009990 tx=0x7fdd0c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 -- 172.21.15.2:0/2956394259 shutdown_connections 2023-06-23T18:23:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 --2- 172.21.15.2:0/2956394259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c100610 0x7fdd1c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 --2- 172.21.15.2:0/2956394259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 0x7fdd1c0feea0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 --2- 172.21.15.2:0/2956394259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd1c0ffc80 0x7fdd1c1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 -- 172.21.15.2:0/2956394259 >> 172.21.15.2:0/2956394259 conn(0x7fdd1c0fa090 msgr2=0x7fdd1c0fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 -- 172.21.15.2:0/2956394259 shutdown_connections 2023-06-23T18:23:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.677+0000 7fdd20a99700 1 -- 172.21.15.2:0/2956394259 wait complete. 2023-06-23T18:23:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd20a99700 1 Processor -- start 2023-06-23T18:23:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd20a99700 1 -- start start 2023-06-23T18:23:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd20a99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 0x7fdd1c1946f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd20a99700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd1c0ffc80 0x7fdd1c194c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd20a99700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c100610 0x7fdd1c19bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd20a99700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19c270 con 0x7fdd1c0fea90 2023-06-23T18:23:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd20a99700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19c3b0 con 0x7fdd1c100610 2023-06-23T18:23:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd20a99700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19c540 con 0x7fdd1c0ffc80 2023-06-23T18:23:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd1a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 0x7fdd1c1946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd1a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 0x7fdd1c1946f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54376/0 (socket says 172.21.15.2:54376) 2023-06-23T18:23:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd1a59c700 1 -- 172.21.15.2:0/682908944 learned_addr learned my addr 172.21.15.2:0/682908944 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.678+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd1c0ffc80 0x7fdd1c194c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd1ad9d700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c100610 0x7fdd1c19bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd1a59c700 1 -- 172.21.15.2:0/682908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd1c0ffc80 msgr2=0x7fdd1c194c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd1c0ffc80 0x7fdd1c194c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd1a59c700 1 -- 172.21.15.2:0/682908944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c100610 msgr2=0x7fdd1c19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c100610 0x7fdd1c19bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd1a59c700 1 -- 172.21.15.2:0/682908944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0095f0 con 0x7fdd1c0fea90 2023-06-23T18:23:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 0x7fdd1c1946f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c003a20 tx=0x7fdd0c003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd0b7fe700 1 -- 172.21.15.2:0/682908944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c01f070 con 0x7fdd1c0fea90 2023-06-23T18:23:17.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd0b7fe700 1 -- 172.21.15.2:0/682908944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd0c00bdf0 con 0x7fdd1c0fea90 2023-06-23T18:23:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd20a99700 1 -- 172.21.15.2:0/682908944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c0684a0 con 0x7fdd1c0fea90 2023-06-23T18:23:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd0b7fe700 1 -- 172.21.15.2:0/682908944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c00f460 con 0x7fdd1c0fea90 2023-06-23T18:23:17.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.679+0000 7fdd20a99700 1 -- 172.21.15.2:0/682908944 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c068950 con 0x7fdd1c0fea90 2023-06-23T18:23:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.681+0000 7fdd0b7fe700 1 -- 172.21.15.2:0/682908944 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdd0c019800 con 0x7fdd1c0fea90 2023-06-23T18:23:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.682+0000 7fdd0b7fe700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd040a1150 0x7fdd040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.682+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd040a1150 0x7fdd040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.682+0000 7fdd0b7fe700 1 -- 172.21.15.2:0/682908944 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdd0c0cfea0 con 0x7fdd1c0fea90 2023-06-23T18:23:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.683+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd040a1150 0x7fdd040a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005950 tx=0x7fdd1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.683+0000 7fdd2cc1eb80 1 Processor -- start 2023-06-23T18:23:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd2cc1eb80 1 -- start start 2023-06-23T18:23:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27cd7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd75d0 0x559b27cd7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd8000 0x559b27cd8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27bdbcb0 con 0x559b27cd75d0 2023-06-23T18:23:17.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27c2ea40 con 0x559b27cd8000 2023-06-23T18:23:17.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27c27ed0 con 0x559b27cd6c80 2023-06-23T18:23:17.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd19d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd75d0 0x559b27cd7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd19d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd75d0 0x559b27cd7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54388/0 (socket says 172.21.15.2:54388) 2023-06-23T18:23:17.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd19d9b700 1 -- 172.21.15.2:0/371091750 learned_addr learned my addr 172.21.15.2:0/371091750 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:17.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd1ad9d700 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd8000 0x559b27cd8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27cd7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd19d9b700 1 -- 172.21.15.2:0/371091750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 msgr2=0x559b27cd7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27cd7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd19d9b700 1 -- 172.21.15.2:0/371091750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd8000 msgr2=0x559b27cd8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd8000 0x559b27cd8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.684+0000 7fdd19d9b700 1 -- 172.21.15.2:0/371091750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b27aa9630 con 0x559b27cd75d0 2023-06-23T18:23:17.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.685+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd75d0 0x559b27cd7a70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdd10010fa0 tx=0x7fdd10013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4014a5611a18b3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.685+0000 7fdd08ff9700 1 -- 172.21.15.2:0/371091750 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd10011db0 con 0x559b27cd75d0 2023-06-23T18:23:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.685+0000 7fdd08ff9700 1 -- 172.21.15.2:0/371091750 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd10015070 con 0x559b27cd75d0 2023-06-23T18:23:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.685+0000 7fdd08ff9700 1 -- 172.21.15.2:0/371091750 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1001c610 con 0x559b27cd75d0 2023-06-23T18:23:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.685+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/371091750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd75d0 msgr2=0x559b27cd7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.685+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd75d0 0x559b27cd7a70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdd10010fa0 tx=0x7fdd10013000 comp rx=0 tx=0).stop 2023-06-23T18:23:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/371091750 shutdown_connections 2023-06-23T18:23:17.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd8000 0x559b27cd8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27cd7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/371091750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd75d0 0x559b27cd7a70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/371091750 >> 172.21.15.2:0/371091750 conn(0x559b27ccc3f0 msgr2=0x559b27cd1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/371091750 shutdown_connections 2023-06-23T18:23:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/371091750 wait complete. 2023-06-23T18:23:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 -- start start 2023-06-23T18:23:17.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 Processor -- start 2023-06-23T18:23:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.686+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd6c80 0x559b27ce0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 0x559b27ce0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd8000 0x559b27ce7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27bc7ac0 con 0x559b27cd6c80 2023-06-23T18:23:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27bc77c0 con 0x559b27cd75d0 2023-06-23T18:23:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27aab920 con 0x559b27cd8000 2023-06-23T18:23:17.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd2cc1eb80 10 client.0 _ll_register_callbacks cb 0x559b27bcd260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd6c80 0x559b27ce0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd6c80 0x559b27ce0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54390/0 (socket says 172.21.15.2:54390) 2023-06-23T18:23:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 -- 172.21.15.2:0/3317093684 learned_addr learned my addr 172.21.15.2:0/3317093684 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1ad9d700 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd8000 0x559b27ce7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 0x559b27ce0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 -- 172.21.15.2:0/3317093684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd8000 msgr2=0x559b27ce7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:17.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd8000 0x559b27ce7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 -- 172.21.15.2:0/3317093684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 msgr2=0x559b27ce0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 0x559b27ce0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 -- 172.21.15.2:0/3317093684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd10010c50 con 0x559b27cd6c80 2023-06-23T18:23:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd6c80 0x559b27ce0500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c09d040 tx=0x7fdd0c09d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190b80b7203176e3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd01ffb700 1 -- 172.21.15.2:0/3317093684 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c019600 con 0x559b27cd6c80 2023-06-23T18:23:17.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd01ffb700 1 -- 172.21.15.2:0/3317093684 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdd0c09d340 con 0x559b27cd6c80 2023-06-23T18:23:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.687+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3317093684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b27cc9bb0 con 0x559b27cd6c80 2023-06-23T18:23:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd01ffb700 1 -- 172.21.15.2:0/3317093684 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd0c0a1370 con 0x559b27cd6c80 2023-06-23T18:23:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd2cc1eb80 10 client.15525 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3317093684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559b27a55de0 con 0x559b27cd6c80 2023-06-23T18:23:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd01ffb700 1 -- 172.21.15.2:0/3317093684 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559b27cc9bb0 con 0x559b27cd6c80 2023-06-23T18:23:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd01ffb700 1 -- 172.21.15.2:0/3317093684 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdd0c0a5390 con 0x559b27cd6c80 2023-06-23T18:23:17.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd2cc1eb80 10 client.15525 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd2cc1eb80 10 client.15525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd2cc1eb80 20 client.15525 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559b27cebe10 0x559b27cee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd2cc1eb80 4 client.15525 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:23:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3317093684 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b27c20eb0 con 0x559b27cebe10 2023-06-23T18:23:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.688+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559b27cebe10 0x559b27cee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.689+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559b27cebe10 0x559b27cee220 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65720ac1ae79d0bc server_cookie=a9d703a19f056c96 in_seq=0 out_seq=0 2023-06-23T18:23:17.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.689+0000 7fdd01ffb700 10 client.15525 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:17.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd01ffb700 1 -- 172.21.15.2:0/3317093684 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559b27c20eb0 con 0x559b27cebe10 2023-06-23T18:23:17.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd01ffb700 10 client.15525 handle_command_reply: tid=0 2023-06-23T18:23:17.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 client.15525 shutdown 2023-06-23T18:23:17.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3317093684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd6c80 msgr2=0x559b27ce0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd6c80 0x559b27ce0500 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c09d040 tx=0x7fdd0c09d880 comp rx=0 tx=0).stop 2023-06-23T18:23:17.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3317093684 shutdown_connections 2023-06-23T18:23:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559b27cebe10 0x559b27cee220 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd8000 0x559b27ce7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd01ffb700 0 client.15525 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd6c80 0x559b27ce0500 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3317093684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 0x559b27ce0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3317093684 >> 172.21.15.2:0/3317093684 conn(0x559b27ccc3f0 msgr2=0x559b27cd0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:17.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3317093684 shutdown_connections 2023-06-23T18:23:17.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3317093684 wait complete. 2023-06-23T18:23:17.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.690+0000 7fdd2cc1eb80 20 client.15525 trim_cache size 0 max 16384 2023-06-23T18:23:17.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd2cc1eb80 1 Processor -- start 2023-06-23T18:23:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd2cc1eb80 1 -- start start 2023-06-23T18:23:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd6c80 0x559b27cf2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd75d0 0x559b27ce7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27ce00d0 con 0x559b27cd8000 2023-06-23T18:23:17.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27cdec40 con 0x559b27cd6c80 2023-06-23T18:23:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27cd2810 con 0x559b27cd75d0 2023-06-23T18:23:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd1ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd1ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54400/0 (socket says 172.21.15.2:54400) 2023-06-23T18:23:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd1ad9d700 1 -- 172.21.15.2:0/3009903244 learned_addr learned my addr 172.21.15.2:0/3009903244 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd6c80 0x559b27cf2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd75d0 0x559b27ce7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd1ad9d700 1 -- 172.21.15.2:0/3009903244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd75d0 msgr2=0x559b27ce7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:17.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.712+0000 7fdd1ad9d700 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd75d0 0x559b27ce7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.713+0000 7fdd1ad9d700 1 -- 172.21.15.2:0/3009903244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd6c80 msgr2=0x559b27cf2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:17.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.713+0000 7fdd1ad9d700 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd6c80 0x559b27cf2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.713+0000 7fdd1ad9d700 1 -- 172.21.15.2:0/3009903244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b27a3ec60 con 0x559b27cd8000 2023-06-23T18:23:17.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.713+0000 7fdd1ad9d700 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce8370 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdd14011960 tx=0x7fdd14009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e49b84ddad6a35 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:17.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.713+0000 7fdd097fa700 1 -- 172.21.15.2:0/3009903244 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400e040 con 0x559b27cd8000 2023-06-23T18:23:17.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.713+0000 7fdd097fa700 1 -- 172.21.15.2:0/3009903244 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd1400f040 con 0x559b27cd8000 2023-06-23T18:23:17.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.713+0000 7fdd097fa700 1 -- 172.21.15.2:0/3009903244 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd14017c60 con 0x559b27cd8000 2023-06-23T18:23:17.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3009903244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 msgr2=0x559b27ce8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce8370 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdd14011960 tx=0x7fdd14009920 comp rx=0 tx=0).stop 2023-06-23T18:23:17.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3009903244 shutdown_connections 2023-06-23T18:23:17.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce8370 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd6c80 0x559b27cf2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/3009903244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd75d0 0x559b27ce7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3009903244 >> 172.21.15.2:0/3009903244 conn(0x559b27ccc3f0 msgr2=0x559b27ccd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3009903244 shutdown_connections 2023-06-23T18:23:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.714+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/3009903244 wait complete. 2023-06-23T18:23:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 1 -- start start 2023-06-23T18:23:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 1 Processor -- start 2023-06-23T18:23:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27ce7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 0x559b27ce78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27ce4430 con 0x559b27cd8000 2023-06-23T18:23:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27ce45a0 con 0x559b27cd75d0 2023-06-23T18:23:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b27ce4760 con 0x559b27cd6c80 2023-06-23T18:23:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd2cc1eb80 10 client.0 _ll_register_callbacks cb 0x559b27bcd260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd1a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27ce7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd1ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd1a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27ce7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:54742/0 (socket says 172.21.15.2:54742) 2023-06-23T18:23:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.715+0000 7fdd1a59c700 1 -- 172.21.15.2:0/2841908334 learned_addr learned my addr 172.21.15.2:0/2841908334 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd1a59c700 1 -- 172.21.15.2:0/2841908334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 msgr2=0x559b27ce78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 0x559b27ce78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd1a59c700 1 -- 172.21.15.2:0/2841908334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 msgr2=0x559b27ce3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd1a59c700 1 -- 172.21.15.2:0/2841908334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd14011610 con 0x559b27cd6c80 2023-06-23T18:23:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd1a59c700 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27ce7370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0997f0 tx=0x7fdd0c0a54a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6651b40962c2a5b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:23:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd01ffb700 1 -- 172.21.15.2:0/2841908334 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c00bb60 con 0x559b27cd6c80 2023-06-23T18:23:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd01ffb700 1 -- 172.21.15.2:0/2841908334 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdd0c099480 con 0x559b27cd6c80 2023-06-23T18:23:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/2841908334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd10015070 con 0x559b27cd6c80 2023-06-23T18:23:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd01ffb700 1 -- 172.21.15.2:0/2841908334 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd0c086d00 con 0x559b27cd6c80 2023-06-23T18:23:17.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd2cc1eb80 10 client.25300 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:17.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.716+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/2841908334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x559b27a7c250 con 0x559b27cd6c80 2023-06-23T18:23:17.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd01ffb700 1 -- 172.21.15.2:0/2841908334 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd0c0cb080 con 0x559b27cd6c80 2023-06-23T18:23:17.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd01ffb700 1 -- 172.21.15.2:0/2841908334 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdd0c09c2a0 con 0x559b27cd6c80 2023-06-23T18:23:17.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd2cc1eb80 10 client.25300 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:17.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd2cc1eb80 10 client.25300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:17.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd2cc1eb80 20 client.25300 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:17.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559b27d84db0 0x559b27d871c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd2cc1eb80 4 client.25300 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:23:17.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/2841908334 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b27be3050 con 0x559b27d84db0 2023-06-23T18:23:17.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.717+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559b27d84db0 0x559b27d871c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd01ffb700 10 client.25300 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:17.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd19d9b700 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559b27d84db0 0x559b27d871c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba9ea3b71fc92ea server_cookie=5958c2c3bf53fae7 in_seq=0 out_seq=0 2023-06-23T18:23:17.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd01ffb700 1 -- 172.21.15.2:0/2841908334 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559b27be3050 con 0x559b27d84db0 2023-06-23T18:23:17.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd01ffb700 10 client.25300 handle_command_reply: tid=0 2023-06-23T18:23:17.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 client.25300 shutdown 2023-06-23T18:23:17.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/2841908334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 msgr2=0x559b27ce7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27ce7370 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0997f0 tx=0x7fdd0c0a54a0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/2841908334 shutdown_connections 2023-06-23T18:23:17.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559b27d84db0 0x559b27d871c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559b27cd8000 0x559b27ce3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b27cd6c80 0x559b27ce7370 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd01ffb700 0 client.25300 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:17.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 --2- 172.21.15.2:0/2841908334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559b27cd75d0 0x559b27ce78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/2841908334 >> 172.21.15.2:0/2841908334 conn(0x559b27ccc3f0 msgr2=0x559b27cf2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:17.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/2841908334 shutdown_connections 2023-06-23T18:23:17.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 1 -- 172.21.15.2:0/2841908334 wait complete. 2023-06-23T18:23:17.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.718+0000 7fdd2cc1eb80 20 client.25300 trim_cache size 0 max 16384 2023-06-23T18:23:17.736 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:23:17.736 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:23:17.736 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:23:17.736 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:23:17.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 -- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd040a1150 msgr2=0x7fdd040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd040a1150 0x7fdd040a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd10005950 tx=0x7fdd1000b410 comp rx=0 tx=0).stop 2023-06-23T18:23:17.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 -- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 msgr2=0x7fdd1c1946f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 0x7fdd1c1946f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c003a20 tx=0x7fdd0c003a50 comp rx=0 tx=0).stop 2023-06-23T18:23:17.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 -- 172.21.15.2:0/682908944 shutdown_connections 2023-06-23T18:23:17.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdd040a1150 0x7fdd040a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd1c100610 0x7fdd1c19bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdd1c0fea90 0x7fdd1c1946f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 --2- 172.21.15.2:0/682908944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd1c0ffc80 0x7fdd1c194c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 -- 172.21.15.2:0/682908944 >> 172.21.15.2:0/682908944 conn(0x7fdd1c0fa090 msgr2=0x7fdd1c0fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 -- 172.21.15.2:0/682908944 shutdown_connections 2023-06-23T18:23:17.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.719+0000 7fdd20a99700 1 -- 172.21.15.2:0/682908944 wait complete. 2023-06-23T18:23:17.740 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:23:17.740 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:23:17.794 INFO:teuthology.orchestra.run.smithi002.stderr:3700000 of 50000000 tuples (7%) done (elapsed 41.04 s, remaining 513.50 s) 2023-06-23T18:23:17.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 -- 172.21.15.2:0/624791125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 msgr2=0x7f78b00fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 --2- 172.21.15.2:0/624791125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 0x7f78b00fee70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f78ac009990 tx=0x7f78ac009ca0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 -- 172.21.15.2:0/624791125 shutdown_connections 2023-06-23T18:23:17.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 --2- 172.21.15.2:0/624791125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b01005e0 0x7f78b0106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 --2- 172.21.15.2:0/624791125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 0x7f78b00fee70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 --2- 172.21.15.2:0/624791125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78b00ffc50 0x7f78b01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 -- 172.21.15.2:0/624791125 >> 172.21.15.2:0/624791125 conn(0x7f78b00fa0e0 msgr2=0x7f78b00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 -- 172.21.15.2:0/624791125 shutdown_connections 2023-06-23T18:23:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 -- 172.21.15.2:0/624791125 wait complete. 2023-06-23T18:23:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 Processor -- start 2023-06-23T18:23:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.875+0000 7f78b856d700 1 -- start start 2023-06-23T18:23:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b856d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 0x7f78b0194650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b856d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b00ffc50 0x7f78b0194b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b856d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78b01005e0 0x7f78b019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b856d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b019c1d0 con 0x7f78b00fea60 2023-06-23T18:23:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b856d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b019c310 con 0x7f78b01005e0 2023-06-23T18:23:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b856d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b019c4a0 con 0x7f78b00ffc50 2023-06-23T18:23:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 0x7f78b0194650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b5b08700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b00ffc50 0x7f78b0194b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 0x7f78b0194650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54418/0 (socket says 172.21.15.2:54418) 2023-06-23T18:23:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 -- 172.21.15.2:0/2192548649 learned_addr learned my addr 172.21.15.2:0/2192548649 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6b0a700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78b01005e0 0x7f78b019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 -- 172.21.15.2:0/2192548649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b00ffc50 msgr2=0x7f78b0194b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b00ffc50 0x7f78b0194b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 -- 172.21.15.2:0/2192548649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78b01005e0 msgr2=0x7f78b019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78b01005e0 0x7f78b019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 -- 172.21.15.2:0/2192548649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0095f0 con 0x7f78b00fea60 2023-06-23T18:23:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.876+0000 7f78b6309700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 0x7f78b0194650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00b5c0 tx=0x7f78ac005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.877+0000 7f78a77fe700 1 -- 172.21.15.2:0/2192548649 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac01d070 con 0x7f78b00fea60 2023-06-23T18:23:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.877+0000 7f78a77fe700 1 -- 172.21.15.2:0/2192548649 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f78ac004000 con 0x7f78b00fea60 2023-06-23T18:23:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.877+0000 7f78a77fe700 1 -- 172.21.15.2:0/2192548649 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac022d00 con 0x7f78b00fea60 2023-06-23T18:23:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.877+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b019c720 con 0x7f78b00fea60 2023-06-23T18:23:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.877+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b019caf0 con 0x7f78b00fea60 2023-06-23T18:23:17.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.878+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b010a0e0 con 0x7f78b00fea60 2023-06-23T18:23:17.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.879+0000 7f78a77fe700 1 -- 172.21.15.2:0/2192548649 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f78ac003b70 con 0x7f78b00fea60 2023-06-23T18:23:17.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.880+0000 7f78a77fe700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f789c0a11d0 0x7f789c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:17.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.880+0000 7f78a77fe700 1 -- 172.21.15.2:0/2192548649 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78ac0ce360 con 0x7f78b00fea60 2023-06-23T18:23:17.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.880+0000 7f78b5b08700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f789c0a11d0 0x7f789c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:17.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.880+0000 7f78b5b08700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f789c0a11d0 0x7f789c0a3680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f78a000ac50 tx=0x7f78a000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:17.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:17.884+0000 7f78a77fe700 1 -- 172.21.15.2:0/2192548649 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f78ac097b40 con 0x7f78b00fea60 2023-06-23T18:23:18.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:17 smithi082 ceph-mon[97889]: pgmap v334: 97 pgs: 97 active+clean; 806 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 28 op/s 2023-06-23T18:23:18.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:17 smithi017 ceph-mon[97472]: pgmap v334: 97 pgs: 97 active+clean; 806 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 28 op/s 2023-06-23T18:23:18.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.200+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78b0105950 con 0x7f78b00fea60 2023-06-23T18:23:18.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.201+0000 7f78a77fe700 1 -- 172.21.15.2:0/2192548649 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10780 (secure 0 0 0) 0x7f78ac022e60 con 0x7f78b00fea60 2023-06-23T18:23:18.207 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:23:18.207 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:22:50.581884+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:23:18.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.203+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f789c0a11d0 msgr2=0x7f789c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.203+0000 7f78b856d700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f789c0a11d0 0x7f789c0a3680 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f78a000ac50 tx=0x7f78a000a380 comp rx=0 tx=0).stop 2023-06-23T18:23:18.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.203+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 msgr2=0x7f78b0194650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.203+0000 7f78b856d700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 0x7f78b0194650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00b5c0 tx=0x7f78ac005740 comp rx=0 tx=0).stop 2023-06-23T18:23:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.204+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 shutdown_connections 2023-06-23T18:23:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.204+0000 7f78b856d700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f789c0a11d0 0x7f789c0a3680 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.204+0000 7f78b856d700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78b01005e0 0x7f78b019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.204+0000 7f78b856d700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78b00fea60 0x7f78b0194650 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.204+0000 7f78b856d700 1 --2- 172.21.15.2:0/2192548649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b00ffc50 0x7f78b0194b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.204+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 >> 172.21.15.2:0/2192548649 conn(0x7f78b00fa0e0 msgr2=0x7f78b00fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.204+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 shutdown_connections 2023-06-23T18:23:18.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.204+0000 7f78b856d700 1 -- 172.21.15.2:0/2192548649 wait complete. 2023-06-23T18:23:18.211 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 24 2023-06-23T18:23:18.221 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:17 smithi002 ceph-mon[95947]: pgmap v334: 97 pgs: 97 active+clean; 806 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 41 MiB/s wr, 28 op/s 2023-06-23T18:23:18.223 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:23:18.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 -- 172.21.15.2:0/1726419043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368100d80 msgr2=0x7f43681011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 --2- 172.21.15.2:0/1726419043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368100d80 0x7f43681011d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4358009990 tx=0x7f4358009ca0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 -- 172.21.15.2:0/1726419043 shutdown_connections 2023-06-23T18:23:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 --2- 172.21.15.2:0/1726419043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4368101710 0x7f4368107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 --2- 172.21.15.2:0/1726419043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43680ffb90 0x7f43680fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 --2- 172.21.15.2:0/1726419043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368100d80 0x7f43681011d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 -- 172.21.15.2:0/1726419043 >> 172.21.15.2:0/1726419043 conn(0x7f43680fb1b0 msgr2=0x7f43680fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 -- 172.21.15.2:0/1726419043 shutdown_connections 2023-06-23T18:23:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 -- 172.21.15.2:0/1726419043 wait complete. 2023-06-23T18:23:18.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 Processor -- start 2023-06-23T18:23:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.368+0000 7f436f59a700 1 -- start start 2023-06-23T18:23:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436f59a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43680ffb90 0x7f4368195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436f59a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4368100d80 0x7f4368195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436f59a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368101710 0x7f436819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436f59a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436819d380 con 0x7f4368101710 2023-06-23T18:23:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436f59a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436819d4c0 con 0x7f4368100d80 2023-06-23T18:23:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436f59a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436819d650 con 0x7f43680ffb90 2023-06-23T18:23:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368101710 0x7f436819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368101710 0x7f436819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54436/0 (socket says 172.21.15.2:54436) 2023-06-23T18:23:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 -- 172.21.15.2:0/3594034448 learned_addr learned my addr 172.21.15.2:0/3594034448 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436cb35700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4368100d80 0x7f4368195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436d336700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43680ffb90 0x7f4368195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 -- 172.21.15.2:0/3594034448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43680ffb90 msgr2=0x7f4368195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43680ffb90 0x7f4368195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 -- 172.21.15.2:0/3594034448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4368100d80 msgr2=0x7f4368195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4368100d80 0x7f4368195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 -- 172.21.15.2:0/3594034448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43580095f0 con 0x7f4368101710 2023-06-23T18:23:18.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f436db37700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368101710 0x7f436819cdb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4360007ee0 tx=0x7f436000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.369+0000 7f435e7fc700 1 -- 172.21.15.2:0/3594034448 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000dd20 con 0x7f4368101710 2023-06-23T18:23:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.370+0000 7f435e7fc700 1 -- 172.21.15.2:0/3594034448 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f436000f040 con 0x7f4368101710 2023-06-23T18:23:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.370+0000 7f436f59a700 1 -- 172.21.15.2:0/3594034448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436819d930 con 0x7f4368101710 2023-06-23T18:23:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.370+0000 7f435e7fc700 1 -- 172.21.15.2:0/3594034448 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360004750 con 0x7f4368101710 2023-06-23T18:23:18.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.370+0000 7f436f59a700 1 -- 172.21.15.2:0/3594034448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436819de90 con 0x7f4368101710 2023-06-23T18:23:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.372+0000 7f435e7fc700 1 -- 172.21.15.2:0/3594034448 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4360003a00 con 0x7f4368101710 2023-06-23T18:23:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.373+0000 7f435e7fc700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43540a11d0 0x7f43540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.373+0000 7f435e7fc700 1 -- 172.21.15.2:0/3594034448 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f43600cbfa0 con 0x7f4368101710 2023-06-23T18:23:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.373+0000 7f436d336700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43540a11d0 0x7f43540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f436d336700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43540a11d0 0x7f43540a3680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4368100be0 tx=0x7f4364007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f437b71fb80 1 Processor -- start 2023-06-23T18:23:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f437b71fb80 1 -- start start 2023-06-23T18:23:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390ec80 0x55eec390f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390f5d0 0x55eec390fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec3910000 0x55eec3910470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f437b71fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec3813cb0 con 0x55eec3910000 2023-06-23T18:23:18.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f437b71fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec3866a40 con 0x55eec390f5d0 2023-06-23T18:23:18.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.374+0000 7f437b71fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec385fed0 con 0x55eec390ec80 2023-06-23T18:23:18.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436db37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec3910000 0x55eec3910470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436d336700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390ec80 0x55eec390f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436d336700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390ec80 0x55eec390f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:54778/0 (socket says 172.21.15.2:54778) 2023-06-23T18:23:18.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436db37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec3910000 0x55eec3910470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54444/0 (socket says 172.21.15.2:54444) 2023-06-23T18:23:18.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436d336700 1 -- 172.21.15.2:0/1813203932 learned_addr learned my addr 172.21.15.2:0/1813203932 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436cb35700 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390f5d0 0x55eec390fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436db37700 1 -- 172.21.15.2:0/1813203932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390ec80 msgr2=0x55eec390f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436db37700 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390ec80 0x55eec390f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436db37700 1 -- 172.21.15.2:0/1813203932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390f5d0 msgr2=0x55eec390fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436db37700 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390f5d0 0x55eec390fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436db37700 1 -- 172.21.15.2:0/1813203932 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eec36e1630 con 0x55eec3910000 2023-06-23T18:23:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436d336700 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390ec80 0x55eec390f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f436db37700 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec3910000 0x55eec3910470 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4360095950 tx=0x7f4360099320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2fcd3dc7d9d380 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f43537fe700 1 -- 172.21.15.2:0/1813203932 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360003cb0 con 0x55eec3910000 2023-06-23T18:23:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f43537fe700 1 -- 172.21.15.2:0/1813203932 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43600048b0 con 0x55eec3910000 2023-06-23T18:23:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.375+0000 7f43537fe700 1 -- 172.21.15.2:0/1813203932 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360009410 con 0x55eec3910000 2023-06-23T18:23:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.376+0000 7f437b71fb80 1 -- 172.21.15.2:0/1813203932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec3910000 msgr2=0x55eec3910470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.376+0000 7f437b71fb80 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec3910000 0x55eec3910470 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4360095950 tx=0x7f4360099320 comp rx=0 tx=0).stop 2023-06-23T18:23:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.376+0000 7f437b71fb80 1 -- 172.21.15.2:0/1813203932 shutdown_connections 2023-06-23T18:23:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.376+0000 7f437b71fb80 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec3910000 0x55eec3910470 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.376+0000 7f437b71fb80 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390ec80 0x55eec390f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.376+0000 7f437b71fb80 1 --2- 172.21.15.2:0/1813203932 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390f5d0 0x55eec390fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.376+0000 7f437b71fb80 1 -- 172.21.15.2:0/1813203932 >> 172.21.15.2:0/1813203932 conn(0x55eec39043f0 msgr2=0x55eec3909520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.377+0000 7f437b71fb80 1 -- 172.21.15.2:0/1813203932 shutdown_connections 2023-06-23T18:23:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.377+0000 7f437b71fb80 1 -- 172.21.15.2:0/1813203932 wait complete. 2023-06-23T18:23:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.377+0000 7f437b71fb80 1 -- start start 2023-06-23T18:23:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.377+0000 7f437b71fb80 1 Processor -- start 2023-06-23T18:23:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.377+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec3918550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec3918a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec391fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f437b71fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec37ffac0 con 0x55eec390ec80 2023-06-23T18:23:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f437b71fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec37ff7c0 con 0x55eec3910000 2023-06-23T18:23:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f437b71fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec36e3920 con 0x55eec390f5d0 2023-06-23T18:23:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f437b71fb80 10 client.0 _ll_register_callbacks cb 0x55eec3805260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec3918550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec3918550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54446/0 (socket says 172.21.15.2:54446) 2023-06-23T18:23:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 -- 172.21.15.2:0/3052201192 learned_addr learned my addr 172.21.15.2:0/3052201192 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436db37700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec391fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436cb35700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec3918a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 -- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 msgr2=0x55eec3918a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec3918a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 -- 172.21.15.2:0/3052201192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 msgr2=0x55eec391fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec391fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 -- 172.21.15.2:0/3052201192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4364010b30 con 0x55eec390ec80 2023-06-23T18:23:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436cb35700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec3918a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:23:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436db37700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec391fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:23:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f436d336700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec3918550 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4364011880 tx=0x7f43640047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2013a384525a745b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f4350ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f4350ff9700 1 -- 172.21.15.2:0/3052201192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364011ea0 con 0x55eec390ec80 2023-06-23T18:23:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f4350ff9700 1 -- 172.21.15.2:0/3052201192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4364017070 con 0x55eec390ec80 2023-06-23T18:23:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.378+0000 7f437b71fb80 1 -- 172.21.15.2:0/3052201192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eec3901bb0 con 0x55eec390ec80 2023-06-23T18:23:18.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f4350ff9700 1 -- 172.21.15.2:0/3052201192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4364004460 con 0x55eec390ec80 2023-06-23T18:23:18.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f437b71fb80 10 client.15549 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:18.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f437b71fb80 1 -- 172.21.15.2:0/3052201192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f436819de90 con 0x55eec390ec80 2023-06-23T18:23:18.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f4350ff9700 1 -- 172.21.15.2:0/3052201192 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eec3901bb0 con 0x55eec390ec80 2023-06-23T18:23:18.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f4350ff9700 1 -- 172.21.15.2:0/3052201192 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f436401f5e0 con 0x55eec390ec80 2023-06-23T18:23:18.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f437b71fb80 10 client.15549 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:18.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f437b71fb80 10 client.15549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:23:18.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f437b71fb80 20 client.15549 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:18.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eec39243b0 0x55eec39267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f437b71fb80 4 client.15549 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:23:18.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f437b71fb80 1 -- 172.21.15.2:0/3052201192 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eec3858eb0 con 0x55eec39243b0 2023-06-23T18:23:18.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.379+0000 7f436cb35700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eec39243b0 0x55eec39267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.380+0000 7f436cb35700 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eec39243b0 0x55eec39267c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b177fc9932f41274 server_cookie=fe90025146d43750 in_seq=0 out_seq=0 2023-06-23T18:23:18.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.380+0000 7f4350ff9700 10 client.15549 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:23:18.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.380+0000 7f4350ff9700 1 -- 172.21.15.2:0/3052201192 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eec3858eb0 con 0x55eec39243b0 2023-06-23T18:23:18.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.380+0000 7f4350ff9700 10 client.15549 handle_command_reply: tid=0 2023-06-23T18:23:18.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.380+0000 7f437b71fb80 1 client.15549 shutdown 2023-06-23T18:23:18.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 -- 172.21.15.2:0/3052201192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 msgr2=0x55eec3918550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec3918550 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4364011880 tx=0x7f43640047c0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 -- 172.21.15.2:0/3052201192 shutdown_connections 2023-06-23T18:23:18.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eec39243b0 0x55eec39267c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec391fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec3918550 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f4350ff9700 0 client.15549 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:23:18.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3052201192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec3918a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 -- 172.21.15.2:0/3052201192 >> 172.21.15.2:0/3052201192 conn(0x55eec39043f0 msgr2=0x55eec3908c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 -- 172.21.15.2:0/3052201192 shutdown_connections 2023-06-23T18:23:18.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 1 -- 172.21.15.2:0/3052201192 wait complete. 2023-06-23T18:23:18.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.381+0000 7f437b71fb80 20 client.15549 trim_cache size 0 max 16384 2023-06-23T18:23:18.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.395+0000 7f437b71fb80 1 Processor -- start 2023-06-23T18:23:18.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.395+0000 7f437b71fb80 1 -- start start 2023-06-23T18:23:18.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.395+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec39058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.395+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec391fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.395+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec392b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.395+0000 7f437b71fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec3917a00 con 0x55eec390ec80 2023-06-23T18:23:18.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.395+0000 7f437b71fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec390a680 con 0x55eec3910000 2023-06-23T18:23:18.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.395+0000 7f437b71fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec390afb0 con 0x55eec390f5d0 2023-06-23T18:23:18.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec39058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec39058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54454/0 (socket says 172.21.15.2:54454) 2023-06-23T18:23:18.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 -- 172.21.15.2:0/3760554480 learned_addr learned my addr 172.21.15.2:0/3760554480 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436cb35700 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec391fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436db37700 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec392b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 -- 172.21.15.2:0/3760554480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 msgr2=0x55eec391fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec391fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 -- 172.21.15.2:0/3760554480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 msgr2=0x55eec392b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec392b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 -- 172.21.15.2:0/3760554480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eec368dde0 con 0x55eec390ec80 2023-06-23T18:23:18.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f436d336700 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec39058c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4364000c00 tx=0x7f43640109d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bb056ac9d60737 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f4353fff700 1 -- 172.21.15.2:0/3760554480 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436401f730 con 0x55eec390ec80 2023-06-23T18:23:18.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f4353fff700 1 -- 172.21.15.2:0/3760554480 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4364015070 con 0x55eec390ec80 2023-06-23T18:23:18.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.396+0000 7f4353fff700 1 -- 172.21.15.2:0/3760554480 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364003aa0 con 0x55eec390ec80 2023-06-23T18:23:18.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 -- 172.21.15.2:0/3760554480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 msgr2=0x55eec39058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec39058c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4364000c00 tx=0x7f43640109d0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 -- 172.21.15.2:0/3760554480 shutdown_connections 2023-06-23T18:23:18.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec3910000 0x55eec392b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390ec80 0x55eec39058c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 --2- 172.21.15.2:0/3760554480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec390f5d0 0x55eec391fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 -- 172.21.15.2:0/3760554480 >> 172.21.15.2:0/3760554480 conn(0x55eec39043f0 msgr2=0x55eec39094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 -- 172.21.15.2:0/3760554480 shutdown_connections 2023-06-23T18:23:18.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 -- 172.21.15.2:0/3760554480 wait complete. 2023-06-23T18:23:18.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 -- start start 2023-06-23T18:23:18.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.397+0000 7f437b71fb80 1 Processor -- start 2023-06-23T18:23:18.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390ec80 0x55eec391b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390f5d0 0x55eec391bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f437b71fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec3910000 0x55eec391ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f437b71fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec391f260 con 0x55eec390f5d0 2023-06-23T18:23:18.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f437b71fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec391f3d0 con 0x55eec390ec80 2023-06-23T18:23:18.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f437b71fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eec391f590 con 0x55eec3910000 2023-06-23T18:23:18.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f437b71fb80 10 client.0 _ll_register_callbacks cb 0x55eec3805260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:18.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436cb35700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390f5d0 0x55eec391bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436cb35700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390f5d0 0x55eec391bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54462/0 (socket says 172.21.15.2:54462) 2023-06-23T18:23:18.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436cb35700 1 -- 172.21.15.2:0/2631869363 learned_addr learned my addr 172.21.15.2:0/2631869363 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436d336700 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390ec80 0x55eec391b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436db37700 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec3910000 0x55eec391ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436cb35700 1 -- 172.21.15.2:0/2631869363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec3910000 msgr2=0x55eec391ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436cb35700 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec3910000 0x55eec391ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436cb35700 1 -- 172.21.15.2:0/2631869363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390ec80 msgr2=0x55eec391b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436cb35700 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390ec80 0x55eec391b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436cb35700 1 -- 172.21.15.2:0/2631869363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436819de90 con 0x55eec390f5d0 2023-06-23T18:23:18.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.398+0000 7f436d336700 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390ec80 0x55eec391b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:18.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f436cb35700 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390f5d0 0x55eec391bd00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4358009990 tx=0x7f4358024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936f840b2ed02ef4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f4350ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:18.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f4350ff9700 1 -- 172.21.15.2:0/2631869363 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435801d040 con 0x55eec390f5d0 2023-06-23T18:23:18.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f4350ff9700 1 -- 172.21.15.2:0/2631869363 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f435802c5c0 con 0x55eec390f5d0 2023-06-23T18:23:18.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f437b71fb80 1 -- 172.21.15.2:0/2631869363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f43600048b0 con 0x55eec390f5d0 2023-06-23T18:23:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f4350ff9700 1 -- 172.21.15.2:0/2631869363 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4358027040 con 0x55eec390f5d0 2023-06-23T18:23:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f437b71fb80 10 client.15555 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f437b71fb80 1 -- 172.21.15.2:0/2631869363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55eec3676c60 con 0x55eec390f5d0 2023-06-23T18:23:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f4350ff9700 1 -- 172.21.15.2:0/2631869363 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43600048b0 con 0x55eec390f5d0 2023-06-23T18:23:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f4350ff9700 1 -- 172.21.15.2:0/2631869363 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43580225c0 con 0x55eec390f5d0 2023-06-23T18:23:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f437b71fb80 10 client.15555 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f437b71fb80 10 client.15555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:23:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f437b71fb80 20 client.15555 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:18.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.399+0000 7f437b71fb80 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eec3979270 0x55eec397b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.400+0000 7f437b71fb80 4 client.15555 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:23:18.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.400+0000 7f437b71fb80 1 -- 172.21.15.2:0/2631869363 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eec381b050 con 0x55eec3979270 2023-06-23T18:23:18.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.400+0000 7f436d336700 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eec3979270 0x55eec397b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.400+0000 7f4350ff9700 10 client.15555 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:23:18.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.400+0000 7f436d336700 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eec3979270 0x55eec397b680 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fedb99dc0b3d3aa server_cookie=162ebb255346a1bf in_seq=0 out_seq=0 2023-06-23T18:23:18.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f4350ff9700 1 -- 172.21.15.2:0/2631869363 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eec381b050 con 0x55eec3979270 2023-06-23T18:23:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f4350ff9700 10 client.15555 handle_command_reply: tid=0 2023-06-23T18:23:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 client.15555 shutdown 2023-06-23T18:23:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 -- 172.21.15.2:0/2631869363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390f5d0 msgr2=0x55eec391bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390f5d0 0x55eec391bd00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4358009990 tx=0x7f4358024040 comp rx=0 tx=0).stop 2023-06-23T18:23:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 -- 172.21.15.2:0/2631869363 shutdown_connections 2023-06-23T18:23:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55eec3979270 0x55eec397b680 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eec3910000 0x55eec391ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eec390ec80 0x55eec391b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 --2- 172.21.15.2:0/2631869363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eec390f5d0 0x55eec391bd00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 -- 172.21.15.2:0/2631869363 >> 172.21.15.2:0/2631869363 conn(0x55eec39043f0 msgr2=0x55eec3917260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f4350ff9700 0 client.15555 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:23:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 -- 172.21.15.2:0/2631869363 shutdown_connections 2023-06-23T18:23:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 1 -- 172.21.15.2:0/2631869363 wait complete. 2023-06-23T18:23:18.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f437b71fb80 20 client.15555 trim_cache size 0 max 16384 2023-06-23T18:23:18.418 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:23:18.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.401+0000 7f436f59a700 1 -- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43540a11d0 msgr2=0x7f43540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43540a11d0 0x7f43540a3680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4368100be0 tx=0x7f4364007400 comp rx=0 tx=0).stop 2023-06-23T18:23:18.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 -- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368101710 msgr2=0x7f436819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368101710 0x7f436819cdb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4360007ee0 tx=0x7f436000d3b0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 -- 172.21.15.2:0/3594034448 shutdown_connections 2023-06-23T18:23:18.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4368101710 0x7f436819cdb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43680ffb90 0x7f4368195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43540a11d0 0x7f43540a3680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 --2- 172.21.15.2:0/3594034448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4368100d80 0x7f4368195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 -- 172.21.15.2:0/3594034448 >> 172.21.15.2:0/3594034448 conn(0x7f43680fb1b0 msgr2=0x7f43680fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 -- 172.21.15.2:0/3594034448 shutdown_connections 2023-06-23T18:23:18.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.402+0000 7f436f59a700 1 -- 172.21.15.2:0/3594034448 wait complete. 2023-06-23T18:23:18.421 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:23:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 -- 172.21.15.2:0/3684948614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 msgr2=0x7f20fc0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 --2- 172.21.15.2:0/3684948614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 0x7f20fc0fee70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009990 tx=0x7f20f8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 -- 172.21.15.2:0/3684948614 shutdown_connections 2023-06-23T18:23:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 --2- 172.21.15.2:0/3684948614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20fc1006a0 0x7f20fc106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 --2- 172.21.15.2:0/3684948614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 0x7f20fc0fee70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 --2- 172.21.15.2:0/3684948614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20fc0ffcf0 0x7f20fc100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 -- 172.21.15.2:0/3684948614 >> 172.21.15.2:0/3684948614 conn(0x7f20fc0fa080 msgr2=0x7f20fc0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 -- 172.21.15.2:0/3684948614 shutdown_connections 2023-06-23T18:23:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.557+0000 7f2103b23700 1 -- 172.21.15.2:0/3684948614 wait complete. 2023-06-23T18:23:18.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f2103b23700 1 Processor -- start 2023-06-23T18:23:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f2103b23700 1 -- start start 2023-06-23T18:23:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f2103b23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 0x7f20fc194750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f2103b23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20fc0ffcf0 0x7f20fc194c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f2103b23700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20fc1006a0 0x7f20fc19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f2103b23700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19c2d0 con 0x7f20fc0fea60 2023-06-23T18:23:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f2103b23700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19c410 con 0x7f20fc1006a0 2023-06-23T18:23:18.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f2103b23700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19c5a0 con 0x7f20fc0ffcf0 2023-06-23T18:23:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21018bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 0x7f20fc194750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21018bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 0x7f20fc194750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54474/0 (socket says 172.21.15.2:54474) 2023-06-23T18:23:18.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21018bf700 1 -- 172.21.15.2:0/1207543203 learned_addr learned my addr 172.21.15.2:0/1207543203 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21020c0700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20fc1006a0 0x7f20fc19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21010be700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20fc0ffcf0 0x7f20fc194c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21018bf700 1 -- 172.21.15.2:0/1207543203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20fc0ffcf0 msgr2=0x7f20fc194c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21018bf700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20fc0ffcf0 0x7f20fc194c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21018bf700 1 -- 172.21.15.2:0/1207543203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20fc1006a0 msgr2=0x7f20fc19bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21018bf700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20fc1006a0 0x7f20fc19bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.558+0000 7f21018bf700 1 -- 172.21.15.2:0/1207543203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f80095f0 con 0x7f20fc0fea60 2023-06-23T18:23:18.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.559+0000 7f21018bf700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 0x7f20fc194750 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f20f8000c00 tx=0x7f20f8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.559+0000 7f20f2ffd700 1 -- 172.21.15.2:0/1207543203 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f801f070 con 0x7f20fc0fea60 2023-06-23T18:23:18.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.559+0000 7f20f2ffd700 1 -- 172.21.15.2:0/1207543203 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20f800be70 con 0x7f20fc0fea60 2023-06-23T18:23:18.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.559+0000 7f2103b23700 1 -- 172.21.15.2:0/1207543203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc068950 con 0x7f20fc0fea60 2023-06-23T18:23:18.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.559+0000 7f20f2ffd700 1 -- 172.21.15.2:0/1207543203 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f800f460 con 0x7f20fc0fea60 2023-06-23T18:23:18.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.559+0000 7f2103b23700 1 -- 172.21.15.2:0/1207543203 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc068e00 con 0x7f20fc0fea60 2023-06-23T18:23:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.562+0000 7f20f2ffd700 1 -- 172.21.15.2:0/1207543203 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f20f8019800 con 0x7f20fc0fea60 2023-06-23T18:23:18.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.562+0000 7f20f2ffd700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20e80a11a0 0x7f20e80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.562+0000 7f20f2ffd700 1 -- 172.21.15.2:0/1207543203 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f20f80cfea0 con 0x7f20fc0fea60 2023-06-23T18:23:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.562+0000 7f21010be700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20e80a11a0 0x7f20e80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.563+0000 7f21010be700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20e80a11a0 0x7f20e80a3650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009c60 tx=0x7f20ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.563+0000 7f210fca8b80 1 Processor -- start 2023-06-23T18:23:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f210fca8b80 1 -- start start 2023-06-23T18:23:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6804fc80 0x559a68050090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68050a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a68051000 0x559a68051470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f210fca8b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a67f54cb0 con 0x559a6804fc80 2023-06-23T18:23:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f210fca8b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a67fa7a40 con 0x559a68051000 2023-06-23T18:23:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f210fca8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a67fa0ed0 con 0x559a680505d0 2023-06-23T18:23:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21018bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6804fc80 0x559a68050090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21018bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6804fc80 0x559a68050090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54482/0 (socket says 172.21.15.2:54482) 2023-06-23T18:23:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21018bf700 1 -- 172.21.15.2:0/3117418668 learned_addr learned my addr 172.21.15.2:0/3117418668 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21010be700 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68050a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21020c0700 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a68051000 0x559a68051470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21010be700 1 -- 172.21.15.2:0/3117418668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a68051000 msgr2=0x559a68051470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21010be700 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a68051000 0x559a68051470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21010be700 1 -- 172.21.15.2:0/3117418668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6804fc80 msgr2=0x559a68050090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21010be700 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6804fc80 0x559a68050090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21010be700 1 -- 172.21.15.2:0/3117418668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a67e22630 con 0x559a680505d0 2023-06-23T18:23:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.564+0000 7f21018bf700 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6804fc80 0x559a68050090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.565+0000 7f21010be700 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68050a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00e3b0 tx=0x7f20ec014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492a0034a4403e52 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.565+0000 7f20e7fff700 1 -- 172.21.15.2:0/3117418668 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00ed50 con 0x559a680505d0 2023-06-23T18:23:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.565+0000 7f20e7fff700 1 -- 172.21.15.2:0/3117418668 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20ec021070 con 0x559a680505d0 2023-06-23T18:23:18.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.565+0000 7f20e7fff700 1 -- 172.21.15.2:0/3117418668 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec003e10 con 0x559a680505d0 2023-06-23T18:23:18.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 -- 172.21.15.2:0/3117418668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 msgr2=0x559a68050a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68050a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00e3b0 tx=0x7f20ec014000 comp rx=0 tx=0).stop 2023-06-23T18:23:18.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 -- 172.21.15.2:0/3117418668 shutdown_connections 2023-06-23T18:23:18.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a68051000 0x559a68051470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a6804fc80 0x559a68050090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 --2- 172.21.15.2:0/3117418668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68050a70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 -- 172.21.15.2:0/3117418668 >> 172.21.15.2:0/3117418668 conn(0x559a680453f0 msgr2=0x559a6804a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 -- 172.21.15.2:0/3117418668 shutdown_connections 2023-06-23T18:23:18.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 -- 172.21.15.2:0/3117418668 wait complete. 2023-06-23T18:23:18.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 -- start start 2023-06-23T18:23:18.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.566+0000 7f210fca8b80 1 Processor -- start 2023-06-23T18:23:18.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68059570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a68059ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 0x559a68060af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f210fca8b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a67f40ac0 con 0x559a680505d0 2023-06-23T18:23:18.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f210fca8b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a67f407c0 con 0x559a6804fc80 2023-06-23T18:23:18.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f210fca8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a67e24920 con 0x559a68051000 2023-06-23T18:23:18.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f210fca8b80 10 client.0 _ll_register_callbacks cb 0x559a67f46260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:18.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a68059ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a68059ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54490/0 (socket says 172.21.15.2:54490) 2023-06-23T18:23:18.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 -- 172.21.15.2:0/2326172628 learned_addr learned my addr 172.21.15.2:0/2326172628 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21018bf700 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68059570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21020c0700 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 0x559a68060af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 -- 172.21.15.2:0/2326172628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 msgr2=0x559a68060af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 0x559a68060af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 -- 172.21.15.2:0/2326172628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 msgr2=0x559a68059570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68059570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 -- 172.21.15.2:0/2326172628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc068e00 con 0x559a680505d0 2023-06-23T18:23:18.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f21010be700 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a68059ab0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f20ec003c70 tx=0x7f20ec003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744aaea254f56201 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.567+0000 7f20e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:18.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f20e57fa700 1 -- 172.21.15.2:0/2326172628 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec02a040 con 0x559a680505d0 2023-06-23T18:23:18.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f20e57fa700 1 -- 172.21.15.2:0/2326172628 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f20ec003700 con 0x559a680505d0 2023-06-23T18:23:18.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 1 -- 172.21.15.2:0/2326172628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a68042bb0 con 0x559a680505d0 2023-06-23T18:23:18.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f20e57fa700 1 -- 172.21.15.2:0/2326172628 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20ec0054d0 con 0x559a680505d0 2023-06-23T18:23:18.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 10 client.15567 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:18.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 1 -- 172.21.15.2:0/2326172628 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f20ec017070 con 0x559a680505d0 2023-06-23T18:23:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f20e57fa700 1 -- 172.21.15.2:0/2326172628 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559a68042bb0 con 0x559a680505d0 2023-06-23T18:23:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f20e57fa700 1 -- 172.21.15.2:0/2326172628 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f20ec03a430 con 0x559a680505d0 2023-06-23T18:23:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 10 client.15567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:23:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 20 client.15567 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x559a68065210 0x559a68067620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 4 client.15567 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:23:18.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.568+0000 7f210fca8b80 1 -- 172.21.15.2:0/2326172628 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a67f99eb0 con 0x559a68065210 2023-06-23T18:23:18.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.569+0000 7f21018bf700 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x559a68065210 0x559a68067620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.569+0000 7f20e57fa700 10 client.15567 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:23:18.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.569+0000 7f21018bf700 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x559a68065210 0x559a68067620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9cec05c023680f8 server_cookie=8ea78d6779cd344a in_seq=0 out_seq=0 2023-06-23T18:23:18.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.569+0000 7f20e57fa700 1 -- 172.21.15.2:0/2326172628 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559a67f99eb0 con 0x559a68065210 2023-06-23T18:23:18.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.569+0000 7f20e57fa700 10 client.15567 handle_command_reply: tid=0 2023-06-23T18:23:18.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.569+0000 7f210fca8b80 1 client.15567 shutdown 2023-06-23T18:23:18.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 -- 172.21.15.2:0/2326172628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 msgr2=0x559a68059ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a68059ab0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f20ec003c70 tx=0x7f20ec003ca0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 -- 172.21.15.2:0/2326172628 shutdown_connections 2023-06-23T18:23:18.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x559a68065210 0x559a68067620 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 0x559a68060af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68059570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f20e57fa700 0 client.15567 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:23:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 --2- 172.21.15.2:0/2326172628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a68059ab0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 -- 172.21.15.2:0/2326172628 >> 172.21.15.2:0/2326172628 conn(0x559a680453f0 msgr2=0x559a68049c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 -- 172.21.15.2:0/2326172628 shutdown_connections 2023-06-23T18:23:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 1 -- 172.21.15.2:0/2326172628 wait complete. 2023-06-23T18:23:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.570+0000 7f210fca8b80 20 client.15567 trim_cache size 0 max 16384 2023-06-23T18:23:18.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f210fca8b80 1 Processor -- start 2023-06-23T18:23:18.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f210fca8b80 1 -- start start 2023-06-23T18:23:18.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68058260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68060a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a68051000 0x559a68053ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f210fca8b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a68064d00 con 0x559a68051000 2023-06-23T18:23:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f210fca8b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6805a410 con 0x559a6804fc80 2023-06-23T18:23:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f210fca8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a6804afa0 con 0x559a680505d0 2023-06-23T18:23:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f21020c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a68051000 0x559a68053ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f21018bf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68058260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f21020c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a68051000 0x559a68053ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54500/0 (socket says 172.21.15.2:54500) 2023-06-23T18:23:18.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f21010be700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68060a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f21020c0700 1 -- 172.21.15.2:0/3328592205 learned_addr learned my addr 172.21.15.2:0/3328592205 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.586+0000 7f21018bf700 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68058260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45088/0 (socket says 172.21.15.2:45088) 2023-06-23T18:23:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f21020c0700 1 -- 172.21.15.2:0/3328592205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 msgr2=0x559a68060a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f21020c0700 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68060a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f21020c0700 1 -- 172.21.15.2:0/3328592205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 msgr2=0x559a68058260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f21020c0700 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68058260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f21020c0700 1 -- 172.21.15.2:0/3328592205 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a67dcede0 con 0x559a68051000 2023-06-23T18:23:18.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f21018bf700 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68058260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:18.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f21010be700 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68060a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:23:18.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f21020c0700 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a68051000 0x559a68053ec0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009860 tx=0x7f20f4009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64be05a2dbd1780d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f20f0dac700 1 -- 172.21.15.2:0/3328592205 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4004370 con 0x559a68051000 2023-06-23T18:23:18.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f20f0dac700 1 -- 172.21.15.2:0/3328592205 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20f400f030 con 0x559a68051000 2023-06-23T18:23:18.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.587+0000 7f20f0dac700 1 -- 172.21.15.2:0/3328592205 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4007e90 con 0x559a68051000 2023-06-23T18:23:18.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 -- 172.21.15.2:0/3328592205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a68051000 msgr2=0x559a68053ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a68051000 0x559a68053ec0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009860 tx=0x7f20f4009130 comp rx=0 tx=0).stop 2023-06-23T18:23:18.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 -- 172.21.15.2:0/3328592205 shutdown_connections 2023-06-23T18:23:18.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a68051000 0x559a68053ec0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a68058260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 --2- 172.21.15.2:0/3328592205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a680505d0 0x559a68060a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 -- 172.21.15.2:0/3328592205 >> 172.21.15.2:0/3328592205 conn(0x559a680453f0 msgr2=0x559a6806bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 -- 172.21.15.2:0/3328592205 shutdown_connections 2023-06-23T18:23:18.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 -- 172.21.15.2:0/3328592205 wait complete. 2023-06-23T18:23:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.588+0000 7f210fca8b80 1 -- start start 2023-06-23T18:23:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f210fca8b80 1 Processor -- start 2023-06-23T18:23:18.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a6805c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a6805ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f210fca8b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 0x559a6805fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f210fca8b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a68060380 con 0x559a680505d0 2023-06-23T18:23:18.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f210fca8b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a680604f0 con 0x559a6804fc80 2023-06-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f210fca8b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a680606b0 con 0x559a68051000 2023-06-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f210fca8b80 10 client.0 _ll_register_callbacks cb 0x559a67f46260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:18.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f21010be700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a6805ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f21010be700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a6805ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54504/0 (socket says 172.21.15.2:54504) 2023-06-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f21010be700 1 -- 172.21.15.2:0/292753546 learned_addr learned my addr 172.21.15.2:0/292753546 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f21018bf700 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a6805c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.589+0000 7f21020c0700 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 0x559a6805fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f21010be700 1 -- 172.21.15.2:0/292753546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 msgr2=0x559a6805fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f21010be700 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 0x559a6805fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f21010be700 1 -- 172.21.15.2:0/292753546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 msgr2=0x559a6805c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f21010be700 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a6805c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f21010be700 1 -- 172.21.15.2:0/292753546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc068950 con 0x559a680505d0 2023-06-23T18:23:18.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f21010be700 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a6805ce20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f20ec005090 tx=0x7f20ec00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0658359ef92cebd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f20e57fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f20e57fa700 1 -- 172.21.15.2:0/292753546 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec02a040 con 0x559a680505d0 2023-06-23T18:23:18.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f20e57fa700 1 -- 172.21.15.2:0/292753546 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f20ec035030 con 0x559a680505d0 2023-06-23T18:23:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f210fca8b80 1 -- 172.21.15.2:0/292753546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ec021070 con 0x559a680505d0 2023-06-23T18:23:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f20e57fa700 1 -- 172.21.15.2:0/292753546 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20ec003880 con 0x559a680505d0 2023-06-23T18:23:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f210fca8b80 10 client.15573 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:18.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f210fca8b80 1 -- 172.21.15.2:0/292753546 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f20f4003cb0 con 0x559a680505d0 2023-06-23T18:23:18.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.590+0000 7f20e57fa700 1 -- 172.21.15.2:0/292753546 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20ec021070 con 0x559a680505d0 2023-06-23T18:23:18.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f20e57fa700 1 -- 172.21.15.2:0/292753546 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f20ec026970 con 0x559a680505d0 2023-06-23T18:23:18.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f210fca8b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:18.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f210fca8b80 10 client.15573 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:23:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f210fca8b80 20 client.15573 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f210fca8b80 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x559a680b9da0 0x559a680bc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f210fca8b80 4 client.15573 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:23:18.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f210fca8b80 1 -- 172.21.15.2:0/292753546 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559a67f5c050 con 0x559a680b9da0 2023-06-23T18:23:18.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f21018bf700 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x559a680b9da0 0x559a680bc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f20e57fa700 10 client.15573 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:23:18.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.591+0000 7f21018bf700 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x559a680b9da0 0x559a680bc1b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dac44003a9a56ec server_cookie=199db8a61d14ef8f in_seq=0 out_seq=0 2023-06-23T18:23:18.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f20e57fa700 1 -- 172.21.15.2:0/292753546 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559a67f5c050 con 0x559a680b9da0 2023-06-23T18:23:18.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f20e57fa700 10 client.15573 handle_command_reply: tid=0 2023-06-23T18:23:18.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 client.15573 shutdown 2023-06-23T18:23:18.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 -- 172.21.15.2:0/292753546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 msgr2=0x559a6805ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a6805ce20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f20ec005090 tx=0x7f20ec00eed0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 -- 172.21.15.2:0/292753546 shutdown_connections 2023-06-23T18:23:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x559a680b9da0 0x559a680bc1b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559a68051000 0x559a6805fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559a6804fc80 0x559a6805c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 --2- 172.21.15.2:0/292753546 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559a680505d0 0x559a6805ce20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 -- 172.21.15.2:0/292753546 >> 172.21.15.2:0/292753546 conn(0x559a680453f0 msgr2=0x559a68058260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f20e57fa700 0 client.15573 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:23:18.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 -- 172.21.15.2:0/292753546 shutdown_connections 2023-06-23T18:23:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 1 -- 172.21.15.2:0/292753546 wait complete. 2023-06-23T18:23:18.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f210fca8b80 20 client.15573 trim_cache size 0 max 16384 2023-06-23T18:23:18.623 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:23:18.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f2103b23700 1 -- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20e80a11a0 msgr2=0x7f20e80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f2103b23700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20e80a11a0 0x7f20e80a3650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009c60 tx=0x7f20ec009260 comp rx=0 tx=0).stop 2023-06-23T18:23:18.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f2103b23700 1 -- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 msgr2=0x7f20fc194750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.592+0000 7f2103b23700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 0x7f20fc194750 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f20f8000c00 tx=0x7f20f8009580 comp rx=0 tx=0).stop 2023-06-23T18:23:18.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.593+0000 7f2103b23700 1 -- 172.21.15.2:0/1207543203 shutdown_connections 2023-06-23T18:23:18.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.593+0000 7f2103b23700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20e80a11a0 0x7f20e80a3650 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.593+0000 7f2103b23700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20fc1006a0 0x7f20fc19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.593+0000 7f2103b23700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20fc0fea60 0x7f20fc194750 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.593+0000 7f2103b23700 1 --2- 172.21.15.2:0/1207543203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20fc0ffcf0 0x7f20fc194c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.593+0000 7f2103b23700 1 -- 172.21.15.2:0/1207543203 >> 172.21.15.2:0/1207543203 conn(0x7f20fc0fa080 msgr2=0x7f20fc1032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.593+0000 7f2103b23700 1 -- 172.21.15.2:0/1207543203 shutdown_connections 2023-06-23T18:23:18.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.593+0000 7f2103b23700 1 -- 172.21.15.2:0/1207543203 wait complete. 2023-06-23T18:23:18.626 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:23:18.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 -- 172.21.15.2:0/1496523294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 msgr2=0x7f6314106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 --2- 172.21.15.2:0/1496523294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 0x7f6314106c50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b0a0 tx=0x7f630c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 -- 172.21.15.2:0/1496523294 shutdown_connections 2023-06-23T18:23:18.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 --2- 172.21.15.2:0/1496523294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 0x7f6314106c50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 --2- 172.21.15.2:0/1496523294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63140ffda0 0x7f63141001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 --2- 172.21.15.2:0/1496523294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63140fe9f0 0x7f63140fee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 -- 172.21.15.2:0/1496523294 >> 172.21.15.2:0/1496523294 conn(0x7f63140f9530 msgr2=0x7f63140fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 -- 172.21.15.2:0/1496523294 shutdown_connections 2023-06-23T18:23:18.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.762+0000 7f631a1a2700 1 -- 172.21.15.2:0/1496523294 wait complete. 2023-06-23T18:23:18.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631a1a2700 1 Processor -- start 2023-06-23T18:23:18.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631a1a2700 1 -- start start 2023-06-23T18:23:18.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631a1a2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63140fe9f0 0x7f63141027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631a1a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 0x7f6314100de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631a1a2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63140ffda0 0x7f6314101320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631a1a2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6314101ac0 con 0x7f63140ff3a0 2023-06-23T18:23:18.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631a1a2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6314101c30 con 0x7f63140ffda0 2023-06-23T18:23:18.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631a1a2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631419f690 con 0x7f63140fe9f0 2023-06-23T18:23:18.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631899f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 0x7f6314100de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f63191a0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63140fe9f0 0x7f63141027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631899f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 0x7f6314100de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54516/0 (socket says 172.21.15.2:54516) 2023-06-23T18:23:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.763+0000 7f631899f700 1 -- 172.21.15.2:0/481104618 learned_addr learned my addr 172.21.15.2:0/481104618 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f63199a1700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63140ffda0 0x7f6314101320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f63191a0700 1 -- 172.21.15.2:0/481104618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63140ffda0 msgr2=0x7f6314101320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f63191a0700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63140ffda0 0x7f6314101320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f63191a0700 1 -- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 msgr2=0x7f6314100de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f63191a0700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 0x7f6314100de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f63191a0700 1 -- 172.21.15.2:0/481104618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c009d00 con 0x7f63140fe9f0 2023-06-23T18:23:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f63199a1700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63140ffda0 0x7f6314101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:23:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f63191a0700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63140fe9f0 0x7f63141027a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f631000e9c0 tx=0x7f631000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f630a7fc700 1 -- 172.21.15.2:0/481104618 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631000cbd0 con 0x7f63140fe9f0 2023-06-23T18:23:18.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f631a1a2700 1 -- 172.21.15.2:0/481104618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631419f970 con 0x7f63140fe9f0 2023-06-23T18:23:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f630a7fc700 1 -- 172.21.15.2:0/481104618 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f631000eeb0 con 0x7f63140fe9f0 2023-06-23T18:23:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f630a7fc700 1 -- 172.21.15.2:0/481104618 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310003c40 con 0x7f63140fe9f0 2023-06-23T18:23:18.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.764+0000 7f631a1a2700 1 -- 172.21.15.2:0/481104618 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631419fed0 con 0x7f63140fe9f0 2023-06-23T18:23:18.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.767+0000 7f630a7fc700 1 -- 172.21.15.2:0/481104618 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6310003da0 con 0x7f63140fe9f0 2023-06-23T18:23:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.767+0000 7f630a7fc700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63000a11d0 0x7f63000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.767+0000 7f630a7fc700 1 -- 172.21.15.2:0/481104618 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6310014070 con 0x7f63140fe9f0 2023-06-23T18:23:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.767+0000 7f631899f700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63000a11d0 0x7f63000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.768+0000 7f631899f700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63000a11d0 0x7f63000a3680 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f630400bd70 tx=0x7f630400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.768+0000 7f6327589b80 1 Processor -- start 2023-06-23T18:23:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.768+0000 7f6327589b80 1 -- start start 2023-06-23T18:23:18.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.768+0000 7f6327589b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 0x55c160d75130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.768+0000 7f6327589b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d75fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.768+0000 7f6327589b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d7cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f6327589b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160c79cb0 con 0x55c160d75b70 2023-06-23T18:23:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f6327589b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160ccca40 con 0x55c160d74d20 2023-06-23T18:23:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f6327589b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160cc5ed0 con 0x55c160d76520 2023-06-23T18:23:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d75fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d75fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54524/0 (socket says 172.21.15.2:54524) 2023-06-23T18:23:18.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 -- 172.21.15.2:0/1263110544 learned_addr learned my addr 172.21.15.2:0/1263110544 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f63199a1700 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d7cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f63191a0700 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 0x55c160d75130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 -- 172.21.15.2:0/1263110544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 msgr2=0x55c160d7cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d7cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 -- 172.21.15.2:0/1263110544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 msgr2=0x55c160d75130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 0x55c160d75130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 -- 172.21.15.2:0/1263110544 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c160b47630 con 0x55c160d75b70 2023-06-23T18:23:18.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f63199a1700 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d7cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:23:18.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f631899f700 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d75fe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6304010fe0 tx=0x7f6304013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ddf99b52e836c0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f62ff7fe700 1 -- 172.21.15.2:0/1263110544 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630401e620 con 0x55c160d75b70 2023-06-23T18:23:18.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f62ff7fe700 1 -- 172.21.15.2:0/1263110544 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6304015070 con 0x55c160d75b70 2023-06-23T18:23:18.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.769+0000 7f62ff7fe700 1 -- 172.21.15.2:0/1263110544 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630401ca80 con 0x55c160d75b70 2023-06-23T18:23:18.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 -- 172.21.15.2:0/1263110544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 msgr2=0x55c160d75fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d75fe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6304010fe0 tx=0x7f6304013000 comp rx=0 tx=0).stop 2023-06-23T18:23:18.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 -- 172.21.15.2:0/1263110544 shutdown_connections 2023-06-23T18:23:18.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d7cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 0x55c160d75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 --2- 172.21.15.2:0/1263110544 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d75fe0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 -- 172.21.15.2:0/1263110544 >> 172.21.15.2:0/1263110544 conn(0x55c160d6a3f0 msgr2=0x55c160d6f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 -- 172.21.15.2:0/1263110544 shutdown_connections 2023-06-23T18:23:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.770+0000 7f6327589b80 1 -- 172.21.15.2:0/1263110544 wait complete. 2023-06-23T18:23:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 1 -- start start 2023-06-23T18:23:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 1 Processor -- start 2023-06-23T18:23:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d74d20 0x55c160d7e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d7ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d76520 0x55c160d85c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160c65ac0 con 0x55c160d75b70 2023-06-23T18:23:18.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160c657c0 con 0x55c160d76520 2023-06-23T18:23:18.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160b49920 con 0x55c160d74d20 2023-06-23T18:23:18.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f6327589b80 10 client.0 _ll_register_callbacks cb 0x55c160c6b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:18.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f63191a0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d74d20 0x55c160d7e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f63191a0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d74d20 0x55c160d7e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:54866/0 (socket says 172.21.15.2:54866) 2023-06-23T18:23:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f631899f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d7ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f63191a0700 1 -- 172.21.15.2:0/71794161 learned_addr learned my addr 172.21.15.2:0/71794161 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.771+0000 7f63199a1700 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d76520 0x55c160d85c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f631899f700 1 -- 172.21.15.2:0/71794161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d74d20 msgr2=0x55c160d7e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f631899f700 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d74d20 0x55c160d7e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f631899f700 1 -- 172.21.15.2:0/71794161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d76520 msgr2=0x55c160d85c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f631899f700 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d76520 0x55c160d85c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f631899f700 1 -- 172.21.15.2:0/71794161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6304010c90 con 0x55c160d75b70 2023-06-23T18:23:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f631899f700 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d7ec40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6304010fb0 tx=0x7f630401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b251109906efdb53 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f62fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f62fcff9700 1 -- 172.21.15.2:0/71794161 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630402c070 con 0x55c160d75b70 2023-06-23T18:23:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f62fcff9700 1 -- 172.21.15.2:0/71794161 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6304017070 con 0x55c160d75b70 2023-06-23T18:23:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f6327589b80 1 -- 172.21.15.2:0/71794161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c160d67bb0 con 0x55c160d75b70 2023-06-23T18:23:18.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f62fcff9700 1 -- 172.21.15.2:0/71794161 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f630401ce30 con 0x55c160d75b70 2023-06-23T18:23:18.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f6327589b80 10 client.15585 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:18.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.772+0000 7f6327589b80 1 -- 172.21.15.2:0/71794161 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c160af3de0 con 0x55c160d75b70 2023-06-23T18:23:18.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f62fcff9700 1 -- 172.21.15.2:0/71794161 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c160d67bb0 con 0x55c160d75b70 2023-06-23T18:23:18.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f62fcff9700 1 -- 172.21.15.2:0/71794161 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6304003c70 con 0x55c160d75b70 2023-06-23T18:23:18.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f6327589b80 10 client.15585 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:18.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f6327589b80 10 client.15585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:18.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f6327589b80 20 client.15585 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:18.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f6327589b80 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c160d8a420 0x55c160d8c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f6327589b80 4 client.15585 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:23:18.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f6327589b80 1 -- 172.21.15.2:0/71794161 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c160cbeeb0 con 0x55c160d8a420 2023-06-23T18:23:18.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.773+0000 7f63191a0700 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c160d8a420 0x55c160d8c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.774+0000 7f63191a0700 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c160d8a420 0x55c160d8c830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da0eed68abd7f9b server_cookie=21ce4e16d798bbfa in_seq=0 out_seq=0 2023-06-23T18:23:18.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.774+0000 7f62fcff9700 10 client.15585 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.774+0000 7f62fcff9700 1 -- 172.21.15.2:0/71794161 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c160cbeeb0 con 0x55c160d8a420 2023-06-23T18:23:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.774+0000 7f62fcff9700 10 client.15585 handle_command_reply: tid=0 2023-06-23T18:23:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.774+0000 7f6327589b80 1 client.15585 shutdown 2023-06-23T18:23:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.774+0000 7f6327589b80 1 -- 172.21.15.2:0/71794161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 msgr2=0x55c160d7ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.774+0000 7f6327589b80 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d7ec40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6304010fb0 tx=0x7f630401ca30 comp rx=0 tx=0).stop 2023-06-23T18:23:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 1 -- 172.21.15.2:0/71794161 shutdown_connections 2023-06-23T18:23:18.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c160d8a420 0x55c160d8c830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d76520 0x55c160d85c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f62fcff9700 0 client.15585 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d74d20 0x55c160d7e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 1 --2- 172.21.15.2:0/71794161 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d75b70 0x55c160d7ec40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 1 -- 172.21.15.2:0/71794161 >> 172.21.15.2:0/71794161 conn(0x55c160d6a3f0 msgr2=0x55c160d6ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 1 -- 172.21.15.2:0/71794161 shutdown_connections 2023-06-23T18:23:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 1 -- 172.21.15.2:0/71794161 wait complete. 2023-06-23T18:23:18.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.775+0000 7f6327589b80 20 client.15585 trim_cache size 0 max 16384 2023-06-23T18:23:18.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f6327589b80 1 Processor -- start 2023-06-23T18:23:18.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f6327589b80 1 -- start start 2023-06-23T18:23:18.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f6327589b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 0x55c160d70330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f6327589b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d75b70 0x55c160d85c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f6327589b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d76520 0x55c160d7c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f6327589b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160d78ff0 con 0x55c160d76520 2023-06-23T18:23:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f6327589b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160d7dbb0 con 0x55c160d74d20 2023-06-23T18:23:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f6327589b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160d7e270 con 0x55c160d75b70 2023-06-23T18:23:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.791+0000 7f63199a1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d76520 0x55c160d7c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63199a1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d76520 0x55c160d7c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54540/0 (socket says 172.21.15.2:54540) 2023-06-23T18:23:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63199a1700 1 -- 172.21.15.2:0/1380652554 learned_addr learned my addr 172.21.15.2:0/1380652554 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63191a0700 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 0x55c160d70330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f631899f700 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d75b70 0x55c160d85c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63199a1700 1 -- 172.21.15.2:0/1380652554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d75b70 msgr2=0x55c160d85c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63199a1700 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d75b70 0x55c160d85c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63199a1700 1 -- 172.21.15.2:0/1380652554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 msgr2=0x55c160d70330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63199a1700 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 0x55c160d70330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63199a1700 1 -- 172.21.15.2:0/1380652554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c160d6bb90 con 0x55c160d76520 2023-06-23T18:23:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f63199a1700 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d76520 0x55c160d7c940 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b070 tx=0x7f630c009490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d691b73c98749a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.792+0000 7f62fffff700 1 -- 172.21.15.2:0/1380652554 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c00e040 con 0x55c160d76520 2023-06-23T18:23:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.793+0000 7f62fffff700 1 -- 172.21.15.2:0/1380652554 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f630c021070 con 0x55c160d76520 2023-06-23T18:23:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.793+0000 7f62fffff700 1 -- 172.21.15.2:0/1380652554 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f630c004dd0 con 0x55c160d76520 2023-06-23T18:23:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.793+0000 7f6327589b80 1 -- 172.21.15.2:0/1380652554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d76520 msgr2=0x55c160d7c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.793+0000 7f6327589b80 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d76520 0x55c160d7c940 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b070 tx=0x7f630c009490 comp rx=0 tx=0).stop 2023-06-23T18:23:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.793+0000 7f6327589b80 1 -- 172.21.15.2:0/1380652554 shutdown_connections 2023-06-23T18:23:18.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.793+0000 7f6327589b80 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d76520 0x55c160d7c940 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.793+0000 7f6327589b80 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d74d20 0x55c160d70330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.793+0000 7f6327589b80 1 --2- 172.21.15.2:0/1380652554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d75b70 0x55c160d85c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.794+0000 7f6327589b80 1 -- 172.21.15.2:0/1380652554 >> 172.21.15.2:0/1380652554 conn(0x55c160d6a3f0 msgr2=0x55c160d7d400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.794+0000 7f6327589b80 1 -- 172.21.15.2:0/1380652554 shutdown_connections 2023-06-23T18:23:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.794+0000 7f6327589b80 1 -- 172.21.15.2:0/1380652554 wait complete. 2023-06-23T18:23:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.794+0000 7f6327589b80 1 -- start start 2023-06-23T18:23:18.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.794+0000 7f6327589b80 1 Processor -- start 2023-06-23T18:23:18.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f6327589b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d74d20 0x55c160d83750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f6327589b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d75b70 0x55c160d83c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f6327589b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d84ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f6327589b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160d851e0 con 0x55c160d74d20 2023-06-23T18:23:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f6327589b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160d85350 con 0x55c160d75b70 2023-06-23T18:23:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f6327589b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c160d85510 con 0x55c160d76520 2023-06-23T18:23:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f6327589b80 10 client.0 _ll_register_callbacks cb 0x55c160c6b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63191a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d74d20 0x55c160d83750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63191a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d74d20 0x55c160d83750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54542/0 (socket says 172.21.15.2:54542) 2023-06-23T18:23:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63191a0700 1 -- 172.21.15.2:0/1369556731 learned_addr learned my addr 172.21.15.2:0/1369556731 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:18.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f631899f700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d75b70 0x55c160d83c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63199a1700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d84ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63191a0700 1 -- 172.21.15.2:0/1369556731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 msgr2=0x55c160d84ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63191a0700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d84ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63191a0700 1 -- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d75b70 msgr2=0x55c160d83c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63191a0700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d75b70 0x55c160d83c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63191a0700 1 -- 172.21.15.2:0/1369556731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6314107550 con 0x55c160d74d20 2023-06-23T18:23:18.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f63199a1700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d84ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:23:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.795+0000 7f631899f700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d75b70 0x55c160d83c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:23:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f63191a0700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d74d20 0x55c160d83750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6310099f00 tx=0x7f631009a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2628c122e29d25e9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f62fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f62fcff9700 1 -- 172.21.15.2:0/1369556731 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63100b8330 con 0x55c160d74d20 2023-06-23T18:23:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f62fcff9700 1 -- 172.21.15.2:0/1369556731 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f63100b6070 con 0x55c160d74d20 2023-06-23T18:23:18.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 1 -- 172.21.15.2:0/1369556731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6304015070 con 0x55c160d74d20 2023-06-23T18:23:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f62fcff9700 1 -- 172.21.15.2:0/1369556731 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63100ca1e0 con 0x55c160d74d20 2023-06-23T18:23:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 10 client.15591 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 1 -- 172.21.15.2:0/1369556731 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c160adcc60 con 0x55c160d74d20 2023-06-23T18:23:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f62fcff9700 1 -- 172.21.15.2:0/1369556731 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6304015070 con 0x55c160d74d20 2023-06-23T18:23:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f62fcff9700 1 -- 172.21.15.2:0/1369556731 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6310099460 con 0x55c160d74d20 2023-06-23T18:23:18.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 10 client.15591 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 10 client.15591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 20 client.15591 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c160ddf050 0x55c160de1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 4 client.15591 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:23:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.796+0000 7f6327589b80 1 -- 172.21.15.2:0/1369556731 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c160c81050 con 0x55c160ddf050 2023-06-23T18:23:18.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.797+0000 7f631899f700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c160ddf050 0x55c160de1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.797+0000 7f62fcff9700 10 client.15591 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.797+0000 7f631899f700 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c160ddf050 0x55c160de1460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33f4dfb1537753 server_cookie=40afb9151e828805 in_seq=0 out_seq=0 2023-06-23T18:23:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.797+0000 7f62fcff9700 1 -- 172.21.15.2:0/1369556731 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c160c81050 con 0x55c160ddf050 2023-06-23T18:23:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.797+0000 7f62fcff9700 10 client.15591 handle_command_reply: tid=0 2023-06-23T18:23:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.797+0000 7f6327589b80 1 client.15591 shutdown 2023-06-23T18:23:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 -- 172.21.15.2:0/1369556731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d74d20 msgr2=0x55c160d83750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d74d20 0x55c160d83750 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6310099f00 tx=0x7f631009a0f0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 -- 172.21.15.2:0/1369556731 shutdown_connections 2023-06-23T18:23:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c160ddf050 0x55c160de1460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c160d76520 0x55c160d84ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c160d74d20 0x55c160d83750 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 --2- 172.21.15.2:0/1369556731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c160d75b70 0x55c160d83c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 -- 172.21.15.2:0/1369556731 >> 172.21.15.2:0/1369556731 conn(0x55c160d6a3f0 msgr2=0x55c160d7c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f62fcff9700 0 client.15591 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:18.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 -- 172.21.15.2:0/1369556731 shutdown_connections 2023-06-23T18:23:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 1 -- 172.21.15.2:0/1369556731 wait complete. 2023-06-23T18:23:18.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f6327589b80 20 client.15591 trim_cache size 0 max 16384 2023-06-23T18:23:18.821 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:23:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f631a1a2700 1 -- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63000a11d0 msgr2=0x7f63000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f631a1a2700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63000a11d0 0x7f63000a3680 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f630400bd70 tx=0x7f630400b4a0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f631a1a2700 1 -- 172.21.15.2:0/481104618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63140fe9f0 msgr2=0x7f63141027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.798+0000 7f631a1a2700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63140fe9f0 0x7f63141027a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f631000e9c0 tx=0x7f631000ecd0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.799+0000 7f631a1a2700 1 -- 172.21.15.2:0/481104618 shutdown_connections 2023-06-23T18:23:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.799+0000 7f631a1a2700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63000a11d0 0x7f63000a3680 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.799+0000 7f631a1a2700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f63140ffda0 0x7f6314101320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.799+0000 7f631a1a2700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f63140fe9f0 0x7f63141027a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.799+0000 7f631a1a2700 1 --2- 172.21.15.2:0/481104618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63140ff3a0 0x7f6314100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:18.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.799+0000 7f631a1a2700 1 -- 172.21.15.2:0/481104618 >> 172.21.15.2:0/481104618 conn(0x7f63140f9530 msgr2=0x7f63140fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:18.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.799+0000 7f631a1a2700 1 -- 172.21.15.2:0/481104618 shutdown_connections 2023-06-23T18:23:18.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:18.799+0000 7f631a1a2700 1 -- 172.21.15.2:0/481104618 wait complete. 2023-06-23T18:23:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:18 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2192548649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:23:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:18 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2192548649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:23:19.130 INFO:teuthology.orchestra.run.smithi002.stderr:3800000 of 50000000 tuples (7%) done (elapsed 42.37 s, remaining 515.14 s) 2023-06-23T18:23:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:18 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2192548649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:23:19.825 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:23:19.826 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:23:19.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 -- 172.21.15.2:0/306392867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 msgr2=0x7fdffc106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 --2- 172.21.15.2:0/306392867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 0x7fdffc106ed0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdff400b0a0 tx=0x7fdff400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:23:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 -- 172.21.15.2:0/306392867 shutdown_connections 2023-06-23T18:23:19.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 --2- 172.21.15.2:0/306392867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 0x7fdffc106ed0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 --2- 172.21.15.2:0/306392867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdffc0fea90 0x7fdffc0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 --2- 172.21.15.2:0/306392867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdffc0ffc80 0x7fdffc1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 -- 172.21.15.2:0/306392867 >> 172.21.15.2:0/306392867 conn(0x7fdffc0fa090 msgr2=0x7fdffc0fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:19.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 -- 172.21.15.2:0/306392867 shutdown_connections 2023-06-23T18:23:19.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.984+0000 7fe0009d6700 1 -- 172.21.15.2:0/306392867 wait complete. 2023-06-23T18:23:19.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.985+0000 7fe0009d6700 1 Processor -- start 2023-06-23T18:23:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.985+0000 7fe0009d6700 1 -- start start 2023-06-23T18:23:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.985+0000 7fe0009d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdffc0fea90 0x7fdffc069010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.985+0000 7fe0009d6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdffc0ffc80 0x7fdffc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.985+0000 7fe0009d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 0x7fdffc19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.985+0000 7fe0009d6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc19d270 con 0x7fdffc100610 2023-06-23T18:23:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.985+0000 7fe0009d6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc19d3e0 con 0x7fdffc0ffc80 2023-06-23T18:23:19.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.985+0000 7fe0009d6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdffc19d5a0 con 0x7fdffc0fea90 2023-06-23T18:23:19.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 0x7fdffc19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:19.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 0x7fdffc19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54558/0 (socket says 172.21.15.2:54558) 2023-06-23T18:23:19.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 -- 172.21.15.2:0/2646107117 learned_addr learned my addr 172.21.15.2:0/2646107117 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:19.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffa59c700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdffc0fea90 0x7fdffc069010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 -- 172.21.15.2:0/2646107117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdffc0fea90 msgr2=0x7fdffc069010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdffc0fea90 0x7fdffc069010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 -- 172.21.15.2:0/2646107117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdffc0ffc80 msgr2=0x7fdffc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdffc0ffc80 0x7fdffc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdffc0ffc80 0x7fdffc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 -- 172.21.15.2:0/2646107117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4009d00 con 0x7fdffc100610 2023-06-23T18:23:19.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdffc0ffc80 0x7fdffc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:23:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.986+0000 7fdffad9d700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 0x7fdffc19cad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdff400bd00 tx=0x7fdff400bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.987+0000 7fdfeb7fe700 1 -- 172.21.15.2:0/2646107117 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff400e040 con 0x7fdffc100610 2023-06-23T18:23:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.987+0000 7fe0009d6700 1 -- 172.21.15.2:0/2646107117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdffc19d820 con 0x7fdffc100610 2023-06-23T18:23:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.987+0000 7fe0009d6700 1 -- 172.21.15.2:0/2646107117 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdffc19dcf0 con 0x7fdffc100610 2023-06-23T18:23:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.987+0000 7fdfeb7fe700 1 -- 172.21.15.2:0/2646107117 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdff40094d0 con 0x7fdffc100610 2023-06-23T18:23:19.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.987+0000 7fdfeb7fe700 1 -- 172.21.15.2:0/2646107117 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff40052a0 con 0x7fdffc100610 2023-06-23T18:23:20.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.989+0000 7fdfeb7fe700 1 -- 172.21.15.2:0/2646107117 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdff401d470 con 0x7fdffc100610 2023-06-23T18:23:20.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.990+0000 7fdfeb7fe700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfe40a1150 0x7fdfe40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.990+0000 7fdfeb7fe700 1 -- 172.21.15.2:0/2646107117 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdff40cdcc0 con 0x7fdffc100610 2023-06-23T18:23:20.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.990+0000 7fdffa59c700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfe40a1150 0x7fdfe40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.991+0000 7fdffa59c700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfe40a1150 0x7fdfe40a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0ffae0 tx=0x7fdfec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.991+0000 7fe00cb5bb80 1 Processor -- start 2023-06-23T18:23:20.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.991+0000 7fe00cb5bb80 1 -- start start 2023-06-23T18:23:20.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f170d00 0x55e91f171110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f171af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f172080 0x55e91f1724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f075d30 con 0x55e91f171650 2023-06-23T18:23:20.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f0c8ac0 con 0x55e91f172080 2023-06-23T18:23:20.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f0c1f50 con 0x55e91f170d00 2023-06-23T18:23:20.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f171af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f171af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54566/0 (socket says 172.21.15.2:54566) 2023-06-23T18:23:20.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 -- 172.21.15.2:0/928636407 learned_addr learned my addr 172.21.15.2:0/928636407 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdffa59c700 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f170d00 0x55e91f171110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdffad9d700 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f172080 0x55e91f1724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 -- 172.21.15.2:0/928636407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f170d00 msgr2=0x55e91f171110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f170d00 0x55e91f171110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 -- 172.21.15.2:0/928636407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f172080 msgr2=0x55e91f1724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f172080 0x55e91f1724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 -- 172.21.15.2:0/928636407 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e91ef436b0 con 0x55e91f171650 2023-06-23T18:23:20.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdffa59c700 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f170d00 0x55e91f171110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:20.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f171af0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009cc0 tx=0x7fdff0010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c2c1762b4ea559 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.992+0000 7fdfe8ff9700 1 -- 172.21.15.2:0/928636407 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0010bc0 con 0x55e91f171650 2023-06-23T18:23:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.993+0000 7fdfe8ff9700 1 -- 172.21.15.2:0/928636407 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdff00044c0 con 0x55e91f171650 2023-06-23T18:23:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.993+0000 7fdfe8ff9700 1 -- 172.21.15.2:0/928636407 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0003d50 con 0x55e91f171650 2023-06-23T18:23:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.993+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/928636407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 msgr2=0x55e91f171af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.993+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f171af0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009cc0 tx=0x7fdff0010370 comp rx=0 tx=0).stop 2023-06-23T18:23:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.993+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/928636407 shutdown_connections 2023-06-23T18:23:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.993+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f172080 0x55e91f1724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.993+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f170d00 0x55e91f171110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.993+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/928636407 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f171af0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.994+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/928636407 >> 172.21.15.2:0/928636407 conn(0x55e91f166470 msgr2=0x55e91f16b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.994+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/928636407 shutdown_connections 2023-06-23T18:23:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.994+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/928636407 wait complete. 2023-06-23T18:23:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.994+0000 7fe00cb5bb80 1 -- start start 2023-06-23T18:23:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.994+0000 7fe00cb5bb80 1 Processor -- start 2023-06-23T18:23:20.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.994+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.994+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.994+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f181b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f061b40 con 0x55e91f171650 2023-06-23T18:23:20.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f061840 con 0x55e91f170d00 2023-06-23T18:23:20.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91ef459a0 con 0x55e91f172080 2023-06-23T18:23:20.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fe00cb5bb80 10 client.0 _ll_register_callbacks cb 0x55e91f0672e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:20.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdff9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdff9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54572/0 (socket says 172.21.15.2:54572) 2023-06-23T18:23:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdff9d9b700 1 -- 172.21.15.2:0/2532298070 learned_addr learned my addr 172.21.15.2:0/2532298070 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdffa59c700 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdffad9d700 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f181b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdffa59c700 1 -- 172.21.15.2:0/2532298070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 msgr2=0x55e91f181b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdffa59c700 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f181b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdffa59c700 1 -- 172.21.15.2:0/2532298070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 msgr2=0x55e91f17ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdffa59c700 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdffa59c700 1 -- 172.21.15.2:0/2532298070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec010860 con 0x55e91f170d00 2023-06-23T18:23:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdffa59c700 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17a5e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009fd0 tx=0x7fdfec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d265eceef9caa92 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdfe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:23:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.995+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/2532298070 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfec012ea0 con 0x55e91f170d00 2023-06-23T18:23:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/2532298070 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdfec004220 con 0x55e91f170d00 2023-06-23T18:23:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/2532298070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e91f163c30 con 0x55e91f170d00 2023-06-23T18:23:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/2532298070 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfec0043e0 con 0x55e91f170d00 2023-06-23T18:23:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 10 client.25205 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/2532298070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fdff0009920 con 0x55e91f170d00 2023-06-23T18:23:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/2532298070 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e91f163c30 con 0x55e91f170d00 2023-06-23T18:23:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/2532298070 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdfec020720 con 0x55e91f170d00 2023-06-23T18:23:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 10 client.25205 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 10 client.25205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 20 client.25205 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e91f186440 0x55e91f188850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 4 client.25205 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:23:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/2532298070 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e91f0baf30 con 0x55e91f186440 2023-06-23T18:23:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.996+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e91f186440 0x55e91f188850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.997+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e91f186440 0x55e91f188850 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3b8d95ccad0bc6 server_cookie=c5a0f5de7e5b4906 in_seq=0 out_seq=0 2023-06-23T18:23:20.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.997+0000 7fdfe1ffb700 10 client.25205 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.997+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/2532298070 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e91f0baf30 con 0x55e91f186440 2023-06-23T18:23:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.997+0000 7fdfe1ffb700 10 client.25205 handle_command_reply: tid=0 2023-06-23T18:23:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 client.25205 shutdown 2023-06-23T18:23:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 msgr2=0x55e91f17a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17a5e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009fd0 tx=0x7fdfec004750 comp rx=0 tx=0).stop 2023-06-23T18:23:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/2532298070 shutdown_connections 2023-06-23T18:23:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e91f186440 0x55e91f188850 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f181b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fdfe1ffb700 0 client.25205 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17a5e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/2532298070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/2532298070 >> 172.21.15.2:0/2532298070 conn(0x55e91f166470 msgr2=0x55e91f16acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/2532298070 shutdown_connections 2023-06-23T18:23:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/2532298070 wait complete. 2023-06-23T18:23:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:19.998+0000 7fe00cb5bb80 20 client.25205 trim_cache size 0 max 16384 2023-06-23T18:23:20.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.017+0000 7fe00cb5bb80 1 Processor -- start 2023-06-23T18:23:20.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fe00cb5bb80 1 -- start start 2023-06-23T18:23:20.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f16b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f181b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f170180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f179a90 con 0x55e91f171650 2023-06-23T18:23:20.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f17b5f0 con 0x55e91f170d00 2023-06-23T18:23:20.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f16c450 con 0x55e91f172080 2023-06-23T18:23:20.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdff9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f181b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdff9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f181b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54580/0 (socket says 172.21.15.2:54580) 2023-06-23T18:23:20.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdff9d9b700 1 -- 172.21.15.2:0/3837496485 learned_addr learned my addr 172.21.15.2:0/3837496485 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdffad9d700 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f170180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdffa59c700 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f16b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdff9d9b700 1 -- 172.21.15.2:0/3837496485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 msgr2=0x55e91f170180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f170180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdff9d9b700 1 -- 172.21.15.2:0/3837496485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 msgr2=0x55e91f16b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f16b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdff9d9b700 1 -- 172.21.15.2:0/3837496485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e91eeefe60 con 0x55e91f171650 2023-06-23T18:23:20.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.018+0000 7fdffa59c700 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f16b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:23:20.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.019+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f181b20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009fd0 tx=0x7fdff0003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54411e802d6fae05 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.019+0000 7fdfe97fa700 1 -- 172.21.15.2:0/3837496485 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0021050 con 0x55e91f171650 2023-06-23T18:23:20.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.019+0000 7fdfe97fa700 1 -- 172.21.15.2:0/3837496485 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdff0022040 con 0x55e91f171650 2023-06-23T18:23:20.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.019+0000 7fdfe97fa700 1 -- 172.21.15.2:0/3837496485 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff0028290 con 0x55e91f171650 2023-06-23T18:23:20.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3837496485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 msgr2=0x55e91f181b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f181b20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009fd0 tx=0x7fdff0003680 comp rx=0 tx=0).stop 2023-06-23T18:23:20.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3837496485 shutdown_connections 2023-06-23T18:23:20.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f170180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f16b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3837496485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f181b20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3837496485 >> 172.21.15.2:0/3837496485 conn(0x55e91f166470 msgr2=0x55e91f16fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3837496485 shutdown_connections 2023-06-23T18:23:20.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3837496485 wait complete. 2023-06-23T18:23:20.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 -- start start 2023-06-23T18:23:20.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 Processor -- start 2023-06-23T18:23:20.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.020+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fe00cb5bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f180d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f181280 con 0x55e91f171650 2023-06-23T18:23:20.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f1813f0 con 0x55e91f170d00 2023-06-23T18:23:20.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fe00cb5bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91f1815b0 con 0x55e91f172080 2023-06-23T18:23:20.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fe00cb5bb80 10 client.0 _ll_register_callbacks cb 0x55e91f0672e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:20.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54586/0 (socket says 172.21.15.2:54586) 2023-06-23T18:23:20.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 -- 172.21.15.2:0/3038000608 learned_addr learned my addr 172.21.15.2:0/3038000608 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdffad9d700 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f180d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdffa59c700 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 -- 172.21.15.2:0/3038000608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 msgr2=0x55e91f180d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f180d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 -- 172.21.15.2:0/3038000608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 msgr2=0x55e91f17f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 -- 172.21.15.2:0/3038000608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e91ef436b0 con 0x55e91f171650 2023-06-23T18:23:20.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdff9d9b700 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17fd30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009c10 tx=0x7fdff0003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922b5a8b6fdff04d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdfe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:20.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/3038000608 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff00056c0 con 0x55e91f171650 2023-06-23T18:23:20.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/3038000608 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdff0016070 con 0x55e91f171650 2023-06-23T18:23:20.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.021+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3038000608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdff00044c0 con 0x55e91f171650 2023-06-23T18:23:20.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/3038000608 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdff0021020 con 0x55e91f171650 2023-06-23T18:23:20.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fe00cb5bb80 10 client.15609 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:20.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3038000608 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55e91eed8ce0 con 0x55e91f171650 2023-06-23T18:23:20.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/3038000608 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdff00044c0 con 0x55e91f171650 2023-06-23T18:23:20.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/3038000608 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdff00039a0 con 0x55e91f171650 2023-06-23T18:23:20.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fe00cb5bb80 10 client.15609 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:20.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fe00cb5bb80 10 client.15609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:20.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fe00cb5bb80 20 client.15609 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:20.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e91f1dcfb0 0x55e91f1df3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fe00cb5bb80 4 client.15609 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:23:20.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3038000608 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e91f07d0d0 con 0x55e91f1dcfb0 2023-06-23T18:23:20.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.022+0000 7fdffa59c700 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e91f1dcfb0 0x55e91f1df3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.023+0000 7fdfe1ffb700 10 client.15609 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:20.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.023+0000 7fdffa59c700 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e91f1dcfb0 0x55e91f1df3c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88293a96751da12 server_cookie=959a173d98719546 in_seq=0 out_seq=0 2023-06-23T18:23:20.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fdfe1ffb700 1 -- 172.21.15.2:0/3038000608 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e91f07d0d0 con 0x55e91f1dcfb0 2023-06-23T18:23:20.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fdfe1ffb700 10 client.15609 handle_command_reply: tid=0 2023-06-23T18:23:20.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 client.15609 shutdown 2023-06-23T18:23:20.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3038000608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 msgr2=0x55e91f17fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17fd30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fdff0009c10 tx=0x7fdff0003760 comp rx=0 tx=0).stop 2023-06-23T18:23:20.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3038000608 shutdown_connections 2023-06-23T18:23:20.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e91f1dcfb0 0x55e91f1df3c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91f172080 0x55e91f180d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e91f170d00 0x55e91f17f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 --2- 172.21.15.2:0/3038000608 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e91f171650 0x55e91f17fd30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.024+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3038000608 >> 172.21.15.2:0/3038000608 conn(0x55e91f166470 msgr2=0x55e91f17aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fdfe1ffb700 0 client.15609 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:20.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3038000608 shutdown_connections 2023-06-23T18:23:20.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe00cb5bb80 1 -- 172.21.15.2:0/3038000608 wait complete. 2023-06-23T18:23:20.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe00cb5bb80 20 client.15609 trim_cache size 0 max 16384 2023-06-23T18:23:20.039 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:23:20.040 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:23:20.040 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:23:20.040 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:23:20.040 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:23:20.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 -- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfe40a1150 msgr2=0x7fdfe40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfe40a1150 0x7fdfe40a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0ffae0 tx=0x7fdfec009260 comp rx=0 tx=0).stop 2023-06-23T18:23:20.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 -- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 msgr2=0x7fdffc19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 0x7fdffc19cad0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdff400bd00 tx=0x7fdff400bde0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 -- 172.21.15.2:0/2646107117 shutdown_connections 2023-06-23T18:23:20.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdffc100610 0x7fdffc19cad0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdffc0fea90 0x7fdffc069010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdfe40a1150 0x7fdfe40a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 --2- 172.21.15.2:0/2646107117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdffc0ffc80 0x7fdffc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 -- 172.21.15.2:0/2646107117 >> 172.21.15.2:0/2646107117 conn(0x7fdffc0fa090 msgr2=0x7fdffc0fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 -- 172.21.15.2:0/2646107117 shutdown_connections 2023-06-23T18:23:20.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.025+0000 7fe0009d6700 1 -- 172.21.15.2:0/2646107117 wait complete. 2023-06-23T18:23:20.044 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:23:20.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 -- 172.21.15.2:0/3568268619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 msgr2=0x7f5e7c0fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 --2- 172.21.15.2:0/3568268619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 0x7f5e7c0fee90 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009990 tx=0x7f5e6c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 -- 172.21.15.2:0/3568268619 shutdown_connections 2023-06-23T18:23:20.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 --2- 172.21.15.2:0/3568268619 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e7c100600 0x7f5e7c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 --2- 172.21.15.2:0/3568268619 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 0x7f5e7c0fee90 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 --2- 172.21.15.2:0/3568268619 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c0ffc70 0x7f5e7c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 -- 172.21.15.2:0/3568268619 >> 172.21.15.2:0/3568268619 conn(0x7f5e7c0fa0e0 msgr2=0x7f5e7c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 -- 172.21.15.2:0/3568268619 shutdown_connections 2023-06-23T18:23:20.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.202+0000 7f5e81b74700 1 -- 172.21.15.2:0/3568268619 wait complete. 2023-06-23T18:23:20.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e81b74700 1 Processor -- start 2023-06-23T18:23:20.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e81b74700 1 -- start start 2023-06-23T18:23:20.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e81b74700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 0x7f5e7c10a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e81b74700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e7c0ffc70 0x7f5e7c108d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e81b74700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c100600 0x7f5e7c109290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e81b74700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c199660 con 0x7f5e7c0fea80 2023-06-23T18:23:20.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e81b74700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c1097d0 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e81b74700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c109990 con 0x7f5e7c100600 2023-06-23T18:23:20.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 0x7f5e7c10a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e7affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e7c0ffc70 0x7f5e7c108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 0x7f5e7c10a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54598/0 (socket says 172.21.15.2:54598) 2023-06-23T18:23:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/2633379145 learned_addr learned my addr 172.21.15.2:0/2633379145 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.203+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c100600 0x7f5e7c109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e7affd700 1 -- 172.21.15.2:0/2633379145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c100600 msgr2=0x7f5e7c109290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e7affd700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c100600 0x7f5e7c109290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e7affd700 1 -- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 msgr2=0x7f5e7c10a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e7affd700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 0x7f5e7c10a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e7affd700 1 -- 172.21.15.2:0/2633379145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c0095f0 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 0x7f5e7c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e7bfff700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c100600 0x7f5e7c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:20.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e7affd700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e7c0ffc70 0x7f5e7c108d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000e9f0 tx=0x7f5e7000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e78ff9700 1 -- 172.21.15.2:0/2633379145 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000c9c0 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e81b74700 1 -- 172.21.15.2:0/2633379145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c1a4010 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e81b74700 1 -- 172.21.15.2:0/2633379145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c1a4490 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e78ff9700 1 -- 172.21.15.2:0/2633379145 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e70010430 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.204+0000 7f5e78ff9700 1 -- 172.21.15.2:0/2633379145 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70018680 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.207+0000 7f5e78ff9700 1 -- 172.21.15.2:0/2633379145 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5e7000cb20 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.208+0000 7f5e78ff9700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5e640a1150 0x7f5e640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.208+0000 7f5e78ff9700 1 -- 172.21.15.2:0/2633379145 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5e70014070 con 0x7f5e7c0ffc70 2023-06-23T18:23:20.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.208+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5e640a1150 0x7f5e640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5e640a1150 0x7f5e640a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b2e0 tx=0x7f5e6c004500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e8dcf9b80 1 Processor -- start 2023-06-23T18:23:20.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e8dcf9b80 1 -- start start 2023-06-23T18:23:20.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc51090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc51a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc52470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabb55cb0 con 0x563dabc50c80 2023-06-23T18:23:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabba8a40 con 0x563dabc515d0 2023-06-23T18:23:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabba1ed0 con 0x563dabc52000 2023-06-23T18:23:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc51090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc51090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54606/0 (socket says 172.21.15.2:54606) 2023-06-23T18:23:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e7affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc51a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/3658628358 learned_addr learned my addr 172.21.15.2:0/3658628358 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.209+0000 7f5e7bfff700 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc52470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/3658628358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 msgr2=0x563dabc52470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc52470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/3658628358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 msgr2=0x563dabc51a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc51a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/3658628358 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563daba23630 con 0x563dabc50c80 2023-06-23T18:23:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e7affd700 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc51a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc51090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c003b70 tx=0x7f5e6c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690900c5f800fda server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e61ffb700 1 -- 172.21.15.2:0/3658628358 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c02b070 con 0x563dabc50c80 2023-06-23T18:23:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e61ffb700 1 -- 172.21.15.2:0/3658628358 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e6c01e040 con 0x563dabc50c80 2023-06-23T18:23:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.210+0000 7f5e61ffb700 1 -- 172.21.15.2:0/3658628358 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c00fd60 con 0x563dabc50c80 2023-06-23T18:23:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3658628358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 msgr2=0x563dabc51090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc51090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c003b70 tx=0x7f5e6c020000 comp rx=0 tx=0).stop 2023-06-23T18:23:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3658628358 shutdown_connections 2023-06-23T18:23:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc52470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc51090 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/3658628358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc51a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3658628358 >> 172.21.15.2:0/3658628358 conn(0x563dabc463f0 msgr2=0x563dabc4b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3658628358 shutdown_connections 2023-06-23T18:23:20.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3658628358 wait complete. 2023-06-23T18:23:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.211+0000 7f5e8dcf9b80 1 -- start start 2023-06-23T18:23:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e8dcf9b80 1 Processor -- start 2023-06-23T18:23:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc5a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc5a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc52000 0x563dabc619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabb41ac0 con 0x563dabc52000 2023-06-23T18:23:20.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabb417c0 con 0x563dabc515d0 2023-06-23T18:23:20.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563daba25920 con 0x563dabc50c80 2023-06-23T18:23:20.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e8dcf9b80 10 client.0 _ll_register_callbacks cb 0x563dabb47260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:20.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc52000 0x563dabc619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc52000 0x563dabc619d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54616/0 (socket says 172.21.15.2:54616) 2023-06-23T18:23:20.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e7bfff700 1 -- 172.21.15.2:0/319524570 learned_addr learned my addr 172.21.15.2:0/319524570 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e7affd700 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc5a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.212+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc5a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/319524570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 msgr2=0x563dabc5a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc5a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/319524570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc52000 msgr2=0x563dabc619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc52000 0x563dabc619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/319524570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c1a4490 con 0x563dabc50c80 2023-06-23T18:23:20.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e7bfff700 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc52000 0x563dabc619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:20.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc5a450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006010 tx=0x7f5e6c00f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b5628423d83717b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e5affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:23:20.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e5affd700 1 -- 172.21.15.2:0/319524570 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c02b040 con 0x563dabc50c80 2023-06-23T18:23:20.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e5affd700 1 -- 172.21.15.2:0/319524570 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5e6c024410 con 0x563dabc50c80 2023-06-23T18:23:20.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/319524570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563dabc43bb0 con 0x563dabc50c80 2023-06-23T18:23:20.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e5affd700 1 -- 172.21.15.2:0/319524570 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e6c0245d0 con 0x563dabc50c80 2023-06-23T18:23:20.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e8dcf9b80 10 client.25384 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:20.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/319524570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f5e7c1077f0 con 0x563dabc50c80 2023-06-23T18:23:20.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.213+0000 7f5e5affd700 1 -- 172.21.15.2:0/319524570 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563dabc43bb0 con 0x563dabc50c80 2023-06-23T18:23:20.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e5affd700 1 -- 172.21.15.2:0/319524570 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e6c037750 con 0x563dabc50c80 2023-06-23T18:23:20.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e8dcf9b80 10 client.25384 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:20.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e8dcf9b80 10 client.25384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:20.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e8dcf9b80 20 client.25384 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:20.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563dabc65d90 0x563dabc681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e8dcf9b80 4 client.25384 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:23:20.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/319524570 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563dabb9aeb0 con 0x563dabc65d90 2023-06-23T18:23:20.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e7affd700 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563dabc65d90 0x563dabc681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e5affd700 10 client.25384 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:20.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.214+0000 7f5e7affd700 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563dabc65d90 0x563dabc681a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7a26fb494216769 server_cookie=47be9fda48e81334 in_seq=0 out_seq=0 2023-06-23T18:23:20.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e5affd700 1 -- 172.21.15.2:0/319524570 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563dabb9aeb0 con 0x563dabc65d90 2023-06-23T18:23:20.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e5affd700 10 client.25384 handle_command_reply: tid=0 2023-06-23T18:23:20.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 client.25384 shutdown 2023-06-23T18:23:20.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/319524570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 msgr2=0x563dabc5a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc5a450 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006010 tx=0x7f5e6c00f6c0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/319524570 shutdown_connections 2023-06-23T18:23:20.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563dabc65d90 0x563dabc681a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc52000 0x563dabc619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc5a450 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/319524570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc5a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e5affd700 0 client.25384 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:20.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/319524570 >> 172.21.15.2:0/319524570 conn(0x563dabc463f0 msgr2=0x563dabc4ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/319524570 shutdown_connections 2023-06-23T18:23:20.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.215+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/319524570 wait complete. 2023-06-23T18:23:20.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.216+0000 7f5e8dcf9b80 20 client.25384 trim_cache size 0 max 16384 2023-06-23T18:23:20.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e8dcf9b80 1 Processor -- start 2023-06-23T18:23:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e8dcf9b80 1 -- start start 2023-06-23T18:23:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc515d0 0x563dabc61990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc52000 0x563dabc58350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabb9aeb0 con 0x563dabc515d0 2023-06-23T18:23:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabc48470 con 0x563dabc52000 2023-06-23T18:23:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabc5b620 con 0x563dabc50c80 2023-06-23T18:23:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc52000 0x563dabc58350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:54958/0 (socket says 172.21.15.2:54958) 2023-06-23T18:23:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/3260389560 learned_addr learned my addr 172.21.15.2:0/3260389560 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.241+0000 7f5e7affd700 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc515d0 0x563dabc61990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/3260389560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc52000 msgr2=0x563dabc58350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc52000 0x563dabc58350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/3260389560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc515d0 msgr2=0x563dabc61990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc515d0 0x563dabc61990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/3260389560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dab9cfde0 con 0x563dabc50c80 2023-06-23T18:23:20.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e7bfff700 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc52000 0x563dabc58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc622c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c004360 tx=0x7f5e6c00fab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1239ab8fc7769bf9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e627fc700 1 -- 172.21.15.2:0/3260389560 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c02b070 con 0x563dabc50c80 2023-06-23T18:23:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e627fc700 1 -- 172.21.15.2:0/3260389560 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e6c037890 con 0x563dabc50c80 2023-06-23T18:23:20.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.242+0000 7f5e627fc700 1 -- 172.21.15.2:0/3260389560 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c0439e0 con 0x563dabc50c80 2023-06-23T18:23:20.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3260389560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 msgr2=0x563dabc622c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc622c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c004360 tx=0x7f5e6c00fab0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3260389560 shutdown_connections 2023-06-23T18:23:20.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc52000 0x563dabc58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc50c80 0x563dabc622c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/3260389560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc515d0 0x563dabc61990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3260389560 >> 172.21.15.2:0/3260389560 conn(0x563dabc463f0 msgr2=0x563dabc591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3260389560 shutdown_connections 2023-06-23T18:23:20.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/3260389560 wait complete. 2023-06-23T18:23:20.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 -- start start 2023-06-23T18:23:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.243+0000 7f5e8dcf9b80 1 Processor -- start 2023-06-23T18:23:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc61190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e8dcf9b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc5dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabc5e170 con 0x563dabc50c80 2023-06-23T18:23:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabc5e2b0 con 0x563dabc515d0 2023-06-23T18:23:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e8dcf9b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dabc5e470 con 0x563dabc52000 2023-06-23T18:23:20.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e8dcf9b80 10 client.0 _ll_register_callbacks cb 0x563dabb47260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc61190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc61190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54626/0 (socket says 172.21.15.2:54626) 2023-06-23T18:23:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/1976286339 learned_addr learned my addr 172.21.15.2:0/1976286339 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45220/0 (socket says 172.21.15.2:45220) 2023-06-23T18:23:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7bfff700 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc5dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/1976286339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 msgr2=0x563dabc5dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc5dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/1976286339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 msgr2=0x563dabc616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 -- 172.21.15.2:0/1976286339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e74003fc0 con 0x563dabc50c80 2023-06-23T18:23:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7affd700 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:20.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7bfff700 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc5dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e7b7fe700 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc61190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c039040 tx=0x7f5e6c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe229b7eabbf4b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.244+0000 7f5e5affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e5affd700 1 -- 172.21.15.2:0/1976286339 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c02b040 con 0x563dabc50c80 2023-06-23T18:23:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e5affd700 1 -- 172.21.15.2:0/1976286339 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5e6c00f870 con 0x563dabc50c80 2023-06-23T18:23:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/1976286339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e6c01e040 con 0x563dabc50c80 2023-06-23T18:23:20.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e5affd700 1 -- 172.21.15.2:0/1976286339 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e6c043760 con 0x563dabc50c80 2023-06-23T18:23:20.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 10 client.15627 fetch_fsmap learned FSMap version 24 2023-06-23T18:23:20.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/1976286339 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7f5e7c1077f0 con 0x563dabc50c80 2023-06-23T18:23:20.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e5affd700 1 -- 172.21.15.2:0/1976286339 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e6c01e040 con 0x563dabc50c80 2023-06-23T18:23:20.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e5affd700 1 -- 172.21.15.2:0/1976286339 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e6c024840 con 0x563dabc50c80 2023-06-23T18:23:20.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 10 client.15627 fetch_fsmap finished waiting for FSMap version 24 2023-06-23T18:23:20.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:20.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 20 client.15627 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:20.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563dabcbacf0 0x563dabcbd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:20.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 4 client.15627 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:23:20.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/1976286339 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563dabb5d050 con 0x563dabcbacf0 2023-06-23T18:23:20.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.245+0000 7f5e7affd700 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563dabcbacf0 0x563dabcbd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e5affd700 10 client.15627 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e7affd700 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563dabcbacf0 0x563dabcbd100 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58ed9aa44ad242de server_cookie=a0773cc6e693d6aa in_seq=0 out_seq=0 2023-06-23T18:23:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e5affd700 1 -- 172.21.15.2:0/1976286339 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563dabb5d050 con 0x563dabcbacf0 2023-06-23T18:23:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e5affd700 10 client.15627 handle_command_reply: tid=0 2023-06-23T18:23:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e8dcf9b80 1 client.15627 shutdown 2023-06-23T18:23:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/1976286339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 msgr2=0x563dabc61190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc61190 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c039040 tx=0x7f5e6c009290 comp rx=0 tx=0).stop 2023-06-23T18:23:20.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/1976286339 shutdown_connections 2023-06-23T18:23:20.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563dabcbacf0 0x563dabcbd100 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563dabc52000 0x563dabc5dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e5affd700 0 client.15627 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:20.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563dabc50c80 0x563dabc61190 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.246+0000 7f5e8dcf9b80 1 --2- 172.21.15.2:0/1976286339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563dabc515d0 0x563dabc616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/1976286339 >> 172.21.15.2:0/1976286339 conn(0x563dabc463f0 msgr2=0x563dabc54e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/1976286339 shutdown_connections 2023-06-23T18:23:20.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e8dcf9b80 1 -- 172.21.15.2:0/1976286339 wait complete. 2023-06-23T18:23:20.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e8dcf9b80 20 client.15627 trim_cache size 0 max 16384 2023-06-23T18:23:20.265 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:23:20.266 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:23:20.266 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:23:20.266 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:23:20.266 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:23:20.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:23:20.267 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:23:20.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:23:20.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:23:20.268 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:23:20.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 -- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5e640a1150 msgr2=0x7f5e640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5e640a1150 0x7f5e640a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c00b2e0 tx=0x7f5e6c004500 comp rx=0 tx=0).stop 2023-06-23T18:23:20.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 -- 172.21.15.2:0/2633379145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e7c0ffc70 msgr2=0x7f5e7c108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:20.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e7c0ffc70 0x7f5e7c108d50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000e9f0 tx=0x7f5e7000ed00 comp rx=0 tx=0).stop 2023-06-23T18:23:20.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 -- 172.21.15.2:0/2633379145 shutdown_connections 2023-06-23T18:23:20.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5e7c100600 0x7f5e7c109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e7c0fea80 0x7f5e7c10a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f5e640a1150 0x7f5e640a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 --2- 172.21.15.2:0/2633379145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5e7c0ffc70 0x7f5e7c108d50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:20.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 -- 172.21.15.2:0/2633379145 >> 172.21.15.2:0/2633379145 conn(0x7f5e7c0fa0e0 msgr2=0x7f5e7c103220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:20.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 -- 172.21.15.2:0/2633379145 shutdown_connections 2023-06-23T18:23:20.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:20.247+0000 7f5e81b74700 1 -- 172.21.15.2:0/2633379145 wait complete. 2023-06-23T18:23:20.273 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:20 smithi002 ceph-mon[95947]: pgmap v335: 97 pgs: 97 active+clean; 870 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 43 MiB/s wr, 32 op/s 2023-06-23T18:23:20.274 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:23:20.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:20 smithi082 ceph-mon[97889]: pgmap v335: 97 pgs: 97 active+clean; 870 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 43 MiB/s wr, 32 op/s 2023-06-23T18:23:20.432 INFO:teuthology.orchestra.run.smithi002.stderr:3900000 of 50000000 tuples (7%) done (elapsed 43.67 s, remaining 516.24 s) 2023-06-23T18:23:20.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:20 smithi017 ceph-mon[97472]: pgmap v335: 97 pgs: 97 active+clean; 870 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 43 MiB/s wr, 32 op/s 2023-06-23T18:23:21.283 INFO:teuthology.orchestra.run.smithi002.stderr:4000000 of 50000000 tuples (8%) done (elapsed 44.52 s, remaining 512.03 s) 2023-06-23T18:23:21.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:23:21.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:23:21.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:23:21.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:23:21.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:23:21.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:23:21.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:23:21.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:23:21.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:21 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:23:21.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:23:21.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:23:21.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:23:21.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:23:21.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:23:21.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:23:21.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:23:21.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:21 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:23:21.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:23:21.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:23:21.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:23:21.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:23:21.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:23:21.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:23:21.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:23:21.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:21.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:21 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:23:22.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:22 smithi082 ceph-mon[97889]: pgmap v336: 97 pgs: 97 active+clean; 890 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 30 op/s 2023-06-23T18:23:22.426 INFO:teuthology.orchestra.run.smithi002.stderr:4100000 of 50000000 tuples (8%) done (elapsed 45.67 s, remaining 511.25 s) 2023-06-23T18:23:22.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:22 smithi017 ceph-mon[97472]: pgmap v336: 97 pgs: 97 active+clean; 890 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 30 op/s 2023-06-23T18:23:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:22 smithi002 ceph-mon[95947]: pgmap v336: 97 pgs: 97 active+clean; 890 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 39 MiB/s wr, 30 op/s 2023-06-23T18:23:23.302 INFO:teuthology.orchestra.run.smithi002.stderr:4200000 of 50000000 tuples (8%) done (elapsed 46.54 s, remaining 507.55 s) 2023-06-23T18:23:24.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:24 smithi082 ceph-mon[97889]: pgmap v337: 97 pgs: 97 active+clean; 954 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 48 MiB/s wr, 34 op/s 2023-06-23T18:23:24.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:24 smithi017 ceph-mon[97472]: pgmap v337: 97 pgs: 97 active+clean; 954 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 48 MiB/s wr, 34 op/s 2023-06-23T18:23:24.529 INFO:teuthology.orchestra.run.smithi002.stderr:4300000 of 50000000 tuples (8%) done (elapsed 47.77 s, remaining 507.69 s) 2023-06-23T18:23:24.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:24 smithi002 ceph-mon[95947]: pgmap v337: 97 pgs: 97 active+clean; 954 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 48 MiB/s wr, 34 op/s 2023-06-23T18:23:25.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:25 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:23:25.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:25 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:23:25.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:25 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:23:25.835 INFO:teuthology.orchestra.run.smithi002.stderr:4400000 of 50000000 tuples (8%) done (elapsed 49.08 s, remaining 508.61 s) 2023-06-23T18:23:26.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:26 smithi017 ceph-mon[97472]: pgmap v338: 97 pgs: 97 active+clean; 993 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 37 op/s 2023-06-23T18:23:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:26 smithi002 ceph-mon[95947]: pgmap v338: 97 pgs: 97 active+clean; 993 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 37 op/s 2023-06-23T18:23:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:26 smithi082 ceph-mon[97889]: pgmap v338: 97 pgs: 97 active+clean; 993 MiB data, 764 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 MiB/s wr, 37 op/s 2023-06-23T18:23:26.786 INFO:teuthology.orchestra.run.smithi002.stderr:4500000 of 50000000 tuples (9%) done (elapsed 50.03 s, remaining 505.83 s) 2023-06-23T18:23:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:4600000 of 50000000 tuples (9%) done (elapsed 51.34 s, remaining 506.67 s) 2023-06-23T18:23:28.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:28 smithi017 ceph-mon[97472]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:28.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:28 smithi002 ceph-mon[95947]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:28.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:28 smithi082 ceph-mon[97889]: pgmap v339: 97 pgs: 97 active+clean; 1.0 GiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:29.342 INFO:teuthology.orchestra.run.smithi002.stderr:4700000 of 50000000 tuples (9%) done (elapsed 52.58 s, remaining 506.81 s) 2023-06-23T18:23:30.187 INFO:teuthology.orchestra.run.smithi002.stderr:4800000 of 50000000 tuples (9%) done (elapsed 53.43 s, remaining 503.11 s) 2023-06-23T18:23:30.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:30 smithi017 ceph-mon[97472]: pgmap v340: 97 pgs: 97 active+clean; 1.0 GiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 36 op/s 2023-06-23T18:23:30.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:30 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:23:30.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:30 smithi002 ceph-mon[95947]: pgmap v340: 97 pgs: 97 active+clean; 1.0 GiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 36 op/s 2023-06-23T18:23:30.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:30 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:23:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:30 smithi082 ceph-mon[97889]: pgmap v340: 97 pgs: 97 active+clean; 1.0 GiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 36 op/s 2023-06-23T18:23:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:30 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:23:31.432 INFO:teuthology.orchestra.run.smithi002.stderr:4900000 of 50000000 tuples (9%) done (elapsed 54.67 s, remaining 503.21 s) 2023-06-23T18:23:32.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:32 smithi017 ceph-mon[97472]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 35 op/s 2023-06-23T18:23:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:32 smithi002 ceph-mon[95947]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 35 op/s 2023-06-23T18:23:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:32 smithi082 ceph-mon[97889]: pgmap v341: 97 pgs: 97 active+clean; 1.1 GiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 MiB/s wr, 35 op/s 2023-06-23T18:23:32.689 INFO:teuthology.orchestra.run.smithi002.stderr:5000000 of 50000000 tuples (10%) done (elapsed 55.93 s, remaining 503.37 s) 2023-06-23T18:23:33.558 INFO:teuthology.orchestra.run.smithi002.stderr:5100000 of 50000000 tuples (10%) done (elapsed 56.80 s, remaining 500.05 s) 2023-06-23T18:23:34.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:34 smithi017 ceph-mon[97472]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 36 op/s 2023-06-23T18:23:34.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:34 smithi002 ceph-mon[95947]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 36 op/s 2023-06-23T18:23:34.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:34 smithi082 ceph-mon[97889]: pgmap v342: 97 pgs: 97 active+clean; 1.1 GiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 44 MiB/s wr, 36 op/s 2023-06-23T18:23:34.845 INFO:teuthology.orchestra.run.smithi002.stderr:5200000 of 50000000 tuples (10%) done (elapsed 58.09 s, remaining 500.44 s) 2023-06-23T18:23:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:5300000 of 50000000 tuples (10%) done (elapsed 59.36 s, remaining 500.63 s) 2023-06-23T18:23:36.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:36 smithi017 ceph-mon[97472]: pgmap v343: 97 pgs: 97 active+clean; 1.2 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:23:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:36 smithi002 ceph-mon[95947]: pgmap v343: 97 pgs: 97 active+clean; 1.2 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:23:36.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:36 smithi082 ceph-mon[97889]: pgmap v343: 97 pgs: 97 active+clean; 1.2 GiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:23:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:5400000 of 50000000 tuples (10%) done (elapsed 60.23 s, remaining 497.50 s) 2023-06-23T18:23:38.160 INFO:teuthology.orchestra.run.smithi002.stderr:5500000 of 50000000 tuples (11%) done (elapsed 61.40 s, remaining 496.79 s) 2023-06-23T18:23:38.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:38 smithi017 ceph-mon[97472]: pgmap v344: 97 pgs: 97 active+clean; 1.2 GiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 MiB/s wr, 29 op/s 2023-06-23T18:23:38.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:38 smithi002 ceph-mon[95947]: pgmap v344: 97 pgs: 97 active+clean; 1.2 GiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 MiB/s wr, 29 op/s 2023-06-23T18:23:38.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:38 smithi082 ceph-mon[97889]: pgmap v344: 97 pgs: 97 active+clean; 1.2 GiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 MiB/s wr, 29 op/s 2023-06-23T18:23:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:5600000 of 50000000 tuples (11%) done (elapsed 62.28 s, remaining 493.75 s) 2023-06-23T18:23:40.323 INFO:teuthology.orchestra.run.smithi002.stderr:5700000 of 50000000 tuples (11%) done (elapsed 63.56 s, remaining 494.01 s) 2023-06-23T18:23:40.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:40 smithi017 ceph-mon[97472]: pgmap v345: 97 pgs: 97 active+clean; 1.2 GiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:40.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:40 smithi002 ceph-mon[95947]: pgmap v345: 97 pgs: 97 active+clean; 1.2 GiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:40.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:40 smithi082 ceph-mon[97889]: pgmap v345: 97 pgs: 97 active+clean; 1.2 GiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 MiB/s wr, 33 op/s 2023-06-23T18:23:41.575 INFO:teuthology.orchestra.run.smithi002.stderr:5800000 of 50000000 tuples (11%) done (elapsed 64.82 s, remaining 493.95 s) 2023-06-23T18:23:42.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:42 smithi017 ceph-mon[97472]: pgmap v346: 97 pgs: 97 active+clean; 1.3 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 MiB/s wr, 32 op/s 2023-06-23T18:23:42.466 INFO:teuthology.orchestra.run.smithi002.stderr:5900000 of 50000000 tuples (11%) done (elapsed 65.71 s, remaining 491.13 s) 2023-06-23T18:23:42.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:42 smithi002 ceph-mon[95947]: pgmap v346: 97 pgs: 97 active+clean; 1.3 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 MiB/s wr, 32 op/s 2023-06-23T18:23:42.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:42 smithi082 ceph-mon[97889]: pgmap v346: 97 pgs: 97 active+clean; 1.3 GiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 MiB/s wr, 32 op/s 2023-06-23T18:23:43.773 INFO:teuthology.orchestra.run.smithi002.stderr:6000000 of 50000000 tuples (12%) done (elapsed 67.01 s, remaining 491.43 s) 2023-06-23T18:23:44.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:44 smithi017 ceph-mon[97472]: pgmap v347: 97 pgs: 97 active+clean; 1.3 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 MiB/s wr, 34 op/s 2023-06-23T18:23:44.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:44 smithi002 ceph-mon[95947]: pgmap v347: 97 pgs: 97 active+clean; 1.3 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 MiB/s wr, 34 op/s 2023-06-23T18:23:44.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:44 smithi082 ceph-mon[97889]: pgmap v347: 97 pgs: 97 active+clean; 1.3 GiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 MiB/s wr, 34 op/s 2023-06-23T18:23:45.022 INFO:teuthology.orchestra.run.smithi002.stderr:6100000 of 50000000 tuples (12%) done (elapsed 68.26 s, remaining 491.27 s) 2023-06-23T18:23:45.844 INFO:teuthology.orchestra.run.smithi002.stderr:6200000 of 50000000 tuples (12%) done (elapsed 69.08 s, remaining 488.05 s) 2023-06-23T18:23:46.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:46 smithi017 ceph-mon[97472]: pgmap v348: 97 pgs: 97 active+clean; 1.4 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:23:46.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:46 smithi002 ceph-mon[95947]: pgmap v348: 97 pgs: 97 active+clean; 1.4 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:23:46.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:46 smithi082 ceph-mon[97889]: pgmap v348: 97 pgs: 97 active+clean; 1.4 GiB data, 928 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:23:47.134 INFO:teuthology.orchestra.run.smithi002.stderr:6300000 of 50000000 tuples (12%) done (elapsed 70.38 s, remaining 488.16 s) 2023-06-23T18:23:48.402 INFO:teuthology.orchestra.run.smithi002.stderr:6400000 of 50000000 tuples (12%) done (elapsed 71.64 s, remaining 488.07 s) 2023-06-23T18:23:48.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:48 smithi017 ceph-mon[97472]: pgmap v349: 97 pgs: 97 active+clean; 1.4 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 32 op/s 2023-06-23T18:23:48.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:48 smithi002 ceph-mon[95947]: pgmap v349: 97 pgs: 97 active+clean; 1.4 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 32 op/s 2023-06-23T18:23:48.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:48 smithi082 ceph-mon[97889]: pgmap v349: 97 pgs: 97 active+clean; 1.4 GiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 MiB/s wr, 32 op/s 2023-06-23T18:23:49.269 INFO:teuthology.orchestra.run.smithi002.stderr:6500000 of 50000000 tuples (13%) done (elapsed 72.51 s, remaining 485.26 s) 2023-06-23T18:23:50.275 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:23:50.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.413+0000 7f7cb61c1700 1 -- 172.21.15.2:0/2337439851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb0100e20 msgr2=0x7f7cb0101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.413+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/2337439851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb0100e20 0x7f7cb0101270 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4009990 tx=0x7f7ca4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.413+0000 7f7cb61c1700 1 -- 172.21.15.2:0/2337439851 shutdown_connections 2023-06-23T18:23:50.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.413+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/2337439851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb01017b0 0x7f7cb0108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.413+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/2337439851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb00fde50 0x7f7cb00fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.413+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/2337439851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb0100e20 0x7f7cb0101270 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.413+0000 7f7cb61c1700 1 -- 172.21.15.2:0/2337439851 >> 172.21.15.2:0/2337439851 conn(0x7f7cb00f9850 msgr2=0x7f7cb00fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:50.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.414+0000 7f7cb61c1700 1 -- 172.21.15.2:0/2337439851 shutdown_connections 2023-06-23T18:23:50.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.414+0000 7f7cb61c1700 1 -- 172.21.15.2:0/2337439851 wait complete. 2023-06-23T18:23:50.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.414+0000 7f7cb61c1700 1 Processor -- start 2023-06-23T18:23:50.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.414+0000 7f7cb61c1700 1 -- start start 2023-06-23T18:23:50.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7cb61c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb00fde50 0x7f7cb0100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7cb61c1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb0100e20 0x7f7cb00fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7cb61c1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb01017b0 0x7f7cb00ff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7cb61c1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00ffab0 con 0x7f7cb00fde50 2023-06-23T18:23:50.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7cb61c1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00ffc20 con 0x7f7cb01017b0 2023-06-23T18:23:50.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7cb61c1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb019f810 con 0x7f7cb0100e20 2023-06-23T18:23:50.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb00fde50 0x7f7cb0100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb0100e20 0x7f7cb00fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb00fde50 0x7f7cb0100790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54640/0 (socket says 172.21.15.2:54640) 2023-06-23T18:23:50.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 -- 172.21.15.2:0/413892105 learned_addr learned my addr 172.21.15.2:0/413892105 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:50.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caffff700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb01017b0 0x7f7cb00ff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 -- 172.21.15.2:0/413892105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb0100e20 msgr2=0x7f7cb00fedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb0100e20 0x7f7cb00fedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 -- 172.21.15.2:0/413892105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb01017b0 msgr2=0x7f7cb00ff310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb01017b0 0x7f7cb00ff310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 -- 172.21.15.2:0/413892105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca40095f0 con 0x7f7cb00fde50 2023-06-23T18:23:50.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.415+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb00fde50 0x7f7cb0100790 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000e970 tx=0x7f7ca000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:50.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.416+0000 7f7cacff9700 1 -- 172.21.15.2:0/413892105 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0009790 con 0x7f7cb00fde50 2023-06-23T18:23:50.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.416+0000 7f7cacff9700 1 -- 172.21.15.2:0/413892105 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca000c7e0 con 0x7f7cb00fde50 2023-06-23T18:23:50.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.416+0000 7f7cacff9700 1 -- 172.21.15.2:0/413892105 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0003bd0 con 0x7f7cb00fde50 2023-06-23T18:23:50.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.416+0000 7f7cb61c1700 1 -- 172.21.15.2:0/413892105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb019faf0 con 0x7f7cb00fde50 2023-06-23T18:23:50.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.416+0000 7f7cb61c1700 1 -- 172.21.15.2:0/413892105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb019ffb0 con 0x7f7cb00fde50 2023-06-23T18:23:50.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7cc2346b80 1 Processor -- start 2023-06-23T18:23:50.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7cc2346b80 1 -- start start 2023-06-23T18:23:50.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ecc80 0x564f104ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f103f1cb0 con 0x564f104ed5d0 2023-06-23T18:23:50.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f10444a40 con 0x564f104ee000 2023-06-23T18:23:50.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f1043ded0 con 0x564f104ecc80 2023-06-23T18:23:50.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7caffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ecc80 0x564f104ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7caffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ecc80 0x564f104ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:54984/0 (socket says 172.21.15.2:54984) 2023-06-23T18:23:50.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7caeffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7caffff700 1 -- 172.21.15.2:0/329918125 learned_addr learned my addr 172.21.15.2:0/329918125 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:50.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.420+0000 7f7caffff700 1 -- 172.21.15.2:0/329918125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 msgr2=0x564f104ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:50.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7caffff700 1 --2- 172.21.15.2:0/329918125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104ee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7caffff700 1 -- 172.21.15.2:0/329918125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 msgr2=0x564f104eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:50.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7caffff700 1 --2- 172.21.15.2:0/329918125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104eda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7caffff700 1 -- 172.21.15.2:0/329918125 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f102bf630 con 0x564f104ecc80 2023-06-23T18:23:50.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7caeffd700 1 --2- 172.21.15.2:0/329918125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:50.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7caffff700 1 --2- 172.21.15.2:0/329918125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ecc80 0x564f104ed090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8003fc0 tx=0x7f7ca8009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=569ef53e61733f04 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:50.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7c95ffb700 1 -- 172.21.15.2:0/329918125 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8019070 con 0x564f104ecc80 2023-06-23T18:23:50.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7c95ffb700 1 -- 172.21.15.2:0/329918125 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca800f040 con 0x564f104ecc80 2023-06-23T18:23:50.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.421+0000 7f7c95ffb700 1 -- 172.21.15.2:0/329918125 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca80079c0 con 0x564f104ecc80 2023-06-23T18:23:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 -- 172.21.15.2:0/329918125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ecc80 msgr2=0x564f104ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/329918125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ecc80 0x564f104ed090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8003fc0 tx=0x7f7ca8009ba0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 -- 172.21.15.2:0/329918125 shutdown_connections 2023-06-23T18:23:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/329918125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/329918125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ecc80 0x564f104ed090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/329918125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 -- 172.21.15.2:0/329918125 >> 172.21.15.2:0/329918125 conn(0x564f104e23f0 msgr2=0x564f104e7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cacff9700 1 -- 172.21.15.2:0/413892105 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7ca0003d70 con 0x7f7cb00fde50 2023-06-23T18:23:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 -- 172.21.15.2:0/329918125 shutdown_connections 2023-06-23T18:23:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 -- 172.21.15.2:0/329918125 wait complete. 2023-06-23T18:23:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.422+0000 7f7cc2346b80 1 -- start start 2023-06-23T18:23:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cc2346b80 1 Processor -- start 2023-06-23T18:23:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cacff9700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c980a10e0 0x7f7c980a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cacff9700 1 -- 172.21.15.2:0/413892105 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ca0014070 con 0x7f7cb00fde50 2023-06-23T18:23:50.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7caffff700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c980a10e0 0x7f7c980a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104fda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f103ddac0 con 0x564f104ecc80 2023-06-23T18:23:50.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f103dd7c0 con 0x564f104ee000 2023-06-23T18:23:50.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f102c1920 con 0x564f104ed5d0 2023-06-23T18:23:50.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7cc2346b80 10 client.0 _ll_register_callbacks cb 0x564f103e3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54654/0 (socket says 172.21.15.2:54654) 2023-06-23T18:23:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7caf7fe700 1 -- 172.21.15.2:0/3350357406 learned_addr learned my addr 172.21.15.2:0/3350357406 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7caffff700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104fda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.423+0000 7f7caeffd700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caf7fe700 1 -- 172.21.15.2:0/3350357406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 msgr2=0x564f104f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:50.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caf7fe700 1 -- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 msgr2=0x564f104fda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104fda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caf7fe700 1 -- 172.21.15.2:0/3350357406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb01088e0 con 0x564f104ecc80 2023-06-23T18:23:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caffff700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104fda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caeffd700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caffff700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c980a10e0 0x7f7c980a3590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8018000 tx=0x7f7ca8004820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:50.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f6500 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009230 tx=0x7f7ca0099c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f345a8faf79ed3a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7c8effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7c8effd700 1 -- 172.21.15.2:0/3350357406 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca009a4a0 con 0x564f104ecc80 2023-06-23T18:23:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7c8effd700 1 -- 172.21.15.2:0/3350357406 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ca009e070 con 0x564f104ecc80 2023-06-23T18:23:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7cc2346b80 1 -- 172.21.15.2:0/3350357406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f104dfbb0 con 0x564f104ecc80 2023-06-23T18:23:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7c8effd700 1 -- 172.21.15.2:0/3350357406 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca00a2490 con 0x564f104ecc80 2023-06-23T18:23:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7cc2346b80 10 client.15639 fetch_fsmap learned FSMap version 26 2023-06-23T18:23:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.424+0000 7f7cc2346b80 1 -- 172.21.15.2:0/3350357406 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f7ca8003bf0 con 0x564f104ecc80 2023-06-23T18:23:50.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7c8effd700 1 -- 172.21.15.2:0/3350357406 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca00c8080 con 0x564f104ecc80 2023-06-23T18:23:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7c8effd700 1 -- 172.21.15.2:0/3350357406 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7ca00cabb0 con 0x564f104ecc80 2023-06-23T18:23:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7cc2346b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 26 2023-06-23T18:23:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7cc2346b80 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7cc2346b80 20 client.15639 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f105021f0 0x564f10504600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7cc2346b80 4 client.15639 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:23:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7cc2346b80 1 -- 172.21.15.2:0/3350357406 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f10436eb0 con 0x564f105021f0 2023-06-23T18:23:50.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7caeffd700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f105021f0 0x564f10504600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7c8effd700 10 client.15639 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.425+0000 7f7caeffd700 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f105021f0 0x564f10504600 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c6f44bd0858f13a server_cookie=bfebc662a0aed3ac in_seq=0 out_seq=0 2023-06-23T18:23:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7c8effd700 1 -- 172.21.15.2:0/3350357406 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564f10436eb0 con 0x564f105021f0 2023-06-23T18:23:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7c8effd700 10 client.15639 handle_command_reply: tid=0 2023-06-23T18:23:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 client.15639 shutdown 2023-06-23T18:23:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 -- 172.21.15.2:0/3350357406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 msgr2=0x564f104f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f6500 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009230 tx=0x7f7ca0099c70 comp rx=0 tx=0).stop 2023-06-23T18:23:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 -- 172.21.15.2:0/3350357406 shutdown_connections 2023-06-23T18:23:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f105021f0 0x564f10504600 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104fda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f6500 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/3350357406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 -- 172.21.15.2:0/3350357406 >> 172.21.15.2:0/3350357406 conn(0x564f104e23f0 msgr2=0x564f104e6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7c8effd700 0 client.15639 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:50.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 -- 172.21.15.2:0/3350357406 shutdown_connections 2023-06-23T18:23:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.426+0000 7f7cc2346b80 1 -- 172.21.15.2:0/3350357406 wait complete. 2023-06-23T18:23:50.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.427+0000 7f7cc2346b80 20 client.15639 trim_cache size 0 max 16384 2023-06-23T18:23:50.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:50 smithi017 ceph-mon[97472]: pgmap v350: 97 pgs: 97 active+clean; 1.4 GiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 36 op/s 2023-06-23T18:23:50.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.445+0000 7f7cc2346b80 1 Processor -- start 2023-06-23T18:23:50.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.445+0000 7f7cc2346b80 1 -- start start 2023-06-23T18:23:50.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.445+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.445+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104fda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.445+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.445+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f10436eb0 con 0x564f104ecc80 2023-06-23T18:23:50.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.445+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f104e4470 con 0x564f104ee000 2023-06-23T18:23:50.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.445+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f104f59b0 con 0x564f104ed5d0 2023-06-23T18:23:50.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caeffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55002/0 (socket says 172.21.15.2:55002) 2023-06-23T18:23:50.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104f4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45258/0 (socket says 172.21.15.2:45258) 2023-06-23T18:23:50.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54662/0 (socket says 172.21.15.2:54662) 2023-06-23T18:23:50.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 -- 172.21.15.2:0/299489347 learned_addr learned my addr 172.21.15.2:0/299489347 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:50.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 -- 172.21.15.2:0/299489347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 msgr2=0x564f104fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:50.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 --2- 172.21.15.2:0/299489347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104fda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 -- 172.21.15.2:0/299489347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 msgr2=0x564f104f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:50.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 --2- 172.21.15.2:0/299489347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 -- 172.21.15.2:0/299489347 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f104ee8f0 con 0x564f104ee000 2023-06-23T18:23:50.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7caffff700 1 --2- 172.21.15.2:0/299489347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104f4850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8007710 tx=0x7f7ca800a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cb1e5e834138254 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:50.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7c967fc700 1 -- 172.21.15.2:0/299489347 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8028070 con 0x564f104ee000 2023-06-23T18:23:50.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7c967fc700 1 -- 172.21.15.2:0/299489347 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca8007420 con 0x564f104ee000 2023-06-23T18:23:50.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.446+0000 7f7c967fc700 1 -- 172.21.15.2:0/299489347 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8019040 con 0x564f104ee000 2023-06-23T18:23:50.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 -- 172.21.15.2:0/299489347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 msgr2=0x564f104f4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/299489347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104f4850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8007710 tx=0x7f7ca800a000 comp rx=0 tx=0).stop 2023-06-23T18:23:50.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 -- 172.21.15.2:0/299489347 shutdown_connections 2023-06-23T18:23:50.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/299489347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ee000 0x564f104f4850 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/299489347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ecc80 0x564f104f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/299489347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ed5d0 0x564f104fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 -- 172.21.15.2:0/299489347 >> 172.21.15.2:0/299489347 conn(0x564f104e23f0 msgr2=0x564f104fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:50.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 -- 172.21.15.2:0/299489347 shutdown_connections 2023-06-23T18:23:50.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 -- 172.21.15.2:0/299489347 wait complete. 2023-06-23T18:23:50.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.447+0000 7f7cc2346b80 1 -- start start 2023-06-23T18:23:50.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7cc2346b80 1 Processor -- start 2023-06-23T18:23:50.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ecc80 0x564f104f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7cc2346b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ee000 0x564f104fccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f104fd210 con 0x564f104ed5d0 2023-06-23T18:23:50.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f104fd350 con 0x564f104ecc80 2023-06-23T18:23:50.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7cc2346b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f104fd510 con 0x564f104ee000 2023-06-23T18:23:50.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7cc2346b80 10 client.0 _ll_register_callbacks cb 0x564f103e3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:23:50.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7caeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7caf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ecc80 0x564f104f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7caeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104fa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54670/0 (socket says 172.21.15.2:54670) 2023-06-23T18:23:50.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7caeffd700 1 -- 172.21.15.2:0/2196241122 learned_addr learned my addr 172.21.15.2:0/2196241122 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:23:50.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.448+0000 7f7caffff700 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ee000 0x564f104fccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7caeffd700 1 -- 172.21.15.2:0/2196241122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ee000 msgr2=0x564f104fccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:50.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7caeffd700 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ee000 0x564f104fccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7caeffd700 1 -- 172.21.15.2:0/2196241122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ecc80 msgr2=0x564f104f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:23:50.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7caeffd700 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ecc80 0x564f104f9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7caeffd700 1 -- 172.21.15.2:0/2196241122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f102bf630 con 0x564f104ed5d0 2023-06-23T18:23:50.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ecc80 0x564f104f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:23:50.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7caffff700 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ee000 0x564f104fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:23:50.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7caeffd700 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104fa180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40059d0 tx=0x7f7ca40055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deea000dadaea331 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:23:50.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7c8effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:23:50.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7c8effd700 1 -- 172.21.15.2:0/2196241122 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca402c070 con 0x564f104ed5d0 2023-06-23T18:23:50.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7c8effd700 1 -- 172.21.15.2:0/2196241122 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ca4003890 con 0x564f104ed5d0 2023-06-23T18:23:50.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7cc2346b80 1 -- 172.21.15.2:0/2196241122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f103f9050 con 0x564f104ed5d0 2023-06-23T18:23:50.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7c8effd700 1 -- 172.21.15.2:0/2196241122 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca401d040 con 0x564f104ed5d0 2023-06-23T18:23:50.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7cc2346b80 10 client.15645 fetch_fsmap learned FSMap version 26 2023-06-23T18:23:50.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7cc2346b80 1 -- 172.21.15.2:0/2196241122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564f1026bde0 con 0x564f104ed5d0 2023-06-23T18:23:50.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.449+0000 7f7c8effd700 1 -- 172.21.15.2:0/2196241122 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564f103f9050 con 0x564f104ed5d0 2023-06-23T18:23:50.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7c8effd700 1 -- 172.21.15.2:0/2196241122 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7ca400bd10 con 0x564f104ed5d0 2023-06-23T18:23:50.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7cc2346b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 26 2023-06-23T18:23:50.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7cc2346b80 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:23:50.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7cc2346b80 20 client.15645 populate_metadata read hostname 'smithi002' 2023-06-23T18:23:50.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f10557080 0x564f10559490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:23:50.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7cc2346b80 4 client.15645 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:23:50.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7cc2346b80 1 -- 172.21.15.2:0/2196241122 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f1028cf70 con 0x564f10557080 2023-06-23T18:23:50.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f10557080 0x564f10559490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:23:50.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7c8effd700 10 client.15645 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:50.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.450+0000 7f7caf7fe700 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f10557080 0x564f10559490 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b85833846bde7c15 server_cookie=a06eb97708eec3b in_seq=0 out_seq=0 2023-06-23T18:23:50.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7c8effd700 1 -- 172.21.15.2:0/2196241122 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x564f1028cf70 con 0x564f10557080 2023-06-23T18:23:50.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7c8effd700 10 client.15645 handle_command_reply: tid=0 2023-06-23T18:23:50.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 client.15645 shutdown 2023-06-23T18:23:50.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 -- 172.21.15.2:0/2196241122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 msgr2=0x564f104fa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104fa180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40059d0 tx=0x7f7ca40055d0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 -- 172.21.15.2:0/2196241122 shutdown_connections 2023-06-23T18:23:50.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f10557080 0x564f10559490 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f104ee000 0x564f104fccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7c8effd700 0 client.15645 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:23:50.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f104ecc80 0x564f104f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 --2- 172.21.15.2:0/2196241122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f104ed5d0 0x564f104fa180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 -- 172.21.15.2:0/2196241122 >> 172.21.15.2:0/2196241122 conn(0x564f104e23f0 msgr2=0x564f104f0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:50.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 -- 172.21.15.2:0/2196241122 shutdown_connections 2023-06-23T18:23:50.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 1 -- 172.21.15.2:0/2196241122 wait complete. 2023-06-23T18:23:50.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cc2346b80 20 client.15645 trim_cache size 0 max 16384 2023-06-23T18:23:50.467 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:23:50.467 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:23:50.467 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:23:50.467 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:23:50.467 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:23:50.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:23:50.468 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:23:50.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:23:50.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:23:50.468 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:23:50.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cb61c1700 1 -- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c980a10e0 msgr2=0x7f7c980a3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c980a10e0 0x7f7c980a3590 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8018000 tx=0x7f7ca8004820 comp rx=0 tx=0).stop 2023-06-23T18:23:50.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cb61c1700 1 -- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb00fde50 msgr2=0x7f7cb0100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:23:50.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.451+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb00fde50 0x7f7cb0100790 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000e970 tx=0x7f7ca000ec80 comp rx=0 tx=0).stop 2023-06-23T18:23:50.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.452+0000 7f7cb61c1700 1 -- 172.21.15.2:0/413892105 shutdown_connections 2023-06-23T18:23:50.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.452+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c980a10e0 0x7f7c980a3590 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.452+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7cb01017b0 0x7f7cb00ff310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.452+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7cb00fde50 0x7f7cb0100790 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.452+0000 7f7cb61c1700 1 --2- 172.21.15.2:0/413892105 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb0100e20 0x7f7cb00fedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:23:50.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.452+0000 7f7cb61c1700 1 -- 172.21.15.2:0/413892105 >> 172.21.15.2:0/413892105 conn(0x7f7cb00f9850 msgr2=0x7f7cb00fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:23:50.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.452+0000 7f7cb61c1700 1 -- 172.21.15.2:0/413892105 shutdown_connections 2023-06-23T18:23:50.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:23:50.452+0000 7f7cb61c1700 1 -- 172.21.15.2:0/413892105 wait complete. 2023-06-23T18:23:50.472 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:50 smithi002 ceph-mon[95947]: pgmap v350: 97 pgs: 97 active+clean; 1.4 GiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 36 op/s 2023-06-23T18:23:50.472 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:23:50.569 INFO:teuthology.orchestra.run.smithi002.stderr:6600000 of 50000000 tuples (13%) done (elapsed 73.81 s, remaining 485.36 s) 2023-06-23T18:23:50.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:50 smithi082 ceph-mon[97889]: pgmap v350: 97 pgs: 97 active+clean; 1.4 GiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 41 MiB/s wr, 36 op/s 2023-06-23T18:23:51.831 INFO:teuthology.orchestra.run.smithi002.stderr:6700000 of 50000000 tuples (13%) done (elapsed 75.07 s, remaining 485.17 s) 2023-06-23T18:23:52.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:52 smithi017 ceph-mon[97472]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 34 op/s 2023-06-23T18:23:52.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:52 smithi002 ceph-mon[95947]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 34 op/s 2023-06-23T18:23:52.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:52 smithi082 ceph-mon[97889]: pgmap v351: 97 pgs: 97 active+clean; 1.5 GiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 38 MiB/s wr, 34 op/s 2023-06-23T18:23:52.653 INFO:teuthology.orchestra.run.smithi002.stderr:6800000 of 50000000 tuples (13%) done (elapsed 75.89 s, remaining 482.15 s) 2023-06-23T18:23:53.842 INFO:teuthology.orchestra.run.smithi002.stderr:6900000 of 50000000 tuples (13%) done (elapsed 77.08 s, remaining 481.49 s) 2023-06-23T18:23:54.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:54 smithi017 ceph-mon[97472]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 MiB/s wr, 36 op/s 2023-06-23T18:23:54.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:54 smithi002 ceph-mon[95947]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 MiB/s wr, 36 op/s 2023-06-23T18:23:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:54 smithi082 ceph-mon[97889]: pgmap v352: 97 pgs: 97 active+clean; 1.5 GiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 46 MiB/s wr, 36 op/s 2023-06-23T18:23:54.758 INFO:teuthology.orchestra.run.smithi002.stderr:7000000 of 50000000 tuples (14%) done (elapsed 78.00 s, remaining 479.14 s) 2023-06-23T18:23:56.022 INFO:teuthology.orchestra.run.smithi002.stderr:7100000 of 50000000 tuples (14%) done (elapsed 79.26 s, remaining 478.93 s) 2023-06-23T18:23:56.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:56 smithi017 ceph-mon[97472]: pgmap v353: 97 pgs: 97 active+clean; 1.6 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 35 op/s 2023-06-23T18:23:56.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:56 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:23:56.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:56 smithi002 ceph-mon[95947]: pgmap v353: 97 pgs: 97 active+clean; 1.6 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 35 op/s 2023-06-23T18:23:56.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:56 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:23:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:56 smithi082 ceph-mon[97889]: pgmap v353: 97 pgs: 97 active+clean; 1.6 GiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 35 op/s 2023-06-23T18:23:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:56 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:23:57.280 INFO:teuthology.orchestra.run.smithi002.stderr:7200000 of 50000000 tuples (14%) done (elapsed 80.52 s, remaining 478.65 s) 2023-06-23T18:23:57.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:57 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:23:57.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:57 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:23:57.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:57 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:23:57.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:23:57.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:23:57.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:23:57.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:57 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:23:57.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:57 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:23:57.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:57 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:23:58.179 INFO:teuthology.orchestra.run.smithi002.stderr:7300000 of 50000000 tuples (14%) done (elapsed 81.42 s, remaining 476.25 s) 2023-06-23T18:23:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:23:58 smithi017 ceph-mon[97472]: pgmap v354: 97 pgs: 97 active+clean; 1.6 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 43 MiB/s wr, 30 op/s 2023-06-23T18:23:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:23:58 smithi002 ceph-mon[95947]: pgmap v354: 97 pgs: 97 active+clean; 1.6 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 43 MiB/s wr, 30 op/s 2023-06-23T18:23:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:23:58 smithi082 ceph-mon[97889]: pgmap v354: 97 pgs: 97 active+clean; 1.6 GiB data, 1021 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 43 MiB/s wr, 30 op/s 2023-06-23T18:23:59.439 INFO:teuthology.orchestra.run.smithi002.stderr:7400000 of 50000000 tuples (14%) done (elapsed 82.68 s, remaining 475.97 s) 2023-06-23T18:24:00.680 INFO:teuthology.orchestra.run.smithi002.stderr:7500000 of 50000000 tuples (15%) done (elapsed 83.92 s, remaining 475.55 s) 2023-06-23T18:24:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:00 smithi017 ceph-mon[97472]: pgmap v355: 97 pgs: 97 active+clean; 1.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 MiB/s wr, 32 op/s 2023-06-23T18:24:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:00 smithi002 ceph-mon[95947]: pgmap v355: 97 pgs: 97 active+clean; 1.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 MiB/s wr, 32 op/s 2023-06-23T18:24:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:00 smithi082 ceph-mon[97889]: pgmap v355: 97 pgs: 97 active+clean; 1.6 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 41 MiB/s wr, 32 op/s 2023-06-23T18:24:01.485 INFO:teuthology.orchestra.run.smithi002.stderr:7600000 of 50000000 tuples (15%) done (elapsed 84.73 s, remaining 472.68 s) 2023-06-23T18:24:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:02 smithi017 ceph-mon[97472]: pgmap v356: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 30 op/s 2023-06-23T18:24:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:02 smithi002 ceph-mon[95947]: pgmap v356: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 30 op/s 2023-06-23T18:24:02.802 INFO:teuthology.orchestra.run.smithi002.stderr:7700000 of 50000000 tuples (15%) done (elapsed 86.04 s, remaining 472.68 s) 2023-06-23T18:24:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:02 smithi082 ceph-mon[97889]: pgmap v356: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 30 op/s 2023-06-23T18:24:04.063 INFO:teuthology.orchestra.run.smithi002.stderr:7800000 of 50000000 tuples (15%) done (elapsed 87.30 s, remaining 472.34 s) 2023-06-23T18:24:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:04 smithi017 ceph-mon[97472]: pgmap v357: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 46 MiB/s wr, 32 op/s 2023-06-23T18:24:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:04 smithi002 ceph-mon[95947]: pgmap v357: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 46 MiB/s wr, 32 op/s 2023-06-23T18:24:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:04 smithi082 ceph-mon[97889]: pgmap v357: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 46 MiB/s wr, 32 op/s 2023-06-23T18:24:04.900 INFO:teuthology.orchestra.run.smithi002.stderr:7900000 of 50000000 tuples (15%) done (elapsed 88.14 s, remaining 469.72 s) 2023-06-23T18:24:06.195 INFO:teuthology.orchestra.run.smithi002.stderr:8000000 of 50000000 tuples (16%) done (elapsed 89.44 s, remaining 469.54 s) 2023-06-23T18:24:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:06 smithi017 ceph-mon[97472]: pgmap v358: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 33 op/s 2023-06-23T18:24:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:06 smithi002 ceph-mon[95947]: pgmap v358: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 33 op/s 2023-06-23T18:24:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:06 smithi082 ceph-mon[97889]: pgmap v358: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 39 MiB/s wr, 33 op/s 2023-06-23T18:24:07.058 INFO:teuthology.orchestra.run.smithi002.stderr:8100000 of 50000000 tuples (16%) done (elapsed 90.30 s, remaining 467.10 s) 2023-06-23T18:24:08.307 INFO:teuthology.orchestra.run.smithi002.stderr:8200000 of 50000000 tuples (16%) done (elapsed 91.55 s, remaining 466.67 s) 2023-06-23T18:24:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:08 smithi017 ceph-mon[97472]: pgmap v359: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 MiB/s wr, 28 op/s 2023-06-23T18:24:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:08 smithi002 ceph-mon[95947]: pgmap v359: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 MiB/s wr, 28 op/s 2023-06-23T18:24:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:08 smithi082 ceph-mon[97889]: pgmap v359: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 39 MiB/s wr, 28 op/s 2023-06-23T18:24:09.514 INFO:teuthology.orchestra.run.smithi002.stderr:8300000 of 50000000 tuples (16%) done (elapsed 92.76 s, remaining 466.01 s) 2023-06-23T18:24:10.428 INFO:teuthology.orchestra.run.smithi002.stderr:8400000 of 50000000 tuples (16%) done (elapsed 93.67 s, remaining 463.88 s) 2023-06-23T18:24:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:10 smithi017 ceph-mon[97472]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 MiB/s wr, 33 op/s 2023-06-23T18:24:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:10 smithi002 ceph-mon[95947]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 MiB/s wr, 33 op/s 2023-06-23T18:24:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:10 smithi082 ceph-mon[97889]: pgmap v360: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 42 MiB/s wr, 33 op/s 2023-06-23T18:24:11.702 INFO:teuthology.orchestra.run.smithi002.stderr:8500000 of 50000000 tuples (17%) done (elapsed 94.94 s, remaining 463.55 s) 2023-06-23T18:24:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:12 smithi017 ceph-mon[97472]: pgmap v361: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 31 op/s 2023-06-23T18:24:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:12 smithi002 ceph-mon[95947]: pgmap v361: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 31 op/s 2023-06-23T18:24:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:12 smithi082 ceph-mon[97889]: pgmap v361: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 39 MiB/s wr, 31 op/s 2023-06-23T18:24:12.939 INFO:teuthology.orchestra.run.smithi002.stderr:8600000 of 50000000 tuples (17%) done (elapsed 96.18 s, remaining 463.00 s) 2023-06-23T18:24:13.833 INFO:teuthology.orchestra.run.smithi002.stderr:8700000 of 50000000 tuples (17%) done (elapsed 97.07 s, remaining 460.82 s) 2023-06-23T18:24:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:14 smithi017 ceph-mon[97472]: pgmap v362: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 MiB/s wr, 33 op/s 2023-06-23T18:24:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:14 smithi002 ceph-mon[95947]: pgmap v362: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 MiB/s wr, 33 op/s 2023-06-23T18:24:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:14 smithi082 ceph-mon[97889]: pgmap v362: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 47 MiB/s wr, 33 op/s 2023-06-23T18:24:15.067 INFO:teuthology.orchestra.run.smithi002.stderr:8800000 of 50000000 tuples (17%) done (elapsed 98.31 s, remaining 460.26 s) 2023-06-23T18:24:16.331 INFO:teuthology.orchestra.run.smithi002.stderr:8900000 of 50000000 tuples (17%) done (elapsed 99.57 s, remaining 459.82 s) 2023-06-23T18:24:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:16 smithi017 ceph-mon[97472]: pgmap v363: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 34 op/s 2023-06-23T18:24:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:16 smithi002 ceph-mon[95947]: pgmap v363: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 34 op/s 2023-06-23T18:24:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:16 smithi082 ceph-mon[97889]: pgmap v363: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 42 MiB/s wr, 34 op/s 2023-06-23T18:24:17.120 INFO:teuthology.orchestra.run.smithi002.stderr:9000000 of 50000000 tuples (18%) done (elapsed 100.36 s, remaining 457.20 s) 2023-06-23T18:24:18.424 INFO:teuthology.orchestra.run.smithi002.stderr:9100000 of 50000000 tuples (18%) done (elapsed 101.67 s, remaining 456.93 s) 2023-06-23T18:24:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:18 smithi017 ceph-mon[97472]: pgmap v364: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 29 op/s 2023-06-23T18:24:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:18 smithi002 ceph-mon[95947]: pgmap v364: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 29 op/s 2023-06-23T18:24:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:18 smithi082 ceph-mon[97889]: pgmap v364: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 40 MiB/s wr, 29 op/s 2023-06-23T18:24:19.297 INFO:teuthology.orchestra.run.smithi002.stderr:9200000 of 50000000 tuples (18%) done (elapsed 102.54 s, remaining 454.73 s) 2023-06-23T18:24:20.474 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:24:20.545 INFO:teuthology.orchestra.run.smithi002.stderr:9300000 of 50000000 tuples (18%) done (elapsed 103.79 s, remaining 454.21 s) 2023-06-23T18:24:20.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.624+0000 7eff84039700 1 -- 172.21.15.2:0/2917471402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff7c0fea80 msgr2=0x7eff7c0fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.624+0000 7eff84039700 1 --2- 172.21.15.2:0/2917471402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff7c0fea80 0x7eff7c0fee90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7eff78009990 tx=0x7eff78009ca0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.624+0000 7eff84039700 1 -- 172.21.15.2:0/2917471402 shutdown_connections 2023-06-23T18:24:20.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.624+0000 7eff84039700 1 --2- 172.21.15.2:0/2917471402 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff7c100600 0x7eff7c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.624+0000 7eff84039700 1 --2- 172.21.15.2:0/2917471402 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff7c0fea80 0x7eff7c0fee90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.624+0000 7eff84039700 1 --2- 172.21.15.2:0/2917471402 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c0ffc70 0x7eff7c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.624+0000 7eff84039700 1 -- 172.21.15.2:0/2917471402 >> 172.21.15.2:0/2917471402 conn(0x7eff7c0fa0e0 msgr2=0x7eff7c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:20.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 -- 172.21.15.2:0/2917471402 shutdown_connections 2023-06-23T18:24:20.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 -- 172.21.15.2:0/2917471402 wait complete. 2023-06-23T18:24:20.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 Processor -- start 2023-06-23T18:24:20.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 -- start start 2023-06-23T18:24:20.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c0fea80 0x7eff7c195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff7c0ffc70 0x7eff7c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff7c100600 0x7eff7c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c19d3d0 con 0x7eff7c100600 2023-06-23T18:24:20.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c19d510 con 0x7eff7c0ffc70 2023-06-23T18:24:20.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.625+0000 7eff84039700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c19d6a0 con 0x7eff7c0fea80 2023-06-23T18:24:20.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c0fea80 0x7eff7c195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff815d4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff7c0ffc70 0x7eff7c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c0fea80 0x7eff7c195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55020/0 (socket says 172.21.15.2:55020) 2023-06-23T18:24:20.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 -- 172.21.15.2:0/2939745080 learned_addr learned my addr 172.21.15.2:0/2939745080 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:20.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff825d6700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff7c100600 0x7eff7c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 -- 172.21.15.2:0/2939745080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff7c0ffc70 msgr2=0x7eff7c195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff7c0ffc70 0x7eff7c195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 -- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff7c100600 msgr2=0x7eff7c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff7c100600 0x7eff7c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 -- 172.21.15.2:0/2939745080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780095f0 con 0x7eff7c0fea80 2023-06-23T18:24:20.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff81dd5700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c0fea80 0x7eff7c195850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff78003ec0 tx=0x7eff78005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:20.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff72ffd700 1 -- 172.21.15.2:0/2939745080 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7801d070 con 0x7eff7c0fea80 2023-06-23T18:24:20.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff84039700 1 -- 172.21.15.2:0/2939745080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7c19d920 con 0x7eff7c0fea80 2023-06-23T18:24:20.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff72ffd700 1 -- 172.21.15.2:0/2939745080 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff7800bd50 con 0x7eff7c0fea80 2023-06-23T18:24:20.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff72ffd700 1 -- 172.21.15.2:0/2939745080 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78022d00 con 0x7eff7c0fea80 2023-06-23T18:24:20.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.626+0000 7eff84039700 1 -- 172.21.15.2:0/2939745080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c19dda0 con 0x7eff7c0fea80 2023-06-23T18:24:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.629+0000 7eff72ffd700 1 -- 172.21.15.2:0/2939745080 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7eff78022500 con 0x7eff7c0fea80 2023-06-23T18:24:20.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.629+0000 7eff72ffd700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7eff680a1150 0x7eff680a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.629+0000 7eff72ffd700 1 -- 172.21.15.2:0/2939745080 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7eff780cdd90 con 0x7eff7c0fea80 2023-06-23T18:24:20.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.629+0000 7eff815d4700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7eff680a1150 0x7eff680a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.630+0000 7eff815d4700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7eff680a1150 0x7eff680a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009c60 tx=0x7eff6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:20.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.630+0000 7eff901beb80 1 Processor -- start 2023-06-23T18:24:20.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.630+0000 7eff901beb80 1 -- start start 2023-06-23T18:24:20.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff901beb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff901beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff901beb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2c8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff901beb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b1cbcb0 con 0x563e0b2c6c80 2023-06-23T18:24:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff901beb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b21ea40 con 0x563e0b2c8000 2023-06-23T18:24:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff901beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b217ed0 con 0x563e0b2c75d0 2023-06-23T18:24:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54690/0 (socket says 172.21.15.2:54690) 2023-06-23T18:24:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 -- 172.21.15.2:0/1345610260 learned_addr learned my addr 172.21.15.2:0/1345610260 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff825d6700 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff815d4700 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 -- 172.21.15.2:0/1345610260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 msgr2=0x563e0b2c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 -- 172.21.15.2:0/1345610260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 msgr2=0x563e0b2c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 -- 172.21.15.2:0/1345610260 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e0b099630 con 0x563e0b2c6c80 2023-06-23T18:24:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff825d6700 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:24:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2c7090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7eff78097ca0 tx=0x7eff7809adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de376d9f992204ae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff67fff700 1 -- 172.21.15.2:0/1345610260 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7809b230 con 0x563e0b2c6c80 2023-06-23T18:24:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.631+0000 7eff67fff700 1 -- 172.21.15.2:0/1345610260 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff7809f260 con 0x563e0b2c6c80 2023-06-23T18:24:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.632+0000 7eff67fff700 1 -- 172.21.15.2:0/1345610260 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff780a3160 con 0x563e0b2c6c80 2023-06-23T18:24:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.632+0000 7eff901beb80 1 -- 172.21.15.2:0/1345610260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 msgr2=0x563e0b2c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.632+0000 7eff901beb80 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2c7090 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7eff78097ca0 tx=0x7eff7809adf0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.632+0000 7eff901beb80 1 -- 172.21.15.2:0/1345610260 shutdown_connections 2023-06-23T18:24:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.632+0000 7eff901beb80 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.632+0000 7eff901beb80 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2c7090 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.632+0000 7eff901beb80 1 --2- 172.21.15.2:0/1345610260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.632+0000 7eff901beb80 1 -- 172.21.15.2:0/1345610260 >> 172.21.15.2:0/1345610260 conn(0x563e0b2bc3f0 msgr2=0x563e0b2c1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.633+0000 7eff901beb80 1 -- 172.21.15.2:0/1345610260 shutdown_connections 2023-06-23T18:24:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.633+0000 7eff901beb80 1 -- 172.21.15.2:0/1345610260 wait complete. 2023-06-23T18:24:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.633+0000 7eff901beb80 1 -- start start 2023-06-23T18:24:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.633+0000 7eff901beb80 1 Processor -- start 2023-06-23T18:24:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff901beb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff901beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff901beb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2d77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff901beb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b1b7ac0 con 0x563e0b2c6c80 2023-06-23T18:24:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff901beb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b1b77c0 con 0x563e0b2c8000 2023-06-23T18:24:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff901beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b09b920 con 0x563e0b2c75d0 2023-06-23T18:24:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff901beb80 10 client.0 _ll_register_callbacks cb 0x563e0b1bd260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:24:20.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54696/0 (socket says 172.21.15.2:54696) 2023-06-23T18:24:20.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 -- 172.21.15.2:0/1025998804 learned_addr learned my addr 172.21.15.2:0/1025998804 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:20.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff825d6700 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2d77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff815d4700 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 -- 172.21.15.2:0/1025998804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 msgr2=0x563e0b2d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:20.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 -- 172.21.15.2:0/1025998804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 msgr2=0x563e0b2d77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2d77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 -- 172.21.15.2:0/1025998804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff74011230 con 0x563e0b2c6c80 2023-06-23T18:24:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2d0220 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7eff7809b890 tx=0x7eff780a3390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d74e4bf3500b1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff657fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:24:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff657fa700 1 -- 172.21.15.2:0/1025998804 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff780aeea0 con 0x563e0b2c6c80 2023-06-23T18:24:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff657fa700 1 -- 172.21.15.2:0/1025998804 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7eff78095e40 con 0x563e0b2c6c80 2023-06-23T18:24:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.634+0000 7eff901beb80 1 -- 172.21.15.2:0/1025998804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e0b2b9bb0 con 0x563e0b2c6c80 2023-06-23T18:24:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff657fa700 1 -- 172.21.15.2:0/1025998804 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff780a35f0 con 0x563e0b2c6c80 2023-06-23T18:24:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff901beb80 10 client.15657 fetch_fsmap learned FSMap version 26 2023-06-23T18:24:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff901beb80 1 -- 172.21.15.2:0/1025998804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7eff7c19dda0 con 0x563e0b2c6c80 2023-06-23T18:24:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff657fa700 1 -- 172.21.15.2:0/1025998804 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff780c9020 con 0x563e0b2c6c80 2023-06-23T18:24:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff657fa700 1 -- 172.21.15.2:0/1025998804 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7eff780b3c10 con 0x563e0b2c6c80 2023-06-23T18:24:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff901beb80 10 client.15657 fetch_fsmap finished waiting for FSMap version 26 2023-06-23T18:24:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff901beb80 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:24:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff901beb80 20 client.15657 populate_metadata read hostname 'smithi002' 2023-06-23T18:24:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff901beb80 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563e0b2dbb30 0x563e0b2ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff901beb80 4 client.15657 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:24:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff901beb80 1 -- 172.21.15.2:0/1025998804 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e0b210eb0 con 0x563e0b2dbb30 2023-06-23T18:24:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.635+0000 7eff815d4700 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563e0b2dbb30 0x563e0b2ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.636+0000 7eff657fa700 10 client.15657 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:24:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.636+0000 7eff815d4700 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563e0b2dbb30 0x563e0b2ddf40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df82401a429dc29 server_cookie=b6f5dabb3201f590 in_seq=0 out_seq=0 2023-06-23T18:24:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.636+0000 7eff657fa700 1 -- 172.21.15.2:0/1025998804 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563e0b210eb0 con 0x563e0b2dbb30 2023-06-23T18:24:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.636+0000 7eff657fa700 10 client.15657 handle_command_reply: tid=0 2023-06-23T18:24:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 client.15657 shutdown 2023-06-23T18:24:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 -- 172.21.15.2:0/1025998804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 msgr2=0x563e0b2d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2d0220 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7eff7809b890 tx=0x7eff780a3390 comp rx=0 tx=0).stop 2023-06-23T18:24:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 -- 172.21.15.2:0/1025998804 shutdown_connections 2023-06-23T18:24:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563e0b2dbb30 0x563e0b2ddf40 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c8000 0x563e0b2d77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c6c80 0x563e0b2d0220 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff657fa700 0 client.15657 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:24:20.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 --2- 172.21.15.2:0/1025998804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 -- 172.21.15.2:0/1025998804 >> 172.21.15.2:0/1025998804 conn(0x563e0b2bc3f0 msgr2=0x563e0b2c0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:20.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 -- 172.21.15.2:0/1025998804 shutdown_connections 2023-06-23T18:24:20.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 1 -- 172.21.15.2:0/1025998804 wait complete. 2023-06-23T18:24:20.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.637+0000 7eff901beb80 20 client.15657 trim_cache size 0 max 16384 2023-06-23T18:24:20.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff901beb80 1 Processor -- start 2023-06-23T18:24:20.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff901beb80 1 -- start start 2023-06-23T18:24:20.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff901beb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff901beb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c75d0 0x563e0b2d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff901beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c8000 0x563e0b2cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff901beb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b2cf6d0 con 0x563e0b2c75d0 2023-06-23T18:24:20.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff901beb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b2cec00 con 0x563e0b2c6c80 2023-06-23T18:24:20.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff901beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b2ce100 con 0x563e0b2c8000 2023-06-23T18:24:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff815d4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c75d0 0x563e0b2d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff81dd5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff815d4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c75d0 0x563e0b2d7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54706/0 (socket says 172.21.15.2:54706) 2023-06-23T18:24:20.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff825d6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c8000 0x563e0b2cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.658+0000 7eff815d4700 1 -- 172.21.15.2:0/1243181494 learned_addr learned my addr 172.21.15.2:0/1243181494 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:20.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff815d4700 1 -- 172.21.15.2:0/1243181494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c8000 msgr2=0x563e0b2cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:20.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff815d4700 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c8000 0x563e0b2cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff815d4700 1 -- 172.21.15.2:0/1243181494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 msgr2=0x563e0b2d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:20.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff815d4700 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff815d4700 1 -- 172.21.15.2:0/1243181494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e0b045de0 con 0x563e0b2c75d0 2023-06-23T18:24:20.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:24:20.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff825d6700 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c8000 0x563e0b2cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:24:20.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff815d4700 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c75d0 0x563e0b2d7760 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7eff6c013010 tx=0x7eff6c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a0d16d6da84962 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:20.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff70ff9700 1 -- 172.21.15.2:0/1243181494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c022070 con 0x563e0b2c75d0 2023-06-23T18:24:20.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff70ff9700 1 -- 172.21.15.2:0/1243181494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff6c023040 con 0x563e0b2c75d0 2023-06-23T18:24:20.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.659+0000 7eff70ff9700 1 -- 172.21.15.2:0/1243181494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c003d90 con 0x563e0b2c75d0 2023-06-23T18:24:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 -- 172.21.15.2:0/1243181494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c75d0 msgr2=0x563e0b2d7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c75d0 0x563e0b2d7760 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7eff6c013010 tx=0x7eff6c014000 comp rx=0 tx=0).stop 2023-06-23T18:24:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 -- 172.21.15.2:0/1243181494 shutdown_connections 2023-06-23T18:24:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c8000 0x563e0b2cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 --2- 172.21.15.2:0/1243181494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c75d0 0x563e0b2d7760 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 -- 172.21.15.2:0/1243181494 >> 172.21.15.2:0/1243181494 conn(0x563e0b2bc3f0 msgr2=0x563e0b2e28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 -- 172.21.15.2:0/1243181494 shutdown_connections 2023-06-23T18:24:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 -- 172.21.15.2:0/1243181494 wait complete. 2023-06-23T18:24:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 -- start start 2023-06-23T18:24:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.660+0000 7eff901beb80 1 Processor -- start 2023-06-23T18:24:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff901beb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff901beb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff901beb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c8000 0x563e0b2d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff901beb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b2d43c0 con 0x563e0b2c8000 2023-06-23T18:24:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff901beb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b2d4530 con 0x563e0b2c6c80 2023-06-23T18:24:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff901beb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e0b2d46f0 con 0x563e0b2c75d0 2023-06-23T18:24:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff901beb80 10 client.0 _ll_register_callbacks cb 0x563e0b1bd260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:24:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff815d4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff825d6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c8000 0x563e0b2d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff815d4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55050/0 (socket says 172.21.15.2:55050) 2023-06-23T18:24:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff81dd5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff815d4700 1 -- 172.21.15.2:0/1642222731 learned_addr learned my addr 172.21.15.2:0/1642222731 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff815d4700 1 -- 172.21.15.2:0/1642222731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 msgr2=0x563e0b2d7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff815d4700 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff815d4700 1 -- 172.21.15.2:0/1642222731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c8000 msgr2=0x563e0b2d3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff815d4700 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c8000 0x563e0b2d3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.661+0000 7eff815d4700 1 -- 172.21.15.2:0/1642222731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7c19dda0 con 0x563e0b2c75d0 2023-06-23T18:24:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff815d4700 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d76d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009220 tx=0x7eff6c003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc23d27e3714623e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:20.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff657fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:24:20.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff657fa700 1 -- 172.21.15.2:0/1642222731 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6c022070 con 0x563e0b2c75d0 2023-06-23T18:24:20.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 1 -- 172.21.15.2:0/1642222731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff7809f260 con 0x563e0b2c75d0 2023-06-23T18:24:20.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff657fa700 1 -- 172.21.15.2:0/1642222731 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7eff6c003f20 con 0x563e0b2c75d0 2023-06-23T18:24:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff657fa700 1 -- 172.21.15.2:0/1642222731 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff6c02b0f0 con 0x563e0b2c75d0 2023-06-23T18:24:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 10 client.25426 fetch_fsmap learned FSMap version 26 2023-06-23T18:24:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 1 -- 172.21.15.2:0/1642222731 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7eff740091b0 con 0x563e0b2c75d0 2023-06-23T18:24:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff657fa700 1 -- 172.21.15.2:0/1642222731 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7eff7809f260 con 0x563e0b2c75d0 2023-06-23T18:24:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff657fa700 1 -- 172.21.15.2:0/1642222731 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7eff6c033070 con 0x563e0b2c75d0 2023-06-23T18:24:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 10 client.25426 fetch_fsmap finished waiting for FSMap version 26 2023-06-23T18:24:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 10 client.25426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:24:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 20 client.25426 populate_metadata read hostname 'smithi002' 2023-06-23T18:24:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563e0b331150 0x563e0b333560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 4 client.25426 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:24:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.662+0000 7eff901beb80 1 -- 172.21.15.2:0/1642222731 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e0b1d3050 con 0x563e0b331150 2023-06-23T18:24:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563e0b331150 0x563e0b333560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff81dd5700 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563e0b331150 0x563e0b333560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36e050baa627c2a5 server_cookie=be8284eddbc30767 in_seq=0 out_seq=0 2023-06-23T18:24:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff657fa700 10 client.25426 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:24:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff657fa700 1 -- 172.21.15.2:0/1642222731 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x563e0b1d3050 con 0x563e0b331150 2023-06-23T18:24:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff657fa700 10 client.25426 handle_command_reply: tid=0 2023-06-23T18:24:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff901beb80 1 client.25426 shutdown 2023-06-23T18:24:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff901beb80 1 -- 172.21.15.2:0/1642222731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 msgr2=0x563e0b2d76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff901beb80 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d76d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009220 tx=0x7eff6c003680 comp rx=0 tx=0).stop 2023-06-23T18:24:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff901beb80 1 -- 172.21.15.2:0/1642222731 shutdown_connections 2023-06-23T18:24:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.663+0000 7eff901beb80 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563e0b331150 0x563e0b333560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff901beb80 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563e0b2c8000 0x563e0b2d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff657fa700 0 client.25426 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:24:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff901beb80 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563e0b2c6c80 0x563e0b2d7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff901beb80 1 --2- 172.21.15.2:0/1642222731 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563e0b2c75d0 0x563e0b2d76d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff901beb80 1 -- 172.21.15.2:0/1642222731 >> 172.21.15.2:0/1642222731 conn(0x563e0b2bc3f0 msgr2=0x563e0b2cae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff901beb80 1 -- 172.21.15.2:0/1642222731 shutdown_connections 2023-06-23T18:24:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff901beb80 1 -- 172.21.15.2:0/1642222731 wait complete. 2023-06-23T18:24:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff901beb80 20 client.25426 trim_cache size 0 max 16384 2023-06-23T18:24:20.690 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:24:20.690 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:24:20.690 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:24:20.690 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:24:20.690 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:24:20.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:24:20.690 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:24:20.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:24:20.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:24:20.691 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:24:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 -- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7eff680a1150 msgr2=0x7eff680a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7eff680a1150 0x7eff680a3600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff6c009c60 tx=0x7eff6c009260 comp rx=0 tx=0).stop 2023-06-23T18:24:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 -- 172.21.15.2:0/2939745080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c0fea80 msgr2=0x7eff7c195850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c0fea80 0x7eff7c195850 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff78003ec0 tx=0x7eff78005740 comp rx=0 tx=0).stop 2023-06-23T18:24:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 -- 172.21.15.2:0/2939745080 shutdown_connections 2023-06-23T18:24:20.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7eff7c100600 0x7eff7c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7c0fea80 0x7eff7c195850 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7eff680a1150 0x7eff680a3600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 --2- 172.21.15.2:0/2939745080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff7c0ffc70 0x7eff7c195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 -- 172.21.15.2:0/2939745080 >> 172.21.15.2:0/2939745080 conn(0x7eff7c0fa0e0 msgr2=0x7eff7c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 -- 172.21.15.2:0/2939745080 shutdown_connections 2023-06-23T18:24:20.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:20.664+0000 7eff84039700 1 -- 172.21.15.2:0/2939745080 wait complete. 2023-06-23T18:24:20.694 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:20 smithi002 ceph-mon[95947]: pgmap v365: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 34 op/s 2023-06-23T18:24:20.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:20 smithi017 ceph-mon[97472]: pgmap v365: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 34 op/s 2023-06-23T18:24:20.695 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:24:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:20 smithi082 ceph-mon[97889]: pgmap v365: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 MiB/s wr, 34 op/s 2023-06-23T18:24:21.817 INFO:teuthology.orchestra.run.smithi002.stderr:9400000 of 50000000 tuples (18%) done (elapsed 105.06 s, remaining 453.76 s) 2023-06-23T18:24:22.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:22 smithi017 ceph-mon[97472]: pgmap v366: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-06-23T18:24:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:9500000 of 50000000 tuples (19%) done (elapsed 105.94 s, remaining 451.62 s) 2023-06-23T18:24:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:22 smithi002 ceph-mon[95947]: pgmap v366: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-06-23T18:24:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:22 smithi082 ceph-mon[97889]: pgmap v366: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-06-23T18:24:23.975 INFO:teuthology.orchestra.run.smithi002.stderr:9600000 of 50000000 tuples (19%) done (elapsed 107.22 s, remaining 451.20 s) 2023-06-23T18:24:24.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:24 smithi017 ceph-mon[97472]: pgmap v367: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 MiB/s wr, 31 op/s 2023-06-23T18:24:24.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:24 smithi002 ceph-mon[95947]: pgmap v367: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 MiB/s wr, 31 op/s 2023-06-23T18:24:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:24 smithi082 ceph-mon[97889]: pgmap v367: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 MiB/s wr, 31 op/s 2023-06-23T18:24:25.223 INFO:teuthology.orchestra.run.smithi002.stderr:9700000 of 50000000 tuples (19%) done (elapsed 108.46 s, remaining 450.63 s) 2023-06-23T18:24:26.128 INFO:teuthology.orchestra.run.smithi002.stderr:9800000 of 50000000 tuples (19%) done (elapsed 109.37 s, remaining 448.64 s) 2023-06-23T18:24:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:26 smithi017 ceph-mon[97472]: pgmap v368: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 32 op/s 2023-06-23T18:24:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:26 smithi002 ceph-mon[95947]: pgmap v368: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 32 op/s 2023-06-23T18:24:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:26 smithi082 ceph-mon[97889]: pgmap v368: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 39 MiB/s wr, 32 op/s 2023-06-23T18:24:27.057 INFO:teuthology.orchestra.run.smithi002.stderr:9900000 of 50000000 tuples (19%) done (elapsed 110.30 s, remaining 446.76 s) 2023-06-23T18:24:27.968 INFO:teuthology.orchestra.run.smithi002.stderr:10000000 of 50000000 tuples (20%) done (elapsed 111.21 s, remaining 444.84 s) 2023-06-23T18:24:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:28 smithi017 ceph-mon[97472]: pgmap v369: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 28 op/s 2023-06-23T18:24:28.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:28 smithi002 ceph-mon[95947]: pgmap v369: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 28 op/s 2023-06-23T18:24:28.805 INFO:teuthology.orchestra.run.smithi002.stderr:10100000 of 50000000 tuples (20%) done (elapsed 112.05 s, remaining 442.64 s) 2023-06-23T18:24:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:28 smithi082 ceph-mon[97889]: pgmap v369: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 40 MiB/s wr, 28 op/s 2023-06-23T18:24:29.672 INFO:teuthology.orchestra.run.smithi002.stderr:10200000 of 50000000 tuples (20%) done (elapsed 112.91 s, remaining 440.58 s) 2023-06-23T18:24:30.564 INFO:teuthology.orchestra.run.smithi002.stderr:10300000 of 50000000 tuples (20%) done (elapsed 113.81 s, remaining 438.65 s) 2023-06-23T18:24:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:30 smithi017 ceph-mon[97472]: pgmap v370: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 39 op/s 2023-06-23T18:24:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:30 smithi002 ceph-mon[95947]: pgmap v370: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 39 op/s 2023-06-23T18:24:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:30 smithi082 ceph-mon[97889]: pgmap v370: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 39 op/s 2023-06-23T18:24:31.371 INFO:teuthology.orchestra.run.smithi002.stderr:10400000 of 50000000 tuples (20%) done (elapsed 114.61 s, remaining 436.41 s) 2023-06-23T18:24:32.215 INFO:teuthology.orchestra.run.smithi002.stderr:10500000 of 50000000 tuples (21%) done (elapsed 115.46 s, remaining 434.33 s) 2023-06-23T18:24:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:32 smithi017 ceph-mon[97472]: pgmap v371: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 39 MiB/s wr, 47 op/s 2023-06-23T18:24:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:32 smithi002 ceph-mon[95947]: pgmap v371: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 39 MiB/s wr, 47 op/s 2023-06-23T18:24:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:32 smithi082 ceph-mon[97889]: pgmap v371: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 39 MiB/s wr, 47 op/s 2023-06-23T18:24:33.098 INFO:teuthology.orchestra.run.smithi002.stderr:10600000 of 50000000 tuples (21%) done (elapsed 116.34 s, remaining 432.43 s) 2023-06-23T18:24:33.946 INFO:teuthology.orchestra.run.smithi002.stderr:10700000 of 50000000 tuples (21%) done (elapsed 117.19 s, remaining 430.42 s) 2023-06-23T18:24:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:34 smithi017 ceph-mon[97472]: pgmap v372: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 MiB/s wr, 49 op/s 2023-06-23T18:24:34.755 INFO:teuthology.orchestra.run.smithi002.stderr:10800000 of 50000000 tuples (21%) done (elapsed 118.00 s, remaining 428.28 s) 2023-06-23T18:24:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:34 smithi002 ceph-mon[95947]: pgmap v372: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 MiB/s wr, 49 op/s 2023-06-23T18:24:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:34 smithi082 ceph-mon[97889]: pgmap v372: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 46 MiB/s wr, 49 op/s 2023-06-23T18:24:35.617 INFO:teuthology.orchestra.run.smithi002.stderr:10900000 of 50000000 tuples (21%) done (elapsed 118.86 s, remaining 426.36 s) 2023-06-23T18:24:36.445 INFO:teuthology.orchestra.run.smithi002.stderr:11000000 of 50000000 tuples (22%) done (elapsed 119.69 s, remaining 424.34 s) 2023-06-23T18:24:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:36 smithi017 ceph-mon[97472]: pgmap v373: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:24:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:36 smithi002 ceph-mon[95947]: pgmap v373: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:24:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:36 smithi082 ceph-mon[97889]: pgmap v373: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:24:37.414 INFO:teuthology.orchestra.run.smithi002.stderr:11100000 of 50000000 tuples (22%) done (elapsed 120.66 s, remaining 422.84 s) 2023-06-23T18:24:38.243 INFO:teuthology.orchestra.run.smithi002.stderr:11200000 of 50000000 tuples (22%) done (elapsed 121.48 s, remaining 420.86 s) 2023-06-23T18:24:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:38 smithi017 ceph-mon[97472]: pgmap v374: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 42 op/s 2023-06-23T18:24:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:38 smithi002 ceph-mon[95947]: pgmap v374: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 42 op/s 2023-06-23T18:24:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:38 smithi082 ceph-mon[97889]: pgmap v374: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 42 op/s 2023-06-23T18:24:39.086 INFO:teuthology.orchestra.run.smithi002.stderr:11300000 of 50000000 tuples (22%) done (elapsed 122.33 s, remaining 418.95 s) 2023-06-23T18:24:39.903 INFO:teuthology.orchestra.run.smithi002.stderr:11400000 of 50000000 tuples (22%) done (elapsed 123.14 s, remaining 416.96 s) 2023-06-23T18:24:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:40 smithi017 ceph-mon[97472]: pgmap v375: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 45 op/s 2023-06-23T18:24:40.774 INFO:teuthology.orchestra.run.smithi002.stderr:11500000 of 50000000 tuples (23%) done (elapsed 124.02 s, remaining 415.18 s) 2023-06-23T18:24:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:40 smithi002 ceph-mon[95947]: pgmap v375: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 45 op/s 2023-06-23T18:24:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:40 smithi082 ceph-mon[97889]: pgmap v375: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 41 MiB/s wr, 45 op/s 2023-06-23T18:24:41.628 INFO:teuthology.orchestra.run.smithi002.stderr:11600000 of 50000000 tuples (23%) done (elapsed 124.87 s, remaining 413.36 s) 2023-06-23T18:24:42.457 INFO:teuthology.orchestra.run.smithi002.stderr:11700000 of 50000000 tuples (23%) done (elapsed 125.70 s, remaining 411.47 s) 2023-06-23T18:24:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:42 smithi017 ceph-mon[97472]: pgmap v376: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 34 op/s 2023-06-23T18:24:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:42 smithi002 ceph-mon[95947]: pgmap v376: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 34 op/s 2023-06-23T18:24:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:42 smithi082 ceph-mon[97889]: pgmap v376: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 34 op/s 2023-06-23T18:24:43.324 INFO:teuthology.orchestra.run.smithi002.stderr:11800000 of 50000000 tuples (23%) done (elapsed 126.57 s, remaining 409.73 s) 2023-06-23T18:24:44.188 INFO:teuthology.orchestra.run.smithi002.stderr:11900000 of 50000000 tuples (23%) done (elapsed 127.43 s, remaining 407.99 s) 2023-06-23T18:24:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:44 smithi017 ceph-mon[97472]: pgmap v377: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 MiB/s wr, 26 op/s 2023-06-23T18:24:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:44 smithi002 ceph-mon[95947]: pgmap v377: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 MiB/s wr, 26 op/s 2023-06-23T18:24:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:44 smithi082 ceph-mon[97889]: pgmap v377: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 MiB/s wr, 26 op/s 2023-06-23T18:24:44.992 INFO:teuthology.orchestra.run.smithi002.stderr:12000000 of 50000000 tuples (24%) done (elapsed 128.23 s, remaining 406.07 s) 2023-06-23T18:24:45.867 INFO:teuthology.orchestra.run.smithi002.stderr:12100000 of 50000000 tuples (24%) done (elapsed 129.11 s, remaining 404.40 s) 2023-06-23T18:24:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:46 smithi017 ceph-mon[97472]: pgmap v378: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:24:46.731 INFO:teuthology.orchestra.run.smithi002.stderr:12200000 of 50000000 tuples (24%) done (elapsed 129.97 s, remaining 402.70 s) 2023-06-23T18:24:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:46 smithi002 ceph-mon[95947]: pgmap v378: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:24:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:46 smithi082 ceph-mon[97889]: pgmap v378: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:24:47.556 INFO:teuthology.orchestra.run.smithi002.stderr:12300000 of 50000000 tuples (24%) done (elapsed 130.80 s, remaining 400.90 s) 2023-06-23T18:24:48.373 INFO:teuthology.orchestra.run.smithi002.stderr:12400000 of 50000000 tuples (24%) done (elapsed 131.61 s, remaining 399.09 s) 2023-06-23T18:24:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:48 smithi017 ceph-mon[97472]: pgmap v379: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-06-23T18:24:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:48 smithi002 ceph-mon[95947]: pgmap v379: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-06-23T18:24:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:48 smithi082 ceph-mon[97889]: pgmap v379: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-06-23T18:24:49.259 INFO:teuthology.orchestra.run.smithi002.stderr:12500000 of 50000000 tuples (25%) done (elapsed 132.50 s, remaining 397.50 s) 2023-06-23T18:24:50.075 INFO:teuthology.orchestra.run.smithi002.stderr:12600000 of 50000000 tuples (25%) done (elapsed 133.32 s, remaining 395.72 s) 2023-06-23T18:24:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:50 smithi017 ceph-mon[97472]: pgmap v380: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-06-23T18:24:50.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:24:50.723 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:50 smithi002 ceph-mon[95947]: pgmap v380: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-06-23T18:24:50.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.833+0000 7f28d593c700 1 -- 172.21.15.2:0/1637686254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 msgr2=0x7f28d00feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.833+0000 7f28d593c700 1 --2- 172.21.15.2:0/1637686254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 0x7f28d00feea0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009990 tx=0x7f28c0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 -- 172.21.15.2:0/1637686254 shutdown_connections 2023-06-23T18:24:50.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 --2- 172.21.15.2:0/1637686254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0100610 0x7f28d0106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 --2- 172.21.15.2:0/1637686254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 0x7f28d00feea0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 --2- 172.21.15.2:0/1637686254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28d00ffc80 0x7f28d01000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 -- 172.21.15.2:0/1637686254 >> 172.21.15.2:0/1637686254 conn(0x7f28d00fa090 msgr2=0x7f28d00fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 -- 172.21.15.2:0/1637686254 shutdown_connections 2023-06-23T18:24:50.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 -- 172.21.15.2:0/1637686254 wait complete. 2023-06-23T18:24:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 Processor -- start 2023-06-23T18:24:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.834+0000 7f28d593c700 1 -- start start 2023-06-23T18:24:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28d593c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 0x7f28d0195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28d593c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28d00ffc80 0x7f28d0195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28d593c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0100610 0x7f28d019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28d593c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d019d3b0 con 0x7f28d00fea90 2023-06-23T18:24:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28d593c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d019d4f0 con 0x7f28d0100610 2023-06-23T18:24:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28d593c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d019d680 con 0x7f28d00ffc80 2023-06-23T18:24:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 0x7f28d0195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 0x7f28d0195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54724/0 (socket says 172.21.15.2:54724) 2023-06-23T18:24:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 -- 172.21.15.2:0/2783973066 learned_addr learned my addr 172.21.15.2:0/2783973066 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28c67fc700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28d00ffc80 0x7f28d0195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0100610 0x7f28d019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 -- 172.21.15.2:0/2783973066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28d00ffc80 msgr2=0x7f28d0195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28d00ffc80 0x7f28d0195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 -- 172.21.15.2:0/2783973066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0100610 msgr2=0x7f28d019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0100610 0x7f28d019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 -- 172.21.15.2:0/2783973066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c00095f0 con 0x7f28d00fea90 2023-06-23T18:24:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28c67fc700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28d00ffc80 0x7f28d0195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:24:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.835+0000 7f28ceffd700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 0x7f28d0195830 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28c00098e0 tx=0x7f28c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.836+0000 7f28d493a700 1 -- 172.21.15.2:0/2783973066 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c001d070 con 0x7f28d00fea90 2023-06-23T18:24:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.836+0000 7f28d593c700 1 -- 172.21.15.2:0/2783973066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d019d900 con 0x7f28d00fea90 2023-06-23T18:24:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.836+0000 7f28d493a700 1 -- 172.21.15.2:0/2783973066 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c000bd50 con 0x7f28d00fea90 2023-06-23T18:24:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.836+0000 7f28d493a700 1 -- 172.21.15.2:0/2783973066 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0022cf0 con 0x7f28d00fea90 2023-06-23T18:24:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.836+0000 7f28d593c700 1 -- 172.21.15.2:0/2783973066 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d019dd80 con 0x7f28d00fea90 2023-06-23T18:24:50.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.838+0000 7f28d493a700 1 -- 172.21.15.2:0/2783973066 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f28c0022470 con 0x7f28d00fea90 2023-06-23T18:24:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.839+0000 7f28d493a700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28bc0a1150 0x7f28bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.839+0000 7f28d493a700 1 -- 172.21.15.2:0/2783973066 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28c00cdf50 con 0x7f28d00fea90 2023-06-23T18:24:50.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.839+0000 7f28c67fc700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28bc0a1150 0x7f28bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:50 smithi082 ceph-mon[97889]: pgmap v380: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-06-23T18:24:50.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28e1ac1b80 1 Processor -- start 2023-06-23T18:24:50.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28e1ac1b80 1 -- start start 2023-06-23T18:24:50.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28c67fc700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28bc0a1150 0x7f28bc0a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28b8011490 tx=0x7f28b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:50.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab78c80 0x5563fab79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab79a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab7a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563faa7dcb0 con 0x5563fab795d0 2023-06-23T18:24:50.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563faad0a40 con 0x5563fab78c80 2023-06-23T18:24:50.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563faac9ed0 con 0x5563fab7a000 2023-06-23T18:24:50.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28c67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab79a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28c67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab79a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54734/0 (socket says 172.21.15.2:54734) 2023-06-23T18:24:50.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28c67fc700 1 -- 172.21.15.2:0/3036465398 learned_addr learned my addr 172.21.15.2:0/3036465398 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:50.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab7a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28ceffd700 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab78c80 0x5563fab79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28cf7fe700 1 -- 172.21.15.2:0/3036465398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab78c80 msgr2=0x5563fab79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:50.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab78c80 0x5563fab79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28cf7fe700 1 -- 172.21.15.2:0/3036465398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 msgr2=0x5563fab79a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:50.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab79a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28cf7fe700 1 -- 172.21.15.2:0/3036465398 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563fa94b630 con 0x5563fab7a000 2023-06-23T18:24:50.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.840+0000 7f28c67fc700 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab79a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:24:50.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.841+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab7a470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f28c8011580 tx=0x7f28c8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec70d227142a78f0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:50.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.841+0000 7f28c5ffb700 1 -- 172.21.15.2:0/3036465398 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c800e040 con 0x5563fab7a000 2023-06-23T18:24:50.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.841+0000 7f28c5ffb700 1 -- 172.21.15.2:0/3036465398 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c800f040 con 0x5563fab7a000 2023-06-23T18:24:50.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.841+0000 7f28c5ffb700 1 -- 172.21.15.2:0/3036465398 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8004300 con 0x5563fab7a000 2023-06-23T18:24:50.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.841+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/3036465398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 msgr2=0x5563fab7a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.841+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab7a470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f28c8011580 tx=0x7f28c8009920 comp rx=0 tx=0).stop 2023-06-23T18:24:50.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/3036465398 shutdown_connections 2023-06-23T18:24:50.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab7a470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab78c80 0x5563fab79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/3036465398 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab79a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/3036465398 >> 172.21.15.2:0/3036465398 conn(0x5563fab6e3f0 msgr2=0x5563fab73520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:50.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/3036465398 shutdown_connections 2023-06-23T18:24:50.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/3036465398 wait complete. 2023-06-23T18:24:50.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 -- start start 2023-06-23T18:24:50.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 Processor -- start 2023-06-23T18:24:50.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab78c80 0x5563fab82520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab82a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563faa69ac0 con 0x5563fab795d0 2023-06-23T18:24:50.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563faa697c0 con 0x5563fab7a000 2023-06-23T18:24:50.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.842+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563fa94d920 con 0x5563fab78c80 2023-06-23T18:24:50.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28e1ac1b80 10 client.0 _ll_register_callbacks cb 0x5563faa6f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:24:50.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab78c80 0x5563fab82520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab78c80 0x5563fab82520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55076/0 (socket says 172.21.15.2:55076) 2023-06-23T18:24:50.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28ceffd700 1 -- 172.21.15.2:0/2267270225 learned_addr learned my addr 172.21.15.2:0/2267270225 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:50.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28cf7fe700 1 -- 172.21.15.2:0/2267270225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab78c80 msgr2=0x5563fab82520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:50.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab78c80 0x5563fab82520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28cf7fe700 1 -- 172.21.15.2:0/2267270225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 msgr2=0x5563fab82a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:50.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab82a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28cf7fe700 1 -- 172.21.15.2:0/2267270225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8017070 con 0x5563fab7a000 2023-06-23T18:24:50.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000f80 tx=0x7f28c8011f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a57a2a45597707 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:50.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:24:50.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28aeffd700 1 -- 172.21.15.2:0/2267270225 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c800e040 con 0x5563fab7a000 2023-06-23T18:24:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/2267270225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563fab6bbb0 con 0x5563fab7a000 2023-06-23T18:24:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.843+0000 7f28aeffd700 1 -- 172.21.15.2:0/2267270225 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28c80040a0 con 0x5563fab7a000 2023-06-23T18:24:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28aeffd700 1 -- 172.21.15.2:0/2267270225 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28c8004260 con 0x5563fab7a000 2023-06-23T18:24:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28e1ac1b80 10 client.25277 fetch_fsmap learned FSMap version 26 2023-06-23T18:24:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/2267270225 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f28c8011230 con 0x5563fab7a000 2023-06-23T18:24:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28aeffd700 1 -- 172.21.15.2:0/2267270225 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5563fab6bbb0 con 0x5563fab7a000 2023-06-23T18:24:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28aeffd700 1 -- 172.21.15.2:0/2267270225 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f28c801b020 con 0x5563fab7a000 2023-06-23T18:24:50.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28e1ac1b80 10 client.25277 fetch_fsmap finished waiting for FSMap version 26 2023-06-23T18:24:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28e1ac1b80 10 client.25277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:24:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28e1ac1b80 20 client.25277 populate_metadata read hostname 'smithi002' 2023-06-23T18:24:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5563fab8e290 0x5563fab906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28e1ac1b80 4 client.25277 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:24:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/2267270225 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563faac2eb0 con 0x5563fab8e290 2023-06-23T18:24:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.844+0000 7f28ceffd700 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5563fab8e290 0x5563fab906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.845+0000 7f28aeffd700 10 client.25277 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:24:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.845+0000 7f28ceffd700 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5563fab8e290 0x5563fab906a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6891550098b7543 server_cookie=6e68cb478a55268 in_seq=0 out_seq=0 2023-06-23T18:24:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.845+0000 7f28aeffd700 1 -- 172.21.15.2:0/2267270225 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5563faac2eb0 con 0x5563fab8e290 2023-06-23T18:24:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.845+0000 7f28aeffd700 10 client.25277 handle_command_reply: tid=0 2023-06-23T18:24:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 client.25277 shutdown 2023-06-23T18:24:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 msgr2=0x5563fab89aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f28c8000f80 tx=0x7f28c8011f00 comp rx=0 tx=0).stop 2023-06-23T18:24:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/2267270225 shutdown_connections 2023-06-23T18:24:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5563fab8e290 0x5563fab906a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28aeffd700 0 client.25277 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:24:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab78c80 0x5563fab82520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/2267270225 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab795d0 0x5563fab82a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/2267270225 >> 172.21.15.2:0/2267270225 conn(0x5563fab6e3f0 msgr2=0x5563fab72c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/2267270225 shutdown_connections 2023-06-23T18:24:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/2267270225 wait complete. 2023-06-23T18:24:50.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.846+0000 7f28e1ac1b80 20 client.25277 trim_cache size 0 max 16384 2023-06-23T18:24:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.865+0000 7f28e1ac1b80 1 Processor -- start 2023-06-23T18:24:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.865+0000 7f28e1ac1b80 1 -- start start 2023-06-23T18:24:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab80330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab795d0 0x5563fab80870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563fab73f60 con 0x5563fab78c80 2023-06-23T18:24:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563fab819d0 con 0x5563fab7a000 2023-06-23T18:24:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563fab820c0 con 0x5563fab795d0 2023-06-23T18:24:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28c67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab795d0 0x5563fab80870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab80330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28cf7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab80330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54746/0 (socket says 172.21.15.2:54746) 2023-06-23T18:24:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28c67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab795d0 0x5563fab80870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55086/0 (socket says 172.21.15.2:55086) 2023-06-23T18:24:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28ceffd700 1 -- 172.21.15.2:0/1235003296 learned_addr learned my addr 172.21.15.2:0/1235003296 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28c67fc700 1 -- 172.21.15.2:0/1235003296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 msgr2=0x5563fab89a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28c67fc700 1 --2- 172.21.15.2:0/1235003296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28c67fc700 1 -- 172.21.15.2:0/1235003296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 msgr2=0x5563fab80330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28c67fc700 1 --2- 172.21.15.2:0/1235003296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab80330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28c67fc700 1 -- 172.21.15.2:0/1235003296 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563fa8f7de0 con 0x5563fab795d0 2023-06-23T18:24:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.866+0000 7f28c67fc700 1 --2- 172.21.15.2:0/1235003296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab795d0 0x5563fab80870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5563fab799e0 tx=0x7f28b8013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aba33b77a4a0144a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:50.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28c6ffd700 1 -- 172.21.15.2:0/1235003296 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b8003cd0 con 0x5563fab795d0 2023-06-23T18:24:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28c6ffd700 1 -- 172.21.15.2:0/1235003296 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28b8003e30 con 0x5563fab795d0 2023-06-23T18:24:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28c6ffd700 1 -- 172.21.15.2:0/1235003296 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b800e610 con 0x5563fab795d0 2023-06-23T18:24:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1235003296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab795d0 msgr2=0x5563fab80870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1235003296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab795d0 0x5563fab80870 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5563fab799e0 tx=0x7f28b8013040 comp rx=0 tx=0).stop 2023-06-23T18:24:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1235003296 shutdown_connections 2023-06-23T18:24:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1235003296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab7a000 0x5563fab89a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1235003296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.867+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1235003296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab795d0 0x5563fab80870 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1235003296 >> 172.21.15.2:0/1235003296 conn(0x5563fab6e3f0 msgr2=0x5563fab95000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1235003296 shutdown_connections 2023-06-23T18:24:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1235003296 wait complete. 2023-06-23T18:24:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 -- start start 2023-06-23T18:24:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 Processor -- start 2023-06-23T18:24:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab89810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab795d0 0x5563fab862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab867f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563fab86d30 con 0x5563fab78c80 2023-06-23T18:24:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563fab86ea0 con 0x5563fab795d0 2023-06-23T18:24:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.868+0000 7f28e1ac1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563fabe2140 con 0x5563fab7a000 2023-06-23T18:24:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28e1ac1b80 10 client.0 _ll_register_callbacks cb 0x5563faa6f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:24:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab89810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28c67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab795d0 0x5563fab862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab89810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54752/0 (socket says 172.21.15.2:54752) 2023-06-23T18:24:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28c67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab795d0 0x5563fab862b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45346/0 (socket says 172.21.15.2:45346) 2023-06-23T18:24:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 -- 172.21.15.2:0/1302538580 learned_addr learned my addr 172.21.15.2:0/1302538580 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:24:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 -- 172.21.15.2:0/1302538580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 msgr2=0x5563fab867f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab867f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 -- 172.21.15.2:0/1302538580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab795d0 msgr2=0x5563fab862b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:24:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab795d0 0x5563fab862b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 -- 172.21.15.2:0/1302538580 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b8004100 con 0x5563fab78c80 2023-06-23T18:24:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28c67fc700 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab795d0 0x5563fab862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:24:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28cf7fe700 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:24:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28ceffd700 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab89810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f28c0003ec0 tx=0x7f28c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a847f94d79879b3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:24:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:24:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28aeffd700 1 -- 172.21.15.2:0/1302538580 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00860b0 con 0x5563fab78c80 2023-06-23T18:24:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28aeffd700 1 -- 172.21.15.2:0/1302538580 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28c009f030 con 0x5563fab78c80 2023-06-23T18:24:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.869+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1302538580 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28c800f040 con 0x5563fab78c80 2023-06-23T18:24:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28aeffd700 1 -- 172.21.15.2:0/1302538580 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28c009f1f0 con 0x5563fab78c80 2023-06-23T18:24:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28e1ac1b80 10 client.15678 fetch_fsmap learned FSMap version 26 2023-06-23T18:24:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1302538580 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5563fa8e0c60 con 0x5563fab78c80 2023-06-23T18:24:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28aeffd700 1 -- 172.21.15.2:0/1302538580 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c00b2030 con 0x5563fab78c80 2023-06-23T18:24:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28aeffd700 1 -- 172.21.15.2:0/1302538580 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f28c009a970 con 0x5563fab78c80 2023-06-23T18:24:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28e1ac1b80 10 client.15678 fetch_fsmap finished waiting for FSMap version 26 2023-06-23T18:24:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28e1ac1b80 10 client.15678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:24:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28e1ac1b80 20 client.15678 populate_metadata read hostname 'smithi002' 2023-06-23T18:24:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5563fab8e290 0x5563fabe5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:24:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28e1ac1b80 4 client.15678 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:24:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1302538580 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5563faa85050 con 0x5563fab8e290 2023-06-23T18:24:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.870+0000 7f28c67fc700 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5563fab8e290 0x5563fabe5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:24:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28aeffd700 10 client.15678 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:24:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28c67fc700 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5563fab8e290 0x5563fabe5890 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4a7897f53cc71cb server_cookie=fa90b1dbf8dad0bf in_seq=0 out_seq=0 2023-06-23T18:24:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28aeffd700 1 -- 172.21.15.2:0/1302538580 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5563faa85050 con 0x5563fab8e290 2023-06-23T18:24:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28aeffd700 10 client.15678 handle_command_reply: tid=0 2023-06-23T18:24:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 client.15678 shutdown 2023-06-23T18:24:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1302538580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 msgr2=0x5563fab89810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab89810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f28c0003ec0 tx=0x7f28c0005fd0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1302538580 shutdown_connections 2023-06-23T18:24:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5563fab8e290 0x5563fabe5890 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5563fab7a000 0x5563fab867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5563fab78c80 0x5563fab89810 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 --2- 172.21.15.2:0/1302538580 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5563fab795d0 0x5563fab862b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28aeffd700 0 client.15678 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:24:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1302538580 >> 172.21.15.2:0/1302538580 conn(0x5563fab6e3f0 msgr2=0x5563fab80330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1302538580 shutdown_connections 2023-06-23T18:24:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 1 -- 172.21.15.2:0/1302538580 wait complete. 2023-06-23T18:24:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.871+0000 7f28e1ac1b80 20 client.15678 trim_cache size 0 max 16384 2023-06-23T18:24:50.890 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:24:50.890 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:24:50.890 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:24:50.890 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:24:50.890 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:24:50.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:24:50.891 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:24:50.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:24:50.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:24:50.891 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:24:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 -- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28bc0a1150 msgr2=0x7f28bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28bc0a1150 0x7f28bc0a3600 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f28b8011490 tx=0x7f28b8009260 comp rx=0 tx=0).stop 2023-06-23T18:24:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 -- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 msgr2=0x7f28d0195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:24:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 0x7f28d0195830 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28c00098e0 tx=0x7f28c0004730 comp rx=0 tx=0).stop 2023-06-23T18:24:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 -- 172.21.15.2:0/2783973066 shutdown_connections 2023-06-23T18:24:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28bc0a1150 0x7f28bc0a3600 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0100610 0x7f28d019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28d00fea90 0x7f28d0195830 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 --2- 172.21.15.2:0/2783973066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f28d00ffc80 0x7f28d0195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:24:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 -- 172.21.15.2:0/2783973066 >> 172.21.15.2:0/2783973066 conn(0x7f28d00fa090 msgr2=0x7f28d00fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:24:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 -- 172.21.15.2:0/2783973066 shutdown_connections 2023-06-23T18:24:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:24:50.872+0000 7f28d593c700 1 -- 172.21.15.2:0/2783973066 wait complete. 2023-06-23T18:24:50.894 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:24:50.930 INFO:teuthology.orchestra.run.smithi002.stderr:12700000 of 50000000 tuples (25%) done (elapsed 134.17 s, remaining 394.06 s) 2023-06-23T18:24:51.749 INFO:teuthology.orchestra.run.smithi002.stderr:12800000 of 50000000 tuples (25%) done (elapsed 134.99 s, remaining 392.32 s) 2023-06-23T18:24:52.641 INFO:teuthology.orchestra.run.smithi002.stderr:12900000 of 50000000 tuples (25%) done (elapsed 135.88 s, remaining 390.79 s) 2023-06-23T18:24:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:52 smithi017 ceph-mon[97472]: pgmap v381: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:24:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:52 smithi002 ceph-mon[95947]: pgmap v381: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:24:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:52 smithi082 ceph-mon[97889]: pgmap v381: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:24:53.515 INFO:teuthology.orchestra.run.smithi002.stderr:13000000 of 50000000 tuples (26%) done (elapsed 136.76 s, remaining 389.23 s) 2023-06-23T18:24:54.339 INFO:teuthology.orchestra.run.smithi002.stderr:13100000 of 50000000 tuples (26%) done (elapsed 137.58 s, remaining 387.54 s) 2023-06-23T18:24:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:54 smithi017 ceph-mon[97472]: pgmap v382: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:24:54.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:54 smithi002 ceph-mon[95947]: pgmap v382: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:24:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:54 smithi082 ceph-mon[97889]: pgmap v382: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:24:55.213 INFO:teuthology.orchestra.run.smithi002.stderr:13200000 of 50000000 tuples (26%) done (elapsed 138.45 s, remaining 385.99 s) 2023-06-23T18:24:56.041 INFO:teuthology.orchestra.run.smithi002.stderr:13300000 of 50000000 tuples (26%) done (elapsed 139.28 s, remaining 384.33 s) 2023-06-23T18:24:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:56 smithi017 ceph-mon[97472]: pgmap v383: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:24:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:56 smithi002 ceph-mon[95947]: pgmap v383: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:24:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:56 smithi082 ceph-mon[97889]: pgmap v383: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:24:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:13400000 of 50000000 tuples (26%) done (elapsed 140.14 s, remaining 382.77 s) 2023-06-23T18:24:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:57 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:24:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:57 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:57 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:24:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:57.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:57 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:57.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:57 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:24:57.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:57 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:57.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:57 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:57.862 INFO:teuthology.orchestra.run.smithi002.stderr:13500000 of 50000000 tuples (27%) done (elapsed 141.10 s, remaining 381.50 s) 2023-06-23T18:24:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:58 smithi017 ceph-mon[97472]: pgmap v384: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 42 MiB/s wr, 22 op/s 2023-06-23T18:24:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:24:58.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:24:58.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:24:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:58.735 INFO:teuthology.orchestra.run.smithi002.stderr:13600000 of 50000000 tuples (27%) done (elapsed 141.98 s, remaining 379.99 s) 2023-06-23T18:24:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:58 smithi002 ceph-mon[95947]: pgmap v384: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 42 MiB/s wr, 22 op/s 2023-06-23T18:24:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:24:58.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:24:58.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:24:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:58 smithi082 ceph-mon[97889]: pgmap v384: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 42 MiB/s wr, 22 op/s 2023-06-23T18:24:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:24:58.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:24:58.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:24:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:24:59.636 INFO:teuthology.orchestra.run.smithi002.stderr:13700000 of 50000000 tuples (27%) done (elapsed 142.88 s, remaining 378.57 s) 2023-06-23T18:25:00.659 INFO:teuthology.orchestra.run.smithi002.stderr:13800000 of 50000000 tuples (27%) done (elapsed 143.90 s, remaining 377.48 s) 2023-06-23T18:25:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:00 smithi017 ceph-mon[97472]: pgmap v385: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:25:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:00 smithi002 ceph-mon[95947]: pgmap v385: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:25:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:00 smithi082 ceph-mon[97889]: pgmap v385: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:25:01.497 INFO:teuthology.orchestra.run.smithi002.stderr:13900000 of 50000000 tuples (27%) done (elapsed 144.74 s, remaining 375.90 s) 2023-06-23T18:25:02.316 INFO:teuthology.orchestra.run.smithi002.stderr:14000000 of 50000000 tuples (28%) done (elapsed 145.56 s, remaining 374.29 s) 2023-06-23T18:25:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:02 smithi017 ceph-mon[97472]: pgmap v386: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 37 MiB/s wr, 26 op/s 2023-06-23T18:25:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:02 smithi002 ceph-mon[95947]: pgmap v386: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 37 MiB/s wr, 26 op/s 2023-06-23T18:25:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:02 smithi082 ceph-mon[97889]: pgmap v386: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 37 MiB/s wr, 26 op/s 2023-06-23T18:25:03.142 INFO:teuthology.orchestra.run.smithi002.stderr:14100000 of 50000000 tuples (28%) done (elapsed 146.38 s, remaining 372.71 s) 2023-06-23T18:25:04.042 INFO:teuthology.orchestra.run.smithi002.stderr:14200000 of 50000000 tuples (28%) done (elapsed 147.28 s, remaining 371.32 s) 2023-06-23T18:25:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:04 smithi017 ceph-mon[97472]: pgmap v387: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 45 MiB/s wr, 36 op/s 2023-06-23T18:25:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:04 smithi002 ceph-mon[95947]: pgmap v387: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 45 MiB/s wr, 36 op/s 2023-06-23T18:25:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:04 smithi082 ceph-mon[97889]: pgmap v387: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 45 MiB/s wr, 36 op/s 2023-06-23T18:25:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:14300000 of 50000000 tuples (28%) done (elapsed 148.10 s, remaining 369.72 s) 2023-06-23T18:25:05.675 INFO:teuthology.orchestra.run.smithi002.stderr:14400000 of 50000000 tuples (28%) done (elapsed 148.92 s, remaining 368.15 s) 2023-06-23T18:25:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:14500000 of 50000000 tuples (29%) done (elapsed 149.78 s, remaining 366.71 s) 2023-06-23T18:25:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:06 smithi017 ceph-mon[97472]: pgmap v388: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 42 MiB/s wr, 57 op/s 2023-06-23T18:25:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:06 smithi002 ceph-mon[95947]: pgmap v388: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 42 MiB/s wr, 57 op/s 2023-06-23T18:25:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:06 smithi082 ceph-mon[97889]: pgmap v388: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 42 MiB/s wr, 57 op/s 2023-06-23T18:25:07.431 INFO:teuthology.orchestra.run.smithi002.stderr:14600000 of 50000000 tuples (29%) done (elapsed 150.67 s, remaining 365.33 s) 2023-06-23T18:25:08.263 INFO:teuthology.orchestra.run.smithi002.stderr:14700000 of 50000000 tuples (29%) done (elapsed 151.50 s, remaining 363.82 s) 2023-06-23T18:25:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:08 smithi017 ceph-mon[97472]: pgmap v389: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 MiB/s wr, 55 op/s 2023-06-23T18:25:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:08 smithi002 ceph-mon[95947]: pgmap v389: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 MiB/s wr, 55 op/s 2023-06-23T18:25:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:08 smithi082 ceph-mon[97889]: pgmap v389: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 37 MiB/s wr, 55 op/s 2023-06-23T18:25:09.095 INFO:teuthology.orchestra.run.smithi002.stderr:14800000 of 50000000 tuples (29%) done (elapsed 152.34 s, remaining 362.31 s) 2023-06-23T18:25:09.974 INFO:teuthology.orchestra.run.smithi002.stderr:14900000 of 50000000 tuples (29%) done (elapsed 153.22 s, remaining 360.93 s) 2023-06-23T18:25:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:10 smithi017 ceph-mon[97472]: pgmap v390: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 40 MiB/s wr, 58 op/s 2023-06-23T18:25:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:10 smithi002 ceph-mon[95947]: pgmap v390: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 40 MiB/s wr, 58 op/s 2023-06-23T18:25:10.848 INFO:teuthology.orchestra.run.smithi002.stderr:15000000 of 50000000 tuples (30%) done (elapsed 154.09 s, remaining 359.54 s) 2023-06-23T18:25:10.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:10 smithi082 ceph-mon[97889]: pgmap v390: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 40 MiB/s wr, 58 op/s 2023-06-23T18:25:11.668 INFO:teuthology.orchestra.run.smithi002.stderr:15100000 of 50000000 tuples (30%) done (elapsed 154.91 s, remaining 358.03 s) 2023-06-23T18:25:12.545 INFO:teuthology.orchestra.run.smithi002.stderr:15200000 of 50000000 tuples (30%) done (elapsed 155.79 s, remaining 356.67 s) 2023-06-23T18:25:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:12 smithi017 ceph-mon[97472]: pgmap v391: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 35 MiB/s wr, 56 op/s 2023-06-23T18:25:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:12 smithi002 ceph-mon[95947]: pgmap v391: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 35 MiB/s wr, 56 op/s 2023-06-23T18:25:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:12 smithi082 ceph-mon[97889]: pgmap v391: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 35 MiB/s wr, 56 op/s 2023-06-23T18:25:13.443 INFO:teuthology.orchestra.run.smithi002.stderr:15300000 of 50000000 tuples (30%) done (elapsed 156.68 s, remaining 355.36 s) 2023-06-23T18:25:14.286 INFO:teuthology.orchestra.run.smithi002.stderr:15400000 of 50000000 tuples (30%) done (elapsed 157.53 s, remaining 353.93 s) 2023-06-23T18:25:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:14 smithi017 ceph-mon[97472]: pgmap v392: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 47 MiB/s wr, 54 op/s 2023-06-23T18:25:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:14 smithi002 ceph-mon[95947]: pgmap v392: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 47 MiB/s wr, 54 op/s 2023-06-23T18:25:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:14 smithi082 ceph-mon[97889]: pgmap v392: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 47 MiB/s wr, 54 op/s 2023-06-23T18:25:15.112 INFO:teuthology.orchestra.run.smithi002.stderr:15500000 of 50000000 tuples (31%) done (elapsed 158.35 s, remaining 352.46 s) 2023-06-23T18:25:15.995 INFO:teuthology.orchestra.run.smithi002.stderr:15600000 of 50000000 tuples (31%) done (elapsed 159.24 s, remaining 351.14 s) 2023-06-23T18:25:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:16 smithi017 ceph-mon[97472]: pgmap v393: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:25:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:16 smithi002 ceph-mon[95947]: pgmap v393: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:25:16.839 INFO:teuthology.orchestra.run.smithi002.stderr:15700000 of 50000000 tuples (31%) done (elapsed 160.08 s, remaining 349.73 s) 2023-06-23T18:25:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:16 smithi082 ceph-mon[97889]: pgmap v393: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:25:17.657 INFO:teuthology.orchestra.run.smithi002.stderr:15800000 of 50000000 tuples (31%) done (elapsed 160.90 s, remaining 348.27 s) 2023-06-23T18:25:18.567 INFO:teuthology.orchestra.run.smithi002.stderr:15900000 of 50000000 tuples (31%) done (elapsed 161.81 s, remaining 347.02 s) 2023-06-23T18:25:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:18 smithi017 ceph-mon[97472]: pgmap v394: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:25:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:18 smithi002 ceph-mon[95947]: pgmap v394: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:25:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:18 smithi082 ceph-mon[97889]: pgmap v394: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:25:19.394 INFO:teuthology.orchestra.run.smithi002.stderr:16000000 of 50000000 tuples (32%) done (elapsed 162.64 s, remaining 345.60 s) 2023-06-23T18:25:20.198 INFO:teuthology.orchestra.run.smithi002.stderr:16100000 of 50000000 tuples (32%) done (elapsed 163.44 s, remaining 344.14 s) 2023-06-23T18:25:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:20 smithi017 ceph-mon[97472]: pgmap v395: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:25:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:20 smithi002 ceph-mon[95947]: pgmap v395: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:25:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:20 smithi082 ceph-mon[97889]: pgmap v395: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:25:20.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:25:21.039 INFO:teuthology.orchestra.run.smithi002.stderr:16200000 of 50000000 tuples (32%) done (elapsed 164.28 s, remaining 342.76 s) 2023-06-23T18:25:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 -- 172.21.15.2:0/2326536676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1850100600 msgr2=0x7f1850106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 --2- 172.21.15.2:0/2326536676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1850100600 0x7f1850106ec0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f184800b0a0 tx=0x7f184800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 -- 172.21.15.2:0/2326536676 shutdown_connections 2023-06-23T18:25:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 --2- 172.21.15.2:0/2326536676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1850100600 0x7f1850106ec0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 --2- 172.21.15.2:0/2326536676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18500fea80 0x7f18500fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 --2- 172.21.15.2:0/2326536676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18500ffc70 0x7f18501000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 -- 172.21.15.2:0/2326536676 >> 172.21.15.2:0/2326536676 conn(0x7f18500fa0e0 msgr2=0x7f18500fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 -- 172.21.15.2:0/2326536676 shutdown_connections 2023-06-23T18:25:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.039+0000 7f1856ddf700 1 -- 172.21.15.2:0/2326536676 wait complete. 2023-06-23T18:25:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.040+0000 7f1856ddf700 1 Processor -- start 2023-06-23T18:25:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.040+0000 7f1856ddf700 1 -- start start 2023-06-23T18:25:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.040+0000 7f1856ddf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18500fea80 0x7f1850068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1856ddf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18500ffc70 0x7f185019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1856ddf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1850100600 0x7f185019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1856ddf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019d3c0 con 0x7f18500fea80 2023-06-23T18:25:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1856ddf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019d530 con 0x7f1850100600 2023-06-23T18:25:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1856ddf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185019d6f0 con 0x7f18500ffc70 2023-06-23T18:25:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18500fea80 0x7f1850068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18500fea80 0x7f1850068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54766/0 (socket says 172.21.15.2:54766) 2023-06-23T18:25:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 -- 172.21.15.2:0/335718244 learned_addr learned my addr 172.21.15.2:0/335718244 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f184ffff700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18500ffc70 0x7f185019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f185537c700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1850100600 0x7f185019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 -- 172.21.15.2:0/335718244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18500ffc70 msgr2=0x7f185019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18500ffc70 0x7f185019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 -- 172.21.15.2:0/335718244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1850100600 msgr2=0x7f185019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1850100600 0x7f185019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 -- 172.21.15.2:0/335718244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848009d00 con 0x7f18500fea80 2023-06-23T18:25:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1854b7b700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18500fea80 0x7f1850068370 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f184400e970 tx=0x7f184400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f184dffb700 1 -- 172.21.15.2:0/335718244 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844009790 con 0x7f18500fea80 2023-06-23T18:25:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f1856ddf700 1 -- 172.21.15.2:0/335718244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185019d9d0 con 0x7f18500fea80 2023-06-23T18:25:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f184dffb700 1 -- 172.21.15.2:0/335718244 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f184400c7e0 con 0x7f18500fea80 2023-06-23T18:25:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.041+0000 7f184dffb700 1 -- 172.21.15.2:0/335718244 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1844003bd0 con 0x7f18500fea80 2023-06-23T18:25:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.042+0000 7f1856ddf700 1 -- 172.21.15.2:0/335718244 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185019dee0 con 0x7f18500fea80 2023-06-23T18:25:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.044+0000 7f184dffb700 1 -- 172.21.15.2:0/335718244 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f184400c950 con 0x7f18500fea80 2023-06-23T18:25:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.044+0000 7f184dffb700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18400a1150 0x7f18400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.045+0000 7f184dffb700 1 -- 172.21.15.2:0/335718244 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1844014070 con 0x7f18500fea80 2023-06-23T18:25:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.045+0000 7f184ffff700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18400a1150 0x7f18400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.045+0000 7f184ffff700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18400a1150 0x7f18400a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f183c009c60 tx=0x7f183c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.045+0000 7f1862f64b80 1 Processor -- start 2023-06-23T18:25:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.045+0000 7f1862f64b80 1 -- start start 2023-06-23T18:25:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef792b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef792ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f1862f64b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef782fcb0 con 0x557ef792ac80 2023-06-23T18:25:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f1862f64b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef7882a40 con 0x557ef792b5d0 2023-06-23T18:25:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f1862f64b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef787bed0 con 0x557ef792c000 2023-06-23T18:25:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f185537c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f184ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef792ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef792b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f185537c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55116/0 (socket says 172.21.15.2:55116) 2023-06-23T18:25:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f184ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef792ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45368/0 (socket says 172.21.15.2:45368) 2023-06-23T18:25:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f185537c700 1 -- 172.21.15.2:0/4247755633 learned_addr learned my addr 172.21.15.2:0/4247755633 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f184ffff700 1 -- 172.21.15.2:0/4247755633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 msgr2=0x557ef792c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f184ffff700 1 --2- 172.21.15.2:0/4247755633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f184ffff700 1 -- 172.21.15.2:0/4247755633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 msgr2=0x557ef792b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f184ffff700 1 --2- 172.21.15.2:0/4247755633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef792b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f184ffff700 1 -- 172.21.15.2:0/4247755633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef76fd630 con 0x557ef792b5d0 2023-06-23T18:25:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f185537c700 1 --2- 172.21.15.2:0/4247755633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:25:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.046+0000 7f184ffff700 1 --2- 172.21.15.2:0/4247755633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef792ba70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e3b0 tx=0x7f183c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32167d5bfca70745 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f183affd700 1 -- 172.21.15.2:0/4247755633 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c00ed50 con 0x557ef792b5d0 2023-06-23T18:25:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f183affd700 1 -- 172.21.15.2:0/4247755633 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f183c021070 con 0x557ef792b5d0 2023-06-23T18:25:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f183affd700 1 -- 172.21.15.2:0/4247755633 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183c003e80 con 0x557ef792b5d0 2023-06-23T18:25:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f1862f64b80 1 -- 172.21.15.2:0/4247755633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 msgr2=0x557ef792ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f1862f64b80 1 --2- 172.21.15.2:0/4247755633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef792ba70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f183c00e3b0 tx=0x7f183c014000 comp rx=0 tx=0).stop 2023-06-23T18:25:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f1862f64b80 1 -- 172.21.15.2:0/4247755633 shutdown_connections 2023-06-23T18:25:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f1862f64b80 1 --2- 172.21.15.2:0/4247755633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f1862f64b80 1 --2- 172.21.15.2:0/4247755633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef792b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f1862f64b80 1 --2- 172.21.15.2:0/4247755633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef792ba70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.047+0000 7f1862f64b80 1 -- 172.21.15.2:0/4247755633 >> 172.21.15.2:0/4247755633 conn(0x557ef79203f0 msgr2=0x557ef7925520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.048+0000 7f1862f64b80 1 -- 172.21.15.2:0/4247755633 shutdown_connections 2023-06-23T18:25:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.048+0000 7f1862f64b80 1 -- 172.21.15.2:0/4247755633 wait complete. 2023-06-23T18:25:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.048+0000 7f1862f64b80 1 -- start start 2023-06-23T18:25:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.048+0000 7f1862f64b80 1 Processor -- start 2023-06-23T18:25:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef7934540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef7934a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 0x557ef793bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1862f64b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef781bac0 con 0x557ef792c000 2023-06-23T18:25:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1862f64b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef781b7c0 con 0x557ef792b5d0 2023-06-23T18:25:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1862f64b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef76ff920 con 0x557ef792ac80 2023-06-23T18:25:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1862f64b80 10 client.0 _ll_register_callbacks cb 0x557ef7821260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:25:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef7934540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f185537c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 0x557ef793bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef7934540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55118/0 (socket says 172.21.15.2:55118) 2023-06-23T18:25:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 -- 172.21.15.2:0/582213641 learned_addr learned my addr 172.21.15.2:0/582213641 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f184ffff700 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef7934a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 -- 172.21.15.2:0/582213641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 msgr2=0x557ef7934a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef7934a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 -- 172.21.15.2:0/582213641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 msgr2=0x557ef793bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 0x557ef793bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 -- 172.21.15.2:0/582213641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18501077f0 con 0x557ef792ac80 2023-06-23T18:25:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f1854b7b700 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef7934540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f184400cc00 tx=0x7f184400cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16242c726933c8f6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f182ffff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:25:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.049+0000 7f182ffff700 1 -- 172.21.15.2:0/582213641 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184409a370 con 0x557ef792ac80 2023-06-23T18:25:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f182ffff700 1 -- 172.21.15.2:0/582213641 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f18440a2360 con 0x557ef792ac80 2023-06-23T18:25:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 1 -- 172.21.15.2:0/582213641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ef791dbb0 con 0x557ef792ac80 2023-06-23T18:25:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f182ffff700 1 -- 172.21.15.2:0/582213641 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18440b8250 con 0x557ef792ac80 2023-06-23T18:25:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 10 client.25456 fetch_fsmap learned FSMap version 26 2023-06-23T18:25:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 1 -- 172.21.15.2:0/582213641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f183c017070 con 0x557ef792ac80 2023-06-23T18:25:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f182ffff700 1 -- 172.21.15.2:0/582213641 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557ef791dbb0 con 0x557ef792ac80 2023-06-23T18:25:21.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f182ffff700 1 -- 172.21.15.2:0/582213641 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1844099470 con 0x557ef792ac80 2023-06-23T18:25:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 10 client.25456 fetch_fsmap finished waiting for FSMap version 26 2023-06-23T18:25:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 10 client.25456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:25:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 20 client.25456 populate_metadata read hostname 'smithi002' 2023-06-23T18:25:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ef79402d0 0x557ef79426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 4 client.25456 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:25:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f1862f64b80 1 -- 172.21.15.2:0/582213641 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ef7874eb0 con 0x557ef79402d0 2023-06-23T18:25:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.050+0000 7f184ffff700 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ef79402d0 0x557ef79426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.051+0000 7f182ffff700 10 client.25456 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:25:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.051+0000 7f184ffff700 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ef79402d0 0x557ef79426e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f78bcbb817c3b49 server_cookie=51b0dcc71e1b00ea in_seq=0 out_seq=0 2023-06-23T18:25:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f182ffff700 1 -- 172.21.15.2:0/582213641 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557ef7874eb0 con 0x557ef79402d0 2023-06-23T18:25:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f182ffff700 10 client.25456 handle_command_reply: tid=0 2023-06-23T18:25:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 client.25456 shutdown 2023-06-23T18:25:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 -- 172.21.15.2:0/582213641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 msgr2=0x557ef7934540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef7934540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f184400cc00 tx=0x7f184400cc50 comp rx=0 tx=0).stop 2023-06-23T18:25:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 -- 172.21.15.2:0/582213641 shutdown_connections 2023-06-23T18:25:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ef79402d0 0x557ef79426e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 0x557ef793bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef7934540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f182ffff700 0 client.25456 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:25:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 --2- 172.21.15.2:0/582213641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef7934a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 -- 172.21.15.2:0/582213641 >> 172.21.15.2:0/582213641 conn(0x557ef79203f0 msgr2=0x557ef7924c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 -- 172.21.15.2:0/582213641 shutdown_connections 2023-06-23T18:25:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 1 -- 172.21.15.2:0/582213641 wait complete. 2023-06-23T18:25:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.052+0000 7f1862f64b80 20 client.25456 trim_cache size 0 max 16384 2023-06-23T18:25:21.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1862f64b80 1 Processor -- start 2023-06-23T18:25:21.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1862f64b80 1 -- start start 2023-06-23T18:25:21.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef7932320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef793ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1862f64b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef79356c0 con 0x557ef792ac80 2023-06-23T18:25:21.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1862f64b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef7926810 con 0x557ef792b5d0 2023-06-23T18:25:21.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1862f64b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef79339f0 con 0x557ef792c000 2023-06-23T18:25:21.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef7932320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f185537c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef7932320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54788/0 (socket says 172.21.15.2:54788) 2023-06-23T18:25:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f1854b7b700 1 -- 172.21.15.2:0/559454879 learned_addr learned my addr 172.21.15.2:0/559454879 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f184ffff700 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef793ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f185537c700 1 -- 172.21.15.2:0/559454879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 msgr2=0x557ef793ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f185537c700 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef793ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.077+0000 7f185537c700 1 -- 172.21.15.2:0/559454879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 msgr2=0x557ef7932320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:21.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.078+0000 7f185537c700 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef7932320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.078+0000 7f185537c700 1 -- 172.21.15.2:0/559454879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ef792c8f0 con 0x557ef792c000 2023-06-23T18:25:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.078+0000 7f1854b7b700 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef7932320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:25:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.078+0000 7f185537c700 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18480074e0 tx=0x7f1848007510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfcf0400e0e67d3a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.078+0000 7f183b7fe700 1 -- 172.21.15.2:0/559454879 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184800e040 con 0x557ef792c000 2023-06-23T18:25:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.078+0000 7f183b7fe700 1 -- 172.21.15.2:0/559454879 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1848021070 con 0x557ef792c000 2023-06-23T18:25:21.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.078+0000 7f183b7fe700 1 -- 172.21.15.2:0/559454879 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1848003e60 con 0x557ef792c000 2023-06-23T18:25:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 -- 172.21.15.2:0/559454879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 msgr2=0x557ef792ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792ee80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f18480074e0 tx=0x7f1848007510 comp rx=0 tx=0).stop 2023-06-23T18:25:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 -- 172.21.15.2:0/559454879 shutdown_connections 2023-06-23T18:25:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792c000 0x557ef792ee80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792ac80 0x557ef7932320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 --2- 172.21.15.2:0/559454879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef793ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 -- 172.21.15.2:0/559454879 >> 172.21.15.2:0/559454879 conn(0x557ef79203f0 msgr2=0x557ef7925e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 -- 172.21.15.2:0/559454879 shutdown_connections 2023-06-23T18:25:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 -- 172.21.15.2:0/559454879 wait complete. 2023-06-23T18:25:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 -- start start 2023-06-23T18:25:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.079+0000 7f1862f64b80 1 Processor -- start 2023-06-23T18:25:21.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef793b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef79382d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1862f64b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 0x557ef7938810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1862f64b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef7938de0 con 0x557ef792c000 2023-06-23T18:25:21.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1862f64b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef7938f50 con 0x557ef792b5d0 2023-06-23T18:25:21.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1862f64b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ef7994200 con 0x557ef792ac80 2023-06-23T18:25:21.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1862f64b80 10 client.0 _ll_register_callbacks cb 0x557ef7821260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:25:21.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef793b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f185537c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 0x557ef7938810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f184ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef79382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef793b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55132/0 (socket says 172.21.15.2:55132) 2023-06-23T18:25:21.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 -- 172.21.15.2:0/3841246501 learned_addr learned my addr 172.21.15.2:0/3841246501 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:21.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 -- 172.21.15.2:0/3841246501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 msgr2=0x557ef79382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:21.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef79382d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 -- 172.21.15.2:0/3841246501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 msgr2=0x557ef7938810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:21.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 0x557ef7938810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 -- 172.21.15.2:0/3841246501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f183c017070 con 0x557ef792ac80 2023-06-23T18:25:21.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f184ffff700 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef79382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:25:21.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f1854b7b700 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef793b830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1844099ea0 tx=0x7f18440984c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d15732d22255c34 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:21.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.080+0000 7f182ffff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:25:21.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f182ffff700 1 -- 172.21.15.2:0/3841246501 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18440a2360 con 0x557ef792ac80 2023-06-23T18:25:21.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f182ffff700 1 -- 172.21.15.2:0/3841246501 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f18440a2040 con 0x557ef792ac80 2023-06-23T18:25:21.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 1 -- 172.21.15.2:0/3841246501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f183c021070 con 0x557ef792ac80 2023-06-23T18:25:21.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f182ffff700 1 -- 172.21.15.2:0/3841246501 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18440ca1a0 con 0x557ef792ac80 2023-06-23T18:25:21.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 10 client.25462 fetch_fsmap learned FSMap version 26 2023-06-23T18:25:21.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 1 -- 172.21.15.2:0/3841246501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f1848009400 con 0x557ef792ac80 2023-06-23T18:25:21.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f182ffff700 1 -- 172.21.15.2:0/3841246501 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f183c021070 con 0x557ef792ac80 2023-06-23T18:25:21.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f182ffff700 1 -- 172.21.15.2:0/3841246501 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1844095d00 con 0x557ef792ac80 2023-06-23T18:25:21.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 10 client.25462 fetch_fsmap finished waiting for FSMap version 26 2023-06-23T18:25:21.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 10 client.25462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:25:21.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 20 client.25462 populate_metadata read hostname 'smithi002' 2023-06-23T18:25:21.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ef79959e0 0x557ef7997df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:21.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 4 client.25462 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:25:21.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f1862f64b80 1 -- 172.21.15.2:0/3841246501 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ef7837050 con 0x557ef79959e0 2023-06-23T18:25:21.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.081+0000 7f184ffff700 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ef79959e0 0x557ef7997df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:21.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f184ffff700 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ef79959e0 0x557ef7997df0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60d4c6b25f303105 server_cookie=84ab111fe338653b in_seq=0 out_seq=0 2023-06-23T18:25:21.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f182ffff700 10 client.25462 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:25:21.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f182ffff700 1 -- 172.21.15.2:0/3841246501 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557ef7837050 con 0x557ef79959e0 2023-06-23T18:25:21.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f182ffff700 10 client.25462 handle_command_reply: tid=0 2023-06-23T18:25:21.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 client.25462 shutdown 2023-06-23T18:25:21.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 -- 172.21.15.2:0/3841246501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 msgr2=0x557ef793b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef793b830 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1844099ea0 tx=0x7f18440984c0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 -- 172.21.15.2:0/3841246501 shutdown_connections 2023-06-23T18:25:21.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ef79959e0 0x557ef7997df0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ef792c000 0x557ef7938810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ef792ac80 0x557ef793b830 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 --2- 172.21.15.2:0/3841246501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ef792b5d0 0x557ef79382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f182ffff700 0 client.25462 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:25:21.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 -- 172.21.15.2:0/3841246501 >> 172.21.15.2:0/3841246501 conn(0x557ef79203f0 msgr2=0x557ef7932320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:21.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 -- 172.21.15.2:0/3841246501 shutdown_connections 2023-06-23T18:25:21.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 1 -- 172.21.15.2:0/3841246501 wait complete. 2023-06-23T18:25:21.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.082+0000 7f1862f64b80 20 client.25462 trim_cache size 0 max 16384 2023-06-23T18:25:21.099 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:25:21.100 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:25:21.100 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:25:21.100 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:25:21.100 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:25:21.100 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:25:21.101 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:25:21.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:25:21.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:25:21.101 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:25:21.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 -- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18400a1150 msgr2=0x7f18400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18400a1150 0x7f18400a3600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f183c009c60 tx=0x7f183c009260 comp rx=0 tx=0).stop 2023-06-23T18:25:21.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 -- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18500fea80 msgr2=0x7f1850068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:21.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18500fea80 0x7f1850068370 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f184400e970 tx=0x7f184400ec80 comp rx=0 tx=0).stop 2023-06-23T18:25:21.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 -- 172.21.15.2:0/335718244 shutdown_connections 2023-06-23T18:25:21.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f18400a1150 0x7f18400a3600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1850100600 0x7f185019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18500fea80 0x7f1850068370 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 --2- 172.21.15.2:0/335718244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18500ffc70 0x7f185019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:21.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 -- 172.21.15.2:0/335718244 >> 172.21.15.2:0/335718244 conn(0x7f18500fa0e0 msgr2=0x7f18501030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:21.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 -- 172.21.15.2:0/335718244 shutdown_connections 2023-06-23T18:25:21.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:21.083+0000 7f1856ddf700 1 -- 172.21.15.2:0/335718244 wait complete. 2023-06-23T18:25:21.105 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:25:21.911 INFO:teuthology.orchestra.run.smithi002.stderr:16300000 of 50000000 tuples (32%) done (elapsed 165.15 s, remaining 341.45 s) 2023-06-23T18:25:22.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:22 smithi017 ceph-mon[97472]: pgmap v396: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:25:22.730 INFO:teuthology.orchestra.run.smithi002.stderr:16400000 of 50000000 tuples (32%) done (elapsed 165.97 s, remaining 340.04 s) 2023-06-23T18:25:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:22 smithi002 ceph-mon[95947]: pgmap v396: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:25:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:22 smithi082 ceph-mon[97889]: pgmap v396: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:25:23.548 INFO:teuthology.orchestra.run.smithi002.stderr:16500000 of 50000000 tuples (33%) done (elapsed 166.79 s, remaining 338.63 s) 2023-06-23T18:25:24.383 INFO:teuthology.orchestra.run.smithi002.stderr:16600000 of 50000000 tuples (33%) done (elapsed 167.62 s, remaining 337.27 s) 2023-06-23T18:25:24.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:24 smithi002 ceph-mon[95947]: pgmap v397: 97 pgs: 97 active+clean; 3.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:25:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:24 smithi082 ceph-mon[97889]: pgmap v397: 97 pgs: 97 active+clean; 3.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:25:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:24 smithi017 ceph-mon[97472]: pgmap v397: 97 pgs: 97 active+clean; 3.0 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:25:25.272 INFO:teuthology.orchestra.run.smithi002.stderr:16700000 of 50000000 tuples (33%) done (elapsed 168.51 s, remaining 336.02 s) 2023-06-23T18:25:26.147 INFO:teuthology.orchestra.run.smithi002.stderr:16800000 of 50000000 tuples (33%) done (elapsed 169.39 s, remaining 334.74 s) 2023-06-23T18:25:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:26 smithi002 ceph-mon[95947]: pgmap v398: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:26 smithi082 ceph-mon[97889]: pgmap v398: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:26 smithi017 ceph-mon[97472]: pgmap v398: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:26.967 INFO:teuthology.orchestra.run.smithi002.stderr:16900000 of 50000000 tuples (33%) done (elapsed 170.21 s, remaining 333.37 s) 2023-06-23T18:25:27.843 INFO:teuthology.orchestra.run.smithi002.stderr:17000000 of 50000000 tuples (34%) done (elapsed 171.08 s, remaining 332.10 s) 2023-06-23T18:25:28.688 INFO:teuthology.orchestra.run.smithi002.stderr:17100000 of 50000000 tuples (34%) done (elapsed 171.93 s, remaining 330.79 s) 2023-06-23T18:25:28.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:28 smithi002 ceph-mon[95947]: pgmap v399: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-06-23T18:25:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:28 smithi082 ceph-mon[97889]: pgmap v399: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-06-23T18:25:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:28 smithi017 ceph-mon[97472]: pgmap v399: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-06-23T18:25:29.505 INFO:teuthology.orchestra.run.smithi002.stderr:17200000 of 50000000 tuples (34%) done (elapsed 172.75 s, remaining 329.42 s) 2023-06-23T18:25:30.351 INFO:teuthology.orchestra.run.smithi002.stderr:17300000 of 50000000 tuples (34%) done (elapsed 173.59 s, remaining 328.12 s) 2023-06-23T18:25:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:30 smithi002 ceph-mon[95947]: pgmap v400: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:25:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:30 smithi082 ceph-mon[97889]: pgmap v400: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:25:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:30 smithi017 ceph-mon[97472]: pgmap v400: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:25:31.233 INFO:teuthology.orchestra.run.smithi002.stderr:17400000 of 50000000 tuples (34%) done (elapsed 174.47 s, remaining 326.89 s) 2023-06-23T18:25:32.044 INFO:teuthology.orchestra.run.smithi002.stderr:17500000 of 50000000 tuples (35%) done (elapsed 175.29 s, remaining 325.53 s) 2023-06-23T18:25:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:32 smithi002 ceph-mon[95947]: pgmap v401: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:32 smithi082 ceph-mon[97889]: pgmap v401: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:32.939 INFO:teuthology.orchestra.run.smithi002.stderr:17600000 of 50000000 tuples (35%) done (elapsed 176.18 s, remaining 324.33 s) 2023-06-23T18:25:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:32 smithi017 ceph-mon[97472]: pgmap v401: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:34.120 INFO:teuthology.orchestra.run.smithi002.stderr:17700000 of 50000000 tuples (35%) done (elapsed 177.36 s, remaining 323.66 s) 2023-06-23T18:25:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:34 smithi002 ceph-mon[95947]: pgmap v402: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:25:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:34 smithi082 ceph-mon[97889]: pgmap v402: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:25:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:34 smithi017 ceph-mon[97472]: pgmap v402: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:25:34.987 INFO:teuthology.orchestra.run.smithi002.stderr:17800000 of 50000000 tuples (35%) done (elapsed 178.23 s, remaining 322.41 s) 2023-06-23T18:25:35.813 INFO:teuthology.orchestra.run.smithi002.stderr:17900000 of 50000000 tuples (35%) done (elapsed 179.05 s, remaining 321.10 s) 2023-06-23T18:25:36.719 INFO:teuthology.orchestra.run.smithi002.stderr:18000000 of 50000000 tuples (36%) done (elapsed 179.96 s, remaining 319.93 s) 2023-06-23T18:25:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:36 smithi002 ceph-mon[95947]: pgmap v403: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 43 op/s 2023-06-23T18:25:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:36 smithi082 ceph-mon[97889]: pgmap v403: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 43 op/s 2023-06-23T18:25:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:36 smithi017 ceph-mon[97472]: pgmap v403: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 MiB/s wr, 43 op/s 2023-06-23T18:25:37.560 INFO:teuthology.orchestra.run.smithi002.stderr:18100000 of 50000000 tuples (36%) done (elapsed 180.80 s, remaining 318.65 s) 2023-06-23T18:25:37.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:37 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:25:37.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:37 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:25:37.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:37 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:25:38.369 INFO:teuthology.orchestra.run.smithi002.stderr:18200000 of 50000000 tuples (36%) done (elapsed 181.61 s, remaining 317.32 s) 2023-06-23T18:25:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:38 smithi002 ceph-mon[95947]: pgmap v404: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 42 op/s 2023-06-23T18:25:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:38 smithi082 ceph-mon[97889]: pgmap v404: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 42 op/s 2023-06-23T18:25:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:38 smithi017 ceph-mon[97472]: pgmap v404: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 MiB/s wr, 42 op/s 2023-06-23T18:25:39.185 INFO:teuthology.orchestra.run.smithi002.stderr:18300000 of 50000000 tuples (36%) done (elapsed 182.43 s, remaining 316.01 s) 2023-06-23T18:25:40.087 INFO:teuthology.orchestra.run.smithi002.stderr:18400000 of 50000000 tuples (36%) done (elapsed 183.33 s, remaining 314.85 s) 2023-06-23T18:25:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:40 smithi002 ceph-mon[95947]: pgmap v405: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 MiB/s wr, 48 op/s 2023-06-23T18:25:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:40 smithi082 ceph-mon[97889]: pgmap v405: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 MiB/s wr, 48 op/s 2023-06-23T18:25:40.923 INFO:teuthology.orchestra.run.smithi002.stderr:18500000 of 50000000 tuples (37%) done (elapsed 184.16 s, remaining 313.58 s) 2023-06-23T18:25:40.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:40 smithi017 ceph-mon[97472]: pgmap v405: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 44 MiB/s wr, 48 op/s 2023-06-23T18:25:41.729 INFO:teuthology.orchestra.run.smithi002.stderr:18600000 of 50000000 tuples (37%) done (elapsed 184.97 s, remaining 312.26 s) 2023-06-23T18:25:42.632 INFO:teuthology.orchestra.run.smithi002.stderr:18700000 of 50000000 tuples (37%) done (elapsed 185.87 s, remaining 311.11 s) 2023-06-23T18:25:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:42 smithi002 ceph-mon[95947]: pgmap v406: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:25:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:42 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:25:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:42 smithi082 ceph-mon[97889]: pgmap v406: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:25:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:42 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:25:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:42 smithi017 ceph-mon[97472]: pgmap v406: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:25:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:42 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:25:43.441 INFO:teuthology.orchestra.run.smithi002.stderr:18800000 of 50000000 tuples (37%) done (elapsed 186.68 s, remaining 309.81 s) 2023-06-23T18:25:44.331 INFO:teuthology.orchestra.run.smithi002.stderr:18900000 of 50000000 tuples (37%) done (elapsed 187.57 s, remaining 308.65 s) 2023-06-23T18:25:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:44 smithi002 ceph-mon[95947]: pgmap v407: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 45 MiB/s wr, 48 op/s 2023-06-23T18:25:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:44 smithi082 ceph-mon[97889]: pgmap v407: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 45 MiB/s wr, 48 op/s 2023-06-23T18:25:44.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:44 smithi017 ceph-mon[97472]: pgmap v407: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 45 MiB/s wr, 48 op/s 2023-06-23T18:25:45.163 INFO:teuthology.orchestra.run.smithi002.stderr:19000000 of 50000000 tuples (38%) done (elapsed 188.40 s, remaining 307.40 s) 2023-06-23T18:25:46.086 INFO:teuthology.orchestra.run.smithi002.stderr:19100000 of 50000000 tuples (38%) done (elapsed 189.33 s, remaining 306.29 s) 2023-06-23T18:25:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:46 smithi002 ceph-mon[95947]: pgmap v408: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:25:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:46 smithi082 ceph-mon[97889]: pgmap v408: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:25:46.939 INFO:teuthology.orchestra.run.smithi002.stderr:19200000 of 50000000 tuples (38%) done (elapsed 190.18 s, remaining 305.08 s) 2023-06-23T18:25:46.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:46 smithi017 ceph-mon[97472]: pgmap v408: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 MiB/s wr, 47 op/s 2023-06-23T18:25:47.754 INFO:teuthology.orchestra.run.smithi002.stderr:19300000 of 50000000 tuples (38%) done (elapsed 191.00 s, remaining 303.81 s) 2023-06-23T18:25:48.583 INFO:teuthology.orchestra.run.smithi002.stderr:19400000 of 50000000 tuples (38%) done (elapsed 191.82 s, remaining 302.57 s) 2023-06-23T18:25:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:48 smithi002 ceph-mon[95947]: pgmap v409: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 27 op/s 2023-06-23T18:25:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:48 smithi082 ceph-mon[97889]: pgmap v409: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 27 op/s 2023-06-23T18:25:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:48 smithi017 ceph-mon[97472]: pgmap v409: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 42 MiB/s wr, 27 op/s 2023-06-23T18:25:49.459 INFO:teuthology.orchestra.run.smithi002.stderr:19500000 of 50000000 tuples (39%) done (elapsed 192.70 s, remaining 301.40 s) 2023-06-23T18:25:50.260 INFO:teuthology.orchestra.run.smithi002.stderr:19600000 of 50000000 tuples (39%) done (elapsed 193.50 s, remaining 300.12 s) 2023-06-23T18:25:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:50 smithi002 ceph-mon[95947]: pgmap v410: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 29 op/s 2023-06-23T18:25:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:50 smithi082 ceph-mon[97889]: pgmap v410: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 29 op/s 2023-06-23T18:25:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:50 smithi017 ceph-mon[97472]: pgmap v410: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 MiB/s wr, 29 op/s 2023-06-23T18:25:51.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:25:51.125 INFO:teuthology.orchestra.run.smithi002.stderr:19700000 of 50000000 tuples (39%) done (elapsed 194.37 s, remaining 298.95 s) 2023-06-23T18:25:51.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 -- 172.21.15.2:0/1815695346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418071e30 msgr2=0x7fa418110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/1815695346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418071e30 0x7fa418110820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa41000b0a0 tx=0x7fa41000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 -- 172.21.15.2:0/1815695346 shutdown_connections 2023-06-23T18:25:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/1815695346 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418071e30 0x7fa418110820 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/1815695346 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa418107790 0x7fa418107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/1815695346 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa418071480 0x7fa4180718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 -- 172.21.15.2:0/1815695346 >> 172.21.15.2:0/1815695346 conn(0x7fa41806cde0 msgr2=0x7fa41806f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 -- 172.21.15.2:0/1815695346 shutdown_connections 2023-06-23T18:25:51.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.242+0000 7fa41cdf7700 1 -- 172.21.15.2:0/1815695346 wait complete. 2023-06-23T18:25:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.243+0000 7fa41cdf7700 1 Processor -- start 2023-06-23T18:25:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.243+0000 7fa41cdf7700 1 -- start start 2023-06-23T18:25:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.243+0000 7fa41cdf7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418071480 0x7fa41810a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.243+0000 7fa41cdf7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa418071e30 0x7fa4181087a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.243+0000 7fa41cdf7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa418107790 0x7fa418108ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa41cdf7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418109480 con 0x7fa418071480 2023-06-23T18:25:51.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa41cdf7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181095f0 con 0x7fa418107790 2023-06-23T18:25:51.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa41cdf7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4181a8150 con 0x7fa418071e30 2023-06-23T18:25:51.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa418071e30 0x7fa4181087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa41659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418071480 0x7fa41810a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa418071e30 0x7fa4181087a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55148/0 (socket says 172.21.15.2:55148) 2023-06-23T18:25:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 -- 172.21.15.2:0/2934105010 learned_addr learned my addr 172.21.15.2:0/2934105010 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa416d9d700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa418107790 0x7fa418108ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 -- 172.21.15.2:0/2934105010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa418107790 msgr2=0x7fa418108ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa418107790 0x7fa418108ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 -- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418071480 msgr2=0x7fa41810a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418071480 0x7fa41810a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 -- 172.21.15.2:0/2934105010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa410009d00 con 0x7fa418071e30 2023-06-23T18:25:51.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa415d9b700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa418071e30 0x7fa4181087a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c600 tx=0x7fa40c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa4077fe700 1 -- 172.21.15.2:0/2934105010 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c00ce70 con 0x7fa418071e30 2023-06-23T18:25:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa41cdf7700 1 -- 172.21.15.2:0/2934105010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa41819dd70 con 0x7fa418071e30 2023-06-23T18:25:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa4077fe700 1 -- 172.21.15.2:0/2934105010 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa40c01e8c0 con 0x7fa418071e30 2023-06-23T18:25:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa4077fe700 1 -- 172.21.15.2:0/2934105010 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c016570 con 0x7fa418071e30 2023-06-23T18:25:51.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.244+0000 7fa41cdf7700 1 -- 172.21.15.2:0/2934105010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41819e280 con 0x7fa418071e30 2023-06-23T18:25:51.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.247+0000 7fa4077fe700 1 -- 172.21.15.2:0/2934105010 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa40c01ea30 con 0x7fa418071e30 2023-06-23T18:25:51.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.248+0000 7fa4077fe700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4000a1150 0x7fa4000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.248+0000 7fa41659c700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4000a1150 0x7fa4000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.248+0000 7fa4077fe700 1 -- 172.21.15.2:0/2934105010 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa40c0d4b40 con 0x7fa418071e30 2023-06-23T18:25:51.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.248+0000 7fa428f7cb80 1 Processor -- start 2023-06-23T18:25:51.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.248+0000 7fa41659c700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4000a1150 0x7fa4000a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa418069700 tx=0x7fa408009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:51.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.248+0000 7fa428f7cb80 1 -- start start 2023-06-23T18:25:51.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520ec80 0x557da520f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520f5d0 0x557da520fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da5210000 0x557da5210470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da5113cb0 con 0x557da5210000 2023-06-23T18:25:51.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da5166a40 con 0x557da520ec80 2023-06-23T18:25:51.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da515fed0 con 0x557da520f5d0 2023-06-23T18:25:51.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa416d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da5210000 0x557da5210470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520ec80 0x557da520f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa415d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520f5d0 0x557da520fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520ec80 0x557da520f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45408/0 (socket says 172.21.15.2:45408) 2023-06-23T18:25:51.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa415d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520f5d0 0x557da520fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55156/0 (socket says 172.21.15.2:55156) 2023-06-23T18:25:51.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa416d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da5210000 0x557da5210470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54820/0 (socket says 172.21.15.2:54820) 2023-06-23T18:25:51.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 -- 172.21.15.2:0/616591048 learned_addr learned my addr 172.21.15.2:0/616591048 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:51.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 -- 172.21.15.2:0/616591048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520f5d0 msgr2=0x557da520fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:51.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 --2- 172.21.15.2:0/616591048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520f5d0 0x557da520fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 -- 172.21.15.2:0/616591048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da5210000 msgr2=0x557da5210470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 --2- 172.21.15.2:0/616591048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da5210000 0x557da5210470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 -- 172.21.15.2:0/616591048 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557da4fe1630 con 0x557da520ec80 2023-06-23T18:25:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa415d9b700 1 --2- 172.21.15.2:0/616591048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520f5d0 0x557da520fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:25:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.249+0000 7fa41659c700 1 --2- 172.21.15.2:0/616591048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520ec80 0x557da520f090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa408011d50 tx=0x7fa408014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=904ecca209806c65 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa404ff9700 1 -- 172.21.15.2:0/616591048 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40800ed20 con 0x557da520ec80 2023-06-23T18:25:51.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa404ff9700 1 -- 172.21.15.2:0/616591048 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa408021070 con 0x557da520ec80 2023-06-23T18:25:51.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa404ff9700 1 -- 172.21.15.2:0/616591048 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40801d410 con 0x557da520ec80 2023-06-23T18:25:51.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa428f7cb80 1 -- 172.21.15.2:0/616591048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520ec80 msgr2=0x557da520f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/616591048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520ec80 0x557da520f090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa408011d50 tx=0x7fa408014000 comp rx=0 tx=0).stop 2023-06-23T18:25:51.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa428f7cb80 1 -- 172.21.15.2:0/616591048 shutdown_connections 2023-06-23T18:25:51.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/616591048 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da5210000 0x557da5210470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/616591048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520ec80 0x557da520f090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/616591048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520f5d0 0x557da520fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.250+0000 7fa428f7cb80 1 -- 172.21.15.2:0/616591048 >> 172.21.15.2:0/616591048 conn(0x557da52043f0 msgr2=0x557da5209520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:51.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 -- 172.21.15.2:0/616591048 shutdown_connections 2023-06-23T18:25:51.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 -- 172.21.15.2:0/616591048 wait complete. 2023-06-23T18:25:51.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 -- start start 2023-06-23T18:25:51.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 Processor -- start 2023-06-23T18:25:51.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520ec80 0x557da52184c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520f5d0 0x557da5218a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da5210000 0x557da521fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da50ffac0 con 0x557da520f5d0 2023-06-23T18:25:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da50ff7c0 con 0x557da5210000 2023-06-23T18:25:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da4fe3920 con 0x557da520ec80 2023-06-23T18:25:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.251+0000 7fa428f7cb80 10 client.0 _ll_register_callbacks cb 0x557da5105260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:25:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520f5d0 0x557da5218a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa41659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520ec80 0x557da52184c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520f5d0 0x557da5218a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54824/0 (socket says 172.21.15.2:54824) 2023-06-23T18:25:51.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa41659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520ec80 0x557da52184c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55160/0 (socket says 172.21.15.2:55160) 2023-06-23T18:25:51.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 -- 172.21.15.2:0/3717311122 learned_addr learned my addr 172.21.15.2:0/3717311122 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:51.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa416d9d700 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da5210000 0x557da521fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 -- 172.21.15.2:0/3717311122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520ec80 msgr2=0x557da52184c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:51.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520ec80 0x557da52184c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 -- 172.21.15.2:0/3717311122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da5210000 msgr2=0x557da521fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:51.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da5210000 0x557da521fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 -- 172.21.15.2:0/3717311122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa41819e280 con 0x557da520f5d0 2023-06-23T18:25:51.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa41659c700 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520ec80 0x557da52184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:25:51.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa415d9b700 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520f5d0 0x557da5218a00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa40c01ece0 tx=0x7fa40c0a2490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fef12461175be4a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:51.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa3fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:25:51.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3717311122 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0aa110 con 0x557da520f5d0 2023-06-23T18:25:51.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3717311122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557da5201bb0 con 0x557da520f5d0 2023-06-23T18:25:51.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3717311122 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa40c0a03b0 con 0x557da520f5d0 2023-06-23T18:25:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3717311122 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa40c01e430 con 0x557da520f5d0 2023-06-23T18:25:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa428f7cb80 10 client.15708 fetch_fsmap learned FSMap version 28 2023-06-23T18:25:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.252+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3717311122 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fa408017070 con 0x557da520f5d0 2023-06-23T18:25:51.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3717311122 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557da5201bb0 con 0x557da520f5d0 2023-06-23T18:25:51.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3717311122 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa40c0d2b00 con 0x557da520f5d0 2023-06-23T18:25:51.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa428f7cb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2023-06-23T18:25:51.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa428f7cb80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:25:51.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa428f7cb80 20 client.15708 populate_metadata read hostname 'smithi002' 2023-06-23T18:25:51.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557da5224230 0x557da5226640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa428f7cb80 4 client.15708 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:25:51.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3717311122 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557da5158eb0 con 0x557da5224230 2023-06-23T18:25:51.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa41659c700 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557da5224230 0x557da5226640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.253+0000 7fa3fdffb700 10 client.15708 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:25:51.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.254+0000 7fa41659c700 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557da5224230 0x557da5226640 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4ec768c8d0bf11 server_cookie=4280218eca36e980 in_seq=0 out_seq=0 2023-06-23T18:25:51.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.254+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3717311122 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557da5158eb0 con 0x557da5224230 2023-06-23T18:25:51.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.254+0000 7fa3fdffb700 10 client.15708 handle_command_reply: tid=0 2023-06-23T18:25:51.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.254+0000 7fa428f7cb80 1 client.15708 shutdown 2023-06-23T18:25:51.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.254+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3717311122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520f5d0 msgr2=0x557da5218a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520f5d0 0x557da5218a00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa40c01ece0 tx=0x7fa40c0a2490 comp rx=0 tx=0).stop 2023-06-23T18:25:51.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3717311122 shutdown_connections 2023-06-23T18:25:51.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557da5224230 0x557da5226640 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da5210000 0x557da521fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa3fdffb700 0 client.15708 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:25:51.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da520ec80 0x557da52184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3717311122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520f5d0 0x557da5218a00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3717311122 >> 172.21.15.2:0/3717311122 conn(0x557da52043f0 msgr2=0x557da5208c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:51.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3717311122 shutdown_connections 2023-06-23T18:25:51.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3717311122 wait complete. 2023-06-23T18:25:51.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.255+0000 7fa428f7cb80 20 client.15708 trim_cache size 0 max 16384 2023-06-23T18:25:51.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.276+0000 7fa428f7cb80 1 Processor -- start 2023-06-23T18:25:51.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.276+0000 7fa428f7cb80 1 -- start start 2023-06-23T18:25:51.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.276+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da5212e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.276+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.276+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da52133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.276+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da5158eb0 con 0x557da520ec80 2023-06-23T18:25:51.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.276+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da52196b0 con 0x557da520f5d0 2023-06-23T18:25:51.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.276+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da52179c0 con 0x557da5210000 2023-06-23T18:25:51.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa41659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da5212e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa415d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa41659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da5212e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54830/0 (socket says 172.21.15.2:54830) 2023-06-23T18:25:51.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa41659c700 1 -- 172.21.15.2:0/2859625643 learned_addr learned my addr 172.21.15.2:0/2859625643 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:51.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa415d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45426/0 (socket says 172.21.15.2:45426) 2023-06-23T18:25:51.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa415d9b700 1 -- 172.21.15.2:0/2859625643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 msgr2=0x557da52133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:51.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa416d9d700 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da52133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa415d9b700 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da52133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa415d9b700 1 -- 172.21.15.2:0/2859625643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 msgr2=0x557da5212e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:51.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa415d9b700 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da5212e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa415d9b700 1 -- 172.21.15.2:0/2859625643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557da4f8dde0 con 0x557da520f5d0 2023-06-23T18:25:51.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa41659c700 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da5212e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:25:51.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa416d9d700 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da52133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:25:51.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.277+0000 7fa415d9b700 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521f9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0a1ee0 tx=0x7fa40c0a1bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=600965eb91e0a354 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:51.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.278+0000 7fa4057fa700 1 -- 172.21.15.2:0/2859625643 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c09dde0 con 0x557da520f5d0 2023-06-23T18:25:51.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.278+0000 7fa4057fa700 1 -- 172.21.15.2:0/2859625643 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa40c0a03b0 con 0x557da520f5d0 2023-06-23T18:25:51.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.278+0000 7fa4057fa700 1 -- 172.21.15.2:0/2859625643 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0aa1e0 con 0x557da520f5d0 2023-06-23T18:25:51.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.278+0000 7fa428f7cb80 1 -- 172.21.15.2:0/2859625643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 msgr2=0x557da521f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.278+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521f9d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0a1ee0 tx=0x7fa40c0a1bc0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 -- 172.21.15.2:0/2859625643 shutdown_connections 2023-06-23T18:25:51.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da52133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da5212e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/2859625643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521f9d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 -- 172.21.15.2:0/2859625643 >> 172.21.15.2:0/2859625643 conn(0x557da52043f0 msgr2=0x557da522afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:51.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 -- 172.21.15.2:0/2859625643 shutdown_connections 2023-06-23T18:25:51.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 -- 172.21.15.2:0/2859625643 wait complete. 2023-06-23T18:25:51.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 -- start start 2023-06-23T18:25:51.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 Processor -- start 2023-06-23T18:25:51.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da521b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.279+0000 7fa428f7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da521ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da521efb0 con 0x557da520ec80 2023-06-23T18:25:51.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da521f120 con 0x557da520f5d0 2023-06-23T18:25:51.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa428f7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557da521f2e0 con 0x557da5210000 2023-06-23T18:25:51.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa428f7cb80 10 client.0 _ll_register_callbacks cb 0x557da5105260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:25:51.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da521b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da521b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54836/0 (socket says 172.21.15.2:54836) 2023-06-23T18:25:51.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 -- 172.21.15.2:0/3277995696 learned_addr learned my addr 172.21.15.2:0/3277995696 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:25:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa415d9b700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa416d9d700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da521ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 -- 172.21.15.2:0/3277995696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 msgr2=0x557da521ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da521ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 -- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 msgr2=0x557da521ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:25:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 -- 172.21.15.2:0/3277995696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40802a040 con 0x557da520ec80 2023-06-23T18:25:51.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa416d9d700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da521ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:25:51.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa415d9b700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:25:51.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa41659c700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da521b510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa408026000 tx=0x7fa40800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfb5f6e62828431 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:25:51.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.280+0000 7fa3fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:25:51.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3277995696 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080324f0 con 0x557da520ec80 2023-06-23T18:25:51.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3277995696 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa40800eba0 con 0x557da520ec80 2023-06-23T18:25:51.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3277995696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa408021070 con 0x557da520ec80 2023-06-23T18:25:51.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3277995696 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa40800ed60 con 0x557da520ec80 2023-06-23T18:25:51.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 10 client.15714 fetch_fsmap learned FSMap version 28 2023-06-23T18:25:51.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3277995696 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7fa408017070 con 0x557da520ec80 2023-06-23T18:25:51.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3277995696 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa408021070 con 0x557da520ec80 2023-06-23T18:25:51.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3277995696 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa4080046f0 con 0x557da520ec80 2023-06-23T18:25:51.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2023-06-23T18:25:51.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:25:51.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 20 client.15714 populate_metadata read hostname 'smithi002' 2023-06-23T18:25:51.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557da5278d70 0x557da527b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:25:51.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 4 client.15714 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:25:51.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3277995696 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557da511b050 con 0x557da5278d70 2023-06-23T18:25:51.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.281+0000 7fa415d9b700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557da5278d70 0x557da527b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:25:51.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa3fdffb700 10 client.15714 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:25:51.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa415d9b700 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557da5278d70 0x557da527b180 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8bf80cc86473f05 server_cookie=15b48fa52bb38e42 in_seq=0 out_seq=0 2023-06-23T18:25:51.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa3fdffb700 1 -- 172.21.15.2:0/3277995696 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557da511b050 con 0x557da5278d70 2023-06-23T18:25:51.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa3fdffb700 10 client.15714 handle_command_reply: tid=0 2023-06-23T18:25:51.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa428f7cb80 1 client.15714 shutdown 2023-06-23T18:25:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3277995696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 msgr2=0x557da521b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da521b510 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa408026000 tx=0x7fa40800e9a0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3277995696 shutdown_connections 2023-06-23T18:25:51.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557da5278d70 0x557da527b180 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557da5210000 0x557da521ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557da520ec80 0x557da521b510 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa3fdffb700 0 client.15714 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:25:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.282+0000 7fa428f7cb80 1 --2- 172.21.15.2:0/3277995696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557da520f5d0 0x557da521ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.283+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3277995696 >> 172.21.15.2:0/3277995696 conn(0x557da52043f0 msgr2=0x557da5216320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.283+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3277995696 shutdown_connections 2023-06-23T18:25:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.283+0000 7fa428f7cb80 1 -- 172.21.15.2:0/3277995696 wait complete. 2023-06-23T18:25:51.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.283+0000 7fa428f7cb80 20 client.15714 trim_cache size 0 max 16384 2023-06-23T18:25:51.308 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:25:51.308 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:25:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:25:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:25:51.309 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:25:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:25:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:25:51.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:25:51.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:25:51.311 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:25:51.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.283+0000 7fa41cdf7700 1 -- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4000a1150 msgr2=0x7fa4000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.283+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4000a1150 0x7fa4000a3600 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa418069700 tx=0x7fa408009260 comp rx=0 tx=0).stop 2023-06-23T18:25:51.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.283+0000 7fa41cdf7700 1 -- 172.21.15.2:0/2934105010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa418071e30 msgr2=0x7fa4181087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:25:51.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.283+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa418071e30 0x7fa4181087a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c600 tx=0x7fa40c00c910 comp rx=0 tx=0).stop 2023-06-23T18:25:51.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.284+0000 7fa41cdf7700 1 -- 172.21.15.2:0/2934105010 shutdown_connections 2023-06-23T18:25:51.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.284+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa4000a1150 0x7fa4000a3600 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.284+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa418107790 0x7fa418108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.284+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa418071480 0x7fa41810a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.284+0000 7fa41cdf7700 1 --2- 172.21.15.2:0/2934105010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa418071e30 0x7fa4181087a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:25:51.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.284+0000 7fa41cdf7700 1 -- 172.21.15.2:0/2934105010 >> 172.21.15.2:0/2934105010 conn(0x7fa41806cde0 msgr2=0x7fa41806f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:25:51.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.284+0000 7fa41cdf7700 1 -- 172.21.15.2:0/2934105010 shutdown_connections 2023-06-23T18:25:51.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:25:51.284+0000 7fa41cdf7700 1 -- 172.21.15.2:0/2934105010 wait complete. 2023-06-23T18:25:51.314 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:25:52.024 INFO:teuthology.orchestra.run.smithi002.stderr:19800000 of 50000000 tuples (39%) done (elapsed 195.27 s, remaining 297.83 s) 2023-06-23T18:25:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:52 smithi002 ceph-mon[95947]: pgmap v411: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:52.832 INFO:teuthology.orchestra.run.smithi002.stderr:19900000 of 50000000 tuples (39%) done (elapsed 196.07 s, remaining 296.57 s) 2023-06-23T18:25:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:52 smithi082 ceph-mon[97889]: pgmap v411: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:52 smithi017 ceph-mon[97472]: pgmap v411: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:25:53.682 INFO:teuthology.orchestra.run.smithi002.stderr:20000000 of 50000000 tuples (40%) done (elapsed 196.92 s, remaining 295.39 s) 2023-06-23T18:25:54.566 INFO:teuthology.orchestra.run.smithi002.stderr:20100000 of 50000000 tuples (40%) done (elapsed 197.81 s, remaining 294.25 s) 2023-06-23T18:25:54.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:54 smithi002 ceph-mon[95947]: pgmap v412: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:25:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:54 smithi082 ceph-mon[97889]: pgmap v412: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:25:54.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:54 smithi017 ceph-mon[97472]: pgmap v412: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:25:55.412 INFO:teuthology.orchestra.run.smithi002.stderr:20200000 of 50000000 tuples (40%) done (elapsed 198.65 s, remaining 293.06 s) 2023-06-23T18:25:56.225 INFO:teuthology.orchestra.run.smithi002.stderr:20300000 of 50000000 tuples (40%) done (elapsed 199.47 s, remaining 291.83 s) 2023-06-23T18:25:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:56 smithi002 ceph-mon[95947]: pgmap v413: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:25:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:56 smithi082 ceph-mon[97889]: pgmap v413: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:25:56.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:56 smithi017 ceph-mon[97472]: pgmap v413: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:25:57.044 INFO:teuthology.orchestra.run.smithi002.stderr:20400000 of 50000000 tuples (40%) done (elapsed 200.29 s, remaining 290.61 s) 2023-06-23T18:25:57.943 INFO:teuthology.orchestra.run.smithi002.stderr:20500000 of 50000000 tuples (41%) done (elapsed 201.18 s, remaining 289.51 s) 2023-06-23T18:25:58.797 INFO:teuthology.orchestra.run.smithi002.stderr:20600000 of 50000000 tuples (41%) done (elapsed 202.04 s, remaining 288.35 s) 2023-06-23T18:25:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:58 smithi002 ceph-mon[95947]: pgmap v414: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 40 MiB/s wr, 21 op/s 2023-06-23T18:25:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:25:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:58 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:58 smithi082 ceph-mon[97889]: pgmap v414: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 40 MiB/s wr, 21 op/s 2023-06-23T18:25:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:25:58.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:58 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:58 smithi017 ceph-mon[97472]: pgmap v414: 97 pgs: 97 active+clean; 3.5 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 40 MiB/s wr, 21 op/s 2023-06-23T18:25:58.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:25:58.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:58.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:58 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:59.672 INFO:teuthology.orchestra.run.smithi002.stderr:20700000 of 50000000 tuples (41%) done (elapsed 202.91 s, remaining 287.22 s) 2023-06-23T18:25:59.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:25:59.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:25:59.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:59 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:59.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:25:59 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:25:59.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:25:59.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:25:59.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:59 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:59.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:25:59 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:25:59.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:25:59.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:25:59.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:59 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:25:59.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:25:59 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:26:00.563 INFO:teuthology.orchestra.run.smithi002.stderr:20800000 of 50000000 tuples (41%) done (elapsed 203.80 s, remaining 286.11 s) 2023-06-23T18:26:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:00 smithi002 ceph-mon[95947]: pgmap v415: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:26:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:00 smithi082 ceph-mon[97889]: pgmap v415: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:26:00.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:00 smithi017 ceph-mon[97472]: pgmap v415: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:26:01.383 INFO:teuthology.orchestra.run.smithi002.stderr:20900000 of 50000000 tuples (41%) done (elapsed 204.62 s, remaining 284.91 s) 2023-06-23T18:26:02.236 INFO:teuthology.orchestra.run.smithi002.stderr:21000000 of 50000000 tuples (42%) done (elapsed 205.48 s, remaining 283.75 s) 2023-06-23T18:26:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:02 smithi002 ceph-mon[95947]: pgmap v416: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 38 MiB/s wr, 21 op/s 2023-06-23T18:26:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:02 smithi082 ceph-mon[97889]: pgmap v416: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 38 MiB/s wr, 21 op/s 2023-06-23T18:26:02.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:02 smithi017 ceph-mon[97472]: pgmap v416: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 38 MiB/s wr, 21 op/s 2023-06-23T18:26:03.058 INFO:teuthology.orchestra.run.smithi002.stderr:21100000 of 50000000 tuples (42%) done (elapsed 206.30 s, remaining 282.56 s) 2023-06-23T18:26:03.968 INFO:teuthology.orchestra.run.smithi002.stderr:21200000 of 50000000 tuples (42%) done (elapsed 207.21 s, remaining 281.49 s) 2023-06-23T18:26:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:04 smithi002 ceph-mon[95947]: pgmap v417: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:26:04.814 INFO:teuthology.orchestra.run.smithi002.stderr:21300000 of 50000000 tuples (42%) done (elapsed 208.06 s, remaining 280.34 s) 2023-06-23T18:26:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:04 smithi082 ceph-mon[97889]: pgmap v417: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:26:04.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:04 smithi017 ceph-mon[97472]: pgmap v417: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:26:05.664 INFO:teuthology.orchestra.run.smithi002.stderr:21400000 of 50000000 tuples (42%) done (elapsed 208.91 s, remaining 279.19 s) 2023-06-23T18:26:06.521 INFO:teuthology.orchestra.run.smithi002.stderr:21500000 of 50000000 tuples (43%) done (elapsed 209.76 s, remaining 278.06 s) 2023-06-23T18:26:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:06 smithi002 ceph-mon[95947]: pgmap v418: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:26:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:06 smithi082 ceph-mon[97889]: pgmap v418: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:26:06.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:06 smithi017 ceph-mon[97472]: pgmap v418: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:26:07.498 INFO:teuthology.orchestra.run.smithi002.stderr:21600000 of 50000000 tuples (43%) done (elapsed 210.74 s, remaining 277.08 s) 2023-06-23T18:26:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:08 smithi002 ceph-mon[95947]: pgmap v419: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 42 MiB/s wr, 21 op/s 2023-06-23T18:26:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:08 smithi082 ceph-mon[97889]: pgmap v419: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 42 MiB/s wr, 21 op/s 2023-06-23T18:26:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:08 smithi017 ceph-mon[97472]: pgmap v419: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 42 MiB/s wr, 21 op/s 2023-06-23T18:26:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:21700000 of 50000000 tuples (43%) done (elapsed 212.41 s, remaining 277.02 s) 2023-06-23T18:26:09.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:09 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:26:09.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:09 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:26:09.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:09 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:26:09.978 INFO:teuthology.orchestra.run.smithi002.stderr:21800000 of 50000000 tuples (43%) done (elapsed 213.22 s, remaining 275.82 s) 2023-06-23T18:26:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:10 smithi002 ceph-mon[95947]: pgmap v420: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 31 op/s 2023-06-23T18:26:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:10 smithi082 ceph-mon[97889]: pgmap v420: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 31 op/s 2023-06-23T18:26:10.873 INFO:teuthology.orchestra.run.smithi002.stderr:21900000 of 50000000 tuples (43%) done (elapsed 214.11 s, remaining 274.73 s) 2023-06-23T18:26:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:10 smithi017 ceph-mon[97472]: pgmap v420: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 41 MiB/s wr, 31 op/s 2023-06-23T18:26:11.687 INFO:teuthology.orchestra.run.smithi002.stderr:22000000 of 50000000 tuples (44%) done (elapsed 214.93 s, remaining 273.54 s) 2023-06-23T18:26:12.480 INFO:teuthology.orchestra.run.smithi002.stderr:22100000 of 50000000 tuples (44%) done (elapsed 215.72 s, remaining 272.34 s) 2023-06-23T18:26:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:12 smithi002 ceph-mon[95947]: pgmap v421: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 41 op/s 2023-06-23T18:26:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:12 smithi082 ceph-mon[97889]: pgmap v421: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 41 op/s 2023-06-23T18:26:12.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:12 smithi017 ceph-mon[97472]: pgmap v421: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 36 MiB/s wr, 41 op/s 2023-06-23T18:26:13.399 INFO:teuthology.orchestra.run.smithi002.stderr:22200000 of 50000000 tuples (44%) done (elapsed 216.64 s, remaining 271.29 s) 2023-06-23T18:26:14.235 INFO:teuthology.orchestra.run.smithi002.stderr:22300000 of 50000000 tuples (44%) done (elapsed 217.48 s, remaining 270.14 s) 2023-06-23T18:26:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:14 smithi002 ceph-mon[95947]: pgmap v422: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 44 op/s 2023-06-23T18:26:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:14 smithi082 ceph-mon[97889]: pgmap v422: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 44 op/s 2023-06-23T18:26:14.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:14 smithi017 ceph-mon[97472]: pgmap v422: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 43 MiB/s wr, 44 op/s 2023-06-23T18:26:15.057 INFO:teuthology.orchestra.run.smithi002.stderr:22400000 of 50000000 tuples (44%) done (elapsed 218.30 s, remaining 268.97 s) 2023-06-23T18:26:15.922 INFO:teuthology.orchestra.run.smithi002.stderr:22500000 of 50000000 tuples (45%) done (elapsed 219.16 s, remaining 267.87 s) 2023-06-23T18:26:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:16 smithi002 ceph-mon[95947]: pgmap v423: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:26:16.804 INFO:teuthology.orchestra.run.smithi002.stderr:22600000 of 50000000 tuples (45%) done (elapsed 220.05 s, remaining 266.78 s) 2023-06-23T18:26:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:16 smithi082 ceph-mon[97889]: pgmap v423: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:26:16.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:16 smithi017 ceph-mon[97472]: pgmap v423: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:26:17.638 INFO:teuthology.orchestra.run.smithi002.stderr:22700000 of 50000000 tuples (45%) done (elapsed 220.88 s, remaining 265.64 s) 2023-06-23T18:26:18.458 INFO:teuthology.orchestra.run.smithi002.stderr:22800000 of 50000000 tuples (45%) done (elapsed 221.70 s, remaining 264.48 s) 2023-06-23T18:26:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:18 smithi002 ceph-mon[95947]: pgmap v424: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 41 op/s 2023-06-23T18:26:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:18 smithi082 ceph-mon[97889]: pgmap v424: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 41 op/s 2023-06-23T18:26:18.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:18 smithi017 ceph-mon[97472]: pgmap v424: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 36 MiB/s wr, 41 op/s 2023-06-23T18:26:19.349 INFO:teuthology.orchestra.run.smithi002.stderr:22900000 of 50000000 tuples (45%) done (elapsed 222.59 s, remaining 263.41 s) 2023-06-23T18:26:20.170 INFO:teuthology.orchestra.run.smithi002.stderr:23000000 of 50000000 tuples (46%) done (elapsed 223.41 s, remaining 262.27 s) 2023-06-23T18:26:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:20 smithi002 ceph-mon[95947]: pgmap v425: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:26:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:20 smithi082 ceph-mon[97889]: pgmap v425: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:26:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:20 smithi017 ceph-mon[97472]: pgmap v425: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:26:20.971 INFO:teuthology.orchestra.run.smithi002.stderr:23100000 of 50000000 tuples (46%) done (elapsed 224.21 s, remaining 261.10 s) 2023-06-23T18:26:21.315 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:26:21.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.463+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4232090032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 msgr2=0x7f1ff40fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.463+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4232090032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 0x7f1ff40fff60 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009990 tx=0x7f1fe4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4232090032 shutdown_connections 2023-06-23T18:26:21.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4232090032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff4101790 0x7f1ff4108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4232090032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 0x7f1ff40fff60 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4232090032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ff4100de0 0x7f1ff4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4232090032 >> 172.21.15.2:0/4232090032 conn(0x7f1ff40fb150 msgr2=0x7f1ff40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:21.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4232090032 shutdown_connections 2023-06-23T18:26:21.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4232090032 wait complete. 2023-06-23T18:26:21.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 Processor -- start 2023-06-23T18:26:21.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 -- start start 2023-06-23T18:26:21.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.464+0000 7f1ffa86f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 0x7f1ff4074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ffa86f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff4100de0 0x7f1ff40728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ffa86f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ff4101790 0x7f1ff4072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ffa86f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff40735a0 con 0x7f1ff40ffb50 2023-06-23T18:26:21.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ffa86f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff4073710 con 0x7f1ff4101790 2023-06-23T18:26:21.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ffa86f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff419f810 con 0x7f1ff4100de0 2023-06-23T18:26:21.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 0x7f1ff4074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 0x7f1ff4074280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54850/0 (socket says 172.21.15.2:54850) 2023-06-23T18:26:21.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4225773712 learned_addr learned my addr 172.21.15.2:0/4225773712 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:21.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff4100de0 0x7f1ff40728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff8e0c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ff4101790 0x7f1ff4072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4225773712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff4100de0 msgr2=0x7f1ff40728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff4100de0 0x7f1ff40728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4225773712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ff4101790 msgr2=0x7f1ff4072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ff4101790 0x7f1ff4072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4225773712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe40095f0 con 0x7f1ff40ffb50 2023-06-23T18:26:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 0x7f1ff4074280 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40059d0 tx=0x7f1fe40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff17fa700 1 -- 172.21.15.2:0/4225773712 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe401d070 con 0x7f1ff40ffb50 2023-06-23T18:26:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff17fa700 1 -- 172.21.15.2:0/4225773712 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1fe400bdb0 con 0x7f1ff40ffb50 2023-06-23T18:26:21.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4225773712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff419fa90 con 0x7f1ff40ffb50 2023-06-23T18:26:21.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ff17fa700 1 -- 172.21.15.2:0/4225773712 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe400fd90 con 0x7f1ff40ffb50 2023-06-23T18:26:21.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.465+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4225773712 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff419ff40 con 0x7f1ff40ffb50 2023-06-23T18:26:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.468+0000 7f1ff17fa700 1 -- 172.21.15.2:0/4225773712 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1fe400f560 con 0x7f1ff40ffb50 2023-06-23T18:26:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.468+0000 7f1ff17fa700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.468+0000 7f1ff17fa700 1 -- 172.21.15.2:0/4225773712 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1fe40cdce0 con 0x7f1ff40ffb50 2023-06-23T18:26:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.468+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009c60 tx=0x7f1fe8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f20069f4b80 1 Processor -- start 2023-06-23T18:26:21.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f20069f4b80 1 -- start start 2023-06-23T18:26:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf95e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf95ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf95f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f20069f4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf862cb0 con 0x5587bf95dc80 2023-06-23T18:26:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f20069f4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf8b5a40 con 0x5587bf95f000 2023-06-23T18:26:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f20069f4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf8aeed0 con 0x5587bf95e5d0 2023-06-23T18:26:21.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf95e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf95e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54856/0 (socket says 172.21.15.2:54856) 2023-06-23T18:26:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf95ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4062710792 learned_addr learned my addr 172.21.15.2:0/4062710792 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.469+0000 7f1ff8e0c700 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf95f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4062710792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 msgr2=0x5587bf95ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf95ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4062710792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 msgr2=0x5587bf95f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:21.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf95f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4062710792 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587bf730630 con 0x5587bf95dc80 2023-06-23T18:26:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf95ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:26:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf95e090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1fe409b110 tx=0x7f1fe409afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10e8599dc93d372 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1fda7fc700 1 -- 172.21.15.2:0/4062710792 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe409aae0 con 0x5587bf95dc80 2023-06-23T18:26:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1fda7fc700 1 -- 172.21.15.2:0/4062710792 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1fe409b180 con 0x5587bf95dc80 2023-06-23T18:26:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.470+0000 7f1fda7fc700 1 -- 172.21.15.2:0/4062710792 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe40a30b0 con 0x5587bf95dc80 2023-06-23T18:26:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 -- 172.21.15.2:0/4062710792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 msgr2=0x5587bf95e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf95e090 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1fe409b110 tx=0x7f1fe409afd0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 -- 172.21.15.2:0/4062710792 shutdown_connections 2023-06-23T18:26:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf95f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf95e090 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4062710792 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf95ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 -- 172.21.15.2:0/4062710792 >> 172.21.15.2:0/4062710792 conn(0x5587bf9533f0 msgr2=0x5587bf958520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 -- 172.21.15.2:0/4062710792 shutdown_connections 2023-06-23T18:26:21.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 -- 172.21.15.2:0/4062710792 wait complete. 2023-06-23T18:26:21.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 -- start start 2023-06-23T18:26:21.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.471+0000 7f20069f4b80 1 Processor -- start 2023-06-23T18:26:21.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf967190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf9676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf96e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f20069f4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf84eac0 con 0x5587bf95dc80 2023-06-23T18:26:21.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f20069f4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf84e7c0 con 0x5587bf95f000 2023-06-23T18:26:21.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f20069f4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf732920 con 0x5587bf95e5d0 2023-06-23T18:26:21.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f20069f4b80 10 client.0 _ll_register_callbacks cb 0x5587bf854260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:26:21.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf9676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf967190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf9676d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55202/0 (socket says 172.21.15.2:55202) 2023-06-23T18:26:21.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff37fe700 1 -- 172.21.15.2:0/4047391363 learned_addr learned my addr 172.21.15.2:0/4047391363 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:21.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff8e0c700 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf96e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4047391363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 msgr2=0x5587bf9676d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf9676d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4047391363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 msgr2=0x5587bf96e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf96e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff3fff700 1 -- 172.21.15.2:0/4047391363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe8017070 con 0x5587bf95dc80 2023-06-23T18:26:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf9676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:26:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf967190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400b520 tx=0x7f1fe40094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a229739216bec3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1fd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:26:21.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1fd37fe700 1 -- 172.21.15.2:0/4047391363 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4099520 con 0x5587bf95dc80 2023-06-23T18:26:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f1fd37fe700 1 -- 172.21.15.2:0/4047391363 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1fe40a3540 con 0x5587bf95dc80 2023-06-23T18:26:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.472+0000 7f20069f4b80 1 -- 172.21.15.2:0/4047391363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587bf950bb0 con 0x5587bf95dc80 2023-06-23T18:26:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f1fd37fe700 1 -- 172.21.15.2:0/4047391363 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe409aae0 con 0x5587bf95dc80 2023-06-23T18:26:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f20069f4b80 10 client.15726 fetch_fsmap learned FSMap version 30 2023-06-23T18:26:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f20069f4b80 1 -- 172.21.15.2:0/4047391363 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1ff419ff40 con 0x5587bf95dc80 2023-06-23T18:26:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f1fd37fe700 1 -- 172.21.15.2:0/4047391363 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5587bf950bb0 con 0x5587bf95dc80 2023-06-23T18:26:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f1fd37fe700 1 -- 172.21.15.2:0/4047391363 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1fe4097100 con 0x5587bf95dc80 2023-06-23T18:26:21.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f20069f4b80 10 client.15726 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:26:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f20069f4b80 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:26:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f20069f4b80 20 client.15726 populate_metadata read hostname 'smithi002' 2023-06-23T18:26:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5587bf972ab0 0x5587bf974e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f20069f4b80 4 client.15726 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:26:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f20069f4b80 1 -- 172.21.15.2:0/4047391363 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587bf8a7eb0 con 0x5587bf972ab0 2023-06-23T18:26:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5587bf972ab0 0x5587bf974e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f1fd37fe700 10 client.15726 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:26:21.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.473+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5587bf972ab0 0x5587bf974e90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0f848dd956a9b server_cookie=76a1dce41ed447d6 in_seq=0 out_seq=0 2023-06-23T18:26:21.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f1fd37fe700 1 -- 172.21.15.2:0/4047391363 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5587bf8a7eb0 con 0x5587bf972ab0 2023-06-23T18:26:21.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f1fd37fe700 10 client.15726 handle_command_reply: tid=0 2023-06-23T18:26:21.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 client.15726 shutdown 2023-06-23T18:26:21.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 -- 172.21.15.2:0/4047391363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 msgr2=0x5587bf967190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf967190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1fe400b520 tx=0x7f1fe40094b0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 -- 172.21.15.2:0/4047391363 shutdown_connections 2023-06-23T18:26:21.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5587bf972ab0 0x5587bf974e90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf96e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf967190 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f1fd37fe700 0 client.15726 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:26:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 --2- 172.21.15.2:0/4047391363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf9676d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 -- 172.21.15.2:0/4047391363 >> 172.21.15.2:0/4047391363 conn(0x5587bf9533f0 msgr2=0x5587bf957c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.474+0000 7f20069f4b80 1 -- 172.21.15.2:0/4047391363 shutdown_connections 2023-06-23T18:26:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.475+0000 7f20069f4b80 1 -- 172.21.15.2:0/4047391363 wait complete. 2023-06-23T18:26:21.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.475+0000 7f20069f4b80 20 client.15726 trim_cache size 0 max 16384 2023-06-23T18:26:21.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.491+0000 7f20069f4b80 1 Processor -- start 2023-06-23T18:26:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.491+0000 7f20069f4b80 1 -- start start 2023-06-23T18:26:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95dc80 0x5587bf96f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95e5d0 0x5587bf96e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95f000 0x5587bf966f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f20069f4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf8a7eb0 con 0x5587bf95e5d0 2023-06-23T18:26:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f20069f4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf955470 con 0x5587bf95dc80 2023-06-23T18:26:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f20069f4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf966640 con 0x5587bf95f000 2023-06-23T18:26:21.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95e5d0 0x5587bf96e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95e5d0 0x5587bf96e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54872/0 (socket says 172.21.15.2:54872) 2023-06-23T18:26:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff37fe700 1 -- 172.21.15.2:0/3864020008 learned_addr learned my addr 172.21.15.2:0/3864020008 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff8e0c700 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95f000 0x5587bf966f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95dc80 0x5587bf96f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff8e0c700 1 -- 172.21.15.2:0/3864020008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95dc80 msgr2=0x5587bf96f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff8e0c700 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95dc80 0x5587bf96f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff8e0c700 1 -- 172.21.15.2:0/3864020008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95e5d0 msgr2=0x5587bf96e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff8e0c700 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95e5d0 0x5587bf96e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff8e0c700 1 -- 172.21.15.2:0/3864020008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587bf6dcde0 con 0x5587bf95f000 2023-06-23T18:26:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95e5d0 0x5587bf96e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:26:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1ff8e0c700 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95f000 0x5587bf966f40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40748e0 tx=0x7f1fec009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97d1356bf0d25e3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1fdaffd700 1 -- 172.21.15.2:0/3864020008 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00e040 con 0x5587bf95f000 2023-06-23T18:26:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1fdaffd700 1 -- 172.21.15.2:0/3864020008 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1fec00f040 con 0x5587bf95f000 2023-06-23T18:26:21.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.492+0000 7f1fdaffd700 1 -- 172.21.15.2:0/3864020008 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec007500 con 0x5587bf95f000 2023-06-23T18:26:21.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 -- 172.21.15.2:0/3864020008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95f000 msgr2=0x5587bf966f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95f000 0x5587bf966f40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40748e0 tx=0x7f1fec009920 comp rx=0 tx=0).stop 2023-06-23T18:26:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 -- 172.21.15.2:0/3864020008 shutdown_connections 2023-06-23T18:26:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95f000 0x5587bf966f40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95dc80 0x5587bf96f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 --2- 172.21.15.2:0/3864020008 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95e5d0 0x5587bf96e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 -- 172.21.15.2:0/3864020008 >> 172.21.15.2:0/3864020008 conn(0x5587bf9533f0 msgr2=0x5587bf9797c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:21.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 -- 172.21.15.2:0/3864020008 shutdown_connections 2023-06-23T18:26:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 -- 172.21.15.2:0/3864020008 wait complete. 2023-06-23T18:26:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.493+0000 7f20069f4b80 1 -- start start 2023-06-23T18:26:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f20069f4b80 1 Processor -- start 2023-06-23T18:26:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf96a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf96aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f20069f4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf96df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f20069f4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf96e450 con 0x5587bf95dc80 2023-06-23T18:26:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f20069f4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf96e5c0 con 0x5587bf95f000 2023-06-23T18:26:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f20069f4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587bf96e780 con 0x5587bf95e5d0 2023-06-23T18:26:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f20069f4b80 10 client.0 _ll_register_callbacks cb 0x5587bf854260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:26:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf96aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf96a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf96aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55216/0 (socket says 172.21.15.2:55216) 2023-06-23T18:26:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff37fe700 1 -- 172.21.15.2:0/2341774190 learned_addr learned my addr 172.21.15.2:0/2341774190 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:21.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff8e0c700 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf96df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff37fe700 1 -- 172.21.15.2:0/2341774190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 msgr2=0x5587bf96df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf96df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff37fe700 1 -- 172.21.15.2:0/2341774190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 msgr2=0x5587bf96a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf96a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.494+0000 7f1ff37fe700 1 -- 172.21.15.2:0/2341774190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe802bab0 con 0x5587bf95e5d0 2023-06-23T18:26:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f1ff37fe700 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf96aef0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5587bf975710 tx=0x7f1fe80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e1f349f2cf54258 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:21.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f1fd37fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:26:21.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f1fd37fe700 1 -- 172.21.15.2:0/2341774190 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8022040 con 0x5587bf95e5d0 2023-06-23T18:26:21.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f1fd37fe700 1 -- 172.21.15.2:0/2341774190 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1fe8003de0 con 0x5587bf95e5d0 2023-06-23T18:26:21.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 1 -- 172.21.15.2:0/2341774190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fe409b180 con 0x5587bf95e5d0 2023-06-23T18:26:21.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f1fd37fe700 1 -- 172.21.15.2:0/2341774190 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fe802b040 con 0x5587bf95e5d0 2023-06-23T18:26:21.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 10 client.25492 fetch_fsmap learned FSMap version 30 2023-06-23T18:26:21.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 1 -- 172.21.15.2:0/2341774190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1fec0114d0 con 0x5587bf95e5d0 2023-06-23T18:26:21.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f1fd37fe700 1 -- 172.21.15.2:0/2341774190 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1fe409b180 con 0x5587bf95e5d0 2023-06-23T18:26:21.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f1fd37fe700 1 -- 172.21.15.2:0/2341774190 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1fe8034a90 con 0x5587bf95e5d0 2023-06-23T18:26:21.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 10 client.25492 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:26:21.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 10 client.25492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:26:21.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 20 client.25492 populate_metadata read hostname 'smithi002' 2023-06-23T18:26:21.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5587bf9c7d70 0x5587bf9ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:21.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 4 client.25492 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:26:21.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.495+0000 7f20069f4b80 1 -- 172.21.15.2:0/2341774190 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5587bf86a050 con 0x5587bf9c7d70 2023-06-23T18:26:21.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.496+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5587bf9c7d70 0x5587bf9ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.496+0000 7f1fd37fe700 10 client.25492 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:26:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.496+0000 7f1ff3fff700 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5587bf9c7d70 0x5587bf9ca180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b37554906a33288a server_cookie=f46d83eab22d2fca in_seq=0 out_seq=0 2023-06-23T18:26:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.496+0000 7f1fd37fe700 1 -- 172.21.15.2:0/2341774190 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5587bf86a050 con 0x5587bf9c7d70 2023-06-23T18:26:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.496+0000 7f1fd37fe700 10 client.25492 handle_command_reply: tid=0 2023-06-23T18:26:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.496+0000 7f20069f4b80 1 client.25492 shutdown 2023-06-23T18:26:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.496+0000 7f20069f4b80 1 -- 172.21.15.2:0/2341774190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 msgr2=0x5587bf96aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.496+0000 7f20069f4b80 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf96aef0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x5587bf975710 tx=0x7f1fe80047c0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 1 -- 172.21.15.2:0/2341774190 shutdown_connections 2023-06-23T18:26:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5587bf9c7d70 0x5587bf9ca180 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5587bf95f000 0x5587bf96df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f1fd37fe700 0 client.25492 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:26:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5587bf95dc80 0x5587bf96a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 1 --2- 172.21.15.2:0/2341774190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5587bf95e5d0 0x5587bf96aef0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 1 -- 172.21.15.2:0/2341774190 >> 172.21.15.2:0/2341774190 conn(0x5587bf9533f0 msgr2=0x5587bf961e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 1 -- 172.21.15.2:0/2341774190 shutdown_connections 2023-06-23T18:26:21.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 1 -- 172.21.15.2:0/2341774190 wait complete. 2023-06-23T18:26:21.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f20069f4b80 20 client.25492 trim_cache size 0 max 16384 2023-06-23T18:26:21.512 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:26:21.512 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:26:21.512 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:26:21.513 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:26:21.513 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:26:21.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:26:21.513 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:26:21.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:26:21.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:26:21.513 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:26:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1fdc0a1150 msgr2=0x7f1fdc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009c60 tx=0x7f1fe8009260 comp rx=0 tx=0).stop 2023-06-23T18:26:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 msgr2=0x7f1ff4074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.497+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 0x7f1ff4074280 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1fe40059d0 tx=0x7f1fe40047b0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.498+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4225773712 shutdown_connections 2023-06-23T18:26:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.498+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1fdc0a1150 0x7f1fdc0a3600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.498+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ff4101790 0x7f1ff4072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.498+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ff40ffb50 0x7f1ff4074280 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.498+0000 7f1ffa86f700 1 --2- 172.21.15.2:0/4225773712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff4100de0 0x7f1ff40728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.498+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4225773712 >> 172.21.15.2:0/4225773712 conn(0x7f1ff40fb150 msgr2=0x7f1ff40fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.498+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4225773712 shutdown_connections 2023-06-23T18:26:21.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:21.498+0000 7f1ffa86f700 1 -- 172.21.15.2:0/4225773712 wait complete. 2023-06-23T18:26:21.517 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:26:21.883 INFO:teuthology.orchestra.run.smithi002.stderr:23200000 of 50000000 tuples (46%) done (elapsed 225.12 s, remaining 260.06 s) 2023-06-23T18:26:22.757 INFO:teuthology.orchestra.run.smithi002.stderr:23300000 of 50000000 tuples (46%) done (elapsed 226.00 s, remaining 258.98 s) 2023-06-23T18:26:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:22 smithi002 ceph-mon[95947]: pgmap v426: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 MiB/s wr, 34 op/s 2023-06-23T18:26:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:22 smithi082 ceph-mon[97889]: pgmap v426: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 MiB/s wr, 34 op/s 2023-06-23T18:26:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:22 smithi017 ceph-mon[97472]: pgmap v426: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 MiB/s wr, 34 op/s 2023-06-23T18:26:23.605 INFO:teuthology.orchestra.run.smithi002.stderr:23400000 of 50000000 tuples (46%) done (elapsed 226.85 s, remaining 257.87 s) 2023-06-23T18:26:24.447 INFO:teuthology.orchestra.run.smithi002.stderr:23500000 of 50000000 tuples (47%) done (elapsed 227.69 s, remaining 256.76 s) 2023-06-23T18:26:24.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:24 smithi002 ceph-mon[95947]: pgmap v427: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:26:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:24 smithi082 ceph-mon[97889]: pgmap v427: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:26:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:24 smithi017 ceph-mon[97472]: pgmap v427: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:26:25.326 INFO:teuthology.orchestra.run.smithi002.stderr:23600000 of 50000000 tuples (47%) done (elapsed 228.57 s, remaining 255.69 s) 2023-06-23T18:26:26.179 INFO:teuthology.orchestra.run.smithi002.stderr:23700000 of 50000000 tuples (47%) done (elapsed 229.42 s, remaining 254.59 s) 2023-06-23T18:26:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:26 smithi002 ceph-mon[95947]: pgmap v428: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:26:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:26 smithi082 ceph-mon[97889]: pgmap v428: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:26:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:26 smithi017 ceph-mon[97472]: pgmap v428: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:26:26.990 INFO:teuthology.orchestra.run.smithi002.stderr:23800000 of 50000000 tuples (47%) done (elapsed 230.23 s, remaining 253.45 s) 2023-06-23T18:26:27.853 INFO:teuthology.orchestra.run.smithi002.stderr:23900000 of 50000000 tuples (47%) done (elapsed 231.09 s, remaining 252.37 s) 2023-06-23T18:26:28.747 INFO:teuthology.orchestra.run.smithi002.stderr:24000000 of 50000000 tuples (48%) done (elapsed 231.99 s, remaining 251.32 s) 2023-06-23T18:26:28.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:28 smithi002 ceph-mon[95947]: pgmap v429: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 41 MiB/s wr, 21 op/s 2023-06-23T18:26:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:28 smithi082 ceph-mon[97889]: pgmap v429: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 41 MiB/s wr, 21 op/s 2023-06-23T18:26:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:28 smithi017 ceph-mon[97472]: pgmap v429: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 41 MiB/s wr, 21 op/s 2023-06-23T18:26:29.627 INFO:teuthology.orchestra.run.smithi002.stderr:24100000 of 50000000 tuples (48%) done (elapsed 232.87 s, remaining 250.26 s) 2023-06-23T18:26:30.515 INFO:teuthology.orchestra.run.smithi002.stderr:24200000 of 50000000 tuples (48%) done (elapsed 233.76 s, remaining 249.21 s) 2023-06-23T18:26:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:30 smithi002 ceph-mon[95947]: pgmap v430: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:26:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:30 smithi082 ceph-mon[97889]: pgmap v430: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:26:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:30 smithi017 ceph-mon[97472]: pgmap v430: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:26:31.413 INFO:teuthology.orchestra.run.smithi002.stderr:24300000 of 50000000 tuples (48%) done (elapsed 234.65 s, remaining 248.17 s) 2023-06-23T18:26:32.223 INFO:teuthology.orchestra.run.smithi002.stderr:24400000 of 50000000 tuples (48%) done (elapsed 235.46 s, remaining 247.04 s) 2023-06-23T18:26:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:32 smithi002 ceph-mon[95947]: pgmap v431: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 37 MiB/s wr, 22 op/s 2023-06-23T18:26:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:32 smithi082 ceph-mon[97889]: pgmap v431: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 37 MiB/s wr, 22 op/s 2023-06-23T18:26:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:32 smithi017 ceph-mon[97472]: pgmap v431: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 37 MiB/s wr, 22 op/s 2023-06-23T18:26:33.050 INFO:teuthology.orchestra.run.smithi002.stderr:24500000 of 50000000 tuples (49%) done (elapsed 236.29 s, remaining 245.94 s) 2023-06-23T18:26:33.890 INFO:teuthology.orchestra.run.smithi002.stderr:24600000 of 50000000 tuples (49%) done (elapsed 237.13 s, remaining 244.84 s) 2023-06-23T18:26:34.759 INFO:teuthology.orchestra.run.smithi002.stderr:24700000 of 50000000 tuples (49%) done (elapsed 238.00 s, remaining 243.78 s) 2023-06-23T18:26:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:34 smithi002 ceph-mon[95947]: pgmap v432: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 47 MiB/s wr, 33 op/s 2023-06-23T18:26:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:34 smithi082 ceph-mon[97889]: pgmap v432: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 47 MiB/s wr, 33 op/s 2023-06-23T18:26:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:34 smithi017 ceph-mon[97472]: pgmap v432: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 47 MiB/s wr, 33 op/s 2023-06-23T18:26:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:24800000 of 50000000 tuples (49%) done (elapsed 238.84 s, remaining 242.69 s) 2023-06-23T18:26:36.419 INFO:teuthology.orchestra.run.smithi002.stderr:24900000 of 50000000 tuples (49%) done (elapsed 239.66 s, remaining 241.58 s) 2023-06-23T18:26:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:36 smithi002 ceph-mon[95947]: pgmap v433: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 41 MiB/s wr, 35 op/s 2023-06-23T18:26:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:36 smithi082 ceph-mon[97889]: pgmap v433: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 41 MiB/s wr, 35 op/s 2023-06-23T18:26:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:36 smithi017 ceph-mon[97472]: pgmap v433: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 41 MiB/s wr, 35 op/s 2023-06-23T18:26:37.325 INFO:teuthology.orchestra.run.smithi002.stderr:25000000 of 50000000 tuples (50%) done (elapsed 240.57 s, remaining 240.57 s) 2023-06-23T18:26:38.169 INFO:teuthology.orchestra.run.smithi002.stderr:25100000 of 50000000 tuples (50%) done (elapsed 241.41 s, remaining 239.49 s) 2023-06-23T18:26:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:38 smithi002 ceph-mon[95947]: pgmap v434: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 42 MiB/s wr, 35 op/s 2023-06-23T18:26:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:38 smithi082 ceph-mon[97889]: pgmap v434: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 42 MiB/s wr, 35 op/s 2023-06-23T18:26:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:38 smithi017 ceph-mon[97472]: pgmap v434: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 42 MiB/s wr, 35 op/s 2023-06-23T18:26:39.000 INFO:teuthology.orchestra.run.smithi002.stderr:25200000 of 50000000 tuples (50%) done (elapsed 242.24 s, remaining 238.40 s) 2023-06-23T18:26:39.916 INFO:teuthology.orchestra.run.smithi002.stderr:25300000 of 50000000 tuples (50%) done (elapsed 243.16 s, remaining 237.39 s) 2023-06-23T18:26:40.796 INFO:teuthology.orchestra.run.smithi002.stderr:25400000 of 50000000 tuples (50%) done (elapsed 244.04 s, remaining 236.35 s) 2023-06-23T18:26:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:40 smithi002 ceph-mon[95947]: pgmap v435: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 38 op/s 2023-06-23T18:26:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:40 smithi082 ceph-mon[97889]: pgmap v435: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 38 op/s 2023-06-23T18:26:40.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:40 smithi017 ceph-mon[97472]: pgmap v435: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 38 op/s 2023-06-23T18:26:41.749 INFO:teuthology.orchestra.run.smithi002.stderr:25500000 of 50000000 tuples (51%) done (elapsed 244.99 s, remaining 235.38 s) 2023-06-23T18:26:42.616 INFO:teuthology.orchestra.run.smithi002.stderr:25600000 of 50000000 tuples (51%) done (elapsed 245.86 s, remaining 234.33 s) 2023-06-23T18:26:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:42 smithi002 ceph-mon[95947]: pgmap v436: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 36 op/s 2023-06-23T18:26:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:42 smithi082 ceph-mon[97889]: pgmap v436: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 36 op/s 2023-06-23T18:26:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:42 smithi017 ceph-mon[97472]: pgmap v436: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 39 MiB/s wr, 36 op/s 2023-06-23T18:26:43.516 INFO:teuthology.orchestra.run.smithi002.stderr:25700000 of 50000000 tuples (51%) done (elapsed 246.76 s, remaining 233.32 s) 2023-06-23T18:26:44.360 INFO:teuthology.orchestra.run.smithi002.stderr:25800000 of 50000000 tuples (51%) done (elapsed 247.60 s, remaining 232.25 s) 2023-06-23T18:26:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:44 smithi002 ceph-mon[95947]: pgmap v437: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 MiB/s wr, 38 op/s 2023-06-23T18:26:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:44 smithi082 ceph-mon[97889]: pgmap v437: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 MiB/s wr, 38 op/s 2023-06-23T18:26:44.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:44 smithi017 ceph-mon[97472]: pgmap v437: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 47 MiB/s wr, 38 op/s 2023-06-23T18:26:45.179 INFO:teuthology.orchestra.run.smithi002.stderr:25900000 of 50000000 tuples (51%) done (elapsed 248.42 s, remaining 231.16 s) 2023-06-23T18:26:46.015 INFO:teuthology.orchestra.run.smithi002.stderr:26000000 of 50000000 tuples (52%) done (elapsed 249.26 s, remaining 230.08 s) 2023-06-23T18:26:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:46 smithi002 ceph-mon[95947]: pgmap v438: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 37 MiB/s wr, 49 op/s 2023-06-23T18:26:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:46 smithi082 ceph-mon[97889]: pgmap v438: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 37 MiB/s wr, 49 op/s 2023-06-23T18:26:46.907 INFO:teuthology.orchestra.run.smithi002.stderr:26100000 of 50000000 tuples (52%) done (elapsed 250.15 s, remaining 229.06 s) 2023-06-23T18:26:46.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:46 smithi017 ceph-mon[97472]: pgmap v438: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 37 MiB/s wr, 49 op/s 2023-06-23T18:26:47.716 INFO:teuthology.orchestra.run.smithi002.stderr:26200000 of 50000000 tuples (52%) done (elapsed 250.96 s, remaining 227.97 s) 2023-06-23T18:26:48.552 INFO:teuthology.orchestra.run.smithi002.stderr:26300000 of 50000000 tuples (52%) done (elapsed 251.79 s, remaining 226.90 s) 2023-06-23T18:26:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:48 smithi002 ceph-mon[95947]: pgmap v439: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 MiB/s wr, 45 op/s 2023-06-23T18:26:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:48 smithi082 ceph-mon[97889]: pgmap v439: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 MiB/s wr, 45 op/s 2023-06-23T18:26:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:48 smithi017 ceph-mon[97472]: pgmap v439: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 MiB/s wr, 45 op/s 2023-06-23T18:26:49.452 INFO:teuthology.orchestra.run.smithi002.stderr:26400000 of 50000000 tuples (52%) done (elapsed 252.69 s, remaining 225.89 s) 2023-06-23T18:26:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:26500000 of 50000000 tuples (53%) done (elapsed 253.52 s, remaining 224.82 s) 2023-06-23T18:26:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:50 smithi082 ceph-mon[97889]: pgmap v440: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 MiB/s wr, 49 op/s 2023-06-23T18:26:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:50 smithi017 ceph-mon[97472]: pgmap v440: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 MiB/s wr, 49 op/s 2023-06-23T18:26:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:50 smithi002 ceph-mon[95947]: pgmap v440: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 42 MiB/s wr, 49 op/s 2023-06-23T18:26:51.142 INFO:teuthology.orchestra.run.smithi002.stderr:26600000 of 50000000 tuples (53%) done (elapsed 254.38 s, remaining 223.78 s) 2023-06-23T18:26:51.518 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:26:51.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.649+0000 7f004be89700 1 -- 172.21.15.2:0/1300986145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00441058a0 msgr2=0x7f0044107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.649+0000 7f004be89700 1 --2- 172.21.15.2:0/1300986145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00441058a0 0x7f0044107ca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f003c009f90 tx=0x7f003c00b330 comp rx=0 tx=0).stop 2023-06-23T18:26:51.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.650+0000 7f004be89700 1 -- 172.21.15.2:0/1300986145 shutdown_connections 2023-06-23T18:26:51.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.650+0000 7f004be89700 1 --2- 172.21.15.2:0/1300986145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f00441058a0 0x7f0044107ca0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.650+0000 7f004be89700 1 --2- 172.21.15.2:0/1300986145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0044100660 0x7f0044102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.650+0000 7f004be89700 1 --2- 172.21.15.2:0/1300986145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0044102f80 0x7f0044105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.650+0000 7f004be89700 1 -- 172.21.15.2:0/1300986145 >> 172.21.15.2:0/1300986145 conn(0x7f00440fa0e0 msgr2=0x7f00440fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:51.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.650+0000 7f004be89700 1 -- 172.21.15.2:0/1300986145 shutdown_connections 2023-06-23T18:26:51.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.650+0000 7f004be89700 1 -- 172.21.15.2:0/1300986145 wait complete. 2023-06-23T18:26:51.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.650+0000 7f004be89700 1 Processor -- start 2023-06-23T18:26:51.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f004be89700 1 -- start start 2023-06-23T18:26:51.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f004be89700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0044100660 0x7f0044195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f004be89700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0044102f80 0x7f0044195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f004be89700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00441058a0 0x7f004419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f004be89700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004419d400 con 0x7f00441058a0 2023-06-23T18:26:51.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f004be89700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004419d540 con 0x7f0044102f80 2023-06-23T18:26:51.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f004be89700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004419d6d0 con 0x7f0044100660 2023-06-23T18:26:51.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0044100660 0x7f0044195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0044100660 0x7f0044195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55228/0 (socket says 172.21.15.2:55228) 2023-06-23T18:26:51.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 -- 172.21.15.2:0/1985169302 learned_addr learned my addr 172.21.15.2:0/1985169302 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:51.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f004a426700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00441058a0 0x7f004419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049424700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0044102f80 0x7f0044195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 -- 172.21.15.2:0/1985169302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0044102f80 msgr2=0x7f0044195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0044102f80 0x7f0044195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 -- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00441058a0 msgr2=0x7f004419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00441058a0 0x7f004419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 -- 172.21.15.2:0/1985169302 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00400095f0 con 0x7f0044100660 2023-06-23T18:26:51.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.651+0000 7f0049c25700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0044100660 0x7f0044195880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f004000ec10 tx=0x7f004000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:51.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.652+0000 7f003affd700 1 -- 172.21.15.2:0/1985169302 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004000cc90 con 0x7f0044100660 2023-06-23T18:26:51.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.652+0000 7f004be89700 1 -- 172.21.15.2:0/1985169302 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c009c40 con 0x7f0044100660 2023-06-23T18:26:51.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.652+0000 7f003affd700 1 -- 172.21.15.2:0/1985169302 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f004000cdf0 con 0x7f0044100660 2023-06-23T18:26:51.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.652+0000 7f003affd700 1 -- 172.21.15.2:0/1985169302 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0040003e80 con 0x7f0044100660 2023-06-23T18:26:51.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.652+0000 7f004be89700 1 -- 172.21.15.2:0/1985169302 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004419dd30 con 0x7f0044100660 2023-06-23T18:26:51.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.654+0000 7f003affd700 1 -- 172.21.15.2:0/1985169302 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0040004000 con 0x7f0044100660 2023-06-23T18:26:51.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.655+0000 7f003affd700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00300a1150 0x7f00300a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.655+0000 7f0049424700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00300a1150 0x7f00300a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.655+0000 7f003affd700 1 -- 172.21.15.2:0/1985169302 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0040014070 con 0x7f0044100660 2023-06-23T18:26:51.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f005800eb80 1 Processor -- start 2023-06-23T18:26:51.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f0049424700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00300a1150 0x7f00300a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0034009c60 tx=0x7f0034009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:51.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f005800eb80 1 -- start start 2023-06-23T18:26:51.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f005800eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 0x55990f7a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f005800eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7a7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f005800eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a8000 0x55990f7a8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f005800eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f6abcb0 con 0x55990f7a8000 2023-06-23T18:26:51.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f005800eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f6fea40 con 0x55990f7a6c80 2023-06-23T18:26:51.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f005800eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f6f7ed0 con 0x55990f7a75d0 2023-06-23T18:26:51.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f0049424700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7a7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f0049c25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 0x55990f7a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f004a426700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a8000 0x55990f7a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f0049424700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7a7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55238/0 (socket says 172.21.15.2:55238) 2023-06-23T18:26:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.656+0000 7f0049424700 1 -- 172.21.15.2:0/1042785829 learned_addr learned my addr 172.21.15.2:0/1042785829 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f0049424700 1 -- 172.21.15.2:0/1042785829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 msgr2=0x55990f7a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f0049424700 1 --2- 172.21.15.2:0/1042785829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 0x55990f7a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f0049424700 1 -- 172.21.15.2:0/1042785829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a8000 msgr2=0x55990f7a8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f0049424700 1 --2- 172.21.15.2:0/1042785829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a8000 0x55990f7a8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f0049424700 1 -- 172.21.15.2:0/1042785829 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55990f579630 con 0x55990f7a75d0 2023-06-23T18:26:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f0049424700 1 --2- 172.21.15.2:0/1042785829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7a7a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0034009f80 tx=0x7f0034014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52304f43fa88108f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:51.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f002ffff700 1 -- 172.21.15.2:0/1042785829 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003400ee50 con 0x55990f7a75d0 2023-06-23T18:26:51.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f002ffff700 1 -- 172.21.15.2:0/1042785829 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0034025070 con 0x55990f7a75d0 2023-06-23T18:26:51.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.657+0000 7f002ffff700 1 -- 172.21.15.2:0/1042785829 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003401cc20 con 0x55990f7a75d0 2023-06-23T18:26:51.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 -- 172.21.15.2:0/1042785829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 msgr2=0x55990f7a7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 --2- 172.21.15.2:0/1042785829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7a7a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0034009f80 tx=0x7f0034014000 comp rx=0 tx=0).stop 2023-06-23T18:26:51.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 -- 172.21.15.2:0/1042785829 shutdown_connections 2023-06-23T18:26:51.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 --2- 172.21.15.2:0/1042785829 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a8000 0x55990f7a8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 --2- 172.21.15.2:0/1042785829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 0x55990f7a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 --2- 172.21.15.2:0/1042785829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7a7a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 -- 172.21.15.2:0/1042785829 >> 172.21.15.2:0/1042785829 conn(0x55990f79c3f0 msgr2=0x55990f7a1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 -- 172.21.15.2:0/1042785829 shutdown_connections 2023-06-23T18:26:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 -- 172.21.15.2:0/1042785829 wait complete. 2023-06-23T18:26:51.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 -- start start 2023-06-23T18:26:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.658+0000 7f005800eb80 1 Processor -- start 2023-06-23T18:26:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f005800eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a6c80 0x55990f7b0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f005800eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f005800eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a8000 0x55990f7b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f005800eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f697ac0 con 0x55990f7a6c80 2023-06-23T18:26:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f005800eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f6977c0 con 0x55990f7a75d0 2023-06-23T18:26:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f005800eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f57b920 con 0x55990f7a8000 2023-06-23T18:26:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f005800eb80 10 client.0 _ll_register_callbacks cb 0x55990f69d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:26:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f0049c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a6c80 0x55990f7b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f0049c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a6c80 0x55990f7b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54904/0 (socket says 172.21.15.2:54904) 2023-06-23T18:26:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.659+0000 7f0049c25700 1 -- 172.21.15.2:0/3176322953 learned_addr learned my addr 172.21.15.2:0/3176322953 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:51.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f0049c25700 1 -- 172.21.15.2:0/3176322953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a8000 msgr2=0x55990f7b7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f0049c25700 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a8000 0x55990f7b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f0049c25700 1 -- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 msgr2=0x55990f7b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f0049424700 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f0049c25700 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f0049c25700 1 -- 172.21.15.2:0/3176322953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0034017070 con 0x55990f7a6c80 2023-06-23T18:26:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f0049424700 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:26:51.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f0049c25700 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a6c80 0x55990f7b0510 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0040099e30 tx=0x7f004009a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2334ffafcc295ea4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f002d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:26:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f002d7fa700 1 -- 172.21.15.2:0/3176322953 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004009a390 con 0x55990f7a6c80 2023-06-23T18:26:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f002d7fa700 1 -- 172.21.15.2:0/3176322953 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f004009a4f0 con 0x55990f7a6c80 2023-06-23T18:26:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f005800eb80 1 -- 172.21.15.2:0/3176322953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55990f799bb0 con 0x55990f7a6c80 2023-06-23T18:26:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f002d7fa700 1 -- 172.21.15.2:0/3176322953 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0040099620 con 0x55990f7a6c80 2023-06-23T18:26:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f005800eb80 10 client.15744 fetch_fsmap learned FSMap version 30 2023-06-23T18:26:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.660+0000 7f005800eb80 1 -- 172.21.15.2:0/3176322953 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55990f525de0 con 0x55990f7a6c80 2023-06-23T18:26:51.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.661+0000 7f002d7fa700 1 -- 172.21.15.2:0/3176322953 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55990f799bb0 con 0x55990f7a6c80 2023-06-23T18:26:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.661+0000 7f002d7fa700 1 -- 172.21.15.2:0/3176322953 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f00400a20f0 con 0x55990f7a6c80 2023-06-23T18:26:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.661+0000 7f005800eb80 10 client.15744 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:26:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.661+0000 7f005800eb80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:26:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.661+0000 7f005800eb80 20 client.15744 populate_metadata read hostname 'smithi002' 2023-06-23T18:26:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.661+0000 7f005800eb80 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55990f7bc2e0 0x55990f7be6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.661+0000 7f005800eb80 4 client.15744 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:26:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.661+0000 7f005800eb80 1 -- 172.21.15.2:0/3176322953 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55990f6f0eb0 con 0x55990f7bc2e0 2023-06-23T18:26:51.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.662+0000 7f0049424700 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55990f7bc2e0 0x55990f7be6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.662+0000 7f002d7fa700 10 client.15744 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:26:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.662+0000 7f0049424700 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55990f7bc2e0 0x55990f7be6f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a65d7f63f43af server_cookie=bf16ce70e83e50dd in_seq=0 out_seq=0 2023-06-23T18:26:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.663+0000 7f002d7fa700 1 -- 172.21.15.2:0/3176322953 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55990f6f0eb0 con 0x55990f7bc2e0 2023-06-23T18:26:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.663+0000 7f002d7fa700 10 client.15744 handle_command_reply: tid=0 2023-06-23T18:26:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.663+0000 7f005800eb80 1 client.15744 shutdown 2023-06-23T18:26:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 -- 172.21.15.2:0/3176322953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a6c80 msgr2=0x55990f7b0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a6c80 0x55990f7b0510 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0040099e30 tx=0x7f004009a260 comp rx=0 tx=0).stop 2023-06-23T18:26:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 -- 172.21.15.2:0/3176322953 shutdown_connections 2023-06-23T18:26:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55990f7bc2e0 0x55990f7be6f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a8000 0x55990f7b7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f7a6c80 0x55990f7b0510 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 --2- 172.21.15.2:0/3176322953 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 -- 172.21.15.2:0/3176322953 >> 172.21.15.2:0/3176322953 conn(0x55990f79c3f0 msgr2=0x55990f7a0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f002d7fa700 0 client.15744 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:26:51.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 -- 172.21.15.2:0/3176322953 shutdown_connections 2023-06-23T18:26:51.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 1 -- 172.21.15.2:0/3176322953 wait complete. 2023-06-23T18:26:51.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.664+0000 7f005800eb80 20 client.15744 trim_cache size 0 max 16384 2023-06-23T18:26:51.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.685+0000 7f005800eb80 1 Processor -- start 2023-06-23T18:26:51.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.685+0000 7f005800eb80 1 -- start start 2023-06-23T18:26:51.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.685+0000 7f005800eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.685+0000 7f005800eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 0x55990f7b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f005800eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f005800eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f7b9780 con 0x55990f79c3f0 2023-06-23T18:26:51.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f005800eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f7a2810 con 0x55990f7a6c80 2023-06-23T18:26:51.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f005800eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f7b0080 con 0x55990f7a75d0 2023-06-23T18:26:51.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7c3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54912/0 (socket says 172.21.15.2:54912) 2023-06-23T18:26:51.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f004a426700 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 -- 172.21.15.2:0/3835048759 learned_addr learned my addr 172.21.15.2:0/3835048759 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:51.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049424700 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 0x55990f7b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 -- 172.21.15.2:0/3835048759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 msgr2=0x55990f7ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:51.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 -- 172.21.15.2:0/3835048759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 msgr2=0x55990f7b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:51.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 0x55990f7b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 -- 172.21.15.2:0/3835048759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55990f50ec60 con 0x55990f79c3f0 2023-06-23T18:26:51.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f004a426700 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:26:51.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0049c25700 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7c3070 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f004009a0c0 tx=0x7f00400965a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fc89749f5a28a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:51.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0038ff9700 1 -- 172.21.15.2:0/3835048759 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00400984d0 con 0x55990f79c3f0 2023-06-23T18:26:51.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.686+0000 7f0038ff9700 1 -- 172.21.15.2:0/3835048759 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00400a20f0 con 0x55990f79c3f0 2023-06-23T18:26:51.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f0038ff9700 1 -- 172.21.15.2:0/3835048759 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004009a390 con 0x55990f79c3f0 2023-06-23T18:26:51.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 -- 172.21.15.2:0/3835048759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 msgr2=0x55990f7c3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7c3070 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f004009a0c0 tx=0x7f00400965a0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 -- 172.21.15.2:0/3835048759 shutdown_connections 2023-06-23T18:26:51.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a75d0 0x55990f7ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7c3070 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 --2- 172.21.15.2:0/3835048759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a6c80 0x55990f7b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 -- 172.21.15.2:0/3835048759 >> 172.21.15.2:0/3835048759 conn(0x55990f7af5c0 msgr2=0x55990f79d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 -- 172.21.15.2:0/3835048759 shutdown_connections 2023-06-23T18:26:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.687+0000 7f005800eb80 1 -- 172.21.15.2:0/3835048759 wait complete. 2023-06-23T18:26:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 1 -- start start 2023-06-23T18:26:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 1 Processor -- start 2023-06-23T18:26:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a6c80 0x55990f7b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f7b7320 con 0x55990f79c3f0 2023-06-23T18:26:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f7b7490 con 0x55990f7a75d0 2023-06-23T18:26:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990f7b7650 con 0x55990f7a6c80 2023-06-23T18:26:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f005800eb80 10 client.0 _ll_register_callbacks cb 0x55990f69d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:26:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f0049c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f0049424700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a6c80 0x55990f7b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f0049c25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54918/0 (socket says 172.21.15.2:54918) 2023-06-23T18:26:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f0049c25700 1 -- 172.21.15.2:0/1860147742 learned_addr learned my addr 172.21.15.2:0/1860147742 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:26:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f0049424700 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a6c80 0x55990f7b3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55258/0 (socket says 172.21.15.2:55258) 2023-06-23T18:26:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.688+0000 7f004a426700 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f0049424700 1 -- 172.21.15.2:0/1860147742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 msgr2=0x55990f7b6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f0049424700 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f0049424700 1 -- 172.21.15.2:0/1860147742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 msgr2=0x55990f7b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:26:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f0049424700 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f0049424700 1 -- 172.21.15.2:0/1860147742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0044108890 con 0x55990f7a6c80 2023-06-23T18:26:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f0049424700 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a6c80 0x55990f7b3dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55990f7bf020 tx=0x7f0034011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed64633afdf65ea server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:26:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f002d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:26:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f002d7fa700 1 -- 172.21.15.2:0/1860147742 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034035a00 con 0x55990f7a6c80 2023-06-23T18:26:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f002d7fa700 1 -- 172.21.15.2:0/1860147742 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0034035b60 con 0x55990f7a6c80 2023-06-23T18:26:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f005800eb80 1 -- 172.21.15.2:0/1860147742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0034025070 con 0x55990f7a6c80 2023-06-23T18:26:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f002d7fa700 1 -- 172.21.15.2:0/1860147742 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0034035d20 con 0x55990f7a6c80 2023-06-23T18:26:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f005800eb80 10 client.25507 fetch_fsmap learned FSMap version 30 2023-06-23T18:26:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f005800eb80 1 -- 172.21.15.2:0/1860147742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55990f525de0 con 0x55990f7a6c80 2023-06-23T18:26:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f002d7fa700 1 -- 172.21.15.2:0/1860147742 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0034044020 con 0x55990f7a6c80 2023-06-23T18:26:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.689+0000 7f002d7fa700 1 -- 172.21.15.2:0/1860147742 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f00340051b0 con 0x55990f7a6c80 2023-06-23T18:26:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f005800eb80 10 client.25507 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:26:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f005800eb80 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:26:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f005800eb80 20 client.25507 populate_metadata read hostname 'smithi002' 2023-06-23T18:26:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f005800eb80 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55990f7a8000 0x55990f813200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:26:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f005800eb80 4 client.25507 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:26:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f005800eb80 1 -- 172.21.15.2:0/1860147742 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55990f6b3050 con 0x55990f7a8000 2023-06-23T18:26:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f0049c25700 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55990f7a8000 0x55990f813200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:26:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f0049c25700 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55990f7a8000 0x55990f813200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c94dc35afb756848 server_cookie=5311eca92f4a3fa8 in_seq=0 out_seq=0 2023-06-23T18:26:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f002d7fa700 10 client.25507 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:26:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f002d7fa700 1 -- 172.21.15.2:0/1860147742 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55990f6b3050 con 0x55990f7a8000 2023-06-23T18:26:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f002d7fa700 10 client.25507 handle_command_reply: tid=0 2023-06-23T18:26:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.690+0000 7f005800eb80 1 client.25507 shutdown 2023-06-23T18:26:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 -- 172.21.15.2:0/1860147742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a6c80 msgr2=0x55990f7b3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a6c80 0x55990f7b3dc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55990f7bf020 tx=0x7f0034011bb0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 -- 172.21.15.2:0/1860147742 shutdown_connections 2023-06-23T18:26:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55990f7a8000 0x55990f813200 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55990f7a75d0 0x55990f7b6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55990f79c3f0 0x55990f7b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 --2- 172.21.15.2:0/1860147742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55990f7a6c80 0x55990f7b3dc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f002d7fa700 0 client.25507 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:26:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 -- 172.21.15.2:0/1860147742 >> 172.21.15.2:0/1860147742 conn(0x55990f7af5c0 msgr2=0x55990f7c3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:51.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 -- 172.21.15.2:0/1860147742 shutdown_connections 2023-06-23T18:26:51.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 1 -- 172.21.15.2:0/1860147742 wait complete. 2023-06-23T18:26:51.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f005800eb80 20 client.25507 trim_cache size 0 max 16384 2023-06-23T18:26:51.705 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:26:51.705 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:26:51.706 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:26:51.706 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:26:51.706 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:26:51.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:26:51.706 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:26:51.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:26:51.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:26:51.706 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:26:51.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f004be89700 1 -- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00300a1150 msgr2=0x7f00300a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f004be89700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00300a1150 0x7f00300a3600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0034009c60 tx=0x7f0034009260 comp rx=0 tx=0).stop 2023-06-23T18:26:51.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.691+0000 7f004be89700 1 -- 172.21.15.2:0/1985169302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0044100660 msgr2=0x7f0044195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:26:51.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0044100660 0x7f0044195880 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f004000ec10 tx=0x7f004000ef20 comp rx=0 tx=0).stop 2023-06-23T18:26:51.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 -- 172.21.15.2:0/1985169302 shutdown_connections 2023-06-23T18:26:51.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f00441058a0 0x7f004419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0044100660 0x7f0044195880 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00300a1150 0x7f00300a3600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 --2- 172.21.15.2:0/1985169302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0044102f80 0x7f0044195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:26:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 -- 172.21.15.2:0/1985169302 >> 172.21.15.2:0/1985169302 conn(0x7f00440fa0e0 msgr2=0x7f0044101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:26:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 -- 172.21.15.2:0/1985169302 shutdown_connections 2023-06-23T18:26:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:26:51.692+0000 7f004be89700 1 -- 172.21.15.2:0/1985169302 wait complete. 2023-06-23T18:26:51.710 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:26:52.043 INFO:teuthology.orchestra.run.smithi002.stderr:26700000 of 50000000 tuples (53%) done (elapsed 255.28 s, remaining 222.78 s) 2023-06-23T18:26:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:52 smithi082 ceph-mon[97889]: pgmap v441: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 39 MiB/s wr, 45 op/s 2023-06-23T18:26:52.861 INFO:teuthology.orchestra.run.smithi002.stderr:26800000 of 50000000 tuples (53%) done (elapsed 256.10 s, remaining 221.70 s) 2023-06-23T18:26:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:52 smithi017 ceph-mon[97472]: pgmap v441: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 39 MiB/s wr, 45 op/s 2023-06-23T18:26:53.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:52 smithi002 ceph-mon[95947]: pgmap v441: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 39 MiB/s wr, 45 op/s 2023-06-23T18:26:53.686 INFO:teuthology.orchestra.run.smithi002.stderr:26900000 of 50000000 tuples (53%) done (elapsed 256.93 s, remaining 220.63 s) 2023-06-23T18:26:54.567 INFO:teuthology.orchestra.run.smithi002.stderr:27000000 of 50000000 tuples (54%) done (elapsed 257.81 s, remaining 219.61 s) 2023-06-23T18:26:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:54 smithi082 ceph-mon[97889]: pgmap v442: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 45 MiB/s wr, 48 op/s 2023-06-23T18:26:54.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:54 smithi017 ceph-mon[97472]: pgmap v442: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 45 MiB/s wr, 48 op/s 2023-06-23T18:26:55.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:54 smithi002 ceph-mon[95947]: pgmap v442: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 45 MiB/s wr, 48 op/s 2023-06-23T18:26:55.402 INFO:teuthology.orchestra.run.smithi002.stderr:27100000 of 50000000 tuples (54%) done (elapsed 258.64 s, remaining 218.56 s) 2023-06-23T18:26:56.285 INFO:teuthology.orchestra.run.smithi002.stderr:27200000 of 50000000 tuples (54%) done (elapsed 259.53 s, remaining 217.54 s) 2023-06-23T18:26:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:56 smithi082 ceph-mon[97889]: pgmap v443: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:26:56.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:56 smithi017 ceph-mon[97472]: pgmap v443: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:26:57.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:56 smithi002 ceph-mon[95947]: pgmap v443: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:26:57.121 INFO:teuthology.orchestra.run.smithi002.stderr:27300000 of 50000000 tuples (54%) done (elapsed 260.36 s, remaining 216.49 s) 2023-06-23T18:26:58.010 INFO:teuthology.orchestra.run.smithi002.stderr:27400000 of 50000000 tuples (54%) done (elapsed 261.25 s, remaining 215.48 s) 2023-06-23T18:26:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:26:58 smithi082 ceph-mon[97889]: pgmap v444: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:26:58.849 INFO:teuthology.orchestra.run.smithi002.stderr:27500000 of 50000000 tuples (55%) done (elapsed 262.09 s, remaining 214.44 s) 2023-06-23T18:26:58.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:26:58 smithi017 ceph-mon[97472]: pgmap v444: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:26:59.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:26:58 smithi002 ceph-mon[95947]: pgmap v444: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:26:59.711 INFO:teuthology.orchestra.run.smithi002.stderr:27600000 of 50000000 tuples (55%) done (elapsed 262.95 s, remaining 213.41 s) 2023-06-23T18:27:00.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:00 smithi017 ceph-mon[97472]: pgmap v445: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:27:00.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:00 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:27:00.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:00 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:27:00.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:00 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:27:00.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:00 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:27:00.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:00 smithi002 ceph-mon[95947]: pgmap v445: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:27:00.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:00 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:27:00.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:00 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:27:00.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:00 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:27:00.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:00 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:27:00.586 INFO:teuthology.orchestra.run.smithi002.stderr:27700000 of 50000000 tuples (55%) done (elapsed 263.83 s, remaining 212.40 s) 2023-06-23T18:27:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:00 smithi082 ceph-mon[97889]: pgmap v445: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:27:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:00 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:27:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:00 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:27:00.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:00 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:27:00.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:00 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:27:01.491 INFO:teuthology.orchestra.run.smithi002.stderr:27800000 of 50000000 tuples (55%) done (elapsed 264.73 s, remaining 211.40 s) 2023-06-23T18:27:02.309 INFO:teuthology.orchestra.run.smithi002.stderr:27900000 of 50000000 tuples (55%) done (elapsed 265.55 s, remaining 210.35 s) 2023-06-23T18:27:02.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:02 smithi017 ceph-mon[97472]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 38 MiB/s wr, 21 op/s 2023-06-23T18:27:02.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:02 smithi002 ceph-mon[95947]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 38 MiB/s wr, 21 op/s 2023-06-23T18:27:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:02 smithi082 ceph-mon[97889]: pgmap v446: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 38 MiB/s wr, 21 op/s 2023-06-23T18:27:03.178 INFO:teuthology.orchestra.run.smithi002.stderr:28000000 of 50000000 tuples (56%) done (elapsed 266.42 s, remaining 209.33 s) 2023-06-23T18:27:04.025 INFO:teuthology.orchestra.run.smithi002.stderr:28100000 of 50000000 tuples (56%) done (elapsed 267.27 s, remaining 208.30 s) 2023-06-23T18:27:04.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:04 smithi017 ceph-mon[97472]: pgmap v447: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:27:04.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:04 smithi002 ceph-mon[95947]: pgmap v447: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:27:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:04 smithi082 ceph-mon[97889]: pgmap v447: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:27:04.902 INFO:teuthology.orchestra.run.smithi002.stderr:28200000 of 50000000 tuples (56%) done (elapsed 268.14 s, remaining 207.29 s) 2023-06-23T18:27:05.719 INFO:teuthology.orchestra.run.smithi002.stderr:28300000 of 50000000 tuples (56%) done (elapsed 268.96 s, remaining 206.23 s) 2023-06-23T18:27:06.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:06 smithi017 ceph-mon[97472]: pgmap v448: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:27:06.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:06 smithi002 ceph-mon[95947]: pgmap v448: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:27:06.554 INFO:teuthology.orchestra.run.smithi002.stderr:28400000 of 50000000 tuples (56%) done (elapsed 269.79 s, remaining 205.20 s) 2023-06-23T18:27:06.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:06 smithi082 ceph-mon[97889]: pgmap v448: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:27:07.445 INFO:teuthology.orchestra.run.smithi002.stderr:28500000 of 50000000 tuples (57%) done (elapsed 270.69 s, remaining 204.20 s) 2023-06-23T18:27:08.270 INFO:teuthology.orchestra.run.smithi002.stderr:28600000 of 50000000 tuples (57%) done (elapsed 271.51 s, remaining 203.16 s) 2023-06-23T18:27:08.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:08 smithi017 ceph-mon[97472]: pgmap v449: 97 pgs: 97 active+clean; 4.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 39 MiB/s wr, 21 op/s 2023-06-23T18:27:08.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:08 smithi002 ceph-mon[95947]: pgmap v449: 97 pgs: 97 active+clean; 4.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 39 MiB/s wr, 21 op/s 2023-06-23T18:27:08.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:08 smithi082 ceph-mon[97889]: pgmap v449: 97 pgs: 97 active+clean; 4.5 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 39 MiB/s wr, 21 op/s 2023-06-23T18:27:09.124 INFO:teuthology.orchestra.run.smithi002.stderr:28700000 of 50000000 tuples (57%) done (elapsed 272.37 s, remaining 202.14 s) 2023-06-23T18:27:09.986 INFO:teuthology.orchestra.run.smithi002.stderr:28800000 of 50000000 tuples (57%) done (elapsed 273.23 s, remaining 201.13 s) 2023-06-23T18:27:10.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:10 smithi017 ceph-mon[97472]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:27:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:10 smithi002 ceph-mon[95947]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:27:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:10 smithi082 ceph-mon[97889]: pgmap v450: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:27:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:28900000 of 50000000 tuples (57%) done (elapsed 274.05 s, remaining 200.09 s) 2023-06-23T18:27:11.623 INFO:teuthology.orchestra.run.smithi002.stderr:29000000 of 50000000 tuples (58%) done (elapsed 274.86 s, remaining 199.04 s) 2023-06-23T18:27:12.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:12 smithi017 ceph-mon[97472]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 38 MiB/s wr, 20 op/s 2023-06-23T18:27:12.522 INFO:teuthology.orchestra.run.smithi002.stderr:29100000 of 50000000 tuples (58%) done (elapsed 275.76 s, remaining 198.06 s) 2023-06-23T18:27:12.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:12 smithi002 ceph-mon[95947]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 38 MiB/s wr, 20 op/s 2023-06-23T18:27:12.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:12 smithi082 ceph-mon[97889]: pgmap v451: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 38 MiB/s wr, 20 op/s 2023-06-23T18:27:13.355 INFO:teuthology.orchestra.run.smithi002.stderr:29200000 of 50000000 tuples (58%) done (elapsed 276.60 s, remaining 197.03 s) 2023-06-23T18:27:14.203 INFO:teuthology.orchestra.run.smithi002.stderr:29300000 of 50000000 tuples (58%) done (elapsed 277.44 s, remaining 196.01 s) 2023-06-23T18:27:14.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:14 smithi017 ceph-mon[97472]: pgmap v452: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:27:14.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:14 smithi002 ceph-mon[95947]: pgmap v452: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:27:14.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:14 smithi082 ceph-mon[97889]: pgmap v452: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:27:15.148 INFO:teuthology.orchestra.run.smithi002.stderr:29400000 of 50000000 tuples (58%) done (elapsed 278.39 s, remaining 195.06 s) 2023-06-23T18:27:16.066 INFO:teuthology.orchestra.run.smithi002.stderr:29500000 of 50000000 tuples (59%) done (elapsed 279.31 s, remaining 194.10 s) 2023-06-23T18:27:16.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:16 smithi017 ceph-mon[97472]: pgmap v453: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 41 MiB/s wr, 26 op/s 2023-06-23T18:27:16.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:16 smithi002 ceph-mon[95947]: pgmap v453: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 41 MiB/s wr, 26 op/s 2023-06-23T18:27:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:16 smithi082 ceph-mon[97889]: pgmap v453: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 41 MiB/s wr, 26 op/s 2023-06-23T18:27:16.929 INFO:teuthology.orchestra.run.smithi002.stderr:29600000 of 50000000 tuples (59%) done (elapsed 280.17 s, remaining 193.09 s) 2023-06-23T18:27:17.774 INFO:teuthology.orchestra.run.smithi002.stderr:29700000 of 50000000 tuples (59%) done (elapsed 281.02 s, remaining 192.07 s) 2023-06-23T18:27:18.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:18 smithi017 ceph-mon[97472]: pgmap v454: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 37 MiB/s wr, 24 op/s 2023-06-23T18:27:18.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:18 smithi002 ceph-mon[95947]: pgmap v454: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 37 MiB/s wr, 24 op/s 2023-06-23T18:27:18.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:18 smithi082 ceph-mon[97889]: pgmap v454: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 37 MiB/s wr, 24 op/s 2023-06-23T18:27:18.653 INFO:teuthology.orchestra.run.smithi002.stderr:29800000 of 50000000 tuples (59%) done (elapsed 281.89 s, remaining 191.08 s) 2023-06-23T18:27:19.494 INFO:teuthology.orchestra.run.smithi002.stderr:29900000 of 50000000 tuples (59%) done (elapsed 282.74 s, remaining 190.07 s) 2023-06-23T18:27:20.307 INFO:teuthology.orchestra.run.smithi002.stderr:30000000 of 50000000 tuples (60%) done (elapsed 283.55 s, remaining 189.03 s) 2023-06-23T18:27:20.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:20 smithi017 ceph-mon[97472]: pgmap v455: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 34 op/s 2023-06-23T18:27:20.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:20 smithi002 ceph-mon[95947]: pgmap v455: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 34 op/s 2023-06-23T18:27:20.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:20 smithi082 ceph-mon[97889]: pgmap v455: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 41 MiB/s wr, 34 op/s 2023-06-23T18:27:21.156 INFO:teuthology.orchestra.run.smithi002.stderr:30100000 of 50000000 tuples (60%) done (elapsed 284.40 s, remaining 188.02 s) 2023-06-23T18:27:21.710 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:27:21.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 -- 172.21.15.2:0/108963560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34240fde70 msgr2=0x7f34240fe280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 --2- 172.21.15.2:0/108963560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34240fde70 0x7f34240fe280 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f34140098f0 tx=0x7f3414009c00 comp rx=0 tx=0).stop 2023-06-23T18:27:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 -- 172.21.15.2:0/108963560 shutdown_connections 2023-06-23T18:27:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 --2- 172.21.15.2:0/108963560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3424101870 0x7f3424108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 --2- 172.21.15.2:0/108963560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34240fde70 0x7f34240fe280 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 --2- 172.21.15.2:0/108963560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3424100ee0 0x7f3424101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 -- 172.21.15.2:0/108963560 >> 172.21.15.2:0/108963560 conn(0x7f34240f9850 msgr2=0x7f34240fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:21.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 -- 172.21.15.2:0/108963560 shutdown_connections 2023-06-23T18:27:21.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.847+0000 7f342956c700 1 -- 172.21.15.2:0/108963560 wait complete. 2023-06-23T18:27:21.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f342956c700 1 Processor -- start 2023-06-23T18:27:21.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f342956c700 1 -- start start 2023-06-23T18:27:21.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f342956c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240fde70 0x7f3424195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f342956c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3424100ee0 0x7f3424195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f342956c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3424101870 0x7f342419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f342956c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342419d390 con 0x7f3424100ee0 2023-06-23T18:27:21.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f342956c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342419d4d0 con 0x7f34240fde70 2023-06-23T18:27:21.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f342956c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342419d660 con 0x7f3424101870 2023-06-23T18:27:21.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f34227fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3424100ee0 0x7f3424195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f34237fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3424101870 0x7f342419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240fde70 0x7f3424195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f34227fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3424100ee0 0x7f3424195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54934/0 (socket says 172.21.15.2:54934) 2023-06-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.848+0000 7f34227fc700 1 -- 172.21.15.2:0/2965790505 learned_addr learned my addr 172.21.15.2:0/2965790505 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f3422ffd700 1 -- 172.21.15.2:0/2965790505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3424101870 msgr2=0x7f342419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f3422ffd700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3424101870 0x7f342419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f3422ffd700 1 -- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3424100ee0 msgr2=0x7f3424195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f3422ffd700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3424100ee0 0x7f3424195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f3422ffd700 1 -- 172.21.15.2:0/2965790505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3414009520 con 0x7f34240fde70 2023-06-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f34227fc700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3424100ee0 0x7f3424195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f34237fe700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3424101870 0x7f342419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f3422ffd700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240fde70 0x7f3424195810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3414009230 tx=0x7f341400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f341bfff700 1 -- 172.21.15.2:0/2965790505 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341401c070 con 0x7f34240fde70 2023-06-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f342956c700 1 -- 172.21.15.2:0/2965790505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34240fecd0 con 0x7f34240fde70 2023-06-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f342956c700 1 -- 172.21.15.2:0/2965790505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34240ff1a0 con 0x7f34240fde70 2023-06-23T18:27:21.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f341bfff700 1 -- 172.21.15.2:0/2965790505 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f341400bb90 con 0x7f34240fde70 2023-06-23T18:27:21.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.849+0000 7f341bfff700 1 -- 172.21.15.2:0/2965790505 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414003b50 con 0x7f34240fde70 2023-06-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.852+0000 7f341bfff700 1 -- 172.21.15.2:0/2965790505 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3414003cb0 con 0x7f34240fde70 2023-06-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.852+0000 7f341bfff700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34100a1150 0x7f34100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.852+0000 7f34227fc700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34100a1150 0x7f34100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.852+0000 7f341bfff700 1 -- 172.21.15.2:0/2965790505 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f34140cd5f0 con 0x7f34240fde70 2023-06-23T18:27:21.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34227fc700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34100a1150 0x7f34100a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f340c011550 tx=0x7f340c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:21.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34356f1b80 1 Processor -- start 2023-06-23T18:27:21.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34356f1b80 1 -- start start 2023-06-23T18:27:21.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f89130c80 0x555f89131090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f891315d0 0x555f89131a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89132470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34356f1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f89035cb0 con 0x555f89132000 2023-06-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34356f1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f89088a40 con 0x555f89130c80 2023-06-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.853+0000 7f34356f1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f89081ed0 con 0x555f891315d0 2023-06-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f891315d0 0x555f89131a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34237fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89132470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f89130c80 0x555f89131090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f891315d0 0x555f89131a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55280/0 (socket says 172.21.15.2:55280) 2023-06-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 -- 172.21.15.2:0/2194397944 learned_addr learned my addr 172.21.15.2:0/2194397944 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 -- 172.21.15.2:0/2194397944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f89130c80 msgr2=0x555f89131090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 --2- 172.21.15.2:0/2194397944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f89130c80 0x555f89131090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 -- 172.21.15.2:0/2194397944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 msgr2=0x555f89132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 --2- 172.21.15.2:0/2194397944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89132470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 -- 172.21.15.2:0/2194397944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f88f03630 con 0x555f891315d0 2023-06-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34237fe700 1 --2- 172.21.15.2:0/2194397944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89132470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:27:21.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34227fc700 1 --2- 172.21.15.2:0/2194397944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f891315d0 0x555f89131a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f340c009880 tx=0x7f340c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cec0120a0cd9c74 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34197fa700 1 -- 172.21.15.2:0/2194397944 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c026630 con 0x555f891315d0 2023-06-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34197fa700 1 -- 172.21.15.2:0/2194397944 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f340c021070 con 0x555f891315d0 2023-06-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.854+0000 7f34197fa700 1 -- 172.21.15.2:0/2194397944 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c01d670 con 0x555f891315d0 2023-06-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 -- 172.21.15.2:0/2194397944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f891315d0 msgr2=0x555f89131a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 --2- 172.21.15.2:0/2194397944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f891315d0 0x555f89131a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f340c009880 tx=0x7f340c014000 comp rx=0 tx=0).stop 2023-06-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 -- 172.21.15.2:0/2194397944 shutdown_connections 2023-06-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 --2- 172.21.15.2:0/2194397944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89132470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 --2- 172.21.15.2:0/2194397944 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f89130c80 0x555f89131090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 --2- 172.21.15.2:0/2194397944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f891315d0 0x555f89131a70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 -- 172.21.15.2:0/2194397944 >> 172.21.15.2:0/2194397944 conn(0x555f891263f0 msgr2=0x555f8912b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 -- 172.21.15.2:0/2194397944 shutdown_connections 2023-06-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.855+0000 7f34356f1b80 1 -- 172.21.15.2:0/2194397944 wait complete. 2023-06-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 1 -- start start 2023-06-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 1 Processor -- start 2023-06-23T18:27:21.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 0x555f89141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f89021ac0 con 0x555f89130c80 2023-06-23T18:27:21.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f890217c0 con 0x555f891315d0 2023-06-23T18:27:21.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f88f05920 con 0x555f89132000 2023-06-23T18:27:21.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34356f1b80 10 client.0 _ll_register_callbacks cb 0x555f89027260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:27:21.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34227fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54946/0 (socket says 172.21.15.2:54946) 2023-06-23T18:27:21.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f3422ffd700 1 -- 172.21.15.2:0/501271034 learned_addr learned my addr 172.21.15.2:0/501271034 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:21.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.856+0000 7f34237fe700 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 0x555f89141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34227fc700 1 -- 172.21.15.2:0/501271034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 msgr2=0x555f89141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:21.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34227fc700 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 0x555f89141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34227fc700 1 -- 172.21.15.2:0/501271034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 msgr2=0x555f8913a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:21.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34227fc700 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34227fc700 1 -- 172.21.15.2:0/501271034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c010880 con 0x555f891315d0 2023-06-23T18:27:21.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f3422ffd700 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:27:21.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34227fc700 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913aa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f340c009cf0 tx=0x7f340c01cfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80834b76d593c31e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:21.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f340a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:27:21.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f340a7fc700 1 -- 172.21.15.2:0/501271034 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c01c720 con 0x555f891315d0 2023-06-23T18:27:21.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34356f1b80 1 -- 172.21.15.2:0/501271034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f89123bb0 con 0x555f891315d0 2023-06-23T18:27:21.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f340a7fc700 1 -- 172.21.15.2:0/501271034 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f340c01c880 con 0x555f891315d0 2023-06-23T18:27:21.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f340a7fc700 1 -- 172.21.15.2:0/501271034 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f340c005040 con 0x555f891315d0 2023-06-23T18:27:21.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34356f1b80 10 client.25364 fetch_fsmap learned FSMap version 30 2023-06-23T18:27:21.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.857+0000 7f34356f1b80 1 -- 172.21.15.2:0/501271034 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f340c017070 con 0x555f891315d0 2023-06-23T18:27:21.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f340a7fc700 1 -- 172.21.15.2:0/501271034 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555f89123bb0 con 0x555f891315d0 2023-06-23T18:27:21.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f340a7fc700 1 -- 172.21.15.2:0/501271034 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f340c004c60 con 0x555f891315d0 2023-06-23T18:27:21.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f34356f1b80 10 client.25364 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:27:21.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f34356f1b80 10 client.25364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:27:21.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f34356f1b80 20 client.25364 populate_metadata read hostname 'smithi002' 2023-06-23T18:27:21.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f34356f1b80 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555f891461d0 0x555f891485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f34356f1b80 4 client.25364 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:27:21.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f34356f1b80 1 -- 172.21.15.2:0/501271034 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f8907aeb0 con 0x555f891461d0 2023-06-23T18:27:21.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f3422ffd700 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555f891461d0 0x555f891485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f340a7fc700 10 client.25364 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:27:21.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.858+0000 7f3422ffd700 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555f891461d0 0x555f891485e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c0b2256edf77fd server_cookie=4236263ed93a5414 in_seq=0 out_seq=0 2023-06-23T18:27:21.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f340a7fc700 1 -- 172.21.15.2:0/501271034 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555f8907aeb0 con 0x555f891461d0 2023-06-23T18:27:21.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f340a7fc700 10 client.25364 handle_command_reply: tid=0 2023-06-23T18:27:21.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 client.25364 shutdown 2023-06-23T18:27:21.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 -- 172.21.15.2:0/501271034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 msgr2=0x555f8913aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913aa70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f340c009cf0 tx=0x7f340c01cfd0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 -- 172.21.15.2:0/501271034 shutdown_connections 2023-06-23T18:27:21.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555f891461d0 0x555f891485e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 0x555f89141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f340a7fc700 0 client.25364 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:27:21.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 --2- 172.21.15.2:0/501271034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913aa70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 -- 172.21.15.2:0/501271034 >> 172.21.15.2:0/501271034 conn(0x555f891263f0 msgr2=0x555f8912ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:21.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 -- 172.21.15.2:0/501271034 shutdown_connections 2023-06-23T18:27:21.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 1 -- 172.21.15.2:0/501271034 wait complete. 2023-06-23T18:27:21.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.859+0000 7f34356f1b80 20 client.25364 trim_cache size 0 max 16384 2023-06-23T18:27:21.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.876+0000 7f34356f1b80 1 Processor -- start 2023-06-23T18:27:21.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.876+0000 7f34356f1b80 1 -- start start 2023-06-23T18:27:21.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89130c80 0x555f891423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f891419d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89134e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f34356f1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f891390b0 con 0x555f89132000 2023-06-23T18:27:21.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f34356f1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8912cd50 con 0x555f891315d0 2023-06-23T18:27:21.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f34356f1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8913b3d0 con 0x555f89130c80 2023-06-23T18:27:21.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89130c80 0x555f891423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89130c80 0x555f891423a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55292/0 (socket says 172.21.15.2:55292) 2023-06-23T18:27:21.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 -- 172.21.15.2:0/4108967460 learned_addr learned my addr 172.21.15.2:0/4108967460 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:21.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f34237fe700 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89134e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f34227fc700 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f891419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 -- 172.21.15.2:0/4108967460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 msgr2=0x555f891419d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:21.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f891419d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 -- 172.21.15.2:0/4108967460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 msgr2=0x555f89134e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:21.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89134e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 -- 172.21.15.2:0/4108967460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f88eafde0 con 0x555f89130c80 2023-06-23T18:27:21.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.877+0000 7f3422ffd700 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89130c80 0x555f891423a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f341400bd20 tx=0x7f3414009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3f77c7aea5c088 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:21.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.878+0000 7f3419ffb700 1 -- 172.21.15.2:0/4108967460 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140b31d0 con 0x555f89130c80 2023-06-23T18:27:21.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.878+0000 7f3419ffb700 1 -- 172.21.15.2:0/4108967460 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34140a2e50 con 0x555f89130c80 2023-06-23T18:27:21.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.878+0000 7f3419ffb700 1 -- 172.21.15.2:0/4108967460 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140b8b90 con 0x555f89130c80 2023-06-23T18:27:21.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.878+0000 7f34356f1b80 1 -- 172.21.15.2:0/4108967460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89130c80 msgr2=0x555f891423a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.878+0000 7f34356f1b80 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89130c80 0x555f891423a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f341400bd20 tx=0x7f3414009260 comp rx=0 tx=0).stop 2023-06-23T18:27:21.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 -- 172.21.15.2:0/4108967460 shutdown_connections 2023-06-23T18:27:21.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89132000 0x555f89134e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89130c80 0x555f891423a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 --2- 172.21.15.2:0/4108967460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f891419d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 -- 172.21.15.2:0/4108967460 >> 172.21.15.2:0/4108967460 conn(0x555f891263f0 msgr2=0x555f8914cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:21.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 -- 172.21.15.2:0/4108967460 shutdown_connections 2023-06-23T18:27:21.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 -- 172.21.15.2:0/4108967460 wait complete. 2023-06-23T18:27:21.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 -- start start 2023-06-23T18:27:21.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 Processor -- start 2023-06-23T18:27:21.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 0x555f89140c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f891411b0 con 0x555f89130c80 2023-06-23T18:27:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f89141320 con 0x555f891315d0 2023-06-23T18:27:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f891414e0 con 0x555f89132000 2023-06-23T18:27:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f34356f1b80 10 client.0 _ll_register_callbacks cb 0x555f89027260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:27:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.879+0000 7f3422ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54960/0 (socket says 172.21.15.2:54960) 2023-06-23T18:27:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f34227fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f3422ffd700 1 -- 172.21.15.2:0/457349711 learned_addr learned my addr 172.21.15.2:0/457349711 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f34237fe700 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 0x555f89140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f3422ffd700 1 -- 172.21.15.2:0/457349711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 msgr2=0x555f89140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f3422ffd700 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 0x555f89140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f3422ffd700 1 -- 172.21.15.2:0/457349711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 msgr2=0x555f8913dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f3422ffd700 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f3422ffd700 1 -- 172.21.15.2:0/457349711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34240ff1a0 con 0x555f89130c80 2023-06-23T18:27:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f3422ffd700 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913d710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f341409a790 tx=0x7f3414009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324a86852c61a854 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:21.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f340a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:27:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f340a7fc700 1 -- 172.21.15.2:0/457349711 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140b8b90 con 0x555f89130c80 2023-06-23T18:27:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f340a7fc700 1 -- 172.21.15.2:0/457349711 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f34140b8cf0 con 0x555f89130c80 2023-06-23T18:27:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f34356f1b80 1 -- 172.21.15.2:0/457349711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f340c021070 con 0x555f89130c80 2023-06-23T18:27:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.880+0000 7f340a7fc700 1 -- 172.21.15.2:0/457349711 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f341409acf0 con 0x555f89130c80 2023-06-23T18:27:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34356f1b80 10 client.15768 fetch_fsmap learned FSMap version 30 2023-06-23T18:27:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34356f1b80 1 -- 172.21.15.2:0/457349711 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555f88e98c60 con 0x555f89130c80 2023-06-23T18:27:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f340a7fc700 1 -- 172.21.15.2:0/457349711 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f340c021070 con 0x555f89130c80 2023-06-23T18:27:21.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f340a7fc700 1 -- 172.21.15.2:0/457349711 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3414015020 con 0x555f89130c80 2023-06-23T18:27:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34356f1b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:27:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34356f1b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:27:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34356f1b80 20 client.15768 populate_metadata read hostname 'smithi002' 2023-06-23T18:27:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34356f1b80 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555f8919a9f0 0x555f8919ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34356f1b80 4 client.15768 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:27:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34356f1b80 1 -- 172.21.15.2:0/457349711 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555f8903d050 con 0x555f8919a9f0 2023-06-23T18:27:21.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34227fc700 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555f8919a9f0 0x555f8919ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f340a7fc700 10 client.15768 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:27:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.881+0000 7f34227fc700 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555f8919a9f0 0x555f8919ce00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca3a7ed063d3089 server_cookie=bbbce22b6c5870d6 in_seq=0 out_seq=0 2023-06-23T18:27:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f340a7fc700 1 -- 172.21.15.2:0/457349711 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555f8903d050 con 0x555f8919a9f0 2023-06-23T18:27:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f340a7fc700 10 client.15768 handle_command_reply: tid=0 2023-06-23T18:27:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 client.15768 shutdown 2023-06-23T18:27:21.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 -- 172.21.15.2:0/457349711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 msgr2=0x555f8913d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913d710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f341409a790 tx=0x7f3414009c00 comp rx=0 tx=0).stop 2023-06-23T18:27:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 -- 172.21.15.2:0/457349711 shutdown_connections 2023-06-23T18:27:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555f8919a9f0 0x555f8919ce00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555f89132000 0x555f89140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555f89130c80 0x555f8913d710 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f340a7fc700 0 client.15768 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:27:21.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 --2- 172.21.15.2:0/457349711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555f891315d0 0x555f8913dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 -- 172.21.15.2:0/457349711 >> 172.21.15.2:0/457349711 conn(0x555f891263f0 msgr2=0x555f89134e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:21.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 -- 172.21.15.2:0/457349711 shutdown_connections 2023-06-23T18:27:21.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 1 -- 172.21.15.2:0/457349711 wait complete. 2023-06-23T18:27:21.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.882+0000 7f34356f1b80 20 client.15768 trim_cache size 0 max 16384 2023-06-23T18:27:21.897 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:27:21.898 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:27:21.898 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:27:21.898 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:27:21.898 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:27:21.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:27:21.898 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:27:21.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:27:21.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:27:21.899 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:27:21.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 -- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34100a1150 msgr2=0x7f34100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34100a1150 0x7f34100a3600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f340c011550 tx=0x7f340c009260 comp rx=0 tx=0).stop 2023-06-23T18:27:21.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 -- 172.21.15.2:0/2965790505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240fde70 msgr2=0x7f3424195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:21.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240fde70 0x7f3424195810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3414009230 tx=0x7f341400b790 comp rx=0 tx=0).stop 2023-06-23T18:27:21.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 -- 172.21.15.2:0/2965790505 shutdown_connections 2023-06-23T18:27:21.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3424101870 0x7f342419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34100a1150 0x7f34100a3600 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34240fde70 0x7f3424195810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 --2- 172.21.15.2:0/2965790505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3424100ee0 0x7f3424195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:21.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 -- 172.21.15.2:0/2965790505 >> 172.21.15.2:0/2965790505 conn(0x7f34240f9850 msgr2=0x7f34240fbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:21.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 -- 172.21.15.2:0/2965790505 shutdown_connections 2023-06-23T18:27:21.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:21.883+0000 7f342956c700 1 -- 172.21.15.2:0/2965790505 wait complete. 2023-06-23T18:27:21.902 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:27:22.077 INFO:teuthology.orchestra.run.smithi002.stderr:30200000 of 50000000 tuples (60%) done (elapsed 285.32 s, remaining 187.06 s) 2023-06-23T18:27:22.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:22 smithi017 ceph-mon[97472]: pgmap v456: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 37 MiB/s wr, 42 op/s 2023-06-23T18:27:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:22 smithi002 ceph-mon[95947]: pgmap v456: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 37 MiB/s wr, 42 op/s 2023-06-23T18:27:22.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:22 smithi082 ceph-mon[97889]: pgmap v456: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 37 MiB/s wr, 42 op/s 2023-06-23T18:27:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:30300000 of 50000000 tuples (60%) done (elapsed 286.14 s, remaining 186.04 s) 2023-06-23T18:27:23.725 INFO:teuthology.orchestra.run.smithi002.stderr:30400000 of 50000000 tuples (60%) done (elapsed 286.97 s, remaining 185.02 s) 2023-06-23T18:27:24.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:24 smithi017 ceph-mon[97472]: pgmap v457: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 45 MiB/s wr, 45 op/s 2023-06-23T18:27:24.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:24 smithi002 ceph-mon[95947]: pgmap v457: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 45 MiB/s wr, 45 op/s 2023-06-23T18:27:24.559 INFO:teuthology.orchestra.run.smithi002.stderr:30500000 of 50000000 tuples (61%) done (elapsed 287.80 s, remaining 184.00 s) 2023-06-23T18:27:24.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:24 smithi082 ceph-mon[97889]: pgmap v457: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 45 MiB/s wr, 45 op/s 2023-06-23T18:27:25.452 INFO:teuthology.orchestra.run.smithi002.stderr:30600000 of 50000000 tuples (61%) done (elapsed 288.69 s, remaining 183.03 s) 2023-06-23T18:27:26.261 INFO:teuthology.orchestra.run.smithi002.stderr:30700000 of 50000000 tuples (61%) done (elapsed 289.50 s, remaining 182.00 s) 2023-06-23T18:27:26.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:26 smithi017 ceph-mon[97472]: pgmap v458: 97 pgs: 97 active+clean; 4.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:27:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:26 smithi002 ceph-mon[95947]: pgmap v458: 97 pgs: 97 active+clean; 4.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:27:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:26 smithi082 ceph-mon[97889]: pgmap v458: 97 pgs: 97 active+clean; 4.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:27:27.120 INFO:teuthology.orchestra.run.smithi002.stderr:30800000 of 50000000 tuples (61%) done (elapsed 290.36 s, remaining 181.00 s) 2023-06-23T18:27:28.001 INFO:teuthology.orchestra.run.smithi002.stderr:30900000 of 50000000 tuples (61%) done (elapsed 291.24 s, remaining 180.02 s) 2023-06-23T18:27:28.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:28 smithi017 ceph-mon[97472]: pgmap v459: 97 pgs: 97 active+clean; 4.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 40 MiB/s wr, 39 op/s 2023-06-23T18:27:28.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:28 smithi002 ceph-mon[95947]: pgmap v459: 97 pgs: 97 active+clean; 4.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 40 MiB/s wr, 39 op/s 2023-06-23T18:27:28.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:28 smithi082 ceph-mon[97889]: pgmap v459: 97 pgs: 97 active+clean; 4.8 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 40 MiB/s wr, 39 op/s 2023-06-23T18:27:28.871 INFO:teuthology.orchestra.run.smithi002.stderr:31000000 of 50000000 tuples (62%) done (elapsed 292.11 s, remaining 179.04 s) 2023-06-23T18:27:29.698 INFO:teuthology.orchestra.run.smithi002.stderr:31100000 of 50000000 tuples (62%) done (elapsed 292.94 s, remaining 178.02 s) 2023-06-23T18:27:30.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:30 smithi017 ceph-mon[97472]: pgmap v460: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 MiB/s wr, 43 op/s 2023-06-23T18:27:30.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:30 smithi002 ceph-mon[95947]: pgmap v460: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 MiB/s wr, 43 op/s 2023-06-23T18:27:30.574 INFO:teuthology.orchestra.run.smithi002.stderr:31200000 of 50000000 tuples (62%) done (elapsed 293.81 s, remaining 177.04 s) 2023-06-23T18:27:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:30 smithi082 ceph-mon[97889]: pgmap v460: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 48 MiB/s wr, 43 op/s 2023-06-23T18:27:31.430 INFO:teuthology.orchestra.run.smithi002.stderr:31300000 of 50000000 tuples (62%) done (elapsed 294.67 s, remaining 176.05 s) 2023-06-23T18:27:32.270 INFO:teuthology.orchestra.run.smithi002.stderr:31400000 of 50000000 tuples (62%) done (elapsed 295.51 s, remaining 175.05 s) 2023-06-23T18:27:32.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:32 smithi017 ceph-mon[97472]: pgmap v461: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 MiB/s wr, 32 op/s 2023-06-23T18:27:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:32 smithi002 ceph-mon[95947]: pgmap v461: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 MiB/s wr, 32 op/s 2023-06-23T18:27:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:32 smithi082 ceph-mon[97889]: pgmap v461: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 MiB/s wr, 32 op/s 2023-06-23T18:27:33.087 INFO:teuthology.orchestra.run.smithi002.stderr:31500000 of 50000000 tuples (63%) done (elapsed 296.33 s, remaining 174.03 s) 2023-06-23T18:27:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:31600000 of 50000000 tuples (63%) done (elapsed 297.22 s, remaining 173.06 s) 2023-06-23T18:27:34.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:34 smithi017 ceph-mon[97472]: pgmap v462: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:27:34.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:34 smithi002 ceph-mon[95947]: pgmap v462: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:27:34.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:34 smithi082 ceph-mon[97889]: pgmap v462: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:27:34.812 INFO:teuthology.orchestra.run.smithi002.stderr:31700000 of 50000000 tuples (63%) done (elapsed 298.05 s, remaining 172.06 s) 2023-06-23T18:27:35.654 INFO:teuthology.orchestra.run.smithi002.stderr:31800000 of 50000000 tuples (63%) done (elapsed 298.90 s, remaining 171.07 s) 2023-06-23T18:27:36.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:36 smithi017 ceph-mon[97472]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:27:36.534 INFO:teuthology.orchestra.run.smithi002.stderr:31900000 of 50000000 tuples (63%) done (elapsed 299.78 s, remaining 170.09 s) 2023-06-23T18:27:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:36 smithi002 ceph-mon[95947]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:27:36.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:36 smithi082 ceph-mon[97889]: pgmap v463: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 23 op/s 2023-06-23T18:27:37.382 INFO:teuthology.orchestra.run.smithi002.stderr:32000000 of 50000000 tuples (64%) done (elapsed 300.62 s, remaining 169.10 s) 2023-06-23T18:27:38.221 INFO:teuthology.orchestra.run.smithi002.stderr:32100000 of 50000000 tuples (64%) done (elapsed 301.46 s, remaining 168.11 s) 2023-06-23T18:27:38.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:38 smithi017 ceph-mon[97472]: pgmap v464: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 39 MiB/s wr, 20 op/s 2023-06-23T18:27:38.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:38 smithi002 ceph-mon[95947]: pgmap v464: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 39 MiB/s wr, 20 op/s 2023-06-23T18:27:38.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:38 smithi082 ceph-mon[97889]: pgmap v464: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 39 MiB/s wr, 20 op/s 2023-06-23T18:27:39.060 INFO:teuthology.orchestra.run.smithi002.stderr:32200000 of 50000000 tuples (64%) done (elapsed 302.30 s, remaining 167.11 s) 2023-06-23T18:27:39.967 INFO:teuthology.orchestra.run.smithi002.stderr:32300000 of 50000000 tuples (64%) done (elapsed 303.21 s, remaining 166.15 s) 2023-06-23T18:27:40.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:40 smithi017 ceph-mon[97472]: pgmap v465: 97 pgs: 97 active+clean; 5.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:27:40.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:40 smithi002 ceph-mon[95947]: pgmap v465: 97 pgs: 97 active+clean; 5.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:27:40.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:40 smithi082 ceph-mon[97889]: pgmap v465: 97 pgs: 97 active+clean; 5.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 MiB/s wr, 25 op/s 2023-06-23T18:27:40.777 INFO:teuthology.orchestra.run.smithi002.stderr:32400000 of 50000000 tuples (64%) done (elapsed 304.02 s, remaining 165.15 s) 2023-06-23T18:27:41.599 INFO:teuthology.orchestra.run.smithi002.stderr:32500000 of 50000000 tuples (65%) done (elapsed 304.84 s, remaining 164.14 s) 2023-06-23T18:27:42.438 INFO:teuthology.orchestra.run.smithi002.stderr:32600000 of 50000000 tuples (65%) done (elapsed 305.68 s, remaining 163.15 s) 2023-06-23T18:27:42.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:42 smithi017 ceph-mon[97472]: pgmap v466: 97 pgs: 97 active+clean; 5.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 MiB/s wr, 22 op/s 2023-06-23T18:27:42.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:42 smithi002 ceph-mon[95947]: pgmap v466: 97 pgs: 97 active+clean; 5.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 MiB/s wr, 22 op/s 2023-06-23T18:27:42.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:42 smithi082 ceph-mon[97889]: pgmap v466: 97 pgs: 97 active+clean; 5.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 38 MiB/s wr, 22 op/s 2023-06-23T18:27:43.345 INFO:teuthology.orchestra.run.smithi002.stderr:32700000 of 50000000 tuples (65%) done (elapsed 306.59 s, remaining 162.20 s) 2023-06-23T18:27:44.167 INFO:teuthology.orchestra.run.smithi002.stderr:32800000 of 50000000 tuples (65%) done (elapsed 307.41 s, remaining 161.20 s) 2023-06-23T18:27:44.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:44 smithi017 ceph-mon[97472]: pgmap v467: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-06-23T18:27:44.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:44 smithi002 ceph-mon[95947]: pgmap v467: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-06-23T18:27:44.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:44 smithi082 ceph-mon[97889]: pgmap v467: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 MiB/s wr, 27 op/s 2023-06-23T18:27:44.974 INFO:teuthology.orchestra.run.smithi002.stderr:32900000 of 50000000 tuples (65%) done (elapsed 308.21 s, remaining 160.20 s) 2023-06-23T18:27:45.845 INFO:teuthology.orchestra.run.smithi002.stderr:33000000 of 50000000 tuples (66%) done (elapsed 309.09 s, remaining 159.23 s) 2023-06-23T18:27:46.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:46 smithi017 ceph-mon[97472]: pgmap v468: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 26 op/s 2023-06-23T18:27:46.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:46 smithi002 ceph-mon[95947]: pgmap v468: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 26 op/s 2023-06-23T18:27:46.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:46 smithi082 ceph-mon[97889]: pgmap v468: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 26 op/s 2023-06-23T18:27:46.718 INFO:teuthology.orchestra.run.smithi002.stderr:33100000 of 50000000 tuples (66%) done (elapsed 309.96 s, remaining 158.26 s) 2023-06-23T18:27:47.556 INFO:teuthology.orchestra.run.smithi002.stderr:33200000 of 50000000 tuples (66%) done (elapsed 310.80 s, remaining 157.27 s) 2023-06-23T18:27:48.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:48 smithi017 ceph-mon[97472]: pgmap v469: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 24 op/s 2023-06-23T18:27:48.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:48 smithi002 ceph-mon[95947]: pgmap v469: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 24 op/s 2023-06-23T18:27:48.565 INFO:teuthology.orchestra.run.smithi002.stderr:33300000 of 50000000 tuples (66%) done (elapsed 311.81 s, remaining 156.37 s) 2023-06-23T18:27:48.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:48 smithi082 ceph-mon[97889]: pgmap v469: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 24 op/s 2023-06-23T18:27:49.427 INFO:teuthology.orchestra.run.smithi002.stderr:33400000 of 50000000 tuples (66%) done (elapsed 312.67 s, remaining 155.40 s) 2023-06-23T18:27:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:33500000 of 50000000 tuples (67%) done (elapsed 313.52 s, remaining 154.42 s) 2023-06-23T18:27:50.442 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:50 smithi017 ceph-mon[97472]: pgmap v470: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 MiB/s wr, 36 op/s 2023-06-23T18:27:50.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:50 smithi002 ceph-mon[95947]: pgmap v470: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 MiB/s wr, 36 op/s 2023-06-23T18:27:50.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:50 smithi082 ceph-mon[97889]: pgmap v470: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 48 MiB/s wr, 36 op/s 2023-06-23T18:27:51.108 INFO:teuthology.orchestra.run.smithi002.stderr:33600000 of 50000000 tuples (67%) done (elapsed 314.35 s, remaining 153.43 s) 2023-06-23T18:27:51.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:27:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:33700000 of 50000000 tuples (67%) done (elapsed 315.22 s, remaining 152.47 s) 2023-06-23T18:27:52.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.040+0000 7f1c75862700 1 -- 172.21.15.2:0/3953039885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700fe1f0 msgr2=0x7f1c700fe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.040+0000 7f1c75862700 1 --2- 172.21.15.2:0/3953039885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700fe1f0 0x7f1c700fe600 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009990 tx=0x7f1c60009ca0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.040+0000 7f1c75862700 1 -- 172.21.15.2:0/3953039885 shutdown_connections 2023-06-23T18:27:52.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.040+0000 7f1c75862700 1 --2- 172.21.15.2:0/3953039885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c700ffd70 0x7f1c70106630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.040+0000 7f1c75862700 1 --2- 172.21.15.2:0/3953039885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700fe1f0 0x7f1c700fe600 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.040+0000 7f1c75862700 1 --2- 172.21.15.2:0/3953039885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c700ff3e0 0x7f1c700ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.040+0000 7f1c75862700 1 -- 172.21.15.2:0/3953039885 >> 172.21.15.2:0/3953039885 conn(0x7f1c700f9810 msgr2=0x7f1c700fbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:52.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.041+0000 7f1c75862700 1 -- 172.21.15.2:0/3953039885 shutdown_connections 2023-06-23T18:27:52.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.041+0000 7f1c75862700 1 -- 172.21.15.2:0/3953039885 wait complete. 2023-06-23T18:27:52.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.041+0000 7f1c75862700 1 Processor -- start 2023-06-23T18:27:52.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.041+0000 7f1c75862700 1 -- start start 2023-06-23T18:27:52.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c75862700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c700fe1f0 0x7f1c700fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c75862700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c700ff3e0 0x7f1c7019a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c75862700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700ffd70 0x7f1c7019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c75862700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7019d310 con 0x7f1c700ffd70 2023-06-23T18:27:52.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c75862700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7019d480 con 0x7f1c700fe1f0 2023-06-23T18:27:52.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c75862700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7019d640 con 0x7f1c700ff3e0 2023-06-23T18:27:52.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c6f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700ffd70 0x7f1c7019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c667fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c700ff3e0 0x7f1c7019a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c6f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700ffd70 0x7f1c7019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:54978/0 (socket says 172.21.15.2:54978) 2023-06-23T18:27:52.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c6f7fe700 1 -- 172.21.15.2:0/4102551729 learned_addr learned my addr 172.21.15.2:0/4102551729 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:52.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c6effd700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c700fe1f0 0x7f1c700fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c667fc700 1 -- 172.21.15.2:0/4102551729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c700fe1f0 msgr2=0x7f1c700fbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c667fc700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c700fe1f0 0x7f1c700fbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c667fc700 1 -- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700ffd70 msgr2=0x7f1c7019cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c667fc700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700ffd70 0x7f1c7019cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c667fc700 1 -- 172.21.15.2:0/4102551729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68009c40 con 0x7f1c700ff3e0 2023-06-23T18:27:52.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c6f7fe700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700ffd70 0x7f1c7019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:27:52.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c667fc700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c700ff3e0 0x7f1c7019a530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e9f0 tx=0x7f1c5800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:52.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.042+0000 7f1c74860700 1 -- 172.21.15.2:0/4102551729 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800c9c0 con 0x7f1c700ff3e0 2023-06-23T18:27:52.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.043+0000 7f1c75862700 1 -- 172.21.15.2:0/4102551729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c600095f0 con 0x7f1c700ff3e0 2023-06-23T18:27:52.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.043+0000 7f1c74860700 1 -- 172.21.15.2:0/4102551729 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c58010430 con 0x7f1c700ff3e0 2023-06-23T18:27:52.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.043+0000 7f1c74860700 1 -- 172.21.15.2:0/4102551729 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58018680 con 0x7f1c700ff3e0 2023-06-23T18:27:52.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.043+0000 7f1c75862700 1 -- 172.21.15.2:0/4102551729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7019dca0 con 0x7f1c700ff3e0 2023-06-23T18:27:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.046+0000 7f1c74860700 1 -- 172.21.15.2:0/4102551729 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1c580105a0 con 0x7f1c700ff3e0 2023-06-23T18:27:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.046+0000 7f1c819e7b80 1 Processor -- start 2023-06-23T18:27:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.046+0000 7f1c819e7b80 1 -- start start 2023-06-23T18:27:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.046+0000 7f1c74860700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c5c0a1150 0x7f1c5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.046+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4563e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.046+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 0x562d4563ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4563f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d45542cb0 con 0x562d4563e5d0 2023-06-23T18:27:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d45595a40 con 0x562d4563f000 2023-06-23T18:27:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4558eed0 con 0x562d4563dc80 2023-06-23T18:27:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6f7fe700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c5c0a1150 0x7f1c5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4563e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c74860700 1 -- 172.21.15.2:0/4102551729 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1c58014070 con 0x7f1c700ff3e0 2023-06-23T18:27:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4563e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55320/0 (socket says 172.21.15.2:55320) 2023-06-23T18:27:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 -- 172.21.15.2:0/4281458508 learned_addr learned my addr 172.21.15.2:0/4281458508 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c667fc700 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 0x562d4563ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6f7fe700 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4563f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 -- 172.21.15.2:0/4281458508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 msgr2=0x562d4563f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4563f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 -- 172.21.15.2:0/4281458508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 msgr2=0x562d4563ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 0x562d4563ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 -- 172.21.15.2:0/4281458508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d45410630 con 0x562d4563dc80 2023-06-23T18:27:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6f7fe700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c5c0a1150 0x7f1c5c0a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009690 tx=0x7f1c68009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c6effd700 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4563e090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009990 tx=0x7f1c60005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2b354b90c3480ae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.047+0000 7f1c65ffb700 1 -- 172.21.15.2:0/4281458508 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6001d070 con 0x562d4563dc80 2023-06-23T18:27:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c65ffb700 1 -- 172.21.15.2:0/4281458508 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c6001e040 con 0x562d4563dc80 2023-06-23T18:27:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c65ffb700 1 -- 172.21.15.2:0/4281458508 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000b910 con 0x562d4563dc80 2023-06-23T18:27:52.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 -- 172.21.15.2:0/4281458508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 msgr2=0x562d4563e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4563e090 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009990 tx=0x7f1c60005c90 comp rx=0 tx=0).stop 2023-06-23T18:27:52.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 -- 172.21.15.2:0/4281458508 shutdown_connections 2023-06-23T18:27:52.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4563f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4563e090 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/4281458508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 0x562d4563ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 -- 172.21.15.2:0/4281458508 >> 172.21.15.2:0/4281458508 conn(0x562d456333f0 msgr2=0x562d45638520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 -- 172.21.15.2:0/4281458508 shutdown_connections 2023-06-23T18:27:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.048+0000 7f1c819e7b80 1 -- 172.21.15.2:0/4281458508 wait complete. 2023-06-23T18:27:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 1 -- start start 2023-06-23T18:27:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 1 Processor -- start 2023-06-23T18:27:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563dc80 0x562d456474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563e5d0 0x562d45647a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d4564ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4552eac0 con 0x562d4563f000 2023-06-23T18:27:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4552e7c0 con 0x562d4563dc80 2023-06-23T18:27:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d45412920 con 0x562d4563e5d0 2023-06-23T18:27:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c819e7b80 10 client.0 _ll_register_callbacks cb 0x562d45534260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:27:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c6effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563dc80 0x562d456474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c6f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d4564ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c6effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563dc80 0x562d456474f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45580/0 (socket says 172.21.15.2:45580) 2023-06-23T18:27:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c6effd700 1 -- 172.21.15.2:0/2447809051 learned_addr learned my addr 172.21.15.2:0/2447809051 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.049+0000 7f1c667fc700 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563e5d0 0x562d45647a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c6effd700 1 -- 172.21.15.2:0/2447809051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563e5d0 msgr2=0x562d45647a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c6effd700 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563e5d0 0x562d45647a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c6effd700 1 -- 172.21.15.2:0/2447809051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 msgr2=0x562d4564ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c6effd700 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d4564ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c6effd700 1 -- 172.21.15.2:0/2447809051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c70106f60 con 0x562d4563dc80 2023-06-23T18:27:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c6effd700 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563dc80 0x562d456474f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c60000c00 tx=0x7f1c600038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20ba3784c065c18 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c4effd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:27:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c4effd700 1 -- 172.21.15.2:0/2447809051 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6001d040 con 0x562d4563dc80 2023-06-23T18:27:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c819e7b80 1 -- 172.21.15.2:0/2447809051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d45630bb0 con 0x562d4563dc80 2023-06-23T18:27:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c4effd700 1 -- 172.21.15.2:0/2447809051 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1c6000bac0 con 0x562d4563dc80 2023-06-23T18:27:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c4effd700 1 -- 172.21.15.2:0/2447809051 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c6000bc80 con 0x562d4563dc80 2023-06-23T18:27:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c819e7b80 10 client.25382 fetch_fsmap learned FSMap version 30 2023-06-23T18:27:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c819e7b80 1 -- 172.21.15.2:0/2447809051 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x562d453bcde0 con 0x562d4563dc80 2023-06-23T18:27:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.050+0000 7f1c4effd700 1 -- 172.21.15.2:0/2447809051 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562d45630bb0 con 0x562d4563dc80 2023-06-23T18:27:52.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c4effd700 1 -- 172.21.15.2:0/2447809051 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1c60034c70 con 0x562d4563dc80 2023-06-23T18:27:52.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c819e7b80 10 client.25382 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:27:52.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c819e7b80 10 client.25382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:27:52.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c819e7b80 20 client.25382 populate_metadata read hostname 'smithi002' 2023-06-23T18:27:52.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562d45652d80 0x562d45655160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c819e7b80 4 client.25382 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:27:52.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c819e7b80 1 -- 172.21.15.2:0/2447809051 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d45587eb0 con 0x562d45652d80 2023-06-23T18:27:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c667fc700 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562d45652d80 0x562d45655160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c4effd700 10 client.25382 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:27:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.051+0000 7f1c667fc700 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562d45652d80 0x562d45655160 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8607af852d0c2078 server_cookie=d8e33370f50320a0 in_seq=0 out_seq=0 2023-06-23T18:27:52.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.052+0000 7f1c4effd700 1 -- 172.21.15.2:0/2447809051 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d45587eb0 con 0x562d45652d80 2023-06-23T18:27:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.052+0000 7f1c4effd700 10 client.25382 handle_command_reply: tid=0 2023-06-23T18:27:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 client.25382 shutdown 2023-06-23T18:27:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 -- 172.21.15.2:0/2447809051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563dc80 msgr2=0x562d456474f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563dc80 0x562d456474f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c60000c00 tx=0x7f1c600038b0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 -- 172.21.15.2:0/2447809051 shutdown_connections 2023-06-23T18:27:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562d45652d80 0x562d45655160 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d4564ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c4effd700 0 client.25382 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:27:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563dc80 0x562d456474f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/2447809051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563e5d0 0x562d45647a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 -- 172.21.15.2:0/2447809051 >> 172.21.15.2:0/2447809051 conn(0x562d456333f0 msgr2=0x562d45637c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 -- 172.21.15.2:0/2447809051 shutdown_connections 2023-06-23T18:27:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 1 -- 172.21.15.2:0/2447809051 wait complete. 2023-06-23T18:27:52.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.053+0000 7f1c819e7b80 20 client.25382 trim_cache size 0 max 16384 2023-06-23T18:27:52.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.073+0000 7f1c819e7b80 1 Processor -- start 2023-06-23T18:27:52.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.073+0000 7f1c819e7b80 1 -- start start 2023-06-23T18:27:52.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.073+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d45646200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.073+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563e5d0 0x562d4564ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.073+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d45641ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.073+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d456393d0 con 0x562d4563f000 2023-06-23T18:27:52.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.073+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d45652b20 con 0x562d4563e5d0 2023-06-23T18:27:52.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.073+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d45642a00 con 0x562d4563dc80 2023-06-23T18:27:52.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d45646200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d45641ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d45646200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55334/0 (socket says 172.21.15.2:55334) 2023-06-23T18:27:52.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 -- 172.21.15.2:0/828837542 learned_addr learned my addr 172.21.15.2:0/828837542 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:52.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c667fc700 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563e5d0 0x562d4564ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 -- 172.21.15.2:0/828837542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563e5d0 msgr2=0x562d4564ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:52.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563e5d0 0x562d4564ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 -- 172.21.15.2:0/828837542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 msgr2=0x562d45641ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:52.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d45641ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 -- 172.21.15.2:0/828837542 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d453a5c60 con 0x562d4563dc80 2023-06-23T18:27:52.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c667fc700 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563e5d0 0x562d4564ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:27:52.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6f7fe700 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d45641ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:27:52.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c6effd700 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d45646200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000b2e0 tx=0x7f1c60003db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cab28a21744e7297 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:52.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c66ffd700 1 -- 172.21.15.2:0/828837542 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000f6d0 con 0x562d4563dc80 2023-06-23T18:27:52.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c66ffd700 1 -- 172.21.15.2:0/828837542 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c6003a030 con 0x562d4563dc80 2023-06-23T18:27:52.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.074+0000 7f1c66ffd700 1 -- 172.21.15.2:0/828837542 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6002b850 con 0x562d4563dc80 2023-06-23T18:27:52.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 -- 172.21.15.2:0/828837542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 msgr2=0x562d45646200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d45646200 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000b2e0 tx=0x7f1c60003db0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 -- 172.21.15.2:0/828837542 shutdown_connections 2023-06-23T18:27:52.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563f000 0x562d45641ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d45646200 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/828837542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563e5d0 0x562d4564ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 -- 172.21.15.2:0/828837542 >> 172.21.15.2:0/828837542 conn(0x562d456333f0 msgr2=0x562d45659ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:52.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 -- 172.21.15.2:0/828837542 shutdown_connections 2023-06-23T18:27:52.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.075+0000 7f1c819e7b80 1 -- 172.21.15.2:0/828837542 wait complete. 2023-06-23T18:27:52.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 1 -- start start 2023-06-23T18:27:52.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 1 Processor -- start 2023-06-23T18:27:52.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4564a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 0x562d4564aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4564def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4564e430 con 0x562d4563e5d0 2023-06-23T18:27:52.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4564e5a0 con 0x562d4563f000 2023-06-23T18:27:52.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d4564e760 con 0x562d4563dc80 2023-06-23T18:27:52.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c819e7b80 10 client.0 _ll_register_callbacks cb 0x562d45534260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:27:52.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c6effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4564a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c6effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4564a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55340/0 (socket says 172.21.15.2:55340) 2023-06-23T18:27:52.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c6effd700 1 -- 172.21.15.2:0/937628852 learned_addr learned my addr 172.21.15.2:0/937628852 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:27:52.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c6f7fe700 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4564def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.076+0000 7f1c667fc700 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 0x562d4564aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c6effd700 1 -- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 msgr2=0x562d4564def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:52.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c6effd700 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4564def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c6effd700 1 -- 172.21.15.2:0/937628852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 msgr2=0x562d4564aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:27:52.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c6effd700 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 0x562d4564aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c6effd700 1 -- 172.21.15.2:0/937628852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7019dca0 con 0x562d4563dc80 2023-06-23T18:27:52.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c6f7fe700 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4564def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:27:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c6effd700 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4564a990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c6004a040 tx=0x7f1c6000f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c469a019d378add4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:27:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c4effd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:27:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c4effd700 1 -- 172.21.15.2:0/937628852 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c6001d070 con 0x562d4563dc80 2023-06-23T18:27:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c4effd700 1 -- 172.21.15.2:0/937628852 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1c60000e40 con 0x562d4563dc80 2023-06-23T18:27:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c819e7b80 1 -- 172.21.15.2:0/937628852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c6001e040 con 0x562d4563dc80 2023-06-23T18:27:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c4effd700 1 -- 172.21.15.2:0/937628852 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c60048da0 con 0x562d4563dc80 2023-06-23T18:27:52.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c819e7b80 10 client.25543 fetch_fsmap learned FSMap version 30 2023-06-23T18:27:52.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c819e7b80 1 -- 172.21.15.2:0/937628852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f1c68003b60 con 0x562d4563dc80 2023-06-23T18:27:52.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c4effd700 1 -- 172.21.15.2:0/937628852 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c60056020 con 0x562d4563dc80 2023-06-23T18:27:52.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.077+0000 7f1c4effd700 1 -- 172.21.15.2:0/937628852 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1c6002ba20 con 0x562d4563dc80 2023-06-23T18:27:52.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c819e7b80 10 client.25543 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:27:52.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c819e7b80 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:27:52.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c819e7b80 20 client.25543 populate_metadata read hostname 'smithi002' 2023-06-23T18:27:52.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562d456a7960 0x562d456a9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:27:52.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c819e7b80 4 client.25543 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:27:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c819e7b80 1 -- 172.21.15.2:0/937628852 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562d4554a050 con 0x562d456a7960 2023-06-23T18:27:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c667fc700 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562d456a7960 0x562d456a9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:27:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c4effd700 10 client.25543 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:27:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c667fc700 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562d456a7960 0x562d456a9d70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6adf565b2f298a server_cookie=ec65874d499947e0 in_seq=0 out_seq=0 2023-06-23T18:27:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c4effd700 1 -- 172.21.15.2:0/937628852 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562d4554a050 con 0x562d456a7960 2023-06-23T18:27:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c4effd700 10 client.25543 handle_command_reply: tid=0 2023-06-23T18:27:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.078+0000 7f1c819e7b80 1 client.25543 shutdown 2023-06-23T18:27:52.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 -- 172.21.15.2:0/937628852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 msgr2=0x562d4564a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4564a990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1c6004a040 tx=0x7f1c6000f740 comp rx=0 tx=0).stop 2023-06-23T18:27:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 -- 172.21.15.2:0/937628852 shutdown_connections 2023-06-23T18:27:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562d456a7960 0x562d456a9d70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d4563f000 0x562d4564def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d4563dc80 0x562d4564a990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c4effd700 0 client.25543 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:27:52.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 --2- 172.21.15.2:0/937628852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d4563e5d0 0x562d4564aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 -- 172.21.15.2:0/937628852 >> 172.21.15.2:0/937628852 conn(0x562d456333f0 msgr2=0x562d456461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:52.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 -- 172.21.15.2:0/937628852 shutdown_connections 2023-06-23T18:27:52.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 1 -- 172.21.15.2:0/937628852 wait complete. 2023-06-23T18:27:52.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c819e7b80 20 client.25543 trim_cache size 0 max 16384 2023-06-23T18:27:52.096 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:27:52.097 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:27:52.097 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:27:52.097 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:27:52.097 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:27:52.097 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:27:52.097 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:27:52.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:27:52.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:27:52.098 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:27:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 -- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c5c0a1150 msgr2=0x7f1c5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c5c0a1150 0x7f1c5c0a3600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009690 tx=0x7f1c68009920 comp rx=0 tx=0).stop 2023-06-23T18:27:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 -- 172.21.15.2:0/4102551729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c700ff3e0 msgr2=0x7f1c7019a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:27:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c700ff3e0 0x7f1c7019a530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800e9f0 tx=0x7f1c5800ed00 comp rx=0 tx=0).stop 2023-06-23T18:27:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 -- 172.21.15.2:0/4102551729 shutdown_connections 2023-06-23T18:27:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1c700ffd70 0x7f1c7019cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1c5c0a1150 0x7f1c5c0a3600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c700fe1f0 0x7f1c700fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 --2- 172.21.15.2:0/4102551729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1c700ff3e0 0x7f1c7019a530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:27:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.079+0000 7f1c75862700 1 -- 172.21.15.2:0/4102551729 >> 172.21.15.2:0/4102551729 conn(0x7f1c700f9810 msgr2=0x7f1c70102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:27:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.080+0000 7f1c75862700 1 -- 172.21.15.2:0/4102551729 shutdown_connections 2023-06-23T18:27:52.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:27:52.080+0000 7f1c75862700 1 -- 172.21.15.2:0/4102551729 wait complete. 2023-06-23T18:27:52.100 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:27:52.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:52 smithi017 ceph-mon[97472]: pgmap v471: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 39 MiB/s wr, 44 op/s 2023-06-23T18:27:52.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:52 smithi002 ceph-mon[95947]: pgmap v471: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 39 MiB/s wr, 44 op/s 2023-06-23T18:27:52.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:52 smithi082 ceph-mon[97889]: pgmap v471: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 39 MiB/s wr, 44 op/s 2023-06-23T18:27:52.839 INFO:teuthology.orchestra.run.smithi002.stderr:33800000 of 50000000 tuples (67%) done (elapsed 316.08 s, remaining 151.49 s) 2023-06-23T18:27:53.663 INFO:teuthology.orchestra.run.smithi002.stderr:33900000 of 50000000 tuples (67%) done (elapsed 316.90 s, remaining 150.51 s) 2023-06-23T18:27:54.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:54 smithi017 ceph-mon[97472]: pgmap v472: 97 pgs: 97 active+clean; 5.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 44 MiB/s wr, 47 op/s 2023-06-23T18:27:54.545 INFO:teuthology.orchestra.run.smithi002.stderr:34000000 of 50000000 tuples (68%) done (elapsed 317.79 s, remaining 149.55 s) 2023-06-23T18:27:54.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:54 smithi002 ceph-mon[95947]: pgmap v472: 97 pgs: 97 active+clean; 5.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 44 MiB/s wr, 47 op/s 2023-06-23T18:27:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:54 smithi082 ceph-mon[97889]: pgmap v472: 97 pgs: 97 active+clean; 5.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 44 MiB/s wr, 47 op/s 2023-06-23T18:27:55.382 INFO:teuthology.orchestra.run.smithi002.stderr:34100000 of 50000000 tuples (68%) done (elapsed 318.62 s, remaining 148.57 s) 2023-06-23T18:27:56.201 INFO:teuthology.orchestra.run.smithi002.stderr:34200000 of 50000000 tuples (68%) done (elapsed 319.44 s, remaining 147.58 s) 2023-06-23T18:27:56.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:56 smithi002 ceph-mon[95947]: pgmap v473: 97 pgs: 97 active+clean; 5.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:27:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:56 smithi082 ceph-mon[97889]: pgmap v473: 97 pgs: 97 active+clean; 5.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:27:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:56 smithi017 ceph-mon[97472]: pgmap v473: 97 pgs: 97 active+clean; 5.2 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:27:57.012 INFO:teuthology.orchestra.run.smithi002.stderr:34300000 of 50000000 tuples (68%) done (elapsed 320.25 s, remaining 146.59 s) 2023-06-23T18:27:57.906 INFO:teuthology.orchestra.run.smithi002.stderr:34400000 of 50000000 tuples (68%) done (elapsed 321.15 s, remaining 145.64 s) 2023-06-23T18:27:58.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:27:58 smithi002 ceph-mon[95947]: pgmap v474: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:27:58.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:27:58 smithi082 ceph-mon[97889]: pgmap v474: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:27:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:27:58 smithi017 ceph-mon[97472]: pgmap v474: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 39 MiB/s wr, 43 op/s 2023-06-23T18:27:58.746 INFO:teuthology.orchestra.run.smithi002.stderr:34500000 of 50000000 tuples (69%) done (elapsed 321.99 s, remaining 144.66 s) 2023-06-23T18:27:59.602 INFO:teuthology.orchestra.run.smithi002.stderr:34600000 of 50000000 tuples (69%) done (elapsed 322.84 s, remaining 143.69 s) 2023-06-23T18:28:00.442 INFO:teuthology.orchestra.run.smithi002.stderr:34700000 of 50000000 tuples (69%) done (elapsed 323.68 s, remaining 142.72 s) 2023-06-23T18:28:00.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:00 smithi002 ceph-mon[95947]: pgmap v475: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 46 op/s 2023-06-23T18:28:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:00 smithi082 ceph-mon[97889]: pgmap v475: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 46 op/s 2023-06-23T18:28:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:00 smithi017 ceph-mon[97472]: pgmap v475: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 46 op/s 2023-06-23T18:28:01.349 INFO:teuthology.orchestra.run.smithi002.stderr:34800000 of 50000000 tuples (69%) done (elapsed 324.59 s, remaining 141.78 s) 2023-06-23T18:28:01.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:28:01.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:28:01.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:28:01.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:28:01.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:28:01.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:28:01.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:28:01.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:28:01.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:28:01.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:28:01.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:28:01.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:28:02.159 INFO:teuthology.orchestra.run.smithi002.stderr:34900000 of 50000000 tuples (69%) done (elapsed 325.40 s, remaining 140.79 s) 2023-06-23T18:28:02.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:02 smithi002 ceph-mon[95947]: pgmap v476: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 35 op/s 2023-06-23T18:28:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:02 smithi082 ceph-mon[97889]: pgmap v476: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 35 op/s 2023-06-23T18:28:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:02 smithi017 ceph-mon[97472]: pgmap v476: 97 pgs: 97 active+clean; 5.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 35 MiB/s wr, 35 op/s 2023-06-23T18:28:03.031 INFO:teuthology.orchestra.run.smithi002.stderr:35000000 of 50000000 tuples (70%) done (elapsed 326.27 s, remaining 139.83 s) 2023-06-23T18:28:03.925 INFO:teuthology.orchestra.run.smithi002.stderr:35100000 of 50000000 tuples (70%) done (elapsed 327.17 s, remaining 138.88 s) 2023-06-23T18:28:04.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:04 smithi002 ceph-mon[95947]: pgmap v477: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:04 smithi082 ceph-mon[97889]: pgmap v477: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:04 smithi017 ceph-mon[97472]: pgmap v477: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:04.759 INFO:teuthology.orchestra.run.smithi002.stderr:35200000 of 50000000 tuples (70%) done (elapsed 328.00 s, remaining 137.91 s) 2023-06-23T18:28:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:35300000 of 50000000 tuples (70%) done (elapsed 328.89 s, remaining 136.96 s) 2023-06-23T18:28:06.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:06 smithi002 ceph-mon[95947]: pgmap v478: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:28:06.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:06 smithi082 ceph-mon[97889]: pgmap v478: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:28:06.633 INFO:teuthology.orchestra.run.smithi002.stderr:35400000 of 50000000 tuples (70%) done (elapsed 329.87 s, remaining 136.05 s) 2023-06-23T18:28:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:06 smithi017 ceph-mon[97472]: pgmap v478: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:28:07.539 INFO:teuthology.orchestra.run.smithi002.stderr:35500000 of 50000000 tuples (71%) done (elapsed 330.78 s, remaining 135.11 s) 2023-06-23T18:28:08.361 INFO:teuthology.orchestra.run.smithi002.stderr:35600000 of 50000000 tuples (71%) done (elapsed 331.60 s, remaining 134.13 s) 2023-06-23T18:28:08.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:08 smithi002 ceph-mon[95947]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 41 MiB/s wr, 21 op/s 2023-06-23T18:28:08.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:08 smithi082 ceph-mon[97889]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 41 MiB/s wr, 21 op/s 2023-06-23T18:28:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:08 smithi017 ceph-mon[97472]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 41 MiB/s wr, 21 op/s 2023-06-23T18:28:09.204 INFO:teuthology.orchestra.run.smithi002.stderr:35700000 of 50000000 tuples (71%) done (elapsed 332.45 s, remaining 133.16 s) 2023-06-23T18:28:10.106 INFO:teuthology.orchestra.run.smithi002.stderr:35800000 of 50000000 tuples (71%) done (elapsed 333.35 s, remaining 132.22 s) 2023-06-23T18:28:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:10 smithi002 ceph-mon[95947]: pgmap v480: 97 pgs: 97 active+clean; 5.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 45 MiB/s wr, 32 op/s 2023-06-23T18:28:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:10 smithi082 ceph-mon[97889]: pgmap v480: 97 pgs: 97 active+clean; 5.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 45 MiB/s wr, 32 op/s 2023-06-23T18:28:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:10 smithi017 ceph-mon[97472]: pgmap v480: 97 pgs: 97 active+clean; 5.4 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 45 MiB/s wr, 32 op/s 2023-06-23T18:28:10.904 INFO:teuthology.orchestra.run.smithi002.stderr:35900000 of 50000000 tuples (71%) done (elapsed 334.15 s, remaining 131.24 s) 2023-06-23T18:28:11.750 INFO:teuthology.orchestra.run.smithi002.stderr:36000000 of 50000000 tuples (72%) done (elapsed 334.99 s, remaining 130.27 s) 2023-06-23T18:28:12.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:12 smithi002 ceph-mon[95947]: pgmap v481: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 40 MiB/s wr, 32 op/s 2023-06-23T18:28:12.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:12 smithi082 ceph-mon[97889]: pgmap v481: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 40 MiB/s wr, 32 op/s 2023-06-23T18:28:12.623 INFO:teuthology.orchestra.run.smithi002.stderr:36100000 of 50000000 tuples (72%) done (elapsed 335.86 s, remaining 129.32 s) 2023-06-23T18:28:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:12 smithi017 ceph-mon[97472]: pgmap v481: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 40 MiB/s wr, 32 op/s 2023-06-23T18:28:13.508 INFO:teuthology.orchestra.run.smithi002.stderr:36200000 of 50000000 tuples (72%) done (elapsed 336.75 s, remaining 128.37 s) 2023-06-23T18:28:14.344 INFO:teuthology.orchestra.run.smithi002.stderr:36300000 of 50000000 tuples (72%) done (elapsed 337.59 s, remaining 127.41 s) 2023-06-23T18:28:14.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:14 smithi002 ceph-mon[95947]: pgmap v482: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 47 MiB/s wr, 38 op/s 2023-06-23T18:28:14.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:14 smithi082 ceph-mon[97889]: pgmap v482: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 47 MiB/s wr, 38 op/s 2023-06-23T18:28:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:14 smithi017 ceph-mon[97472]: pgmap v482: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 47 MiB/s wr, 38 op/s 2023-06-23T18:28:15.233 INFO:teuthology.orchestra.run.smithi002.stderr:36400000 of 50000000 tuples (72%) done (elapsed 338.47 s, remaining 126.46 s) 2023-06-23T18:28:16.074 INFO:teuthology.orchestra.run.smithi002.stderr:36500000 of 50000000 tuples (73%) done (elapsed 339.32 s, remaining 125.50 s) 2023-06-23T18:28:16.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:16 smithi002 ceph-mon[95947]: pgmap v483: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:28:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:16 smithi082 ceph-mon[97889]: pgmap v483: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:28:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:16 smithi017 ceph-mon[97472]: pgmap v483: 97 pgs: 97 active+clean; 5.5 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 36 op/s 2023-06-23T18:28:16.902 INFO:teuthology.orchestra.run.smithi002.stderr:36600000 of 50000000 tuples (73%) done (elapsed 340.14 s, remaining 124.53 s) 2023-06-23T18:28:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:36700000 of 50000000 tuples (73%) done (elapsed 341.01 s, remaining 123.58 s) 2023-06-23T18:28:18.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:18 smithi002 ceph-mon[95947]: pgmap v484: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 40 MiB/s wr, 35 op/s 2023-06-23T18:28:18.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:18 smithi082 ceph-mon[97889]: pgmap v484: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 40 MiB/s wr, 35 op/s 2023-06-23T18:28:18.654 INFO:teuthology.orchestra.run.smithi002.stderr:36800000 of 50000000 tuples (73%) done (elapsed 341.90 s, remaining 122.64 s) 2023-06-23T18:28:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:18 smithi017 ceph-mon[97472]: pgmap v484: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 40 MiB/s wr, 35 op/s 2023-06-23T18:28:19.529 INFO:teuthology.orchestra.run.smithi002.stderr:36900000 of 50000000 tuples (73%) done (elapsed 342.77 s, remaining 121.69 s) 2023-06-23T18:28:20.361 INFO:teuthology.orchestra.run.smithi002.stderr:37000000 of 50000000 tuples (74%) done (elapsed 343.60 s, remaining 120.73 s) 2023-06-23T18:28:20.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:20 smithi002 ceph-mon[95947]: pgmap v485: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 41 MiB/s wr, 37 op/s 2023-06-23T18:28:20.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:20 smithi082 ceph-mon[97889]: pgmap v485: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 41 MiB/s wr, 37 op/s 2023-06-23T18:28:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:20 smithi017 ceph-mon[97472]: pgmap v485: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 41 MiB/s wr, 37 op/s 2023-06-23T18:28:21.228 INFO:teuthology.orchestra.run.smithi002.stderr:37100000 of 50000000 tuples (74%) done (elapsed 344.47 s, remaining 119.78 s) 2023-06-23T18:28:22.102 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:28:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:37200000 of 50000000 tuples (74%) done (elapsed 345.50 s, remaining 118.88 s) 2023-06-23T18:28:22.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.275+0000 7f640fec4700 1 -- 172.21.15.2:0/3607541015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ffc70 msgr2=0x7f64081000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.275+0000 7f640fec4700 1 --2- 172.21.15.2:0/3607541015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ffc70 0x7f64081000c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009990 tx=0x7f63f8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 -- 172.21.15.2:0/3607541015 shutdown_connections 2023-06-23T18:28:22.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 --2- 172.21.15.2:0/3607541015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6408100600 0x7f6408106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 --2- 172.21.15.2:0/3607541015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64080fea80 0x7f64080fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 --2- 172.21.15.2:0/3607541015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080ffc70 0x7f64081000c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 -- 172.21.15.2:0/3607541015 >> 172.21.15.2:0/3607541015 conn(0x7f64080fa0e0 msgr2=0x7f64080fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:22.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 -- 172.21.15.2:0/3607541015 shutdown_connections 2023-06-23T18:28:22.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 -- 172.21.15.2:0/3607541015 wait complete. 2023-06-23T18:28:22.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 Processor -- start 2023-06-23T18:28:22.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.276+0000 7f640fec4700 1 -- start start 2023-06-23T18:28:22.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640fec4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080fea80 0x7f6408102d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640fec4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64080ffc70 0x7f6408101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640fec4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6408100600 0x7f6408101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640fec4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6408102030 con 0x7f6408100600 2023-06-23T18:28:22.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640fec4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64081021a0 con 0x7f64080fea80 2023-06-23T18:28:22.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640fec4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640819f860 con 0x7f64080ffc70 2023-06-23T18:28:22.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640e461700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6408100600 0x7f6408101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640d45f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64080ffc70 0x7f6408101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640dc60700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080fea80 0x7f6408102d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640e461700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6408100600 0x7f6408101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55020/0 (socket says 172.21.15.2:55020) 2023-06-23T18:28:22.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640dc60700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080fea80 0x7f6408102d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45608/0 (socket says 172.21.15.2:45608) 2023-06-23T18:28:22.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640e461700 1 -- 172.21.15.2:0/1367501193 learned_addr learned my addr 172.21.15.2:0/1367501193 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:22.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640d45f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64080ffc70 0x7f6408101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55356/0 (socket says 172.21.15.2:55356) 2023-06-23T18:28:22.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640d45f700 1 -- 172.21.15.2:0/1367501193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080fea80 msgr2=0x7f6408102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640d45f700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080fea80 0x7f6408102d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640d45f700 1 -- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6408100600 msgr2=0x7f6408101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640d45f700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6408100600 0x7f6408101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640d45f700 1 -- 172.21.15.2:0/1367501193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6400009c40 con 0x7f64080ffc70 2023-06-23T18:28:22.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640e461700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6408100600 0x7f6408101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:22.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f640d45f700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64080ffc70 0x7f6408101350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f63f8000c00 tx=0x7f63f8005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:22.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.277+0000 7f63feffd700 1 -- 172.21.15.2:0/1367501193 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f801d070 con 0x7f64080ffc70 2023-06-23T18:28:22.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.278+0000 7f640fec4700 1 -- 172.21.15.2:0/1367501193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f80095f0 con 0x7f64080ffc70 2023-06-23T18:28:22.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.278+0000 7f63feffd700 1 -- 172.21.15.2:0/1367501193 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63f800bd50 con 0x7f64080ffc70 2023-06-23T18:28:22.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.278+0000 7f63feffd700 1 -- 172.21.15.2:0/1367501193 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8022d00 con 0x7f64080ffc70 2023-06-23T18:28:22.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.278+0000 7f640fec4700 1 -- 172.21.15.2:0/1367501193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640819fdd0 con 0x7f64080ffc70 2023-06-23T18:28:22.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.280+0000 7f63feffd700 1 -- 172.21.15.2:0/1367501193 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f63f8022500 con 0x7f64080ffc70 2023-06-23T18:28:22.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.281+0000 7f63feffd700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63f40a11d0 0x7f63f40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.281+0000 7f63feffd700 1 -- 172.21.15.2:0/1367501193 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f63f80cdd00 con 0x7f64080ffc70 2023-06-23T18:28:22.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.281+0000 7f640dc60700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63f40a11d0 0x7f63f40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f640dc60700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63f40a11d0 0x7f63f40a3680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64080ffad0 tx=0x7f6404009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:22.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f641c049b80 1 Processor -- start 2023-06-23T18:28:22.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f641c049b80 1 -- start start 2023-06-23T18:28:22.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f641c049b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbdc80 0x5651ebcbe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f641c049b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcbea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f641c049b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbf000 0x5651ebcbf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f641c049b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebbc2cb0 con 0x5651ebcbdc80 2023-06-23T18:28:22.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f641c049b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebc15a40 con 0x5651ebcbe5d0 2023-06-23T18:28:22.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.282+0000 7f641c049b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebc0eed0 con 0x5651ebcbf000 2023-06-23T18:28:22.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640dc60700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbdc80 0x5651ebcbe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640d45f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcbea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640dc60700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbdc80 0x5651ebcbe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55024/0 (socket says 172.21.15.2:55024) 2023-06-23T18:28:22.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640dc60700 1 -- 172.21.15.2:0/4187929251 learned_addr learned my addr 172.21.15.2:0/4187929251 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:22.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640e461700 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbf000 0x5651ebcbf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640d45f700 1 -- 172.21.15.2:0/4187929251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbf000 msgr2=0x5651ebcbf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:22.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640d45f700 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbf000 0x5651ebcbf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640d45f700 1 -- 172.21.15.2:0/4187929251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbdc80 msgr2=0x5651ebcbe090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:22.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640d45f700 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbdc80 0x5651ebcbe090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640d45f700 1 -- 172.21.15.2:0/4187929251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651eba90630 con 0x5651ebcbe5d0 2023-06-23T18:28:22.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640e461700 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbf000 0x5651ebcbf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:28:22.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640dc60700 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbdc80 0x5651ebcbe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:22.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f640d45f700 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcbea70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f63f809b130 tx=0x7f63f8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a4147ed9eae01cb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:22.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f63f3fff700 1 -- 172.21.15.2:0/4187929251 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f809b1a0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f63f3fff700 1 -- 172.21.15.2:0/4187929251 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63f809f1d0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.283+0000 7f63f3fff700 1 -- 172.21.15.2:0/4187929251 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80a30d0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 -- 172.21.15.2:0/4187929251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 msgr2=0x5651ebcbea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcbea70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f63f809b130 tx=0x7f63f8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 -- 172.21.15.2:0/4187929251 shutdown_connections 2023-06-23T18:28:22.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbf000 0x5651ebcbf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbdc80 0x5651ebcbe090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 --2- 172.21.15.2:0/4187929251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcbea70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 -- 172.21.15.2:0/4187929251 >> 172.21.15.2:0/4187929251 conn(0x5651ebcb33f0 msgr2=0x5651ebcb8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:22.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 -- 172.21.15.2:0/4187929251 shutdown_connections 2023-06-23T18:28:22.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.284+0000 7f641c049b80 1 -- 172.21.15.2:0/4187929251 wait complete. 2023-06-23T18:28:22.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 1 -- start start 2023-06-23T18:28:22.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 1 Processor -- start 2023-06-23T18:28:22.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcc7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebbaeac0 con 0x5651ebcbf000 2023-06-23T18:28:22.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebbae7c0 con 0x5651ebcbdc80 2023-06-23T18:28:22.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651eba92920 con 0x5651ebcbe5d0 2023-06-23T18:28:22.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f641c049b80 10 client.0 _ll_register_callbacks cb 0x5651ebbb4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:28:22.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f640d45f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f640dc60700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcc7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f640d45f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55370/0 (socket says 172.21.15.2:55370) 2023-06-23T18:28:22.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f640dc60700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcc7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45622/0 (socket says 172.21.15.2:45622) 2023-06-23T18:28:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.285+0000 7f640d45f700 1 -- 172.21.15.2:0/3928229833 learned_addr learned my addr 172.21.15.2:0/3928229833 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f640e461700 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f640d45f700 1 -- 172.21.15.2:0/3928229833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 msgr2=0x5651ebcc7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f640d45f700 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcc7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f640d45f700 1 -- 172.21.15.2:0/3928229833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 msgr2=0x5651ebcce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f640d45f700 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f640d45f700 1 -- 172.21.15.2:0/3928229833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6404017070 con 0x5651ebcbe5d0 2023-06-23T18:28:22.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f640dc60700 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcc7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f640d45f700 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc7760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63f8005910 tx=0x7f63f8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6918d65596dd6943 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f63f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:28:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f63f17fa700 1 -- 172.21.15.2:0/3928229833 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f80a3560 con 0x5651ebcbe5d0 2023-06-23T18:28:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f641c049b80 1 -- 172.21.15.2:0/3928229833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651ebcb0bb0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f63f17fa700 1 -- 172.21.15.2:0/3928229833 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f63f800b8c0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f63f17fa700 1 -- 172.21.15.2:0/3928229833 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f800ba80 con 0x5651ebcbe5d0 2023-06-23T18:28:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f641c049b80 10 client.25555 fetch_fsmap learned FSMap version 30 2023-06-23T18:28:22.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.286+0000 7f641c049b80 1 -- 172.21.15.2:0/3928229833 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f640819fdd0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f63f17fa700 1 -- 172.21.15.2:0/3928229833 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5651ebcb0bb0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f63f17fa700 1 -- 172.21.15.2:0/3928229833 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f63f8096f90 con 0x5651ebcbe5d0 2023-06-23T18:28:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f641c049b80 10 client.25555 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:28:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f641c049b80 10 client.25555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:28:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f641c049b80 20 client.25555 populate_metadata read hostname 'smithi002' 2023-06-23T18:28:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f641c049b80 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5651ebcd2ab0 0x5651ebcd4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f641c049b80 4 client.25555 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:28:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f641c049b80 1 -- 172.21.15.2:0/3928229833 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651ebc07eb0 con 0x5651ebcd2ab0 2023-06-23T18:28:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f640dc60700 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5651ebcd2ab0 0x5651ebcd4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f640dc60700 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5651ebcd2ab0 0x5651ebcd4e90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2cf55e468eb7c8 server_cookie=f79400c6cd31de9d in_seq=0 out_seq=0 2023-06-23T18:28:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.287+0000 7f63f17fa700 10 client.25555 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:28:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f63f17fa700 1 -- 172.21.15.2:0/3928229833 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5651ebc07eb0 con 0x5651ebcd2ab0 2023-06-23T18:28:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f63f17fa700 10 client.25555 handle_command_reply: tid=0 2023-06-23T18:28:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 client.25555 shutdown 2023-06-23T18:28:22.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 -- 172.21.15.2:0/3928229833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 msgr2=0x5651ebcc7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc7760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63f8005910 tx=0x7f63f8005c00 comp rx=0 tx=0).stop 2023-06-23T18:28:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 -- 172.21.15.2:0/3928229833 shutdown_connections 2023-06-23T18:28:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5651ebcd2ab0 0x5651ebcd4e90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f63f17fa700 0 client.25555 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:28:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcc7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 --2- 172.21.15.2:0/3928229833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc7760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.288+0000 7f641c049b80 1 -- 172.21.15.2:0/3928229833 >> 172.21.15.2:0/3928229833 conn(0x5651ebcb33f0 msgr2=0x5651ebcb7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.289+0000 7f641c049b80 1 -- 172.21.15.2:0/3928229833 shutdown_connections 2023-06-23T18:28:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.289+0000 7f641c049b80 1 -- 172.21.15.2:0/3928229833 wait complete. 2023-06-23T18:28:22.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.289+0000 7f641c049b80 20 client.25555 trim_cache size 0 max 16384 2023-06-23T18:28:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f641c049b80 1 Processor -- start 2023-06-23T18:28:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f641c049b80 1 -- start start 2023-06-23T18:28:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f641c049b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbdc80 0x5651ebcc1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f641c049b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f641c049b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f641c049b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebcc6d90 con 0x5651ebcbf000 2023-06-23T18:28:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f641c049b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebcc5c10 con 0x5651ebcbe5d0 2023-06-23T18:28:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f641c049b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebcb9fb0 con 0x5651ebcbdc80 2023-06-23T18:28:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640d45f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640e461700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640dc60700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbdc80 0x5651ebcc1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640e461700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55042/0 (socket says 172.21.15.2:55042) 2023-06-23T18:28:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640dc60700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbdc80 0x5651ebcc1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55376/0 (socket says 172.21.15.2:55376) 2023-06-23T18:28:22.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640e461700 1 -- 172.21.15.2:0/2858020088 learned_addr learned my addr 172.21.15.2:0/2858020088 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640d45f700 1 -- 172.21.15.2:0/2858020088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbdc80 msgr2=0x5651ebcc1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640d45f700 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbdc80 0x5651ebcc1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640d45f700 1 -- 172.21.15.2:0/2858020088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 msgr2=0x5651ebcce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640d45f700 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640d45f700 1 -- 172.21.15.2:0/2858020088 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651eba3cde0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640e461700 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.310+0000 7f640dc60700 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbdc80 0x5651ebcc1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.311+0000 7f640d45f700 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc2400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f63f80976a0 tx=0x7f63f8005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6d9b143a0cbb60 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.311+0000 7f63fcff9700 1 -- 172.21.15.2:0/2858020088 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8099800 con 0x5651ebcbe5d0 2023-06-23T18:28:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.311+0000 7f63fcff9700 1 -- 172.21.15.2:0/2858020088 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63f80a3560 con 0x5651ebcbe5d0 2023-06-23T18:28:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.311+0000 7f63fcff9700 1 -- 172.21.15.2:0/2858020088 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f809a1d0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 -- 172.21.15.2:0/2858020088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 msgr2=0x5651ebcc2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc2400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f63f80976a0 tx=0x7f63f8005c70 comp rx=0 tx=0).stop 2023-06-23T18:28:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 -- 172.21.15.2:0/2858020088 shutdown_connections 2023-06-23T18:28:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebcce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbdc80 0x5651ebcc1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 --2- 172.21.15.2:0/2858020088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbe5d0 0x5651ebcc2400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 -- 172.21.15.2:0/2858020088 >> 172.21.15.2:0/2858020088 conn(0x5651ebcb33f0 msgr2=0x5651ebcd97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:22.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 -- 172.21.15.2:0/2858020088 shutdown_connections 2023-06-23T18:28:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 -- 172.21.15.2:0/2858020088 wait complete. 2023-06-23T18:28:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 -- start start 2023-06-23T18:28:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.312+0000 7f641c049b80 1 Processor -- start 2023-06-23T18:28:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f641c049b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcca560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f641c049b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebccaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f641c049b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebccdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f641c049b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebcce000 con 0x5651ebcbf000 2023-06-23T18:28:22.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f641c049b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebcce170 con 0x5651ebcbdc80 2023-06-23T18:28:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f641c049b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651ebcce330 con 0x5651ebcbe5d0 2023-06-23T18:28:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f641c049b80 10 client.0 _ll_register_callbacks cb 0x5651ebbb4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:28:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebccaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640dc60700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebccaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55384/0 (socket says 172.21.15.2:55384) 2023-06-23T18:28:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 -- 172.21.15.2:0/3029446448 learned_addr learned my addr 172.21.15.2:0/3029446448 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640dc60700 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcca560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45636/0 (socket says 172.21.15.2:45636) 2023-06-23T18:28:22.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640e461700 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebccdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 -- 172.21.15.2:0/3029446448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 msgr2=0x5651ebcca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcca560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 -- 172.21.15.2:0/3029446448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 msgr2=0x5651ebccdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebccdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 -- 172.21.15.2:0/3029446448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640401d710 con 0x5651ebcbe5d0 2023-06-23T18:28:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f640d45f700 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebccaaa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63f8097350 tx=0x7f63f8009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e013d9bdee2ca142 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:22.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.313+0000 7f63f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:28:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f63f17fa700 1 -- 172.21.15.2:0/3029446448 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8096f90 con 0x5651ebcbe5d0 2023-06-23T18:28:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f63f17fa700 1 -- 172.21.15.2:0/3029446448 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f63f809b1a0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 1 -- 172.21.15.2:0/3029446448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63f809f1d0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f63f17fa700 1 -- 172.21.15.2:0/3029446448 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63f800b8c0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 10 client.25561 fetch_fsmap learned FSMap version 30 2023-06-23T18:28:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 1 -- 172.21.15.2:0/3029446448 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f640000e6d0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f63f17fa700 1 -- 172.21.15.2:0/3029446448 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63f809f1d0 con 0x5651ebcbe5d0 2023-06-23T18:28:22.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f63f17fa700 1 -- 172.21.15.2:0/3029446448 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f63f809c180 con 0x5651ebcbe5d0 2023-06-23T18:28:22.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 10 client.25561 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:28:22.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:28:22.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 20 client.25561 populate_metadata read hostname 'smithi002' 2023-06-23T18:28:22.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5651ebd273c0 0x5651ebd297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:22.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 4 client.25561 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:28:22.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f641c049b80 1 -- 172.21.15.2:0/3029446448 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5651ebbca050 con 0x5651ebd273c0 2023-06-23T18:28:22.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.314+0000 7f640dc60700 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5651ebd273c0 0x5651ebd297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:22.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f63f17fa700 10 client.25561 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:28:22.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f640dc60700 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5651ebd273c0 0x5651ebd297d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1ffcb5bf9043414 server_cookie=ca3fd7d8a1eb8e07 in_seq=0 out_seq=0 2023-06-23T18:28:22.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f63f17fa700 1 -- 172.21.15.2:0/3029446448 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5651ebbca050 con 0x5651ebd273c0 2023-06-23T18:28:22.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f63f17fa700 10 client.25561 handle_command_reply: tid=0 2023-06-23T18:28:22.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 client.25561 shutdown 2023-06-23T18:28:22.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 -- 172.21.15.2:0/3029446448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 msgr2=0x5651ebccaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebccaaa0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f63f8097350 tx=0x7f63f8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 -- 172.21.15.2:0/3029446448 shutdown_connections 2023-06-23T18:28:22.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5651ebd273c0 0x5651ebd297d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5651ebcbf000 0x5651ebccdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f63f17fa700 0 client.25561 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:28:22.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5651ebcbdc80 0x5651ebcca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 --2- 172.21.15.2:0/3029446448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5651ebcbe5d0 0x5651ebccaaa0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 -- 172.21.15.2:0/3029446448 >> 172.21.15.2:0/3029446448 conn(0x5651ebcb33f0 msgr2=0x5651ebcc1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:22.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 -- 172.21.15.2:0/3029446448 shutdown_connections 2023-06-23T18:28:22.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.315+0000 7f641c049b80 1 -- 172.21.15.2:0/3029446448 wait complete. 2023-06-23T18:28:22.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f641c049b80 20 client.25561 trim_cache size 0 max 16384 2023-06-23T18:28:22.330 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:28:22.330 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:28:22.330 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:28:22.330 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:28:22.330 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:28:22.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:28:22.331 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:28:22.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:28:22.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:28:22.331 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:28:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 -- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63f40a11d0 msgr2=0x7f63f40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63f40a11d0 0x7f63f40a3680 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f64080ffad0 tx=0x7f6404009260 comp rx=0 tx=0).stop 2023-06-23T18:28:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 -- 172.21.15.2:0/1367501193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64080ffc70 msgr2=0x7f6408101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64080ffc70 0x7f6408101350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f63f8000c00 tx=0x7f63f8005740 comp rx=0 tx=0).stop 2023-06-23T18:28:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 -- 172.21.15.2:0/1367501193 shutdown_connections 2023-06-23T18:28:22.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6408100600 0x7f6408101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f63f40a11d0 0x7f63f40a3680 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64080fea80 0x7f6408102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 --2- 172.21.15.2:0/1367501193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64080ffc70 0x7f6408101350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 -- 172.21.15.2:0/1367501193 >> 172.21.15.2:0/1367501193 conn(0x7f64080fa0e0 msgr2=0x7f64080fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 -- 172.21.15.2:0/1367501193 shutdown_connections 2023-06-23T18:28:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:22.316+0000 7f640fec4700 1 -- 172.21.15.2:0/1367501193 wait complete. 2023-06-23T18:28:22.334 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:28:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:22 smithi002 ceph-mon[95947]: pgmap v486: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 37 MiB/s wr, 27 op/s 2023-06-23T18:28:22.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:22 smithi082 ceph-mon[97889]: pgmap v486: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 37 MiB/s wr, 27 op/s 2023-06-23T18:28:22.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:22 smithi017 ceph-mon[97472]: pgmap v486: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 37 MiB/s wr, 27 op/s 2023-06-23T18:28:23.115 INFO:teuthology.orchestra.run.smithi002.stderr:37300000 of 50000000 tuples (74%) done (elapsed 346.36 s, remaining 117.93 s) 2023-06-23T18:28:23.983 INFO:teuthology.orchestra.run.smithi002.stderr:37400000 of 50000000 tuples (74%) done (elapsed 347.22 s, remaining 116.98 s) 2023-06-23T18:28:24.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:24 smithi002 ceph-mon[95947]: pgmap v487: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 44 MiB/s wr, 28 op/s 2023-06-23T18:28:24.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:24 smithi082 ceph-mon[97889]: pgmap v487: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 44 MiB/s wr, 28 op/s 2023-06-23T18:28:24.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:24 smithi017 ceph-mon[97472]: pgmap v487: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 44 MiB/s wr, 28 op/s 2023-06-23T18:28:24.854 INFO:teuthology.orchestra.run.smithi002.stderr:37500000 of 50000000 tuples (75%) done (elapsed 348.10 s, remaining 116.03 s) 2023-06-23T18:28:25.702 INFO:teuthology.orchestra.run.smithi002.stderr:37600000 of 50000000 tuples (75%) done (elapsed 348.94 s, remaining 115.08 s) 2023-06-23T18:28:26.509 INFO:teuthology.orchestra.run.smithi002.stderr:37700000 of 50000000 tuples (75%) done (elapsed 349.75 s, remaining 114.11 s) 2023-06-23T18:28:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:26 smithi002 ceph-mon[95947]: pgmap v488: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 41 MiB/s wr, 46 op/s 2023-06-23T18:28:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:26 smithi082 ceph-mon[97889]: pgmap v488: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 41 MiB/s wr, 46 op/s 2023-06-23T18:28:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:26 smithi017 ceph-mon[97472]: pgmap v488: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 41 MiB/s wr, 46 op/s 2023-06-23T18:28:27.400 INFO:teuthology.orchestra.run.smithi002.stderr:37800000 of 50000000 tuples (75%) done (elapsed 350.64 s, remaining 113.17 s) 2023-06-23T18:28:28.204 INFO:teuthology.orchestra.run.smithi002.stderr:37900000 of 50000000 tuples (75%) done (elapsed 351.45 s, remaining 112.20 s) 2023-06-23T18:28:28.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:28 smithi002 ceph-mon[95947]: pgmap v489: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 41 MiB/s wr, 44 op/s 2023-06-23T18:28:28.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:28 smithi082 ceph-mon[97889]: pgmap v489: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 41 MiB/s wr, 44 op/s 2023-06-23T18:28:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:28 smithi017 ceph-mon[97472]: pgmap v489: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 41 MiB/s wr, 44 op/s 2023-06-23T18:28:29.029 INFO:teuthology.orchestra.run.smithi002.stderr:38000000 of 50000000 tuples (76%) done (elapsed 352.27 s, remaining 111.24 s) 2023-06-23T18:28:29.845 INFO:teuthology.orchestra.run.smithi002.stderr:38100000 of 50000000 tuples (76%) done (elapsed 353.09 s, remaining 110.28 s) 2023-06-23T18:28:30.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:30 smithi002 ceph-mon[95947]: pgmap v490: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 44 MiB/s wr, 46 op/s 2023-06-23T18:28:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:30 smithi082 ceph-mon[97889]: pgmap v490: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 44 MiB/s wr, 46 op/s 2023-06-23T18:28:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:30 smithi017 ceph-mon[97472]: pgmap v490: 97 pgs: 97 active+clean; 5.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 44 MiB/s wr, 46 op/s 2023-06-23T18:28:30.715 INFO:teuthology.orchestra.run.smithi002.stderr:38200000 of 50000000 tuples (76%) done (elapsed 353.96 s, remaining 109.34 s) 2023-06-23T18:28:31.542 INFO:teuthology.orchestra.run.smithi002.stderr:38300000 of 50000000 tuples (76%) done (elapsed 354.78 s, remaining 108.38 s) 2023-06-23T18:28:32.356 INFO:teuthology.orchestra.run.smithi002.stderr:38400000 of 50000000 tuples (76%) done (elapsed 355.60 s, remaining 107.42 s) 2023-06-23T18:28:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:32 smithi002 ceph-mon[95947]: pgmap v491: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 45 op/s 2023-06-23T18:28:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:32 smithi082 ceph-mon[97889]: pgmap v491: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 45 op/s 2023-06-23T18:28:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:32 smithi017 ceph-mon[97472]: pgmap v491: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 45 op/s 2023-06-23T18:28:33.240 INFO:teuthology.orchestra.run.smithi002.stderr:38500000 of 50000000 tuples (77%) done (elapsed 356.48 s, remaining 106.48 s) 2023-06-23T18:28:34.089 INFO:teuthology.orchestra.run.smithi002.stderr:38600000 of 50000000 tuples (77%) done (elapsed 357.33 s, remaining 105.53 s) 2023-06-23T18:28:34.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:34 smithi002 ceph-mon[95947]: pgmap v492: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 47 op/s 2023-06-23T18:28:34.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:34 smithi082 ceph-mon[97889]: pgmap v492: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 47 op/s 2023-06-23T18:28:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:34 smithi017 ceph-mon[97472]: pgmap v492: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 45 MiB/s wr, 47 op/s 2023-06-23T18:28:34.929 INFO:teuthology.orchestra.run.smithi002.stderr:38700000 of 50000000 tuples (77%) done (elapsed 358.17 s, remaining 104.58 s) 2023-06-23T18:28:35.747 INFO:teuthology.orchestra.run.smithi002.stderr:38800000 of 50000000 tuples (77%) done (elapsed 358.99 s, remaining 103.63 s) 2023-06-23T18:28:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:36 smithi002 ceph-mon[95947]: pgmap v493: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 45 op/s 2023-06-23T18:28:36.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:36 smithi082 ceph-mon[97889]: pgmap v493: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 45 op/s 2023-06-23T18:28:36.635 INFO:teuthology.orchestra.run.smithi002.stderr:38900000 of 50000000 tuples (77%) done (elapsed 359.88 s, remaining 102.69 s) 2023-06-23T18:28:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:36 smithi017 ceph-mon[97472]: pgmap v493: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 40 MiB/s wr, 45 op/s 2023-06-23T18:28:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:39000000 of 50000000 tuples (78%) done (elapsed 360.71 s, remaining 101.74 s) 2023-06-23T18:28:38.332 INFO:teuthology.orchestra.run.smithi002.stderr:39100000 of 50000000 tuples (78%) done (elapsed 361.57 s, remaining 100.80 s) 2023-06-23T18:28:38.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:38 smithi002 ceph-mon[95947]: pgmap v494: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 41 MiB/s wr, 22 op/s 2023-06-23T18:28:38.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:38 smithi082 ceph-mon[97889]: pgmap v494: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 41 MiB/s wr, 22 op/s 2023-06-23T18:28:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:38 smithi017 ceph-mon[97472]: pgmap v494: 97 pgs: 97 active+clean; 5.8 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 41 MiB/s wr, 22 op/s 2023-06-23T18:28:39.152 INFO:teuthology.orchestra.run.smithi002.stderr:39200000 of 50000000 tuples (78%) done (elapsed 362.39 s, remaining 99.84 s) 2023-06-23T18:28:40.020 INFO:teuthology.orchestra.run.smithi002.stderr:39300000 of 50000000 tuples (78%) done (elapsed 363.26 s, remaining 98.90 s) 2023-06-23T18:28:40.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:40 smithi002 ceph-mon[95947]: pgmap v495: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 44 MiB/s wr, 25 op/s 2023-06-23T18:28:40.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:40 smithi082 ceph-mon[97889]: pgmap v495: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 44 MiB/s wr, 25 op/s 2023-06-23T18:28:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:40 smithi017 ceph-mon[97472]: pgmap v495: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 44 MiB/s wr, 25 op/s 2023-06-23T18:28:40.930 INFO:teuthology.orchestra.run.smithi002.stderr:39400000 of 50000000 tuples (78%) done (elapsed 364.17 s, remaining 97.98 s) 2023-06-23T18:28:41.774 INFO:teuthology.orchestra.run.smithi002.stderr:39500000 of 50000000 tuples (79%) done (elapsed 365.02 s, remaining 97.03 s) 2023-06-23T18:28:42.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:42 smithi002 ceph-mon[95947]: pgmap v496: 97 pgs: 97 active+clean; 5.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:28:42.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:42 smithi082 ceph-mon[97889]: pgmap v496: 97 pgs: 97 active+clean; 5.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:28:42.689 INFO:teuthology.orchestra.run.smithi002.stderr:39600000 of 50000000 tuples (79%) done (elapsed 365.93 s, remaining 96.10 s) 2023-06-23T18:28:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:42 smithi017 ceph-mon[97472]: pgmap v496: 97 pgs: 97 active+clean; 5.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 39 MiB/s wr, 23 op/s 2023-06-23T18:28:43.515 INFO:teuthology.orchestra.run.smithi002.stderr:39700000 of 50000000 tuples (79%) done (elapsed 366.76 s, remaining 95.15 s) 2023-06-23T18:28:44.376 INFO:teuthology.orchestra.run.smithi002.stderr:39800000 of 50000000 tuples (79%) done (elapsed 367.62 s, remaining 94.21 s) 2023-06-23T18:28:44.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:44 smithi002 ceph-mon[95947]: pgmap v497: 97 pgs: 97 active+clean; 5.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:44.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:44 smithi082 ceph-mon[97889]: pgmap v497: 97 pgs: 97 active+clean; 5.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:44 smithi017 ceph-mon[97472]: pgmap v497: 97 pgs: 97 active+clean; 5.9 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:45.257 INFO:teuthology.orchestra.run.smithi002.stderr:39900000 of 50000000 tuples (79%) done (elapsed 368.50 s, remaining 93.28 s) 2023-06-23T18:28:46.100 INFO:teuthology.orchestra.run.smithi002.stderr:40000000 of 50000000 tuples (80%) done (elapsed 369.34 s, remaining 92.34 s) 2023-06-23T18:28:46.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:46 smithi002 ceph-mon[95947]: pgmap v498: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 42 MiB/s wr, 25 op/s 2023-06-23T18:28:46.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:46 smithi082 ceph-mon[97889]: pgmap v498: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 42 MiB/s wr, 25 op/s 2023-06-23T18:28:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:46 smithi017 ceph-mon[97472]: pgmap v498: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 42 MiB/s wr, 25 op/s 2023-06-23T18:28:46.934 INFO:teuthology.orchestra.run.smithi002.stderr:40100000 of 50000000 tuples (80%) done (elapsed 370.17 s, remaining 91.39 s) 2023-06-23T18:28:47.807 INFO:teuthology.orchestra.run.smithi002.stderr:40200000 of 50000000 tuples (80%) done (elapsed 371.05 s, remaining 90.45 s) 2023-06-23T18:28:48.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:48 smithi002 ceph-mon[95947]: pgmap v499: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:28:48.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:48 smithi082 ceph-mon[97889]: pgmap v499: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:28:48.649 INFO:teuthology.orchestra.run.smithi002.stderr:40300000 of 50000000 tuples (80%) done (elapsed 371.89 s, remaining 89.51 s) 2023-06-23T18:28:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:48 smithi017 ceph-mon[97472]: pgmap v499: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:28:49.473 INFO:teuthology.orchestra.run.smithi002.stderr:40400000 of 50000000 tuples (80%) done (elapsed 372.71 s, remaining 88.57 s) 2023-06-23T18:28:50.288 INFO:teuthology.orchestra.run.smithi002.stderr:40500000 of 50000000 tuples (81%) done (elapsed 373.53 s, remaining 87.62 s) 2023-06-23T18:28:50.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:50 smithi002 ceph-mon[95947]: pgmap v500: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:28:50.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:50 smithi082 ceph-mon[97889]: pgmap v500: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:28:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:50 smithi017 ceph-mon[97472]: pgmap v500: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:28:51.204 INFO:teuthology.orchestra.run.smithi002.stderr:40600000 of 50000000 tuples (81%) done (elapsed 374.44 s, remaining 86.69 s) 2023-06-23T18:28:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:40700000 of 50000000 tuples (81%) done (elapsed 375.25 s, remaining 85.75 s) 2023-06-23T18:28:52.335 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:28:52.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 -- 172.21.15.2:0/1827377503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 msgr2=0x7fe9f40ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/1827377503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 0x7fe9f40ff100 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009990 tx=0x7fe9e0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 -- 172.21.15.2:0/1827377503 shutdown_connections 2023-06-23T18:28:52.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/1827377503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f40ff640 0x7fe9f4106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/1827377503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f4100040 0x7fe9f4100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/1827377503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 0x7fe9f40ff100 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 -- 172.21.15.2:0/1827377503 >> 172.21.15.2:0/1827377503 conn(0x7fe9f40f9850 msgr2=0x7fe9f40fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:52.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 -- 172.21.15.2:0/1827377503 shutdown_connections 2023-06-23T18:28:52.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.474+0000 7fe9faa8b700 1 -- 172.21.15.2:0/1827377503 wait complete. 2023-06-23T18:28:52.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9faa8b700 1 Processor -- start 2023-06-23T18:28:52.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9faa8b700 1 -- start start 2023-06-23T18:28:52.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9faa8b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 0x7fe9f419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9faa8b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f40ff640 0x7fe9f419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9faa8b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f4100040 0x7fe9f41a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9faa8b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f41a39c0 con 0x7fe9f40ff640 2023-06-23T18:28:52.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9faa8b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f41a3b00 con 0x7fe9f4100040 2023-06-23T18:28:52.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9faa8b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f41a3c90 con 0x7fe9f40fec90 2023-06-23T18:28:52.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.475+0000 7fe9f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f40ff640 0x7fe9f419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 0x7fe9f419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 0x7fe9f419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55394/0 (socket says 172.21.15.2:55394) 2023-06-23T18:28:52.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 -- 172.21.15.2:0/3563651046 learned_addr learned my addr 172.21.15.2:0/3563651046 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:52.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f9028700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f4100040 0x7fe9f41a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 -- 172.21.15.2:0/3563651046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f4100040 msgr2=0x7fe9f41a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f4100040 0x7fe9f41a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 -- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f40ff640 msgr2=0x7fe9f419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f40ff640 0x7fe9f419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 -- 172.21.15.2:0/3563651046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e00095f0 con 0x7fe9f40fec90 2023-06-23T18:28:52.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f8827700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 0x7fe9f419be40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800d980 tx=0x7fe9e800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:52.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f1ffb700 1 -- 172.21.15.2:0/3563651046 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8004220 con 0x7fe9f40fec90 2023-06-23T18:28:52.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f1ffb700 1 -- 172.21.15.2:0/3563651046 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9e8007640 con 0x7fe9f40fec90 2023-06-23T18:28:52.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9faa8b700 1 -- 172.21.15.2:0/3563651046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f41a3f70 con 0x7fe9f40fec90 2023-06-23T18:28:52.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.476+0000 7fe9f1ffb700 1 -- 172.21.15.2:0/3563651046 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8005660 con 0x7fe9f40fec90 2023-06-23T18:28:52.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.477+0000 7fe9faa8b700 1 -- 172.21.15.2:0/3563651046 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f41a4390 con 0x7fe9f40fec90 2023-06-23T18:28:52.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.479+0000 7fe9f1ffb700 1 -- 172.21.15.2:0/3563651046 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe9e8004750 con 0x7fe9f40fec90 2023-06-23T18:28:52.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.480+0000 7fe9f1ffb700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9e40a1130 0x7fe9e40a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.480+0000 7fe9f1ffb700 1 -- 172.21.15.2:0/3563651046 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe9e80cc5a0 con 0x7fe9f40fec90 2023-06-23T18:28:52.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.480+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9e40a1130 0x7fe9e40a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.480+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9e40a1130 0x7fe9e40a35e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0006010 tx=0x7fe9e000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:52.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fea06c10b80 1 Processor -- start 2023-06-23T18:28:52.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fea06c10b80 1 -- start start 2023-06-23T18:28:52.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a339090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a339a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fea06c10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a23dcb0 con 0x561f1a3395d0 2023-06-23T18:28:52.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fea06c10b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a290a40 con 0x561f1a33a000 2023-06-23T18:28:52.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fea06c10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a289ed0 con 0x561f1a338c80 2023-06-23T18:28:52.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fe9f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a339a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fe9f8827700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a339090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fe9f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a339a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55066/0 (socket says 172.21.15.2:55066) 2023-06-23T18:28:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fe9f3fff700 1 -- 172.21.15.2:0/2554738248 learned_addr learned my addr 172.21.15.2:0/2554738248 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.481+0000 7fe9f9028700 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9f3fff700 1 -- 172.21.15.2:0/2554738248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 msgr2=0x561f1a339090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a339090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9f3fff700 1 -- 172.21.15.2:0/2554738248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 msgr2=0x561f1a33a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9f3fff700 1 -- 172.21.15.2:0/2554738248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1a10b630 con 0x561f1a3395d0 2023-06-23T18:28:52.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9f9028700 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:52.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9f8827700 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a339090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:52.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a339a70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0015000 tx=0x7fe9e0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8217d1a24b2acfd0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:52.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9deffd700 1 -- 172.21.15.2:0/2554738248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0003d10 con 0x561f1a3395d0 2023-06-23T18:28:52.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9deffd700 1 -- 172.21.15.2:0/2554738248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9e001e040 con 0x561f1a3395d0 2023-06-23T18:28:52.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.482+0000 7fe9deffd700 1 -- 172.21.15.2:0/2554738248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0004880 con 0x561f1a3395d0 2023-06-23T18:28:52.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 -- 172.21.15.2:0/2554738248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 msgr2=0x561f1a339a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a339a70 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0015000 tx=0x7fe9e0020000 comp rx=0 tx=0).stop 2023-06-23T18:28:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 -- 172.21.15.2:0/2554738248 shutdown_connections 2023-06-23T18:28:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a339090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 --2- 172.21.15.2:0/2554738248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a339a70 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 -- 172.21.15.2:0/2554738248 >> 172.21.15.2:0/2554738248 conn(0x561f1a32e3f0 msgr2=0x561f1a333520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 -- 172.21.15.2:0/2554738248 shutdown_connections 2023-06-23T18:28:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 -- 172.21.15.2:0/2554738248 wait complete. 2023-06-23T18:28:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 -- start start 2023-06-23T18:28:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.483+0000 7fea06c10b80 1 Processor -- start 2023-06-23T18:28:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a338c80 0x561f1a342530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a342a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fea06c10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a229ac0 con 0x561f1a3395d0 2023-06-23T18:28:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fea06c10b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a2297c0 con 0x561f1a338c80 2023-06-23T18:28:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fea06c10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a10d920 con 0x561f1a33a000 2023-06-23T18:28:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fea06c10b80 10 client.0 _ll_register_callbacks cb 0x561f1a22f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:28:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a342a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f8827700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a338c80 0x561f1a342530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a342a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55072/0 (socket says 172.21.15.2:55072) 2023-06-23T18:28:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f3fff700 1 -- 172.21.15.2:0/1583122228 learned_addr learned my addr 172.21.15.2:0/1583122228 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f9028700 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f8827700 1 -- 172.21.15.2:0/1583122228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 msgr2=0x561f1a349ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:52.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f8827700 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f8827700 1 -- 172.21.15.2:0/1583122228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 msgr2=0x561f1a342a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:52.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f8827700 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a342a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f8827700 1 -- 172.21.15.2:0/1583122228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec011230 con 0x561f1a338c80 2023-06-23T18:28:52.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9f8827700 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a338c80 0x561f1a342530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80ca0a0 tx=0x7fe9e8004380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b331dae44f79ff7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:52.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.484+0000 7fe9d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:28:52.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fe9d3fff700 1 -- 172.21.15.2:0/1583122228 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80990a0 con 0x561f1a338c80 2023-06-23T18:28:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 1 -- 172.21.15.2:0/1583122228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561f1a32bbb0 con 0x561f1a338c80 2023-06-23T18:28:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fe9d3fff700 1 -- 172.21.15.2:0/1583122228 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe9e80999b0 con 0x561f1a338c80 2023-06-23T18:28:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fe9d3fff700 1 -- 172.21.15.2:0/1583122228 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9e809d9e0 con 0x561f1a338c80 2023-06-23T18:28:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 10 client.25418 fetch_fsmap learned FSMap version 30 2023-06-23T18:28:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 1 -- 172.21.15.2:0/1583122228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fe9f41077e0 con 0x561f1a338c80 2023-06-23T18:28:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fe9d3fff700 1 -- 172.21.15.2:0/1583122228 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561f1a32bbb0 con 0x561f1a338c80 2023-06-23T18:28:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fe9d3fff700 1 -- 172.21.15.2:0/1583122228 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe9e80957a0 con 0x561f1a338c80 2023-06-23T18:28:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 10 client.25418 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:28:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 10 client.25418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:28:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 20 client.25418 populate_metadata read hostname 'smithi002' 2023-06-23T18:28:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561f1a34e1d0 0x561f1a3505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 4 client.25418 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:28:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.485+0000 7fea06c10b80 1 -- 172.21.15.2:0/1583122228 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561f1a282eb0 con 0x561f1a34e1d0 2023-06-23T18:28:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.486+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561f1a34e1d0 0x561f1a3505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.486+0000 7fe9d3fff700 10 client.25418 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:28:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.486+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561f1a34e1d0 0x561f1a3505e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7fe59bc57a52ad server_cookie=d4f670b7aaefb21 in_seq=0 out_seq=0 2023-06-23T18:28:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fe9d3fff700 1 -- 172.21.15.2:0/1583122228 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561f1a282eb0 con 0x561f1a34e1d0 2023-06-23T18:28:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fe9d3fff700 10 client.25418 handle_command_reply: tid=0 2023-06-23T18:28:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 client.25418 shutdown 2023-06-23T18:28:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 -- 172.21.15.2:0/1583122228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a338c80 msgr2=0x561f1a342530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a338c80 0x561f1a342530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80ca0a0 tx=0x7fe9e8004380 comp rx=0 tx=0).stop 2023-06-23T18:28:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 -- 172.21.15.2:0/1583122228 shutdown_connections 2023-06-23T18:28:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561f1a34e1d0 0x561f1a3505e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a338c80 0x561f1a342530 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fe9d3fff700 0 client.25418 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:28:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 --2- 172.21.15.2:0/1583122228 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a342a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 -- 172.21.15.2:0/1583122228 >> 172.21.15.2:0/1583122228 conn(0x561f1a32e3f0 msgr2=0x561f1a332c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 -- 172.21.15.2:0/1583122228 shutdown_connections 2023-06-23T18:28:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 1 -- 172.21.15.2:0/1583122228 wait complete. 2023-06-23T18:28:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.487+0000 7fea06c10b80 20 client.25418 trim_cache size 0 max 16384 2023-06-23T18:28:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.508+0000 7fea06c10b80 1 Processor -- start 2023-06-23T18:28:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.508+0000 7fea06c10b80 1 -- start start 2023-06-23T18:28:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a33cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a349a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fea06c10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a343720 con 0x561f1a3395d0 2023-06-23T18:28:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fea06c10b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a3419e0 con 0x561f1a33a000 2023-06-23T18:28:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fea06c10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a34df70 con 0x561f1a338c80 2023-06-23T18:28:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a349a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a349a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55080/0 (socket says 172.21.15.2:55080) 2023-06-23T18:28:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f3fff700 1 -- 172.21.15.2:0/3414150227 learned_addr learned my addr 172.21.15.2:0/3414150227 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:52.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f8827700 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a33cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f9028700 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f8827700 1 -- 172.21.15.2:0/3414150227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 msgr2=0x561f1a33d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f8827700 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f8827700 1 -- 172.21.15.2:0/3414150227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 msgr2=0x561f1a349a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f8827700 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a349a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f8827700 1 -- 172.21.15.2:0/3414150227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561f1a0b7de0 con 0x561f1a338c80 2023-06-23T18:28:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9f8827700 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a33cf20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8095eb0 tx=0x7fe9e8097f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3849fe2da275c056 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9df7fe700 1 -- 172.21.15.2:0/3414150227 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80999b0 con 0x561f1a338c80 2023-06-23T18:28:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9df7fe700 1 -- 172.21.15.2:0/3414150227 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9e809d9e0 con 0x561f1a338c80 2023-06-23T18:28:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.509+0000 7fe9df7fe700 1 -- 172.21.15.2:0/3414150227 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80a1970 con 0x561f1a338c80 2023-06-23T18:28:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 -- 172.21.15.2:0/3414150227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 msgr2=0x561f1a33cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a33cf20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8095eb0 tx=0x7fe9e8097f60 comp rx=0 tx=0).stop 2023-06-23T18:28:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 -- 172.21.15.2:0/3414150227 shutdown_connections 2023-06-23T18:28:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a33a000 0x561f1a33d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a338c80 0x561f1a33cf20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 --2- 172.21.15.2:0/3414150227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a3395d0 0x561f1a349a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 -- 172.21.15.2:0/3414150227 >> 172.21.15.2:0/3414150227 conn(0x561f1a32e3f0 msgr2=0x561f1a3411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 -- 172.21.15.2:0/3414150227 shutdown_connections 2023-06-23T18:28:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.510+0000 7fea06c10b80 1 -- 172.21.15.2:0/3414150227 wait complete. 2023-06-23T18:28:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 1 -- start start 2023-06-23T18:28:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 1 Processor -- start 2023-06-23T18:28:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a338c80 0x561f1a345c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a3395d0 0x561f1a346170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a3496d0 con 0x561f1a338c80 2023-06-23T18:28:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a349840 con 0x561f1a3395d0 2023-06-23T18:28:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561f1a349a00 con 0x561f1a33a000 2023-06-23T18:28:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fea06c10b80 10 client.0 _ll_register_callbacks cb 0x561f1a22f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:28:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fe9f8827700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a338c80 0x561f1a345c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fe9f3fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a3395d0 0x561f1a346170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fe9f8827700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a338c80 0x561f1a345c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55084/0 (socket says 172.21.15.2:55084) 2023-06-23T18:28:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fe9f8827700 1 -- 172.21.15.2:0/4258293196 learned_addr learned my addr 172.21.15.2:0/4258293196 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:28:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.511+0000 7fe9f9028700 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9f8827700 1 -- 172.21.15.2:0/4258293196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 msgr2=0x561f1a349190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9f8827700 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9f8827700 1 -- 172.21.15.2:0/4258293196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a3395d0 msgr2=0x561f1a346170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:28:52.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9f8827700 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a3395d0 0x561f1a346170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9f8827700 1 -- 172.21.15.2:0/4258293196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f41077e0 con 0x561f1a338c80 2023-06-23T18:28:52.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a3395d0 0x561f1a346170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:52.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9f9028700 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:28:52.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9f8827700 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a338c80 0x561f1a345c30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80a18d0 tx=0x7fe9e80b2290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839b492b015679f8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:28:52.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:28:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9d3fff700 1 -- 172.21.15.2:0/4258293196 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80999b0 con 0x561f1a338c80 2023-06-23T18:28:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9d3fff700 1 -- 172.21.15.2:0/4258293196 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe9e80a1970 con 0x561f1a338c80 2023-06-23T18:28:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fea06c10b80 1 -- 172.21.15.2:0/4258293196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9e001e040 con 0x561f1a338c80 2023-06-23T18:28:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9d3fff700 1 -- 172.21.15.2:0/4258293196 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9e80a1b30 con 0x561f1a338c80 2023-06-23T18:28:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fea06c10b80 10 client.15822 fetch_fsmap learned FSMap version 30 2023-06-23T18:28:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fea06c10b80 1 -- 172.21.15.2:0/4258293196 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561f1a0a0c60 con 0x561f1a338c80 2023-06-23T18:28:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.512+0000 7fe9d3fff700 1 -- 172.21.15.2:0/4258293196 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9e001e040 con 0x561f1a338c80 2023-06-23T18:28:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fe9d3fff700 1 -- 172.21.15.2:0/4258293196 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe9e8004a00 con 0x561f1a338c80 2023-06-23T18:28:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fea06c10b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:28:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fea06c10b80 10 client.15822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:28:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fea06c10b80 20 client.15822 populate_metadata read hostname 'smithi002' 2023-06-23T18:28:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fea06c10b80 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561f1a3a2f50 0x561f1a3a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:28:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fea06c10b80 4 client.15822 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:28:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fea06c10b80 1 -- 172.21.15.2:0/4258293196 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561f1a245050 con 0x561f1a3a2f50 2023-06-23T18:28:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561f1a3a2f50 0x561f1a3a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:28:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fe9d3fff700 10 client.15822 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:28:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.513+0000 7fe9f3fff700 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561f1a3a2f50 0x561f1a3a5330 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a379ebaaa4e0003 server_cookie=d52d0fecae304caa in_seq=0 out_seq=0 2023-06-23T18:28:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fe9d3fff700 1 -- 172.21.15.2:0/4258293196 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x561f1a245050 con 0x561f1a3a2f50 2023-06-23T18:28:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fe9d3fff700 10 client.15822 handle_command_reply: tid=0 2023-06-23T18:28:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 client.15822 shutdown 2023-06-23T18:28:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 -- 172.21.15.2:0/4258293196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a338c80 msgr2=0x561f1a345c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a338c80 0x561f1a345c30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80a18d0 tx=0x7fe9e80b2290 comp rx=0 tx=0).stop 2023-06-23T18:28:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 -- 172.21.15.2:0/4258293196 shutdown_connections 2023-06-23T18:28:52.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561f1a3a2f50 0x561f1a3a5330 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561f1a33a000 0x561f1a349190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561f1a338c80 0x561f1a345c30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fe9d3fff700 0 client.15822 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:28:52.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 --2- 172.21.15.2:0/4258293196 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561f1a3395d0 0x561f1a346170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 -- 172.21.15.2:0/4258293196 >> 172.21.15.2:0/4258293196 conn(0x561f1a32e3f0 msgr2=0x561f1a33cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:52.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 -- 172.21.15.2:0/4258293196 shutdown_connections 2023-06-23T18:28:52.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 1 -- 172.21.15.2:0/4258293196 wait complete. 2023-06-23T18:28:52.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.514+0000 7fea06c10b80 20 client.15822 trim_cache size 0 max 16384 2023-06-23T18:28:52.529 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:28:52.529 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:28:52.530 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:28:52.530 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:28:52.530 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:28:52.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:28:52.530 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:28:52.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:28:52.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:28:52.530 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:28:52.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 -- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9e40a1130 msgr2=0x7fe9e40a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9e40a1130 0x7fe9e40a35e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0006010 tx=0x7fe9e000b260 comp rx=0 tx=0).stop 2023-06-23T18:28:52.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 -- 172.21.15.2:0/3563651046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 msgr2=0x7fe9f419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:28:52.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 0x7fe9f419be40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800d980 tx=0x7fe9e800dc90 comp rx=0 tx=0).stop 2023-06-23T18:28:52.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 -- 172.21.15.2:0/3563651046 shutdown_connections 2023-06-23T18:28:52.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9e40a1130 0x7fe9e40a35e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f4100040 0x7fe9f41a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f40fec90 0x7fe9f419be40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 --2- 172.21.15.2:0/3563651046 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f40ff640 0x7fe9f419c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:28:52.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 -- 172.21.15.2:0/3563651046 >> 172.21.15.2:0/3563651046 conn(0x7fe9f40f9850 msgr2=0x7fe9f410b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:28:52.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 -- 172.21.15.2:0/3563651046 shutdown_connections 2023-06-23T18:28:52.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:28:52.515+0000 7fe9faa8b700 1 -- 172.21.15.2:0/3563651046 wait complete. 2023-06-23T18:28:52.533 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:52 smithi002 ceph-mon[95947]: pgmap v501: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 23 op/s 2023-06-23T18:28:52.534 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:28:52.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:52 smithi082 ceph-mon[97889]: pgmap v501: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 23 op/s 2023-06-23T18:28:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:52 smithi017 ceph-mon[97472]: pgmap v501: 97 pgs: 97 active+clean; 6.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 38 MiB/s wr, 23 op/s 2023-06-23T18:28:52.842 INFO:teuthology.orchestra.run.smithi002.stderr:40800000 of 50000000 tuples (81%) done (elapsed 376.08 s, remaining 84.80 s) 2023-06-23T18:28:53.777 INFO:teuthology.orchestra.run.smithi002.stderr:40900000 of 50000000 tuples (81%) done (elapsed 377.02 s, remaining 83.88 s) 2023-06-23T18:28:54.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:54 smithi002 ceph-mon[95947]: pgmap v502: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:54 smithi082 ceph-mon[97889]: pgmap v502: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:54.673 INFO:teuthology.orchestra.run.smithi002.stderr:41000000 of 50000000 tuples (82%) done (elapsed 377.91 s, remaining 82.96 s) 2023-06-23T18:28:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:54 smithi017 ceph-mon[97472]: pgmap v502: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 MiB/s wr, 26 op/s 2023-06-23T18:28:55.618 INFO:teuthology.orchestra.run.smithi002.stderr:41100000 of 50000000 tuples (82%) done (elapsed 378.86 s, remaining 82.04 s) 2023-06-23T18:28:56.484 INFO:teuthology.orchestra.run.smithi002.stderr:41200000 of 50000000 tuples (82%) done (elapsed 379.73 s, remaining 81.11 s) 2023-06-23T18:28:56.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:56 smithi002 ceph-mon[95947]: pgmap v503: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:28:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:56 smithi082 ceph-mon[97889]: pgmap v503: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:28:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:56 smithi017 ceph-mon[97472]: pgmap v503: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 43 MiB/s wr, 25 op/s 2023-06-23T18:28:57.303 INFO:teuthology.orchestra.run.smithi002.stderr:41300000 of 50000000 tuples (82%) done (elapsed 380.54 s, remaining 80.16 s) 2023-06-23T18:28:58.203 INFO:teuthology.orchestra.run.smithi002.stderr:41400000 of 50000000 tuples (82%) done (elapsed 381.44 s, remaining 79.24 s) 2023-06-23T18:28:58.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:28:58 smithi002 ceph-mon[95947]: pgmap v504: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:28:58.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:28:58 smithi082 ceph-mon[97889]: pgmap v504: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:28:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:28:58 smithi017 ceph-mon[97472]: pgmap v504: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:28:59.075 INFO:teuthology.orchestra.run.smithi002.stderr:41500000 of 50000000 tuples (83%) done (elapsed 382.32 s, remaining 78.31 s) 2023-06-23T18:28:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:41600000 of 50000000 tuples (83%) done (elapsed 383.13 s, remaining 77.36 s) 2023-06-23T18:29:00.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:00 smithi002 ceph-mon[95947]: pgmap v505: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:29:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:00 smithi082 ceph-mon[97889]: pgmap v505: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:29:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:00 smithi017 ceph-mon[97472]: pgmap v505: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:29:00.823 INFO:teuthology.orchestra.run.smithi002.stderr:41700000 of 50000000 tuples (83%) done (elapsed 384.06 s, remaining 76.44 s) 2023-06-23T18:29:01.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:01 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:29:01.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:01 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:29:01.659 INFO:teuthology.orchestra.run.smithi002.stderr:41800000 of 50000000 tuples (83%) done (elapsed 384.90 s, remaining 75.51 s) 2023-06-23T18:29:01.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:01 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:29:02.262 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:02 smithi002 ceph-mon[95947]: pgmap v506: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 37 MiB/s wr, 45 op/s 2023-06-23T18:29:02.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:29:02.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:29:02.263 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:29:02.489 INFO:teuthology.orchestra.run.smithi002.stderr:41900000 of 50000000 tuples (83%) done (elapsed 385.73 s, remaining 74.57 s) 2023-06-23T18:29:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:02 smithi082 ceph-mon[97889]: pgmap v506: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 37 MiB/s wr, 45 op/s 2023-06-23T18:29:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:29:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:29:02.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:29:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:02 smithi017 ceph-mon[97472]: pgmap v506: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 37 MiB/s wr, 45 op/s 2023-06-23T18:29:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:29:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:29:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:29:03.369 INFO:teuthology.orchestra.run.smithi002.stderr:42000000 of 50000000 tuples (84%) done (elapsed 386.61 s, remaining 73.64 s) 2023-06-23T18:29:04.241 INFO:teuthology.orchestra.run.smithi002.stderr:42100000 of 50000000 tuples (84%) done (elapsed 387.48 s, remaining 72.71 s) 2023-06-23T18:29:04.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:04 smithi002 ceph-mon[95947]: pgmap v507: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 45 MiB/s wr, 47 op/s 2023-06-23T18:29:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:04 smithi082 ceph-mon[97889]: pgmap v507: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 45 MiB/s wr, 47 op/s 2023-06-23T18:29:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:04 smithi017 ceph-mon[97472]: pgmap v507: 97 pgs: 97 active+clean; 6.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 45 MiB/s wr, 47 op/s 2023-06-23T18:29:05.069 INFO:teuthology.orchestra.run.smithi002.stderr:42200000 of 50000000 tuples (84%) done (elapsed 388.31 s, remaining 71.77 s) 2023-06-23T18:29:05.977 INFO:teuthology.orchestra.run.smithi002.stderr:42300000 of 50000000 tuples (84%) done (elapsed 389.22 s, remaining 70.85 s) 2023-06-23T18:29:06.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:06 smithi002 ceph-mon[95947]: pgmap v508: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 41 MiB/s wr, 45 op/s 2023-06-23T18:29:06.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:06 smithi082 ceph-mon[97889]: pgmap v508: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 41 MiB/s wr, 45 op/s 2023-06-23T18:29:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:06 smithi017 ceph-mon[97472]: pgmap v508: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 41 MiB/s wr, 45 op/s 2023-06-23T18:29:06.834 INFO:teuthology.orchestra.run.smithi002.stderr:42400000 of 50000000 tuples (84%) done (elapsed 390.08 s, remaining 69.92 s) 2023-06-23T18:29:07.657 INFO:teuthology.orchestra.run.smithi002.stderr:42500000 of 50000000 tuples (85%) done (elapsed 390.90 s, remaining 68.98 s) 2023-06-23T18:29:08.501 INFO:teuthology.orchestra.run.smithi002.stderr:42600000 of 50000000 tuples (85%) done (elapsed 391.74 s, remaining 68.05 s) 2023-06-23T18:29:08.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:08 smithi002 ceph-mon[95947]: pgmap v509: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:29:08.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:08 smithi082 ceph-mon[97889]: pgmap v509: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:29:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:08 smithi017 ceph-mon[97472]: pgmap v509: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:29:09.367 INFO:teuthology.orchestra.run.smithi002.stderr:42700000 of 50000000 tuples (85%) done (elapsed 392.61 s, remaining 67.12 s) 2023-06-23T18:29:10.215 INFO:teuthology.orchestra.run.smithi002.stderr:42800000 of 50000000 tuples (85%) done (elapsed 393.46 s, remaining 66.19 s) 2023-06-23T18:29:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:10 smithi002 ceph-mon[95947]: pgmap v510: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:29:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:10 smithi082 ceph-mon[97889]: pgmap v510: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:29:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:10 smithi017 ceph-mon[97472]: pgmap v510: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:29:11.057 INFO:teuthology.orchestra.run.smithi002.stderr:42900000 of 50000000 tuples (85%) done (elapsed 394.30 s, remaining 65.26 s) 2023-06-23T18:29:11.990 INFO:teuthology.orchestra.run.smithi002.stderr:43000000 of 50000000 tuples (86%) done (elapsed 395.23 s, remaining 64.34 s) 2023-06-23T18:29:12.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:12 smithi002 ceph-mon[95947]: pgmap v511: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:29:12.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:12 smithi082 ceph-mon[97889]: pgmap v511: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:29:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:12 smithi017 ceph-mon[97472]: pgmap v511: 97 pgs: 97 active+clean; 6.3 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 40 MiB/s wr, 44 op/s 2023-06-23T18:29:12.785 INFO:teuthology.orchestra.run.smithi002.stderr:43100000 of 50000000 tuples (86%) done (elapsed 396.03 s, remaining 63.40 s) 2023-06-23T18:29:13.608 INFO:teuthology.orchestra.run.smithi002.stderr:43200000 of 50000000 tuples (86%) done (elapsed 396.85 s, remaining 62.47 s) 2023-06-23T18:29:14.464 INFO:teuthology.orchestra.run.smithi002.stderr:43300000 of 50000000 tuples (86%) done (elapsed 397.71 s, remaining 61.54 s) 2023-06-23T18:29:14.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:14 smithi002 ceph-mon[95947]: pgmap v512: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:29:14.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:14 smithi082 ceph-mon[97889]: pgmap v512: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:29:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:14 smithi017 ceph-mon[97472]: pgmap v512: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:29:15.344 INFO:teuthology.orchestra.run.smithi002.stderr:43400000 of 50000000 tuples (86%) done (elapsed 398.59 s, remaining 60.61 s) 2023-06-23T18:29:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:43500000 of 50000000 tuples (87%) done (elapsed 399.47 s, remaining 59.69 s) 2023-06-23T18:29:16.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:16 smithi002 ceph-mon[95947]: pgmap v513: 97 pgs: 97 active+clean; 6.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:29:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:16 smithi082 ceph-mon[97889]: pgmap v513: 97 pgs: 97 active+clean; 6.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:29:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:16 smithi017 ceph-mon[97472]: pgmap v513: 97 pgs: 97 active+clean; 6.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 42 MiB/s wr, 24 op/s 2023-06-23T18:29:17.073 INFO:teuthology.orchestra.run.smithi002.stderr:43600000 of 50000000 tuples (87%) done (elapsed 400.31 s, remaining 58.76 s) 2023-06-23T18:29:17.868 INFO:teuthology.orchestra.run.smithi002.stderr:43700000 of 50000000 tuples (87%) done (elapsed 401.11 s, remaining 57.83 s) 2023-06-23T18:29:18.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:18 smithi002 ceph-mon[95947]: pgmap v514: 97 pgs: 97 active+clean; 6.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:29:18.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:18 smithi082 ceph-mon[97889]: pgmap v514: 97 pgs: 97 active+clean; 6.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:29:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:18 smithi017 ceph-mon[97472]: pgmap v514: 97 pgs: 97 active+clean; 6.4 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:29:18.724 INFO:teuthology.orchestra.run.smithi002.stderr:43800000 of 50000000 tuples (87%) done (elapsed 401.97 s, remaining 56.90 s) 2023-06-23T18:29:19.574 INFO:teuthology.orchestra.run.smithi002.stderr:43900000 of 50000000 tuples (87%) done (elapsed 402.82 s, remaining 55.97 s) 2023-06-23T18:29:20.380 INFO:teuthology.orchestra.run.smithi002.stderr:44000000 of 50000000 tuples (88%) done (elapsed 403.62 s, remaining 55.04 s) 2023-06-23T18:29:20.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:20 smithi002 ceph-mon[95947]: pgmap v515: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:29:20.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:20 smithi082 ceph-mon[97889]: pgmap v515: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:29:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:20 smithi017 ceph-mon[97472]: pgmap v515: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:29:21.305 INFO:teuthology.orchestra.run.smithi002.stderr:44100000 of 50000000 tuples (88%) done (elapsed 404.55 s, remaining 54.12 s) 2023-06-23T18:29:22.155 INFO:teuthology.orchestra.run.smithi002.stderr:44200000 of 50000000 tuples (88%) done (elapsed 405.40 s, remaining 53.20 s) 2023-06-23T18:29:22.535 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:29:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:22 smithi002 ceph-mon[95947]: pgmap v516: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-06-23T18:29:22.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:22 smithi082 ceph-mon[97889]: pgmap v516: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-06-23T18:29:22.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.679+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/1453554559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 msgr2=0x7f3a28105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.679+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/1453554559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 0x7f3a28105300 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009990 tx=0x7f3a1c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.679+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/1453554559 shutdown_connections 2023-06-23T18:29:22.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.679+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/1453554559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a28105840 0x7f3a28107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.679+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/1453554559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a28100600 0x7f3a281029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.679+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/1453554559 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 0x7f3a28105300 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.679+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/1453554559 >> 172.21.15.2:0/1453554559 conn(0x7f3a280fa0e0 msgr2=0x7f3a280fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:22.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.679+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/1453554559 shutdown_connections 2023-06-23T18:29:22.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.680+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/1453554559 wait complete. 2023-06-23T18:29:22.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.680+0000 7f3a2cd2c700 1 Processor -- start 2023-06-23T18:29:22.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2cd2c700 1 -- start start 2023-06-23T18:29:22.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2cd2c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a28100600 0x7f3a28195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2cd2c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 0x7f3a28195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2cd2c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a28105840 0x7f3a2819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2cd2c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2819d300 con 0x7f3a28102f20 2023-06-23T18:29:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2cd2c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2819d440 con 0x7f3a28100600 2023-06-23T18:29:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2cd2c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2819d5d0 con 0x7f3a28105840 2023-06-23T18:29:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a25d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 0x7f3a28195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a28100600 0x7f3a28195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a25d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 0x7f3a28195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55100/0 (socket says 172.21.15.2:55100) 2023-06-23T18:29:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a26d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a28105840 0x7f3a2819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a28100600 0x7f3a28195780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45690/0 (socket says 172.21.15.2:45690) 2023-06-23T18:29:22.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a25d9b700 1 -- 172.21.15.2:0/3370681515 learned_addr learned my addr 172.21.15.2:0/3370681515 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:22.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2659c700 1 -- 172.21.15.2:0/3370681515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a28105840 msgr2=0x7f3a2819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2659c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a28105840 0x7f3a2819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2659c700 1 -- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 msgr2=0x7f3a28195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2659c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 0x7f3a28195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.681+0000 7f3a2659c700 1 -- 172.21.15.2:0/3370681515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1c0095f0 con 0x7f3a28100600 2023-06-23T18:29:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.682+0000 7f3a2659c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a28100600 0x7f3a28195780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e9f0 tx=0x7f3a1800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.682+0000 7f3a177fe700 1 -- 172.21.15.2:0/3370681515 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800c9c0 con 0x7f3a28100600 2023-06-23T18:29:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.682+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/3370681515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2819d8b0 con 0x7f3a28100600 2023-06-23T18:29:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.682+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/3370681515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2819de10 con 0x7f3a28100600 2023-06-23T18:29:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.682+0000 7f3a177fe700 1 -- 172.21.15.2:0/3370681515 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a18010430 con 0x7f3a28100600 2023-06-23T18:29:22.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.682+0000 7f3a177fe700 1 -- 172.21.15.2:0/3370681515 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18018680 con 0x7f3a28100600 2023-06-23T18:29:22.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:22 smithi017 ceph-mon[97472]: pgmap v516: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 40 MiB/s wr, 23 op/s 2023-06-23T18:29:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.684+0000 7f3a177fe700 1 -- 172.21.15.2:0/3370681515 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3a18010830 con 0x7f3a28100600 2023-06-23T18:29:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.685+0000 7f3a177fe700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a100a1150 0x7f3a100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.685+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a100a1150 0x7f3a100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.685+0000 7f3a177fe700 1 -- 172.21.15.2:0/3370681515 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3a18014070 con 0x7f3a28100600 2023-06-23T18:29:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.686+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a100a1150 0x7f3a100a3600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00b2e0 tx=0x7f3a1c005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.686+0000 7f3a38eb1b80 1 Processor -- start 2023-06-23T18:29:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.686+0000 7f3a38eb1b80 1 -- start start 2023-06-23T18:29:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adf8c80 0x56392adf9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392adf9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adfa000 0x56392adfa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392acfdcb0 con 0x56392adf95d0 2023-06-23T18:29:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ad50a40 con 0x56392adf8c80 2023-06-23T18:29:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ad49ed0 con 0x56392adfa000 2023-06-23T18:29:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adf8c80 0x56392adf9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a26d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adfa000 0x56392adfa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a25d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392adf9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adf8c80 0x56392adf9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45698/0 (socket says 172.21.15.2:45698) 2023-06-23T18:29:22.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 -- 172.21.15.2:0/1597117726 learned_addr learned my addr 172.21.15.2:0/1597117726 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:22.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a26d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adfa000 0x56392adfa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55446/0 (socket says 172.21.15.2:55446) 2023-06-23T18:29:22.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 -- 172.21.15.2:0/1597117726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adfa000 msgr2=0x56392adfa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:22.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 --2- 172.21.15.2:0/1597117726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adfa000 0x56392adfa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 -- 172.21.15.2:0/1597117726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 msgr2=0x56392adf9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:22.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 --2- 172.21.15.2:0/1597117726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392adf9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 -- 172.21.15.2:0/1597117726 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56392abcb630 con 0x56392adf8c80 2023-06-23T18:29:22.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/1597117726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392adf9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:29:22.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.687+0000 7f3a2659c700 1 --2- 172.21.15.2:0/1597117726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adf8c80 0x56392adf9090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a18096310 tx=0x7f3a18099a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71da90b9c178f70 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:22.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a14ff9700 1 -- 172.21.15.2:0/1597117726 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1809de40 con 0x56392adf8c80 2023-06-23T18:29:22.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a14ff9700 1 -- 172.21.15.2:0/1597117726 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a180a2260 con 0x56392adf8c80 2023-06-23T18:29:22.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a14ff9700 1 -- 172.21.15.2:0/1597117726 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180a1dd0 con 0x56392adf8c80 2023-06-23T18:29:22.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1597117726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adf8c80 msgr2=0x56392adf9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/1597117726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adf8c80 0x56392adf9090 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a18096310 tx=0x7f3a18099a50 comp rx=0 tx=0).stop 2023-06-23T18:29:22.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1597117726 shutdown_connections 2023-06-23T18:29:22.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/1597117726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adfa000 0x56392adfa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/1597117726 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adf8c80 0x56392adf9090 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/1597117726 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392adf9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.688+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1597117726 >> 172.21.15.2:0/1597117726 conn(0x56392adee3f0 msgr2=0x56392adf3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:22.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1597117726 shutdown_connections 2023-06-23T18:29:22.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1597117726 wait complete. 2023-06-23T18:29:22.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 -- start start 2023-06-23T18:29:22.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 Processor -- start 2023-06-23T18:29:22.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae025f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae02b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ace9ac0 con 0x56392adf95d0 2023-06-23T18:29:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ace97c0 con 0x56392adfa000 2023-06-23T18:29:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392abcd920 con 0x56392adf8c80 2023-06-23T18:29:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a38eb1b80 10 client.0 _ll_register_callbacks cb 0x56392acef260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:29:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae025f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae025f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55450/0 (socket says 172.21.15.2:55450) 2023-06-23T18:29:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a2659c700 1 -- 172.21.15.2:0/771277962 learned_addr learned my addr 172.21.15.2:0/771277962 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a26d9d700 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.689+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae02b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a2659c700 1 -- 172.21.15.2:0/771277962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 msgr2=0x56392ae09b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:22.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a2659c700 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a2659c700 1 -- 172.21.15.2:0/771277962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 msgr2=0x56392ae02b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:22.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a2659c700 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae02b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a2659c700 1 -- 172.21.15.2:0/771277962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a28108890 con 0x56392adf8c80 2023-06-23T18:29:22.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a2659c700 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae025f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a18098400 tx=0x7f3a18010ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59a6480600ed464a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:22.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:29:22.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a0dffb700 1 -- 172.21.15.2:0/771277962 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180981b0 con 0x56392adf8c80 2023-06-23T18:29:22.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a0dffb700 1 -- 172.21.15.2:0/771277962 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3a1809de40 con 0x56392adf8c80 2023-06-23T18:29:22.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/771277962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56392adebbb0 con 0x56392adf8c80 2023-06-23T18:29:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a0dffb700 1 -- 172.21.15.2:0/771277962 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a180ca1f0 con 0x56392adf8c80 2023-06-23T18:29:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a38eb1b80 10 client.25591 fetch_fsmap learned FSMap version 30 2023-06-23T18:29:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/771277962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f3a2819de10 con 0x56392adf8c80 2023-06-23T18:29:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.690+0000 7f3a0dffb700 1 -- 172.21.15.2:0/771277962 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a18063020 con 0x56392adf8c80 2023-06-23T18:29:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a0dffb700 1 -- 172.21.15.2:0/771277962 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3a180b2290 con 0x56392adf8c80 2023-06-23T18:29:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a38eb1b80 10 client.25591 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:29:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a38eb1b80 10 client.25591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:29:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a38eb1b80 20 client.25591 populate_metadata read hostname 'smithi002' 2023-06-23T18:29:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56392ae0e110 0x56392ae10520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a38eb1b80 4 client.25591 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:29:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/771277962 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56392ad42eb0 con 0x56392ae0e110 2023-06-23T18:29:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56392ae0e110 0x56392ae10520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a0dffb700 10 client.25591 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:29:22.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.691+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56392ae0e110 0x56392ae10520 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39eebacbe89921b1 server_cookie=7e2cd438e9904d25 in_seq=0 out_seq=0 2023-06-23T18:29:22.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.692+0000 7f3a0dffb700 1 -- 172.21.15.2:0/771277962 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56392ad42eb0 con 0x56392ae0e110 2023-06-23T18:29:22.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.692+0000 7f3a0dffb700 10 client.25591 handle_command_reply: tid=0 2023-06-23T18:29:22.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.692+0000 7f3a38eb1b80 1 client.25591 shutdown 2023-06-23T18:29:22.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/771277962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 msgr2=0x56392ae025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae025f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a18098400 tx=0x7f3a18010ae0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/771277962 shutdown_connections 2023-06-23T18:29:22.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56392ae0e110 0x56392ae10520 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae025f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a0dffb700 0 client.25591 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:29:22.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/771277962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae02b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/771277962 >> 172.21.15.2:0/771277962 conn(0x56392adee3f0 msgr2=0x56392adf2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:22.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/771277962 shutdown_connections 2023-06-23T18:29:22.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/771277962 wait complete. 2023-06-23T18:29:22.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.693+0000 7f3a38eb1b80 20 client.25591 trim_cache size 0 max 16384 2023-06-23T18:29:22.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.714+0000 7f3a38eb1b80 1 Processor -- start 2023-06-23T18:29:22.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a38eb1b80 1 -- start start 2023-06-23T18:29:22.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf8c80 0x56392ae00440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf95d0 0x56392ae09b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392adfcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ae021c0 con 0x56392adf8c80 2023-06-23T18:29:22.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ad42eb0 con 0x56392adfa000 2023-06-23T18:29:22.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ae010a0 con 0x56392adf95d0 2023-06-23T18:29:22.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf8c80 0x56392ae00440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a25d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf95d0 0x56392ae09b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf8c80 0x56392ae00440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55120/0 (socket says 172.21.15.2:55120) 2023-06-23T18:29:22.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a2659c700 1 -- 172.21.15.2:0/1481122353 learned_addr learned my addr 172.21.15.2:0/1481122353 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:22.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a26d9d700 1 --2- 172.21.15.2:0/1481122353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392adfcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a2659c700 1 -- 172.21.15.2:0/1481122353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf95d0 msgr2=0x56392ae09b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:22.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a2659c700 1 --2- 172.21.15.2:0/1481122353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf95d0 0x56392ae09b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a2659c700 1 -- 172.21.15.2:0/1481122353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 msgr2=0x56392adfcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:22.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a2659c700 1 --2- 172.21.15.2:0/1481122353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392adfcf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.715+0000 7f3a2659c700 1 -- 172.21.15.2:0/1481122353 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56392ab77de0 con 0x56392adf8c80 2023-06-23T18:29:22.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.716+0000 7f3a2659c700 1 --2- 172.21.15.2:0/1481122353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf8c80 0x56392ae00440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a18099da0 tx=0x7f3a18099f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f416f1ce4f9d3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:22.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.716+0000 7f3a157fa700 1 -- 172.21.15.2:0/1481122353 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18099300 con 0x56392adf8c80 2023-06-23T18:29:22.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.716+0000 7f3a157fa700 1 -- 172.21.15.2:0/1481122353 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a18063030 con 0x56392adf8c80 2023-06-23T18:29:22.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.716+0000 7f3a157fa700 1 -- 172.21.15.2:0/1481122353 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180b27f0 con 0x56392adf8c80 2023-06-23T18:29:22.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.716+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1481122353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf8c80 msgr2=0x56392ae00440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.716+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/1481122353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf8c80 0x56392ae00440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3a18099da0 tx=0x7f3a18099f90 comp rx=0 tx=0).stop 2023-06-23T18:29:22.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1481122353 shutdown_connections 2023-06-23T18:29:22.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/1481122353 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392adfcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/1481122353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf8c80 0x56392ae00440 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/1481122353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf95d0 0x56392ae09b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1481122353 >> 172.21.15.2:0/1481122353 conn(0x56392adee3f0 msgr2=0x56392ae14e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:22.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1481122353 shutdown_connections 2023-06-23T18:29:22.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/1481122353 wait complete. 2023-06-23T18:29:22.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 -- start start 2023-06-23T18:29:22.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 Processor -- start 2023-06-23T18:29:22.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae05bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae06130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ae09690 con 0x56392adf95d0 2023-06-23T18:29:22.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ae09800 con 0x56392adfa000 2023-06-23T18:29:22.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56392ae099c0 con 0x56392adf8c80 2023-06-23T18:29:22.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a38eb1b80 10 client.0 _ll_register_callbacks cb 0x56392acef260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:29:22.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a25d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae06130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae05bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.717+0000 7f3a25d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae06130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55128/0 (socket says 172.21.15.2:55128) 2023-06-23T18:29:22.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a2659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae05bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55464/0 (socket says 172.21.15.2:55464) 2023-06-23T18:29:22.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a25d9b700 1 -- 172.21.15.2:0/2865321569 learned_addr learned my addr 172.21.15.2:0/2865321569 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:22.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a26d9d700 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a2659c700 1 -- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 msgr2=0x56392ae09150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:22.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a2659c700 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a2659c700 1 -- 172.21.15.2:0/2865321569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 msgr2=0x56392ae06130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:22.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a2659c700 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae06130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a2659c700 1 -- 172.21.15.2:0/2865321569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2819de10 con 0x56392adf8c80 2023-06-23T18:29:22.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae06130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:29:22.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a26d9d700 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:29:22.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a2659c700 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae05bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3a18099b10 tx=0x7f3a180a2190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c7c76f0ccd8c812 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:22.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:29:22.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a0dffb700 1 -- 172.21.15.2:0/2865321569 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180106a0 con 0x56392adf8c80 2023-06-23T18:29:22.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/2865321569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a180a2260 con 0x56392adf8c80 2023-06-23T18:29:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a0dffb700 1 -- 172.21.15.2:0/2865321569 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3a18064080 con 0x56392adf8c80 2023-06-23T18:29:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a0dffb700 1 -- 172.21.15.2:0/2865321569 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a180ca3c0 con 0x56392adf8c80 2023-06-23T18:29:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a38eb1b80 10 client.25597 fetch_fsmap learned FSMap version 30 2023-06-23T18:29:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.718+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/2865321569 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56392ab60c60 con 0x56392adf8c80 2023-06-23T18:29:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a0dffb700 1 -- 172.21.15.2:0/2865321569 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a180bd080 con 0x56392adf8c80 2023-06-23T18:29:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a0dffb700 1 -- 172.21.15.2:0/2865321569 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3a180b9bc0 con 0x56392adf8c80 2023-06-23T18:29:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a38eb1b80 10 client.25597 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:29:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a38eb1b80 10 client.25597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:29:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a38eb1b80 20 client.25597 populate_metadata read hostname 'smithi002' 2023-06-23T18:29:22.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56392ae62d00 0x56392ae65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:22.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a38eb1b80 4 client.25597 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:29:22.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/2865321569 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56392ad05050 con 0x56392ae62d00 2023-06-23T18:29:22.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56392ae62d00 0x56392ae65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:22.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a0dffb700 10 client.25597 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:29:22.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.719+0000 7f3a25d9b700 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56392ae62d00 0x56392ae65110 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=457e3f50b0e5cca6 server_cookie=b8770e10be9f8e52 in_seq=0 out_seq=0 2023-06-23T18:29:22.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a0dffb700 1 -- 172.21.15.2:0/2865321569 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56392ad05050 con 0x56392ae62d00 2023-06-23T18:29:22.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a0dffb700 10 client.25597 handle_command_reply: tid=0 2023-06-23T18:29:22.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 client.25597 shutdown 2023-06-23T18:29:22.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/2865321569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 msgr2=0x56392ae05bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae05bf0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3a18099b10 tx=0x7f3a180a2190 comp rx=0 tx=0).stop 2023-06-23T18:29:22.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/2865321569 shutdown_connections 2023-06-23T18:29:22.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56392ae62d00 0x56392ae65110 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56392adfa000 0x56392ae09150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a0dffb700 0 client.25597 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:29:22.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56392adf8c80 0x56392ae05bf0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 --2- 172.21.15.2:0/2865321569 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56392adf95d0 0x56392ae06130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/2865321569 >> 172.21.15.2:0/2865321569 conn(0x56392adee3f0 msgr2=0x56392ae01290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:22.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/2865321569 shutdown_connections 2023-06-23T18:29:22.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 1 -- 172.21.15.2:0/2865321569 wait complete. 2023-06-23T18:29:22.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.720+0000 7f3a38eb1b80 20 client.25597 trim_cache size 0 max 16384 2023-06-23T18:29:22.738 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:29:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:29:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:29:22.738 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:29:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:29:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:29:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:29:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:29:22.739 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:29:22.739 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:29:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a100a1150 msgr2=0x7f3a100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a100a1150 0x7f3a100a3600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00b2e0 tx=0x7f3a1c005a90 comp rx=0 tx=0).stop 2023-06-23T18:29:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/3370681515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a28100600 msgr2=0x7f3a28195780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a28100600 0x7f3a28195780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e9f0 tx=0x7f3a1800ed00 comp rx=0 tx=0).stop 2023-06-23T18:29:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/3370681515 shutdown_connections 2023-06-23T18:29:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3a28105840 0x7f3a2819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f3a100a1150 0x7f3a100a3600 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a28100600 0x7f3a28195780 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 --2- 172.21.15.2:0/3370681515 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a28102f20 0x7f3a28195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:22.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/3370681515 >> 172.21.15.2:0/3370681515 conn(0x7f3a280fa0e0 msgr2=0x7f3a28101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:22.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/3370681515 shutdown_connections 2023-06-23T18:29:22.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:22.721+0000 7f3a2cd2c700 1 -- 172.21.15.2:0/3370681515 wait complete. 2023-06-23T18:29:22.742 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:29:22.996 INFO:teuthology.orchestra.run.smithi002.stderr:44300000 of 50000000 tuples (88%) done (elapsed 406.24 s, remaining 52.27 s) 2023-06-23T18:29:23.881 INFO:teuthology.orchestra.run.smithi002.stderr:44400000 of 50000000 tuples (88%) done (elapsed 407.12 s, remaining 51.35 s) 2023-06-23T18:29:24.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:24 smithi002 ceph-mon[95947]: pgmap v517: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:29:24.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:24 smithi082 ceph-mon[97889]: pgmap v517: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:29:24.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:24 smithi017 ceph-mon[97472]: pgmap v517: 97 pgs: 97 active+clean; 6.5 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:29:24.703 INFO:teuthology.orchestra.run.smithi002.stderr:44500000 of 50000000 tuples (89%) done (elapsed 407.94 s, remaining 50.42 s) 2023-06-23T18:29:25.558 INFO:teuthology.orchestra.run.smithi002.stderr:44600000 of 50000000 tuples (89%) done (elapsed 408.80 s, remaining 49.50 s) 2023-06-23T18:29:26.368 INFO:teuthology.orchestra.run.smithi002.stderr:44700000 of 50000000 tuples (89%) done (elapsed 409.61 s, remaining 48.57 s) 2023-06-23T18:29:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:26 smithi002 ceph-mon[95947]: pgmap v518: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 42 MiB/s wr, 22 op/s 2023-06-23T18:29:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:26 smithi082 ceph-mon[97889]: pgmap v518: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 42 MiB/s wr, 22 op/s 2023-06-23T18:29:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:26 smithi017 ceph-mon[97472]: pgmap v518: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 42 MiB/s wr, 22 op/s 2023-06-23T18:29:27.276 INFO:teuthology.orchestra.run.smithi002.stderr:44800000 of 50000000 tuples (89%) done (elapsed 410.52 s, remaining 47.65 s) 2023-06-23T18:29:28.155 INFO:teuthology.orchestra.run.smithi002.stderr:44900000 of 50000000 tuples (89%) done (elapsed 411.40 s, remaining 46.73 s) 2023-06-23T18:29:28.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:28 smithi002 ceph-mon[95947]: pgmap v519: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 40 MiB/s wr, 20 op/s 2023-06-23T18:29:28.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:28 smithi082 ceph-mon[97889]: pgmap v519: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 40 MiB/s wr, 20 op/s 2023-06-23T18:29:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:28 smithi017 ceph-mon[97472]: pgmap v519: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 40 MiB/s wr, 20 op/s 2023-06-23T18:29:30.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:30 smithi002 ceph-mon[95947]: pgmap v520: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:29:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:30 smithi082 ceph-mon[97889]: pgmap v520: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:29:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:30 smithi017 ceph-mon[97472]: pgmap v520: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 40 MiB/s wr, 22 op/s 2023-06-23T18:29:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:32 smithi002 ceph-mon[95947]: pgmap v521: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 33 op/s 2023-06-23T18:29:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:32 smithi082 ceph-mon[97889]: pgmap v521: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 33 op/s 2023-06-23T18:29:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:32 smithi017 ceph-mon[97472]: pgmap v521: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 MiB/s wr, 33 op/s 2023-06-23T18:29:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:45000000 of 50000000 tuples (90%) done (elapsed 416.96 s, remaining 46.33 s) 2023-06-23T18:29:34.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:34 smithi002 ceph-mon[95947]: pgmap v522: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 34 op/s 2023-06-23T18:29:34.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:34 smithi082 ceph-mon[97889]: pgmap v522: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 34 op/s 2023-06-23T18:29:34.659 INFO:teuthology.orchestra.run.smithi002.stderr:45100000 of 50000000 tuples (90%) done (elapsed 417.90 s, remaining 45.40 s) 2023-06-23T18:29:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:34 smithi017 ceph-mon[97472]: pgmap v522: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 MiB/s wr, 34 op/s 2023-06-23T18:29:35.492 INFO:teuthology.orchestra.run.smithi002.stderr:45200000 of 50000000 tuples (90%) done (elapsed 418.73 s, remaining 44.47 s) 2023-06-23T18:29:36.344 INFO:teuthology.orchestra.run.smithi002.stderr:45300000 of 50000000 tuples (90%) done (elapsed 419.59 s, remaining 43.53 s) 2023-06-23T18:29:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:36 smithi002 ceph-mon[95947]: pgmap v523: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 31 MiB/s wr, 39 op/s 2023-06-23T18:29:36.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:36 smithi082 ceph-mon[97889]: pgmap v523: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 31 MiB/s wr, 39 op/s 2023-06-23T18:29:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:36 smithi017 ceph-mon[97472]: pgmap v523: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 31 MiB/s wr, 39 op/s 2023-06-23T18:29:37.192 INFO:teuthology.orchestra.run.smithi002.stderr:45400000 of 50000000 tuples (90%) done (elapsed 420.43 s, remaining 42.60 s) 2023-06-23T18:29:38.104 INFO:teuthology.orchestra.run.smithi002.stderr:45500000 of 50000000 tuples (91%) done (elapsed 421.35 s, remaining 41.67 s) 2023-06-23T18:29:38.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:38 smithi002 ceph-mon[95947]: pgmap v524: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 25 MiB/s wr, 37 op/s 2023-06-23T18:29:38.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:38 smithi082 ceph-mon[97889]: pgmap v524: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 25 MiB/s wr, 37 op/s 2023-06-23T18:29:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:38 smithi017 ceph-mon[97472]: pgmap v524: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 25 MiB/s wr, 37 op/s 2023-06-23T18:29:38.943 INFO:teuthology.orchestra.run.smithi002.stderr:45600000 of 50000000 tuples (91%) done (elapsed 422.18 s, remaining 40.74 s) 2023-06-23T18:29:39.771 INFO:teuthology.orchestra.run.smithi002.stderr:45700000 of 50000000 tuples (91%) done (elapsed 423.01 s, remaining 39.80 s) 2023-06-23T18:29:40.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:40 smithi002 ceph-mon[95947]: pgmap v525: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 39 op/s 2023-06-23T18:29:40.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:40 smithi082 ceph-mon[97889]: pgmap v525: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 39 op/s 2023-06-23T18:29:40.605 INFO:teuthology.orchestra.run.smithi002.stderr:45800000 of 50000000 tuples (91%) done (elapsed 423.85 s, remaining 38.87 s) 2023-06-23T18:29:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:40 smithi017 ceph-mon[97472]: pgmap v525: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 26 MiB/s wr, 39 op/s 2023-06-23T18:29:41.468 INFO:teuthology.orchestra.run.smithi002.stderr:45900000 of 50000000 tuples (91%) done (elapsed 424.71 s, remaining 37.94 s) 2023-06-23T18:29:42.342 INFO:teuthology.orchestra.run.smithi002.stderr:46000000 of 50000000 tuples (92%) done (elapsed 425.58 s, remaining 37.01 s) 2023-06-23T18:29:42.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:42 smithi082 ceph-mon[97889]: pgmap v526: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 24 MiB/s wr, 39 op/s 2023-06-23T18:29:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:42 smithi017 ceph-mon[97472]: pgmap v526: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 24 MiB/s wr, 39 op/s 2023-06-23T18:29:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:42 smithi002 ceph-mon[95947]: pgmap v526: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 24 MiB/s wr, 39 op/s 2023-06-23T18:29:43.176 INFO:teuthology.orchestra.run.smithi002.stderr:46100000 of 50000000 tuples (92%) done (elapsed 426.42 s, remaining 36.07 s) 2023-06-23T18:29:44.041 INFO:teuthology.orchestra.run.smithi002.stderr:46200000 of 50000000 tuples (92%) done (elapsed 427.28 s, remaining 35.14 s) 2023-06-23T18:29:44.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:44 smithi082 ceph-mon[97889]: pgmap v527: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 31 op/s 2023-06-23T18:29:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:44 smithi017 ceph-mon[97472]: pgmap v527: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 31 op/s 2023-06-23T18:29:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:44 smithi002 ceph-mon[95947]: pgmap v527: 97 pgs: 97 active+clean; 6.7 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 31 op/s 2023-06-23T18:29:44.895 INFO:teuthology.orchestra.run.smithi002.stderr:46300000 of 50000000 tuples (92%) done (elapsed 428.14 s, remaining 34.21 s) 2023-06-23T18:29:45.721 INFO:teuthology.orchestra.run.smithi002.stderr:46400000 of 50000000 tuples (92%) done (elapsed 428.96 s, remaining 33.28 s) 2023-06-23T18:29:46.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:46 smithi082 ceph-mon[97889]: pgmap v528: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 30 op/s 2023-06-23T18:29:46.648 INFO:teuthology.orchestra.run.smithi002.stderr:46500000 of 50000000 tuples (93%) done (elapsed 429.89 s, remaining 32.36 s) 2023-06-23T18:29:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:46 smithi017 ceph-mon[97472]: pgmap v528: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 30 op/s 2023-06-23T18:29:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:46 smithi002 ceph-mon[95947]: pgmap v528: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 30 op/s 2023-06-23T18:29:47.466 INFO:teuthology.orchestra.run.smithi002.stderr:46600000 of 50000000 tuples (93%) done (elapsed 430.71 s, remaining 31.43 s) 2023-06-23T18:29:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:46700000 of 50000000 tuples (93%) done (elapsed 431.53 s, remaining 30.49 s) 2023-06-23T18:29:48.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:48 smithi082 ceph-mon[97889]: pgmap v529: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 MiB/s wr, 23 op/s 2023-06-23T18:29:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:48 smithi017 ceph-mon[97472]: pgmap v529: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 MiB/s wr, 23 op/s 2023-06-23T18:29:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:48 smithi002 ceph-mon[95947]: pgmap v529: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 36 MiB/s wr, 23 op/s 2023-06-23T18:29:49.117 INFO:teuthology.orchestra.run.smithi002.stderr:46800000 of 50000000 tuples (93%) done (elapsed 432.36 s, remaining 29.56 s) 2023-06-23T18:29:50.054 INFO:teuthology.orchestra.run.smithi002.stderr:46900000 of 50000000 tuples (93%) done (elapsed 433.30 s, remaining 28.64 s) 2023-06-23T18:29:50.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:50 smithi082 ceph-mon[97889]: pgmap v530: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:29:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:50 smithi017 ceph-mon[97472]: pgmap v530: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:29:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:50 smithi002 ceph-mon[95947]: pgmap v530: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:29:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:47000000 of 50000000 tuples (94%) done (elapsed 434.12 s, remaining 27.71 s) 2023-06-23T18:29:51.738 INFO:teuthology.orchestra.run.smithi002.stderr:47100000 of 50000000 tuples (94%) done (elapsed 434.98 s, remaining 26.78 s) 2023-06-23T18:29:52.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:52 smithi082 ceph-mon[97889]: pgmap v531: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 39 MiB/s wr, 25 op/s 2023-06-23T18:29:52.618 INFO:teuthology.orchestra.run.smithi002.stderr:47200000 of 50000000 tuples (94%) done (elapsed 435.86 s, remaining 25.86 s) 2023-06-23T18:29:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:52 smithi017 ceph-mon[97472]: pgmap v531: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 39 MiB/s wr, 25 op/s 2023-06-23T18:29:52.743 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:29:52.770 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:52 smithi002 ceph-mon[95947]: pgmap v531: 97 pgs: 97 active+clean; 6.8 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 39 MiB/s wr, 25 op/s 2023-06-23T18:29:52.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 -- 172.21.15.2:0/2595594914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80640fea60 msgr2=0x7f80640fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 --2- 172.21.15.2:0/2595594914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80640fea60 0x7f80640fee70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8060009990 tx=0x7f8060009ca0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 -- 172.21.15.2:0/2595594914 shutdown_connections 2023-06-23T18:29:52.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 --2- 172.21.15.2:0/2595594914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80641005e0 0x7f8064106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 --2- 172.21.15.2:0/2595594914 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80640fea60 0x7f80640fee70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 --2- 172.21.15.2:0/2595594914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80640ffc50 0x7f80641000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 -- 172.21.15.2:0/2595594914 >> 172.21.15.2:0/2595594914 conn(0x7f80640fa060 msgr2=0x7f80640fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:52.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 -- 172.21.15.2:0/2595594914 shutdown_connections 2023-06-23T18:29:52.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.897+0000 7f806b5b8700 1 -- 172.21.15.2:0/2595594914 wait complete. 2023-06-23T18:29:52.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f806b5b8700 1 Processor -- start 2023-06-23T18:29:52.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f806b5b8700 1 -- start start 2023-06-23T18:29:52.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f806b5b8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80640fea60 0x7f8064195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f806b5b8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80640ffc50 0x7f8064195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f806b5b8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80641005e0 0x7f806419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f806b5b8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419d390 con 0x7f80641005e0 2023-06-23T18:29:52.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f806b5b8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419d4d0 con 0x7f80640ffc50 2023-06-23T18:29:52.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f806b5b8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419d660 con 0x7f80640fea60 2023-06-23T18:29:52.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f8069354700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80640fea60 0x7f8064195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f8069b55700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80641005e0 0x7f806419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f8069354700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80640fea60 0x7f8064195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55488/0 (socket says 172.21.15.2:55488) 2023-06-23T18:29:52.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f8068b53700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80640ffc50 0x7f8064195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.898+0000 7f8069354700 1 -- 172.21.15.2:0/1508145662 learned_addr learned my addr 172.21.15.2:0/1508145662 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:52.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f8069354700 1 -- 172.21.15.2:0/1508145662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80640ffc50 msgr2=0x7f8064195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f8069354700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80640ffc50 0x7f8064195d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f8069354700 1 -- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80641005e0 msgr2=0x7f806419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f8069354700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80641005e0 0x7f806419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f8069354700 1 -- 172.21.15.2:0/1508145662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c009c40 con 0x7f80640fea60 2023-06-23T18:29:52.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f8069354700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80640fea60 0x7f8064195810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8060003ec0 tx=0x7f8060005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:52.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f805a7fc700 1 -- 172.21.15.2:0/1508145662 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806001d070 con 0x7f80640fea60 2023-06-23T18:29:52.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f806b5b8700 1 -- 172.21.15.2:0/1508145662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80600095f0 con 0x7f80640fea60 2023-06-23T18:29:52.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f805a7fc700 1 -- 172.21.15.2:0/1508145662 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f806000bd50 con 0x7f80640fea60 2023-06-23T18:29:52.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f805a7fc700 1 -- 172.21.15.2:0/1508145662 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060022d30 con 0x7f80640fea60 2023-06-23T18:29:52.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.899+0000 7f806b5b8700 1 -- 172.21.15.2:0/1508145662 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806419dbd0 con 0x7f80640fea60 2023-06-23T18:29:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.902+0000 7f805a7fc700 1 -- 172.21.15.2:0/1508145662 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8060022500 con 0x7f80640fea60 2023-06-23T18:29:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.902+0000 7f805a7fc700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f80500a1150 0x7f80500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.903+0000 7f805a7fc700 1 -- 172.21.15.2:0/1508145662 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f80600cde40 con 0x7f80640fea60 2023-06-23T18:29:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.903+0000 7f8068b53700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f80500a1150 0x7f80500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.903+0000 7f8068b53700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f80500a1150 0x7f80500a3600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8054009c60 tx=0x7f8054009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:52.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.903+0000 7f807773db80 1 Processor -- start 2023-06-23T18:29:52.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.903+0000 7f807773db80 1 -- start start 2023-06-23T18:29:52.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f807773db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de86236090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f807773db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de86236a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f807773db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 0x55de86237470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f807773db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8613acb0 con 0x55de86237000 2023-06-23T18:29:52.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f807773db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8618da40 con 0x55de862365d0 2023-06-23T18:29:52.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f807773db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de86186ed0 con 0x55de86235c80 2023-06-23T18:29:52.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8069354700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de86236090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8068b53700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de86236a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8069354700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de86236090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55496/0 (socket says 172.21.15.2:55496) 2023-06-23T18:29:52.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8069354700 1 -- 172.21.15.2:0/3213309930 learned_addr learned my addr 172.21.15.2:0/3213309930 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:52.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8069b55700 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 0x55de86237470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8068b53700 1 -- 172.21.15.2:0/3213309930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 msgr2=0x55de86236090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:52.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8068b53700 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de86236090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8068b53700 1 -- 172.21.15.2:0/3213309930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 msgr2=0x55de86237470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:52.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8068b53700 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 0x55de86237470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8068b53700 1 -- 172.21.15.2:0/3213309930 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de86008630 con 0x55de862365d0 2023-06-23T18:29:52.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8069354700 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de86236090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:29:52.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.904+0000 7f8068b53700 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de86236a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f805400e3b0 tx=0x7f8054014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332ead3a2e237890 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:52.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.905+0000 7f804f7fe700 1 -- 172.21.15.2:0/3213309930 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805400ed50 con 0x55de862365d0 2023-06-23T18:29:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.905+0000 7f804f7fe700 1 -- 172.21.15.2:0/3213309930 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8054021070 con 0x55de862365d0 2023-06-23T18:29:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.905+0000 7f804f7fe700 1 -- 172.21.15.2:0/3213309930 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054003dd0 con 0x55de862365d0 2023-06-23T18:29:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.905+0000 7f807773db80 1 -- 172.21.15.2:0/3213309930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 msgr2=0x55de86236a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.905+0000 7f807773db80 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de86236a70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f805400e3b0 tx=0x7f8054014000 comp rx=0 tx=0).stop 2023-06-23T18:29:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 -- 172.21.15.2:0/3213309930 shutdown_connections 2023-06-23T18:29:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 0x55de86237470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de86236090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 --2- 172.21.15.2:0/3213309930 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de86236a70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 -- 172.21.15.2:0/3213309930 >> 172.21.15.2:0/3213309930 conn(0x55de8622b3f0 msgr2=0x55de86230520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 -- 172.21.15.2:0/3213309930 shutdown_connections 2023-06-23T18:29:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 -- 172.21.15.2:0/3213309930 wait complete. 2023-06-23T18:29:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 -- start start 2023-06-23T18:29:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 Processor -- start 2023-06-23T18:29:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de8623f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de8623fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 0x55de86246ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.906+0000 7f807773db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de86126ac0 con 0x55de86237000 2023-06-23T18:29:52.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f807773db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de861267c0 con 0x55de862365d0 2023-06-23T18:29:52.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f807773db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8600a920 con 0x55de86235c80 2023-06-23T18:29:52.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f807773db80 10 client.0 _ll_register_callbacks cb 0x55de8612c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:29:52.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de8623f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8068b53700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de8623fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de8623f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55502/0 (socket says 172.21.15.2:55502) 2023-06-23T18:29:52.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069b55700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 0x55de86246ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 -- 172.21.15.2:0/3036748991 learned_addr learned my addr 172.21.15.2:0/3036748991 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:52.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 -- 172.21.15.2:0/3036748991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 msgr2=0x55de8623fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:52.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de8623fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 -- 172.21.15.2:0/3036748991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 msgr2=0x55de86246ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:52.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 0x55de86246ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 -- 172.21.15.2:0/3036748991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806419dbd0 con 0x55de86235c80 2023-06-23T18:29:52.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f8069354700 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de8623f540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8060097d50 tx=0x7f8060009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=791f99b0aec9c4a7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:52.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f804cff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:29:52.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f804cff9700 1 -- 172.21.15.2:0/3036748991 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80600b2070 con 0x55de86235c80 2023-06-23T18:29:52.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f804cff9700 1 -- 172.21.15.2:0/3036748991 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f806009b2e0 con 0x55de86235c80 2023-06-23T18:29:52.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.907+0000 7f807773db80 1 -- 172.21.15.2:0/3036748991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de86228bb0 con 0x55de86235c80 2023-06-23T18:29:52.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f804cff9700 1 -- 172.21.15.2:0/3036748991 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f806009ac40 con 0x55de86235c80 2023-06-23T18:29:52.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f807773db80 10 client.25609 fetch_fsmap learned FSMap version 30 2023-06-23T18:29:52.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f807773db80 1 -- 172.21.15.2:0/3036748991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f8054017070 con 0x55de86235c80 2023-06-23T18:29:52.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f804cff9700 1 -- 172.21.15.2:0/3036748991 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de86228bb0 con 0x55de86235c80 2023-06-23T18:29:52.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f804cff9700 1 -- 172.21.15.2:0/3036748991 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f80600a3210 con 0x55de86235c80 2023-06-23T18:29:52.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f807773db80 10 client.25609 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:29:52.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f807773db80 10 client.25609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:29:52.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f807773db80 20 client.25609 populate_metadata read hostname 'smithi002' 2023-06-23T18:29:52.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f807773db80 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55de8624afb0 0x55de8624d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f807773db80 4 client.25609 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:29:52.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f807773db80 1 -- 172.21.15.2:0/3036748991 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de8617feb0 con 0x55de8624afb0 2023-06-23T18:29:52.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.908+0000 7f8068b53700 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55de8624afb0 0x55de8624d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.909+0000 7f804cff9700 10 client.25609 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:29:52.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.909+0000 7f8068b53700 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55de8624afb0 0x55de8624d3c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9287a809c67b0f server_cookie=56c609a9ec3228bf in_seq=0 out_seq=0 2023-06-23T18:29:52.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.909+0000 7f804cff9700 1 -- 172.21.15.2:0/3036748991 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de8617feb0 con 0x55de8624afb0 2023-06-23T18:29:52.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.909+0000 7f804cff9700 10 client.25609 handle_command_reply: tid=0 2023-06-23T18:29:52.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 client.25609 shutdown 2023-06-23T18:29:52.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 -- 172.21.15.2:0/3036748991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 msgr2=0x55de8623f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de8623f540 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8060097d50 tx=0x7f8060009ca0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 -- 172.21.15.2:0/3036748991 shutdown_connections 2023-06-23T18:29:52.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55de8624afb0 0x55de8624d3c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86237000 0x55de86246ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f804cff9700 0 client.25609 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:29:52.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de86235c80 0x55de8623f540 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 --2- 172.21.15.2:0/3036748991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de862365d0 0x55de8623fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 -- 172.21.15.2:0/3036748991 >> 172.21.15.2:0/3036748991 conn(0x55de8622b3f0 msgr2=0x55de8622fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:52.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 -- 172.21.15.2:0/3036748991 shutdown_connections 2023-06-23T18:29:52.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 1 -- 172.21.15.2:0/3036748991 wait complete. 2023-06-23T18:29:52.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.910+0000 7f807773db80 20 client.25609 trim_cache size 0 max 16384 2023-06-23T18:29:52.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f807773db80 1 Processor -- start 2023-06-23T18:29:52.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f807773db80 1 -- start start 2023-06-23T18:29:52.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f807773db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86235c80 0x55de86239e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f807773db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de862365d0 0x55de8623a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f807773db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86246a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f807773db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8623f110 con 0x55de86235c80 2023-06-23T18:29:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f807773db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8623e9f0 con 0x55de86237000 2023-06-23T18:29:52.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f807773db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de862311f0 con 0x55de862365d0 2023-06-23T18:29:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f8069354700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86235c80 0x55de86239e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f8069354700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86235c80 0x55de86239e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55172/0 (socket says 172.21.15.2:55172) 2023-06-23T18:29:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f8068b53700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de862365d0 0x55de8623a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f8069354700 1 -- 172.21.15.2:0/3228056819 learned_addr learned my addr 172.21.15.2:0/3228056819 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.935+0000 7f8069b55700 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86246a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f8069354700 1 -- 172.21.15.2:0/3228056819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de862365d0 msgr2=0x55de8623a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f8069354700 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de862365d0 0x55de8623a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f8069354700 1 -- 172.21.15.2:0/3228056819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 msgr2=0x55de86246a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:52.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f8069354700 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86246a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f8069354700 1 -- 172.21.15.2:0/3228056819 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de85fb4de0 con 0x55de86235c80 2023-06-23T18:29:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f8068b53700 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de862365d0 0x55de8623a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:29:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f8069b55700 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86246a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:29:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f8069354700 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86235c80 0x55de86239e60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8060097d00 tx=0x7f8060099890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c9e71c45a28b7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f804ffff700 1 -- 172.21.15.2:0/3228056819 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806009ac40 con 0x55de86235c80 2023-06-23T18:29:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f804ffff700 1 -- 172.21.15.2:0/3228056819 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8060022e90 con 0x55de86235c80 2023-06-23T18:29:52.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.936+0000 7f804ffff700 1 -- 172.21.15.2:0/3228056819 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80600a3210 con 0x55de86235c80 2023-06-23T18:29:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 -- 172.21.15.2:0/3228056819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86235c80 msgr2=0x55de86239e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86235c80 0x55de86239e60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8060097d00 tx=0x7f8060099890 comp rx=0 tx=0).stop 2023-06-23T18:29:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 -- 172.21.15.2:0/3228056819 shutdown_connections 2023-06-23T18:29:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86246a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de86235c80 0x55de86239e60 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 --2- 172.21.15.2:0/3228056819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de862365d0 0x55de8623a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 -- 172.21.15.2:0/3228056819 >> 172.21.15.2:0/3228056819 conn(0x55de8622b3f0 msgr2=0x55de86251d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:52.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 -- 172.21.15.2:0/3228056819 shutdown_connections 2023-06-23T18:29:52.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 -- 172.21.15.2:0/3228056819 wait complete. 2023-06-23T18:29:52.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 -- start start 2023-06-23T18:29:52.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.937+0000 7f807773db80 1 Processor -- start 2023-06-23T18:29:52.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f807773db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de862365d0 0x55de86242940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f807773db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86242e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f807773db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8624afb0 0x55de86245ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f807773db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de862463e0 con 0x55de862365d0 2023-06-23T18:29:52.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f807773db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de86246550 con 0x55de86237000 2023-06-23T18:29:52.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f807773db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de86246710 con 0x55de8624afb0 2023-06-23T18:29:52.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f807773db80 10 client.0 _ll_register_callbacks cb 0x55de8612c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:29:52.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8069354700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de862365d0 0x55de86242940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8069354700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de862365d0 0x55de86242940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55178/0 (socket says 172.21.15.2:55178) 2023-06-23T18:29:52.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8069354700 1 -- 172.21.15.2:0/3210771104 learned_addr learned my addr 172.21.15.2:0/3210771104 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:29:52.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8068b53700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86242e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8069b55700 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8624afb0 0x55de86245ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8068b53700 1 -- 172.21.15.2:0/3210771104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8624afb0 msgr2=0x55de86245ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:52.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8068b53700 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8624afb0 0x55de86245ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8068b53700 1 -- 172.21.15.2:0/3210771104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de862365d0 msgr2=0x55de86242940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:29:52.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8068b53700 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de862365d0 0x55de86242940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8068b53700 1 -- 172.21.15.2:0/3210771104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de86008630 con 0x55de86237000 2023-06-23T18:29:52.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8069354700 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de862365d0 0x55de86242940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:29:52.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f8068b53700 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86242e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8054011ab0 tx=0x7f8054003840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926179cf31848487 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:29:52.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.938+0000 7f804cff9700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:29:52.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f804cff9700 1 -- 172.21.15.2:0/3210771104 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805402a040 con 0x55de86237000 2023-06-23T18:29:52.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 1 -- 172.21.15.2:0/3210771104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8054021070 con 0x55de86237000 2023-06-23T18:29:52.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f804cff9700 1 -- 172.21.15.2:0/3210771104 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f805400eb50 con 0x55de86237000 2023-06-23T18:29:52.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f804cff9700 1 -- 172.21.15.2:0/3210771104 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8054011ae0 con 0x55de86237000 2023-06-23T18:29:52.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 10 client.25460 fetch_fsmap learned FSMap version 30 2023-06-23T18:29:52.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 1 -- 172.21.15.2:0/3210771104 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f805c00e630 con 0x55de86237000 2023-06-23T18:29:52.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f804cff9700 1 -- 172.21.15.2:0/3210771104 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8054021070 con 0x55de86237000 2023-06-23T18:29:52.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f804cff9700 1 -- 172.21.15.2:0/3210771104 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 30) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8054003c40 con 0x55de86237000 2023-06-23T18:29:52.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 10 client.25460 fetch_fsmap finished waiting for FSMap version 30 2023-06-23T18:29:52.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 10 client.25460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:29:52.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 20 client.25460 populate_metadata read hostname 'smithi002' 2023-06-23T18:29:52.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55de8629fdb0 0x55de862a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:29:52.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 4 client.25460 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:29:52.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.939+0000 7f807773db80 1 -- 172.21.15.2:0/3210771104 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55de86142050 con 0x55de8629fdb0 2023-06-23T18:29:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f8069354700 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55de8629fdb0 0x55de862a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:29:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f804cff9700 10 client.25460 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:29:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f8069354700 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55de8629fdb0 0x55de862a21c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12f6ca07c4f516d server_cookie=c7e2ce5657290461 in_seq=0 out_seq=0 2023-06-23T18:29:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f804cff9700 1 -- 172.21.15.2:0/3210771104 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55de86142050 con 0x55de8629fdb0 2023-06-23T18:29:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f804cff9700 10 client.25460 handle_command_reply: tid=0 2023-06-23T18:29:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 client.25460 shutdown 2023-06-23T18:29:52.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 -- 172.21.15.2:0/3210771104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 msgr2=0x55de86242e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86242e80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8054011ab0 tx=0x7f8054003840 comp rx=0 tx=0).stop 2023-06-23T18:29:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 -- 172.21.15.2:0/3210771104 shutdown_connections 2023-06-23T18:29:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55de8629fdb0 0x55de862a21c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de8624afb0 0x55de86245ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55de862365d0 0x55de86242940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 --2- 172.21.15.2:0/3210771104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55de86237000 0x55de86242e80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f804cff9700 0 client.25460 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:29:52.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.940+0000 7f807773db80 1 -- 172.21.15.2:0/3210771104 >> 172.21.15.2:0/3210771104 conn(0x55de8622b3f0 msgr2=0x55de8623d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f807773db80 1 -- 172.21.15.2:0/3210771104 shutdown_connections 2023-06-23T18:29:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f807773db80 1 -- 172.21.15.2:0/3210771104 wait complete. 2023-06-23T18:29:52.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f807773db80 20 client.25460 trim_cache size 0 max 16384 2023-06-23T18:29:52.954 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:29:52.954 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "scrub active (2 inodes in the stack)", 2023-06-23T18:29:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:29:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:29:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:29:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:29:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:29:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:29:52.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:29:52.955 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:29:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 -- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f80500a1150 msgr2=0x7f80500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f80500a1150 0x7f80500a3600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8054009c60 tx=0x7f8054009260 comp rx=0 tx=0).stop 2023-06-23T18:29:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 -- 172.21.15.2:0/1508145662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80640fea60 msgr2=0x7f8064195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:29:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80640fea60 0x7f8064195810 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8060003ec0 tx=0x7f8060005740 comp rx=0 tx=0).stop 2023-06-23T18:29:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 -- 172.21.15.2:0/1508145662 shutdown_connections 2023-06-23T18:29:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f80641005e0 0x7f806419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80640fea60 0x7f8064195810 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f80500a1150 0x7f80500a3600 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 --2- 172.21.15.2:0/1508145662 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80640ffc50 0x7f8064195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:29:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 -- 172.21.15.2:0/1508145662 >> 172.21.15.2:0/1508145662 conn(0x7f80640fa060 msgr2=0x7f80640689b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:29:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 -- 172.21.15.2:0/1508145662 shutdown_connections 2023-06-23T18:29:52.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:29:52.941+0000 7f806b5b8700 1 -- 172.21.15.2:0/1508145662 wait complete. 2023-06-23T18:29:52.958 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:29:53.477 INFO:teuthology.orchestra.run.smithi002.stderr:47300000 of 50000000 tuples (94%) done (elapsed 436.72 s, remaining 24.93 s) 2023-06-23T18:29:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:47400000 of 50000000 tuples (94%) done (elapsed 437.65 s, remaining 24.01 s) 2023-06-23T18:29:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:54 smithi082 ceph-mon[97889]: pgmap v532: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 47 MiB/s wr, 27 op/s 2023-06-23T18:29:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:54 smithi017 ceph-mon[97472]: pgmap v532: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 47 MiB/s wr, 27 op/s 2023-06-23T18:29:54.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:54 smithi002 ceph-mon[95947]: pgmap v532: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 47 MiB/s wr, 27 op/s 2023-06-23T18:29:55.310 INFO:teuthology.orchestra.run.smithi002.stderr:47500000 of 50000000 tuples (95%) done (elapsed 438.55 s, remaining 23.08 s) 2023-06-23T18:29:56.136 INFO:teuthology.orchestra.run.smithi002.stderr:47600000 of 50000000 tuples (95%) done (elapsed 439.38 s, remaining 22.15 s) 2023-06-23T18:29:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:56 smithi082 ceph-mon[97889]: pgmap v533: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 26 op/s 2023-06-23T18:29:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:56 smithi017 ceph-mon[97472]: pgmap v533: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 26 op/s 2023-06-23T18:29:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:56 smithi002 ceph-mon[95947]: pgmap v533: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 40 MiB/s wr, 26 op/s 2023-06-23T18:29:56.971 INFO:teuthology.orchestra.run.smithi002.stderr:47700000 of 50000000 tuples (95%) done (elapsed 440.21 s, remaining 21.23 s) 2023-06-23T18:29:57.817 INFO:teuthology.orchestra.run.smithi002.stderr:47800000 of 50000000 tuples (95%) done (elapsed 441.06 s, remaining 20.30 s) 2023-06-23T18:29:58.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:29:58 smithi082 ceph-mon[97889]: pgmap v534: 97 pgs: 97 active+clean; 6.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 42 MiB/s wr, 31 op/s 2023-06-23T18:29:58.654 INFO:teuthology.orchestra.run.smithi002.stderr:47900000 of 50000000 tuples (95%) done (elapsed 441.90 s, remaining 19.37 s) 2023-06-23T18:29:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:29:58 smithi017 ceph-mon[97472]: pgmap v534: 97 pgs: 97 active+clean; 6.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 42 MiB/s wr, 31 op/s 2023-06-23T18:29:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:29:58 smithi002 ceph-mon[95947]: pgmap v534: 97 pgs: 97 active+clean; 6.9 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 42 MiB/s wr, 31 op/s 2023-06-23T18:29:59.585 INFO:teuthology.orchestra.run.smithi002.stderr:48000000 of 50000000 tuples (96%) done (elapsed 442.83 s, remaining 18.45 s) 2023-06-23T18:30:00.402 INFO:teuthology.orchestra.run.smithi002.stderr:48100000 of 50000000 tuples (96%) done (elapsed 443.64 s, remaining 17.52 s) 2023-06-23T18:30:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:00 smithi082 ceph-mon[97889]: pgmap v535: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 43 MiB/s wr, 37 op/s 2023-06-23T18:30:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:00 smithi082 ceph-mon[97889]: overall HEALTH_OK 2023-06-23T18:30:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:00 smithi017 ceph-mon[97472]: pgmap v535: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 43 MiB/s wr, 37 op/s 2023-06-23T18:30:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:00 smithi017 ceph-mon[97472]: overall HEALTH_OK 2023-06-23T18:30:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:00 smithi002 ceph-mon[95947]: pgmap v535: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 43 MiB/s wr, 37 op/s 2023-06-23T18:30:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:00 smithi002 ceph-mon[95947]: overall HEALTH_OK 2023-06-23T18:30:01.249 INFO:teuthology.orchestra.run.smithi002.stderr:48200000 of 50000000 tuples (96%) done (elapsed 444.49 s, remaining 16.60 s) 2023-06-23T18:30:02.130 INFO:teuthology.orchestra.run.smithi002.stderr:48300000 of 50000000 tuples (96%) done (elapsed 445.37 s, remaining 15.68 s) 2023-06-23T18:30:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:02 smithi082 ceph-mon[97889]: pgmap v536: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 35 op/s 2023-06-23T18:30:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:02 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:30:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:02 smithi017 ceph-mon[97472]: pgmap v536: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 35 op/s 2023-06-23T18:30:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:02 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:30:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:02 smithi002 ceph-mon[95947]: pgmap v536: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 36 MiB/s wr, 35 op/s 2023-06-23T18:30:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:02 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:30:03.013 INFO:teuthology.orchestra.run.smithi002.stderr:48400000 of 50000000 tuples (96%) done (elapsed 446.25 s, remaining 14.75 s) 2023-06-23T18:30:03.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:30:03.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:30:03.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:30:03.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:30:03.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:30:03.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:30:03.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:30:03.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:30:03.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:30:03.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:30:03.865 INFO:teuthology.orchestra.run.smithi002.stderr:48500000 of 50000000 tuples (97%) done (elapsed 447.11 s, remaining 13.83 s) 2023-06-23T18:30:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:04 smithi082 ceph-mon[97889]: pgmap v537: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 45 MiB/s wr, 38 op/s 2023-06-23T18:30:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:04 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-06-23T18:30:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:04 smithi082 ceph-mon[97889]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-06-23T18:30:04.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:04 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi002 to 197.2M 2023-06-23T18:30:04.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:04 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi017 to 197.2M 2023-06-23T18:30:04.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:04 smithi082 ceph-mon[97889]: Unable to set osd_memory_target on smithi002 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:30:04.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:04 smithi082 ceph-mon[97889]: Unable to set osd_memory_target on smithi017 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:30:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:04 smithi017 ceph-mon[97472]: pgmap v537: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 45 MiB/s wr, 38 op/s 2023-06-23T18:30:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:04 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-06-23T18:30:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:04 smithi017 ceph-mon[97472]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-06-23T18:30:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:04 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi002 to 197.2M 2023-06-23T18:30:04.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:04 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi017 to 197.2M 2023-06-23T18:30:04.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:04 smithi017 ceph-mon[97472]: Unable to set osd_memory_target on smithi002 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:30:04.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:04 smithi017 ceph-mon[97472]: Unable to set osd_memory_target on smithi017 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:30:04.745 INFO:teuthology.orchestra.run.smithi002.stderr:48600000 of 50000000 tuples (97%) done (elapsed 447.99 s, remaining 12.90 s) 2023-06-23T18:30:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:04 smithi002 ceph-mon[95947]: pgmap v537: 97 pgs: 97 active+clean; 7.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 45 MiB/s wr, 38 op/s 2023-06-23T18:30:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:04 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-06-23T18:30:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:04 smithi002 ceph-mon[95947]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-06-23T18:30:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:04 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi002 to 197.2M 2023-06-23T18:30:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:04 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi017 to 197.2M 2023-06-23T18:30:04.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:04 smithi002 ceph-mon[95947]: Unable to set osd_memory_target on smithi002 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:30:04.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:04 smithi002 ceph-mon[95947]: Unable to set osd_memory_target on smithi017 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:30:05.585 INFO:teuthology.orchestra.run.smithi002.stderr:48700000 of 50000000 tuples (97%) done (elapsed 448.83 s, remaining 11.98 s) 2023-06-23T18:30:06.438 INFO:teuthology.orchestra.run.smithi002.stderr:48800000 of 50000000 tuples (97%) done (elapsed 449.68 s, remaining 11.06 s) 2023-06-23T18:30:06.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:06 smithi082 ceph-mon[97889]: pgmap v538: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 37 op/s 2023-06-23T18:30:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:06 smithi017 ceph-mon[97472]: pgmap v538: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 37 op/s 2023-06-23T18:30:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:06 smithi002 ceph-mon[95947]: pgmap v538: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 39 MiB/s wr, 37 op/s 2023-06-23T18:30:07.449 INFO:teuthology.orchestra.run.smithi002.stderr:48900000 of 50000000 tuples (97%) done (elapsed 450.69 s, remaining 10.14 s) 2023-06-23T18:30:08.310 INFO:teuthology.orchestra.run.smithi002.stderr:49000000 of 50000000 tuples (98%) done (elapsed 451.55 s, remaining 9.22 s) 2023-06-23T18:30:08.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:08 smithi082 ceph-mon[97889]: pgmap v539: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 42 MiB/s wr, 34 op/s 2023-06-23T18:30:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:08 smithi017 ceph-mon[97472]: pgmap v539: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 42 MiB/s wr, 34 op/s 2023-06-23T18:30:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:08 smithi002 ceph-mon[95947]: pgmap v539: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 42 MiB/s wr, 34 op/s 2023-06-23T18:30:09.173 INFO:teuthology.orchestra.run.smithi002.stderr:49100000 of 50000000 tuples (98%) done (elapsed 452.41 s, remaining 8.29 s) 2023-06-23T18:30:10.010 INFO:teuthology.orchestra.run.smithi002.stderr:49200000 of 50000000 tuples (98%) done (elapsed 453.25 s, remaining 7.37 s) 2023-06-23T18:30:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:10 smithi082 ceph-mon[97889]: pgmap v540: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 43 MiB/s wr, 30 op/s 2023-06-23T18:30:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:10 smithi017 ceph-mon[97472]: pgmap v540: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 43 MiB/s wr, 30 op/s 2023-06-23T18:30:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:10 smithi002 ceph-mon[95947]: pgmap v540: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 43 MiB/s wr, 30 op/s 2023-06-23T18:30:10.879 INFO:teuthology.orchestra.run.smithi002.stderr:49300000 of 50000000 tuples (98%) done (elapsed 454.12 s, remaining 6.45 s) 2023-06-23T18:30:11.719 INFO:teuthology.orchestra.run.smithi002.stderr:49400000 of 50000000 tuples (98%) done (elapsed 454.96 s, remaining 5.53 s) 2023-06-23T18:30:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:49500000 of 50000000 tuples (99%) done (elapsed 455.76 s, remaining 4.60 s) 2023-06-23T18:30:12.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:12 smithi082 ceph-mon[97889]: pgmap v541: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 39 MiB/s wr, 46 op/s 2023-06-23T18:30:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:12 smithi017 ceph-mon[97472]: pgmap v541: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 39 MiB/s wr, 46 op/s 2023-06-23T18:30:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:12 smithi002 ceph-mon[95947]: pgmap v541: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 39 MiB/s wr, 46 op/s 2023-06-23T18:30:13.395 INFO:teuthology.orchestra.run.smithi002.stderr:49600000 of 50000000 tuples (99%) done (elapsed 456.64 s, remaining 3.68 s) 2023-06-23T18:30:14.255 INFO:teuthology.orchestra.run.smithi002.stderr:49700000 of 50000000 tuples (99%) done (elapsed 457.50 s, remaining 2.76 s) 2023-06-23T18:30:14.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:14 smithi082 ceph-mon[97889]: pgmap v542: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 47 MiB/s wr, 48 op/s 2023-06-23T18:30:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:14 smithi017 ceph-mon[97472]: pgmap v542: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 47 MiB/s wr, 48 op/s 2023-06-23T18:30:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:14 smithi002 ceph-mon[95947]: pgmap v542: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 47 MiB/s wr, 48 op/s 2023-06-23T18:30:15.077 INFO:teuthology.orchestra.run.smithi002.stderr:49800000 of 50000000 tuples (99%) done (elapsed 458.32 s, remaining 1.84 s) 2023-06-23T18:30:15.911 INFO:teuthology.orchestra.run.smithi002.stderr:49900000 of 50000000 tuples (99%) done (elapsed 459.15 s, remaining 0.92 s) 2023-06-23T18:30:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:16 smithi082 ceph-mon[97889]: pgmap v543: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:30:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:16 smithi017 ceph-mon[97472]: pgmap v543: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:30:16.799 INFO:teuthology.orchestra.run.smithi002.stderr:50000000 of 50000000 tuples (100%) done (elapsed 460.04 s, remaining 0.00 s) 2023-06-23T18:30:16.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:16 smithi002 ceph-mon[95947]: pgmap v543: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 42 MiB/s wr, 46 op/s 2023-06-23T18:30:16.854 INFO:teuthology.orchestra.run.smithi002.stderr:vacuum... 2023-06-23T18:30:18.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:18 smithi082 ceph-mon[97889]: pgmap v544: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:30:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:18 smithi017 ceph-mon[97472]: pgmap v544: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:30:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:18 smithi002 ceph-mon[95947]: pgmap v544: 97 pgs: 97 active+clean; 7.2 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 42 MiB/s wr, 45 op/s 2023-06-23T18:30:20.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:20 smithi082 ceph-mon[97889]: pgmap v545: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 48 op/s 2023-06-23T18:30:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:20 smithi017 ceph-mon[97472]: pgmap v545: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 48 op/s 2023-06-23T18:30:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:20 smithi002 ceph-mon[95947]: pgmap v545: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 41 MiB/s wr, 48 op/s 2023-06-23T18:30:22.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:22 smithi017 ceph-mon[97472]: pgmap v546: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 38 MiB/s wr, 47 op/s 2023-06-23T18:30:22.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:22 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:30:22.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:22 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:30:22.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:22 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:22.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:22 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:22 smithi002 ceph-mon[95947]: pgmap v546: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 38 MiB/s wr, 47 op/s 2023-06-23T18:30:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:22 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:30:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:22 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:30:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:22 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:22 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:22 smithi082 ceph-mon[97889]: pgmap v546: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 38 MiB/s wr, 47 op/s 2023-06-23T18:30:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:22 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:30:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:22 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:30:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:22 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:22 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:22.960 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:30:23.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 -- 172.21.15.2:0/3607823636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c0fece0 msgr2=0x7ff46c0ff150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 --2- 172.21.15.2:0/3607823636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c0fece0 0x7ff46c0ff150 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009990 tx=0x7ff45c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 -- 172.21.15.2:0/3607823636 shutdown_connections 2023-06-23T18:30:23.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 --2- 172.21.15.2:0/3607823636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff46c0ff690 0x7ff46c106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 --2- 172.21.15.2:0/3607823636 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff46c100120 0x7ff46c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 --2- 172.21.15.2:0/3607823636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c0fece0 0x7ff46c0ff150 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 -- 172.21.15.2:0/3607823636 >> 172.21.15.2:0/3607823636 conn(0x7ff46c0f9820 msgr2=0x7ff46c0fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:23.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 -- 172.21.15.2:0/3607823636 shutdown_connections 2023-06-23T18:30:23.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.097+0000 7ff473fc3700 1 -- 172.21.15.2:0/3607823636 wait complete. 2023-06-23T18:30:23.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff473fc3700 1 Processor -- start 2023-06-23T18:30:23.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff473fc3700 1 -- start start 2023-06-23T18:30:23.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff473fc3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff46c0fece0 0x7ff46c195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff473fc3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff46c0ff690 0x7ff46c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff473fc3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c100120 0x7ff46c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff473fc3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c19d3c0 con 0x7ff46c100120 2023-06-23T18:30:23.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff473fc3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c19d500 con 0x7ff46c0ff690 2023-06-23T18:30:23.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff473fc3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c19d690 con 0x7ff46c0fece0 2023-06-23T18:30:23.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff471d5f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff46c0fece0 0x7ff46c195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff472560700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c100120 0x7ff46c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff471d5f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff46c0fece0 0x7ff46c195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55530/0 (socket says 172.21.15.2:55530) 2023-06-23T18:30:23.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff472560700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c100120 0x7ff46c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55196/0 (socket says 172.21.15.2:55196) 2023-06-23T18:30:23.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff471d5f700 1 -- 172.21.15.2:0/1220965676 learned_addr learned my addr 172.21.15.2:0/1220965676 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:23.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.098+0000 7ff47155e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff46c0ff690 0x7ff46c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff471d5f700 1 -- 172.21.15.2:0/1220965676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff46c0ff690 msgr2=0x7ff46c195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff471d5f700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff46c0ff690 0x7ff46c195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff471d5f700 1 -- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c100120 msgr2=0x7ff46c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff471d5f700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c100120 0x7ff46c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff471d5f700 1 -- 172.21.15.2:0/1220965676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45c0095f0 con 0x7ff46c0fece0 2023-06-23T18:30:23.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff472560700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c100120 0x7ff46c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:23.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff471d5f700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff46c0fece0 0x7ff46c195840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff46800d950 tx=0x7ff46800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:23.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff462ffd700 1 -- 172.21.15.2:0/1220965676 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468009940 con 0x7ff46c0fece0 2023-06-23T18:30:23.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff473fc3700 1 -- 172.21.15.2:0/1220965676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c19d970 con 0x7ff46c0fece0 2023-06-23T18:30:23.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff473fc3700 1 -- 172.21.15.2:0/1220965676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c19ded0 con 0x7ff46c0fece0 2023-06-23T18:30:23.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff462ffd700 1 -- 172.21.15.2:0/1220965676 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff468010460 con 0x7ff46c0fece0 2023-06-23T18:30:23.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.099+0000 7ff462ffd700 1 -- 172.21.15.2:0/1220965676 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46800f620 con 0x7ff46c0fece0 2023-06-23T18:30:23.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.102+0000 7ff462ffd700 1 -- 172.21.15.2:0/1220965676 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff4680105d0 con 0x7ff46c0fece0 2023-06-23T18:30:23.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.102+0000 7ff480148b80 1 Processor -- start 2023-06-23T18:30:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff480148b80 1 -- start start 2023-06-23T18:30:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff462ffd700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4580a1150 0x7ff4580a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff480148b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a2090 0x55b0131a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff480148b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff480148b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a3410 0x55b0131a3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff480148b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0130a2cb0 con 0x55b0131a2090 2023-06-23T18:30:23.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff480148b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0130f5a40 con 0x55b0131a3410 2023-06-23T18:30:23.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff480148b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0130eeed0 con 0x55b0131a29e0 2023-06-23T18:30:23.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff462ffd700 1 -- 172.21.15.2:0/1220965676 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff4680cc680 con 0x7ff46c0fece0 2023-06-23T18:30:23.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff472560700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4580a1150 0x7ff4580a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff47155e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff47155e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55538/0 (socket says 172.21.15.2:55538) 2023-06-23T18:30:23.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff47155e700 1 -- 172.21.15.2:0/2852911023 learned_addr learned my addr 172.21.15.2:0/2852911023 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:23.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff471d5f700 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a2090 0x55b0131a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.103+0000 7ff472560700 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a3410 0x55b0131a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff47155e700 1 -- 172.21.15.2:0/2852911023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a3410 msgr2=0x55b0131a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:23.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff47155e700 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a3410 0x55b0131a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff47155e700 1 -- 172.21.15.2:0/2852911023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a2090 msgr2=0x55b0131a24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:23.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff47155e700 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a2090 0x55b0131a24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff47155e700 1 -- 172.21.15.2:0/2852911023 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b012f70630 con 0x55b0131a29e0 2023-06-23T18:30:23.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff472560700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4580a1150 0x7ff4580a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4640108c0 tx=0x7ff464010c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:23.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff47155e700 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131a2e80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009990 tx=0x7ff45c005cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fa7b521182bf950 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff457fff700 1 -- 172.21.15.2:0/2852911023 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c01d070 con 0x55b0131a29e0 2023-06-23T18:30:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff457fff700 1 -- 172.21.15.2:0/2852911023 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff45c01e040 con 0x55b0131a29e0 2023-06-23T18:30:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.104+0000 7ff457fff700 1 -- 172.21.15.2:0/2852911023 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c00b7b0 con 0x55b0131a29e0 2023-06-23T18:30:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 -- 172.21.15.2:0/2852911023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 msgr2=0x55b0131a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131a2e80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff45c009990 tx=0x7ff45c005cb0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 -- 172.21.15.2:0/2852911023 shutdown_connections 2023-06-23T18:30:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a3410 0x55b0131a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a2090 0x55b0131a24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 --2- 172.21.15.2:0/2852911023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131a2e80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 -- 172.21.15.2:0/2852911023 >> 172.21.15.2:0/2852911023 conn(0x55b0131933f0 msgr2=0x55b013198520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:23.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 -- 172.21.15.2:0/2852911023 shutdown_connections 2023-06-23T18:30:23.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 -- 172.21.15.2:0/2852911023 wait complete. 2023-06-23T18:30:23.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 -- start start 2023-06-23T18:30:23.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.105+0000 7ff480148b80 1 Processor -- start 2023-06-23T18:30:23.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff480148b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a2090 0x55b0131ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff480148b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a29e0 0x55b0131abde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff480148b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff480148b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01308eac0 con 0x55b0131a3410 2023-06-23T18:30:23.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff480148b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b01308e7c0 con 0x55b0131a29e0 2023-06-23T18:30:23.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff480148b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b012f72920 con 0x55b0131a2090 2023-06-23T18:30:23.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff480148b80 10 client.0 _ll_register_callbacks cb 0x55b013094260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:23.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff472560700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff472560700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55210/0 (socket says 172.21.15.2:55210) 2023-06-23T18:30:23.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff472560700 1 -- 172.21.15.2:0/2797287197 learned_addr learned my addr 172.21.15.2:0/2797287197 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:23.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff471d5f700 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a2090 0x55b0131ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff47155e700 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a29e0 0x55b0131abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff472560700 1 -- 172.21.15.2:0/2797287197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a2090 msgr2=0x55b0131ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff472560700 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a2090 0x55b0131ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff472560700 1 -- 172.21.15.2:0/2797287197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a29e0 msgr2=0x55b0131abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff472560700 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a29e0 0x55b0131abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.106+0000 7ff472560700 1 -- 172.21.15.2:0/2797287197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c1078f0 con 0x55b0131a3410 2023-06-23T18:30:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff472560700 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131b2e20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff464017dd0 tx=0x7ff46400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51407cdc7c8a472e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff4557fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff4557fa700 1 -- 172.21.15.2:0/2797287197 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46400e040 con 0x55b0131a3410 2023-06-23T18:30:23.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff480148b80 1 -- 172.21.15.2:0/2797287197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b013190bb0 con 0x55b0131a3410 2023-06-23T18:30:23.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff4557fa700 1 -- 172.21.15.2:0/2797287197 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff464007ba0 con 0x55b0131a3410 2023-06-23T18:30:23.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff4557fa700 1 -- 172.21.15.2:0/2797287197 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff464007da0 con 0x55b0131a3410 2023-06-23T18:30:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff480148b80 10 client.15870 fetch_fsmap learned FSMap version 31 2023-06-23T18:30:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff480148b80 1 -- 172.21.15.2:0/2797287197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b012f1cde0 con 0x55b0131a3410 2023-06-23T18:30:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.107+0000 7ff4557fa700 1 -- 172.21.15.2:0/2797287197 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff464034020 con 0x55b0131a3410 2023-06-23T18:30:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff4557fa700 1 -- 172.21.15.2:0/2797287197 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 31) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff464019430 con 0x55b0131a3410 2023-06-23T18:30:23.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff480148b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 31 2023-06-23T18:30:23.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff480148b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:23.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff480148b80 20 client.15870 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:23.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff480148b80 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55b0131b7750 0x55b0131b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff480148b80 4 client.15870 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:30:23.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff480148b80 1 -- 172.21.15.2:0/2797287197 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0130e7eb0 con 0x55b0131b7750 2023-06-23T18:30:23.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff471d5f700 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55b0131b7750 0x55b0131b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff4557fa700 10 client.15870 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:23.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.108+0000 7ff471d5f700 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55b0131b7750 0x55b0131b9b60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eacdf4d343d43f6 server_cookie=8bfbfe1269dd43fb in_seq=0 out_seq=0 2023-06-23T18:30:23.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff4557fa700 1 -- 172.21.15.2:0/2797287197 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b0130e7eb0 con 0x55b0131b7750 2023-06-23T18:30:23.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff4557fa700 10 client.15870 handle_command_reply: tid=0 2023-06-23T18:30:23.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 client.15870 shutdown 2023-06-23T18:30:23.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 -- 172.21.15.2:0/2797287197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 msgr2=0x55b0131b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131b2e20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff464017dd0 tx=0x7ff46400b040 comp rx=0 tx=0).stop 2023-06-23T18:30:23.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 -- 172.21.15.2:0/2797287197 shutdown_connections 2023-06-23T18:30:23.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55b0131b7750 0x55b0131b9b60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131b2e20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff4557fa700 0 client.15870 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:23.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a2090 0x55b0131ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 --2- 172.21.15.2:0/2797287197 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a29e0 0x55b0131abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 -- 172.21.15.2:0/2797287197 >> 172.21.15.2:0/2797287197 conn(0x55b0131933f0 msgr2=0x55b013197c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:23.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 -- 172.21.15.2:0/2797287197 shutdown_connections 2023-06-23T18:30:23.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.109+0000 7ff480148b80 1 -- 172.21.15.2:0/2797287197 wait complete. 2023-06-23T18:30:23.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.110+0000 7ff480148b80 20 client.15870 trim_cache size 0 max 16384 2023-06-23T18:30:23.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.135+0000 7ff480148b80 1 Processor -- start 2023-06-23T18:30:23.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff480148b80 1 -- start start 2023-06-23T18:30:23.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff480148b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff480148b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff480148b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff480148b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0131ab470 con 0x55b0131a3410 2023-06-23T18:30:23.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff480148b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b013199030 con 0x55b0131a2090 2023-06-23T18:30:23.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff480148b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0131aa460 con 0x55b0131a29e0 2023-06-23T18:30:23.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff472560700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff472560700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131a96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55218/0 (socket says 172.21.15.2:55218) 2023-06-23T18:30:23.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff472560700 1 -- 172.21.15.2:0/440606011 learned_addr learned my addr 172.21.15.2:0/440606011 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:23.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff471d5f700 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff47155e700 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff471d5f700 1 -- 172.21.15.2:0/440606011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 msgr2=0x55b0131b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:23.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff471d5f700 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff471d5f700 1 -- 172.21.15.2:0/440606011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 msgr2=0x55b0131a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:23.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff471d5f700 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff471d5f700 1 -- 172.21.15.2:0/440606011 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b012f05c60 con 0x55b0131a2090 2023-06-23T18:30:23.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff472560700 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.136+0000 7ff47155e700 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.137+0000 7ff471d5f700 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b3710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff468099790 tx=0x7ff468095e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a63e0d8fcee311b1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.137+0000 7ff460ff9700 1 -- 172.21.15.2:0/440606011 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468099a90 con 0x55b0131a2090 2023-06-23T18:30:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.137+0000 7ff460ff9700 1 -- 172.21.15.2:0/440606011 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff46809dac0 con 0x55b0131a2090 2023-06-23T18:30:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.137+0000 7ff460ff9700 1 -- 172.21.15.2:0/440606011 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468010b30 con 0x55b0131a2090 2023-06-23T18:30:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 -- 172.21.15.2:0/440606011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 msgr2=0x55b0131b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b3710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff468099790 tx=0x7ff468095e50 comp rx=0 tx=0).stop 2023-06-23T18:30:23.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 -- 172.21.15.2:0/440606011 shutdown_connections 2023-06-23T18:30:23.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b3710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 --2- 172.21.15.2:0/440606011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 -- 172.21.15.2:0/440606011 >> 172.21.15.2:0/440606011 conn(0x55b0131933f0 msgr2=0x55b0131aa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:23.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 -- 172.21.15.2:0/440606011 shutdown_connections 2023-06-23T18:30:23.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 -- 172.21.15.2:0/440606011 wait complete. 2023-06-23T18:30:23.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 -- start start 2023-06-23T18:30:23.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.138+0000 7ff480148b80 1 Processor -- start 2023-06-23T18:30:23.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff480148b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff480148b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff480148b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131af330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff480148b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0131afa40 con 0x55b0131a3410 2023-06-23T18:30:23.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff480148b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0131afbb0 con 0x55b0131a2090 2023-06-23T18:30:23.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff480148b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0131afd70 con 0x55b0131a29e0 2023-06-23T18:30:23.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff480148b80 10 client.0 _ll_register_callbacks cb 0x55b013094260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:23.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff472560700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45812/0 (socket says 172.21.15.2:45812) 2023-06-23T18:30:23.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 -- 172.21.15.2:0/2988250799 learned_addr learned my addr 172.21.15.2:0/2988250799 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:23.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff47155e700 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 -- 172.21.15.2:0/2988250799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 msgr2=0x55b0131b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:23.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 -- 172.21.15.2:0/2988250799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 msgr2=0x55b0131af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:23.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131af330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 -- 172.21.15.2:0/2988250799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b012f70630 con 0x55b0131a2090 2023-06-23T18:30:23.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff47155e700 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:23.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff472560700 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:23.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff471d5f700 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b2830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff468010880 tx=0x7ff468097eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1741191488a716ee server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:23.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.139+0000 7ff4557fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:30:23.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff4557fa700 1 -- 172.21.15.2:0/2988250799 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468010ad0 con 0x55b0131a2090 2023-06-23T18:30:23.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff4557fa700 1 -- 172.21.15.2:0/2988250799 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff46804e070 con 0x55b0131a2090 2023-06-23T18:30:23.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 1 -- 172.21.15.2:0/2988250799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff45c01e040 con 0x55b0131a2090 2023-06-23T18:30:23.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff4557fa700 1 -- 172.21.15.2:0/2988250799 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4680b2230 con 0x55b0131a2090 2023-06-23T18:30:23.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 10 client.25478 fetch_fsmap learned FSMap version 31 2023-06-23T18:30:23.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 1 -- 172.21.15.2:0/2988250799 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b012f1cde0 con 0x55b0131a2090 2023-06-23T18:30:23.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff4557fa700 1 -- 172.21.15.2:0/2988250799 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff45c01e040 con 0x55b0131a2090 2023-06-23T18:30:23.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff4557fa700 1 -- 172.21.15.2:0/2988250799 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 31) v1 ==== 7466+0+0 (secure 0 0 0) 0x7ff468099010 con 0x55b0131a2090 2023-06-23T18:30:23.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 10 client.25478 fetch_fsmap finished waiting for FSMap version 31 2023-06-23T18:30:23.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 10 client.25478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:23.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 20 client.25478 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:23.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55b01320c8b0 0x55b01320ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:23.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 4 client.25478 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:30:23.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff480148b80 1 -- 172.21.15.2:0/2988250799 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b0130aa050 con 0x55b01320c8b0 2023-06-23T18:30:23.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.140+0000 7ff47155e700 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55b01320c8b0 0x55b01320ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:23.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff47155e700 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55b01320c8b0 0x55b01320ecc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48d690a2da3d4fe server_cookie=fd62ff486743ba81 in_seq=0 out_seq=0 2023-06-23T18:30:23.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff4557fa700 10 client.25478 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:23.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff4557fa700 1 -- 172.21.15.2:0/2988250799 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b0130aa050 con 0x55b01320c8b0 2023-06-23T18:30:23.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff4557fa700 10 client.25478 handle_command_reply: tid=0 2023-06-23T18:30:23.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 client.25478 shutdown 2023-06-23T18:30:23.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 -- 172.21.15.2:0/2988250799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 msgr2=0x55b0131b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b2830 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff468010880 tx=0x7ff468097eb0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 -- 172.21.15.2:0/2988250799 shutdown_connections 2023-06-23T18:30:23.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55b01320c8b0 0x55b01320ecc0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b0131a3410 0x55b0131af330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b0131a2090 0x55b0131b2830 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff4557fa700 0 client.25478 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 --2- 172.21.15.2:0/2988250799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b0131a29e0 0x55b0131b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 -- 172.21.15.2:0/2988250799 >> 172.21.15.2:0/2988250799 conn(0x55b0131933f0 msgr2=0x55b0131a61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.141+0000 7ff480148b80 1 -- 172.21.15.2:0/2988250799 shutdown_connections 2023-06-23T18:30:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff480148b80 1 -- 172.21.15.2:0/2988250799 wait complete. 2023-06-23T18:30:23.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff480148b80 20 client.25478 trim_cache size 0 max 16384 2023-06-23T18:30:23.158 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:30:23.158 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:30:23.158 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:30:23.158 INFO:teuthology.orchestra.run.smithi002.stdout: "7c1baf78-91be-46f9-bb0f-5830f8d5d098": { 2023-06-23T18:30:23.158 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:30:23.158 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7c1baf78-91be-46f9-bb0f-5830f8d5d098", 2023-06-23T18:30:23.159 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:30:23.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:30:23.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:30:23.159 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:30:23.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 -- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4580a1150 msgr2=0x7ff4580a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4580a1150 0x7ff4580a3600 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff4640108c0 tx=0x7ff464010c80 comp rx=0 tx=0).stop 2023-06-23T18:30:23.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 -- 172.21.15.2:0/1220965676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff46c0fece0 msgr2=0x7ff46c195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:23.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff46c0fece0 0x7ff46c195840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff46800d950 tx=0x7ff46800dc60 comp rx=0 tx=0).stop 2023-06-23T18:30:23.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 -- 172.21.15.2:0/1220965676 shutdown_connections 2023-06-23T18:30:23.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff46c100120 0x7ff46c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff46c0fece0 0x7ff46c195840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4580a1150 0x7ff4580a3600 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 --2- 172.21.15.2:0/1220965676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff46c0ff690 0x7ff46c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:23.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 -- 172.21.15.2:0/1220965676 >> 172.21.15.2:0/1220965676 conn(0x7ff46c0f9820 msgr2=0x7ff46c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:23.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 -- 172.21.15.2:0/1220965676 shutdown_connections 2023-06-23T18:30:23.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:23.142+0000 7ff473fc3700 1 -- 172.21.15.2:0/1220965676 wait complete. 2023-06-23T18:30:23.162 INFO:tasks.cephfs.filesystem:scrub status for tag:7c1baf78-91be-46f9-bb0f-5830f8d5d098 - {'path': '/', 'tag': '7c1baf78-91be-46f9-bb0f-5830f8d5d098', 'options': 'recursive,force'} 2023-06-23T18:30:24.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:24 smithi017 ceph-mon[97472]: pgmap v547: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 45 MiB/s wr, 30 op/s 2023-06-23T18:30:24.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:24 smithi002 ceph-mon[95947]: pgmap v547: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 45 MiB/s wr, 30 op/s 2023-06-23T18:30:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:24 smithi082 ceph-mon[97889]: pgmap v547: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 45 MiB/s wr, 30 op/s 2023-06-23T18:30:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:26 smithi017 ceph-mon[97472]: pgmap v548: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 29 op/s 2023-06-23T18:30:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:26 smithi002 ceph-mon[95947]: pgmap v548: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 29 op/s 2023-06-23T18:30:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:26 smithi082 ceph-mon[97889]: pgmap v548: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 45 MiB/s wr, 29 op/s 2023-06-23T18:30:27.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:27 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:30:27.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:27 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:30:27.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:27 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:30:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:28 smithi017 ceph-mon[97472]: pgmap v549: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 MiB/s wr, 29 op/s 2023-06-23T18:30:28.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:28 smithi002 ceph-mon[95947]: pgmap v549: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 MiB/s wr, 29 op/s 2023-06-23T18:30:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:28 smithi082 ceph-mon[97889]: pgmap v549: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 44 MiB/s wr, 29 op/s 2023-06-23T18:30:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:30 smithi017 ceph-mon[97472]: pgmap v550: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 31 op/s 2023-06-23T18:30:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:30 smithi002 ceph-mon[95947]: pgmap v550: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 31 op/s 2023-06-23T18:30:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:30 smithi082 ceph-mon[97889]: pgmap v550: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 48 MiB/s wr, 31 op/s 2023-06-23T18:30:31.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:31 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:30:31.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:31 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:30:31.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:31 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:30:31.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:31 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:30:31.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:31 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:30:31.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:31 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:30:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:32 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:30:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:32 smithi017 ceph-mon[97472]: pgmap v551: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 MiB/s wr, 28 op/s 2023-06-23T18:30:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:32 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:30:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:32 smithi002 ceph-mon[95947]: pgmap v551: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 MiB/s wr, 28 op/s 2023-06-23T18:30:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:32 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:30:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:32 smithi082 ceph-mon[97889]: pgmap v551: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 44 MiB/s wr, 28 op/s 2023-06-23T18:30:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:34 smithi017 ceph-mon[97472]: pgmap v552: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 32 op/s 2023-06-23T18:30:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:34 smithi002 ceph-mon[95947]: pgmap v552: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 32 op/s 2023-06-23T18:30:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:34 smithi082 ceph-mon[97889]: pgmap v552: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 53 MiB/s wr, 32 op/s 2023-06-23T18:30:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:36 smithi017 ceph-mon[97472]: pgmap v553: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:36 smithi002 ceph-mon[95947]: pgmap v553: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:36 smithi082 ceph-mon[97889]: pgmap v553: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:38 smithi017 ceph-mon[97472]: pgmap v554: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:38 smithi002 ceph-mon[95947]: pgmap v554: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:38 smithi082 ceph-mon[97889]: pgmap v554: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:40 smithi017 ceph-mon[97472]: pgmap v555: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:40 smithi002 ceph-mon[95947]: pgmap v555: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:40 smithi082 ceph-mon[97889]: pgmap v555: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 48 MiB/s wr, 26 op/s 2023-06-23T18:30:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:42 smithi017 ceph-mon[97472]: pgmap v556: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 42 MiB/s wr, 25 op/s 2023-06-23T18:30:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:42 smithi002 ceph-mon[95947]: pgmap v556: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 42 MiB/s wr, 25 op/s 2023-06-23T18:30:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:42 smithi082 ceph-mon[97889]: pgmap v556: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 42 MiB/s wr, 25 op/s 2023-06-23T18:30:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:44 smithi017 ceph-mon[97472]: pgmap v557: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 55 MiB/s wr, 32 op/s 2023-06-23T18:30:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:44 smithi002 ceph-mon[95947]: pgmap v557: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 55 MiB/s wr, 32 op/s 2023-06-23T18:30:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:44 smithi082 ceph-mon[97889]: pgmap v557: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 55 MiB/s wr, 32 op/s 2023-06-23T18:30:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:46 smithi017 ceph-mon[97472]: pgmap v558: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:30:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:46 smithi002 ceph-mon[95947]: pgmap v558: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:30:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:46 smithi082 ceph-mon[97889]: pgmap v558: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:30:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:48 smithi017 ceph-mon[97472]: pgmap v559: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:30:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:48 smithi002 ceph-mon[95947]: pgmap v559: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:30:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:48 smithi082 ceph-mon[97889]: pgmap v559: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:30:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:50 smithi017 ceph-mon[97472]: pgmap v560: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 52 MiB/s wr, 31 op/s 2023-06-23T18:30:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:50 smithi002 ceph-mon[95947]: pgmap v560: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 52 MiB/s wr, 31 op/s 2023-06-23T18:30:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:50 smithi082 ceph-mon[97889]: pgmap v560: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 52 MiB/s wr, 31 op/s 2023-06-23T18:30:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:52 smithi017 ceph-mon[97472]: pgmap v561: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 MiB/s wr, 27 op/s 2023-06-23T18:30:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:52 smithi002 ceph-mon[95947]: pgmap v561: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 MiB/s wr, 27 op/s 2023-06-23T18:30:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:52 smithi082 ceph-mon[97889]: pgmap v561: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 47 MiB/s wr, 27 op/s 2023-06-23T18:30:53.164 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:30:53.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.300+0000 7fec60b11700 1 -- 172.21.15.2:0/1478856333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 msgr2=0x7fec5c1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.300+0000 7fec60b11700 1 --2- 172.21.15.2:0/1478856333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 0x7fec5c1029e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009990 tx=0x7fec4c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.301+0000 7fec60b11700 1 -- 172.21.15.2:0/1478856333 shutdown_connections 2023-06-23T18:30:53.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.301+0000 7fec60b11700 1 --2- 172.21.15.2:0/1478856333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c105840 0x7fec5c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.301+0000 7fec60b11700 1 --2- 172.21.15.2:0/1478856333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 0x7fec5c1029e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.301+0000 7fec60b11700 1 --2- 172.21.15.2:0/1478856333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec5c102f20 0x7fec5c105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.301+0000 7fec60b11700 1 -- 172.21.15.2:0/1478856333 >> 172.21.15.2:0/1478856333 conn(0x7fec5c0fa0e0 msgr2=0x7fec5c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.301+0000 7fec60b11700 1 -- 172.21.15.2:0/1478856333 shutdown_connections 2023-06-23T18:30:53.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.301+0000 7fec60b11700 1 -- 172.21.15.2:0/1478856333 wait complete. 2023-06-23T18:30:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.301+0000 7fec60b11700 1 Processor -- start 2023-06-23T18:30:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec60b11700 1 -- start start 2023-06-23T18:30:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec60b11700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 0x7fec5c195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec60b11700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c102f20 0x7fec5c195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec60b11700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec5c105840 0x7fec5c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec60b11700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c19d310 con 0x7fec5c100600 2023-06-23T18:30:53.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec60b11700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c19d450 con 0x7fec5c102f20 2023-06-23T18:30:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec60b11700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5c19d5e0 con 0x7fec5c105840 2023-06-23T18:30:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec5a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 0x7fec5c195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec5a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 0x7fec5c195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55234/0 (socket says 172.21.15.2:55234) 2023-06-23T18:30:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec5a59c700 1 -- 172.21.15.2:0/3724227590 learned_addr learned my addr 172.21.15.2:0/3724227590 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec5a59c700 1 -- 172.21.15.2:0/3724227590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec5c105840 msgr2=0x7fec5c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec5a59c700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec5c105840 0x7fec5c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec5a59c700 1 -- 172.21.15.2:0/3724227590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c102f20 msgr2=0x7fec5c195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec5a59c700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c102f20 0x7fec5c195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.302+0000 7fec5a59c700 1 -- 172.21.15.2:0/3724227590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c0095f0 con 0x7fec5c100600 2023-06-23T18:30:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.303+0000 7fec5a59c700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 0x7fec5c195790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fec4c006010 tx=0x7fec4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.303+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3724227590 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c01d070 con 0x7fec5c100600 2023-06-23T18:30:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.303+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3724227590 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec4c0041b0 con 0x7fec5c100600 2023-06-23T18:30:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.303+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3724227590 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c022dc0 con 0x7fec5c100600 2023-06-23T18:30:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.303+0000 7fec60b11700 1 -- 172.21.15.2:0/3724227590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5c19d860 con 0x7fec5c100600 2023-06-23T18:30:53.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.303+0000 7fec60b11700 1 -- 172.21.15.2:0/3724227590 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c19dc30 con 0x7fec5c100600 2023-06-23T18:30:53.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.305+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3724227590 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fec4c022470 con 0x7fec5c100600 2023-06-23T18:30:53.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.306+0000 7fec4b7fe700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec440a1150 0x7fec440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.306+0000 7fec59d9b700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec440a1150 0x7fec440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.306+0000 7fec4b7fe700 1 -- 172.21.15.2:0/3724227590 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fec4c0cdf40 con 0x7fec5c100600 2023-06-23T18:30:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.307+0000 7fec59d9b700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec440a1150 0x7fec440a3600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec50005950 tx=0x7fec500058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.307+0000 7fec6cc96b80 1 Processor -- start 2023-06-23T18:30:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.307+0000 7fec6cc96b80 1 -- start start 2023-06-23T18:30:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334323c80 0x561334324090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x561334324a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334325000 0x561334325470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334228cb0 con 0x561334323c80 2023-06-23T18:30:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56133427ba40 con 0x5613343245d0 2023-06-23T18:30:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334274ed0 con 0x561334325000 2023-06-23T18:30:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334323c80 0x561334324090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334323c80 0x561334324090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55242/0 (socket says 172.21.15.2:55242) 2023-06-23T18:30:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 -- 172.21.15.2:0/1215794307 learned_addr learned my addr 172.21.15.2:0/1215794307 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec59d9b700 1 --2- 172.21.15.2:0/1215794307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x561334324a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 -- 172.21.15.2:0/1215794307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334325000 msgr2=0x561334325470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:53.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 --2- 172.21.15.2:0/1215794307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334325000 0x561334325470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 -- 172.21.15.2:0/1215794307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 msgr2=0x561334324a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:53.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 --2- 172.21.15.2:0/1215794307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x561334324a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 -- 172.21.15.2:0/1215794307 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613340f6630 con 0x561334323c80 2023-06-23T18:30:53.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec5a59c700 1 --2- 172.21.15.2:0/1215794307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334323c80 0x561334324090 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0a3270 tx=0x7fec4c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4864aee0729b917 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.308+0000 7fec48ff9700 1 -- 172.21.15.2:0/1215794307 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c09f410 con 0x561334323c80 2023-06-23T18:30:53.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec48ff9700 1 -- 172.21.15.2:0/1215794307 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec4c0a37a0 con 0x561334323c80 2023-06-23T18:30:53.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec48ff9700 1 -- 172.21.15.2:0/1215794307 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c09ab30 con 0x561334323c80 2023-06-23T18:30:53.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1215794307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334323c80 msgr2=0x561334324090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1215794307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334323c80 0x561334324090 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fec4c0a3270 tx=0x7fec4c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1215794307 shutdown_connections 2023-06-23T18:30:53.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1215794307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334325000 0x561334325470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1215794307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334323c80 0x561334324090 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1215794307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x561334324a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.309+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1215794307 >> 172.21.15.2:0/1215794307 conn(0x5613343193f0 msgr2=0x56133431e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1215794307 shutdown_connections 2023-06-23T18:30:53.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1215794307 wait complete. 2023-06-23T18:30:53.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 -- start start 2023-06-23T18:30:53.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 Processor -- start 2023-06-23T18:30:53.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x56133432d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x56133432d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x5613343347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334214ac0 con 0x561334325000 2023-06-23T18:30:53.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613342147c0 con 0x5613343245d0 2023-06-23T18:30:53.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.310+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613340f8920 con 0x561334323c80 2023-06-23T18:30:53.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec6cc96b80 10 client.0 _ll_register_callbacks cb 0x56133421a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:53.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x56133432d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x56133432d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55586/0 (socket says 172.21.15.2:55586) 2023-06-23T18:30:53.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5a59c700 1 -- 172.21.15.2:0/1089995967 learned_addr learned my addr 172.21.15.2:0/1089995967 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:53.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5ad9d700 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x5613343347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec59d9b700 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x56133432d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5ad9d700 1 -- 172.21.15.2:0/1089995967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 msgr2=0x56133432d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:53.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5ad9d700 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x56133432d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5ad9d700 1 -- 172.21.15.2:0/1089995967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 msgr2=0x56133432d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:53.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5ad9d700 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x56133432d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5ad9d700 1 -- 172.21.15.2:0/1089995967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5c19dc30 con 0x561334325000 2023-06-23T18:30:53.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec5ad9d700 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x5613343347a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fec540077d0 tx=0x7fec54007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc4e95fea594c8a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:53.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec41ffb700 1 -- 172.21.15.2:0/1089995967 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54012760 con 0x561334325000 2023-06-23T18:30:53.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1089995967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561334316bb0 con 0x561334325000 2023-06-23T18:30:53.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec41ffb700 1 -- 172.21.15.2:0/1089995967 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fec540128c0 con 0x561334325000 2023-06-23T18:30:53.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.311+0000 7fec41ffb700 1 -- 172.21.15.2:0/1089995967 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec54012a80 con 0x561334325000 2023-06-23T18:30:53.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec6cc96b80 10 client.15888 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:53.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1089995967 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5613340a2de0 con 0x561334325000 2023-06-23T18:30:53.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec41ffb700 1 -- 172.21.15.2:0/1089995967 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec54022020 con 0x561334325000 2023-06-23T18:30:53.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec41ffb700 1 -- 172.21.15.2:0/1089995967 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec54012070 con 0x561334325000 2023-06-23T18:30:53.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec6cc96b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:53.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec6cc96b80 10 client.15888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:53.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec6cc96b80 20 client.15888 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:53.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5613343390d0 0x56133433b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec6cc96b80 4 client.15888 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:30:53.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1089995967 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56133426deb0 con 0x5613343390d0 2023-06-23T18:30:53.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.312+0000 7fec5a59c700 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5613343390d0 0x56133433b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.313+0000 7fec41ffb700 10 client.15888 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:53.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.313+0000 7fec5a59c700 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5613343390d0 0x56133433b4e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4b8e12b58d396f server_cookie=8caaaa48fe7aefef in_seq=0 out_seq=0 2023-06-23T18:30:53.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.313+0000 7fec41ffb700 1 -- 172.21.15.2:0/1089995967 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56133426deb0 con 0x5613343390d0 2023-06-23T18:30:53.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.313+0000 7fec41ffb700 10 client.15888 handle_command_reply: tid=0 2023-06-23T18:30:53.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.313+0000 7fec6cc96b80 1 client.15888 shutdown 2023-06-23T18:30:53.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1089995967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 msgr2=0x5613343347a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x5613343347a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fec540077d0 tx=0x7fec54007ae0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1089995967 shutdown_connections 2023-06-23T18:30:53.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5613343390d0 0x56133433b4e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x5613343347a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec41ffb700 0 client.15888 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:53.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x56133432d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/1089995967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x56133432d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1089995967 >> 172.21.15.2:0/1089995967 conn(0x5613343193f0 msgr2=0x56133431dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1089995967 shutdown_connections 2023-06-23T18:30:53.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 1 -- 172.21.15.2:0/1089995967 wait complete. 2023-06-23T18:30:53.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.314+0000 7fec6cc96b80 20 client.15888 trim_cache size 0 max 16384 2023-06-23T18:30:53.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec6cc96b80 1 Processor -- start 2023-06-23T18:30:53.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec6cc96b80 1 -- start start 2023-06-23T18:30:53.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x561334327ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x561334328400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x561334334760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334320010 con 0x561334325000 2023-06-23T18:30:53.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334336410 con 0x5613343245d0 2023-06-23T18:30:53.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56133432bbf0 con 0x561334323c80 2023-06-23T18:30:53.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.333+0000 7fec5ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x561334334760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x561334334760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55260/0 (socket says 172.21.15.2:55260) 2023-06-23T18:30:53.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5ad9d700 1 -- 172.21.15.2:0/435986175 learned_addr learned my addr 172.21.15.2:0/435986175 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:53.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5a59c700 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x561334327ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec59d9b700 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x561334328400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5ad9d700 1 -- 172.21.15.2:0/435986175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 msgr2=0x561334327ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:53.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5ad9d700 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x561334327ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5ad9d700 1 -- 172.21.15.2:0/435986175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 msgr2=0x561334328400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:53.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5ad9d700 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x561334328400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5ad9d700 1 -- 172.21.15.2:0/435986175 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56133408bc60 con 0x561334325000 2023-06-23T18:30:53.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec5ad9d700 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x561334334760 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fec54009280 tx=0x7fec54007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367c0e6ebb3fb446 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec497fa700 1 -- 172.21.15.2:0/435986175 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54006000 con 0x561334325000 2023-06-23T18:30:53.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec497fa700 1 -- 172.21.15.2:0/435986175 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec54022020 con 0x561334325000 2023-06-23T18:30:53.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.334+0000 7fec497fa700 1 -- 172.21.15.2:0/435986175 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec54012c80 con 0x561334325000 2023-06-23T18:30:53.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 -- 172.21.15.2:0/435986175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 msgr2=0x561334334760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x561334334760 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fec54009280 tx=0x7fec54007e30 comp rx=0 tx=0).stop 2023-06-23T18:30:53.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 -- 172.21.15.2:0/435986175 shutdown_connections 2023-06-23T18:30:53.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561334325000 0x561334334760 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x561334327ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/435986175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5613343245d0 0x561334328400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 -- 172.21.15.2:0/435986175 >> 172.21.15.2:0/435986175 conn(0x5613343193f0 msgr2=0x56133433fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 -- 172.21.15.2:0/435986175 shutdown_connections 2023-06-23T18:30:53.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 -- 172.21.15.2:0/435986175 wait complete. 2023-06-23T18:30:53.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.335+0000 7fec6cc96b80 1 -- start start 2023-06-23T18:30:53.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec6cc96b80 1 Processor -- start 2023-06-23T18:30:53.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x561334330970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5613343245d0 0x561334330eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec6cc96b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561334325000 0x561334333ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334334410 con 0x5613343245d0 2023-06-23T18:30:53.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334334580 con 0x561334325000 2023-06-23T18:30:53.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec6cc96b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561334334740 con 0x561334323c80 2023-06-23T18:30:53.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec6cc96b80 10 client.0 _ll_register_callbacks cb 0x56133421a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:53.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5613343245d0 0x561334330eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5613343245d0 0x561334330eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55264/0 (socket says 172.21.15.2:55264) 2023-06-23T18:30:53.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 -- 172.21.15.2:0/3650342557 learned_addr learned my addr 172.21.15.2:0/3650342557 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:53.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec5a59c700 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x561334330970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec5ad9d700 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561334325000 0x561334333ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 -- 172.21.15.2:0/3650342557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 msgr2=0x561334330970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:53.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x561334330970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 -- 172.21.15.2:0/3650342557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561334325000 msgr2=0x561334333ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:53.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561334325000 0x561334333ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 -- 172.21.15.2:0/3650342557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613340a2de0 con 0x5613343245d0 2023-06-23T18:30:53.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec59d9b700 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5613343245d0 0x561334330eb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fec500104e0 tx=0x7fec50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a971660002450c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.336+0000 7fec41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:53.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec41ffb700 1 -- 172.21.15.2:0/3650342557 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50024070 con 0x5613343245d0 2023-06-23T18:30:53.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec41ffb700 1 -- 172.21.15.2:0/3650342557 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fec5001e3f0 con 0x5613343245d0 2023-06-23T18:30:53.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 1 -- 172.21.15.2:0/3650342557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec4c0a37a0 con 0x5613343245d0 2023-06-23T18:30:53.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec41ffb700 1 -- 172.21.15.2:0/3650342557 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec5001e5b0 con 0x5613343245d0 2023-06-23T18:30:53.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 10 client.15894 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:53.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 1 -- 172.21.15.2:0/3650342557 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5613340c9250 con 0x5613343245d0 2023-06-23T18:30:53.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec41ffb700 1 -- 172.21.15.2:0/3650342557 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec4c0a37a0 con 0x5613343245d0 2023-06-23T18:30:53.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec41ffb700 1 -- 172.21.15.2:0/3650342557 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec50029780 con 0x5613343245d0 2023-06-23T18:30:53.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:53.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:53.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 20 client.15894 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:53.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56133438e4b0 0x5613343908c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 4 client.15894 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:30:53.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.337+0000 7fec6cc96b80 1 -- 172.21.15.2:0/3650342557 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561334230050 con 0x56133438e4b0 2023-06-23T18:30:53.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec5a59c700 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56133438e4b0 0x5613343908c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec41ffb700 10 client.15894 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:53.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec5a59c700 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56133438e4b0 0x5613343908c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a6d391b27cc019f server_cookie=7e0f1a822af03ac6 in_seq=0 out_seq=0 2023-06-23T18:30:53.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec41ffb700 1 -- 172.21.15.2:0/3650342557 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561334230050 con 0x56133438e4b0 2023-06-23T18:30:53.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec41ffb700 10 client.15894 handle_command_reply: tid=0 2023-06-23T18:30:53.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 client.15894 shutdown 2023-06-23T18:30:53.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 -- 172.21.15.2:0/3650342557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5613343245d0 msgr2=0x561334330eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5613343245d0 0x561334330eb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fec500104e0 tx=0x7fec50004750 comp rx=0 tx=0).stop 2023-06-23T18:30:53.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 -- 172.21.15.2:0/3650342557 shutdown_connections 2023-06-23T18:30:53.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56133438e4b0 0x5613343908c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561334325000 0x561334333ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec41ffb700 0 client.15894 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:53.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561334323c80 0x561334330970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 --2- 172.21.15.2:0/3650342557 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5613343245d0 0x561334330eb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.338+0000 7fec6cc96b80 1 -- 172.21.15.2:0/3650342557 >> 172.21.15.2:0/3650342557 conn(0x5613343193f0 msgr2=0x561334327ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec6cc96b80 1 -- 172.21.15.2:0/3650342557 shutdown_connections 2023-06-23T18:30:53.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec6cc96b80 1 -- 172.21.15.2:0/3650342557 wait complete. 2023-06-23T18:30:53.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec6cc96b80 20 client.15894 trim_cache size 0 max 16384 2023-06-23T18:30:53.355 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:30:53.355 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:30:53.355 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:30:53.355 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:30:53.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 -- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec440a1150 msgr2=0x7fec440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec440a1150 0x7fec440a3600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fec50005950 tx=0x7fec500058e0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 -- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 msgr2=0x7fec5c195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 0x7fec5c195790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fec4c006010 tx=0x7fec4c004730 comp rx=0 tx=0).stop 2023-06-23T18:30:53.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 -- 172.21.15.2:0/3724227590 shutdown_connections 2023-06-23T18:30:53.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec5c105840 0x7fec5c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec5c100600 0x7fec5c195790 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fec440a1150 0x7fec440a3600 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 --2- 172.21.15.2:0/3724227590 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c102f20 0x7fec5c195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 -- 172.21.15.2:0/3724227590 >> 172.21.15.2:0/3724227590 conn(0x7fec5c0fa0e0 msgr2=0x7fec5c101f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 -- 172.21.15.2:0/3724227590 shutdown_connections 2023-06-23T18:30:53.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.339+0000 7fec60b11700 1 -- 172.21.15.2:0/3724227590 wait complete. 2023-06-23T18:30:53.358 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:30:53.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:30:53.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 -- 172.21.15.2:0/3499270516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380ffc50 msgr2=0x7f94381000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 --2- 172.21.15.2:0/3499270516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380ffc50 0x7f94381000a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9424009990 tx=0x7f9424009ca0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 -- 172.21.15.2:0/3499270516 shutdown_connections 2023-06-23T18:30:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 --2- 172.21.15.2:0/3499270516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94381005e0 0x7f9438106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 --2- 172.21.15.2:0/3499270516 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94380fea60 0x7f94380fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 --2- 172.21.15.2:0/3499270516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380ffc50 0x7f94381000a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 -- 172.21.15.2:0/3499270516 >> 172.21.15.2:0/3499270516 conn(0x7f94380fa0e0 msgr2=0x7f94380fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 -- 172.21.15.2:0/3499270516 shutdown_connections 2023-06-23T18:30:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.492+0000 7f943eff3700 1 -- 172.21.15.2:0/3499270516 wait complete. 2023-06-23T18:30:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943eff3700 1 Processor -- start 2023-06-23T18:30:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943eff3700 1 -- start start 2023-06-23T18:30:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943eff3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380fea60 0x7f9438068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943eff3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94380ffc50 0x7f943819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943eff3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94381005e0 0x7f943819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943eff3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943819d220 con 0x7f94380fea60 2023-06-23T18:30:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943eff3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943819d390 con 0x7f94380ffc50 2023-06-23T18:30:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943eff3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943819d550 con 0x7f94381005e0 2023-06-23T18:30:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943cd8f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380fea60 0x7f9438068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943cd8f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380fea60 0x7f9438068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55276/0 (socket says 172.21.15.2:55276) 2023-06-23T18:30:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943cd8f700 1 -- 172.21.15.2:0/2959450723 learned_addr learned my addr 172.21.15.2:0/2959450723 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:53.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f9437fff700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94380ffc50 0x7f943819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.493+0000 7f943d590700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94381005e0 0x7f943819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f943cd8f700 1 -- 172.21.15.2:0/2959450723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94381005e0 msgr2=0x7f943819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f943cd8f700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94381005e0 0x7f943819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f943cd8f700 1 -- 172.21.15.2:0/2959450723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94380ffc50 msgr2=0x7f943819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f943cd8f700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94380ffc50 0x7f943819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f943cd8f700 1 -- 172.21.15.2:0/2959450723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240095f0 con 0x7f94380fea60 2023-06-23T18:30:53.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f943cd8f700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380fea60 0x7f9438068330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f942c00e970 tx=0x7f942c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f9435ffb700 1 -- 172.21.15.2:0/2959450723 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c009790 con 0x7f94380fea60 2023-06-23T18:30:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f9435ffb700 1 -- 172.21.15.2:0/2959450723 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f942c00c7e0 con 0x7f94380fea60 2023-06-23T18:30:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943819d830 con 0x7f94380fea60 2023-06-23T18:30:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f9435ffb700 1 -- 172.21.15.2:0/2959450723 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c003bd0 con 0x7f94380fea60 2023-06-23T18:30:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.494+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943819dd60 con 0x7f94380fea60 2023-06-23T18:30:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.495+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943810a0e0 con 0x7f94380fea60 2023-06-23T18:30:53.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.496+0000 7f9435ffb700 1 -- 172.21.15.2:0/2959450723 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f942c00c950 con 0x7f94380fea60 2023-06-23T18:30:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.497+0000 7f9435ffb700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f94280a11d0 0x7f94280a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.497+0000 7f9435ffb700 1 -- 172.21.15.2:0/2959450723 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f942c014070 con 0x7f94380fea60 2023-06-23T18:30:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.497+0000 7f9437fff700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f94280a11d0 0x7f94280a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.498+0000 7f9437fff700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f94280a11d0 0x7f94280a3680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f942400b2e0 tx=0x7f9424004000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.503+0000 7f9435ffb700 1 -- 172.21.15.2:0/2959450723 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f942c096170 con 0x7f94380fea60 2023-06-23T18:30:53.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.809+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9438105990 con 0x7f94380fea60 2023-06-23T18:30:53.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.810+0000 7f9435ffb700 1 -- 172.21.15.2:0/2959450723 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10780 (secure 0 0 0) 0x7f942c016330 con 0x7f94380fea60 2023-06-23T18:30:53.815 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:30:53.816 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:30:26.353535+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:30:53.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f94280a11d0 msgr2=0x7f94280a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f94280a11d0 0x7f94280a3680 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f942400b2e0 tx=0x7f9424004000 comp rx=0 tx=0).stop 2023-06-23T18:30:53.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380fea60 msgr2=0x7f9438068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380fea60 0x7f9438068330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f942c00e970 tx=0x7f942c00ec80 comp rx=0 tx=0).stop 2023-06-23T18:30:53.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 shutdown_connections 2023-06-23T18:30:53.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94381005e0 0x7f943819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380fea60 0x7f9438068330 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f94280a11d0 0x7f94280a3680 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 --2- 172.21.15.2:0/2959450723 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94380ffc50 0x7f943819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 >> 172.21.15.2:0/2959450723 conn(0x7f94380fa0e0 msgr2=0x7f94380fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 shutdown_connections 2023-06-23T18:30:53.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.813+0000 7f943eff3700 1 -- 172.21.15.2:0/2959450723 wait complete. 2023-06-23T18:30:53.820 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 32 2023-06-23T18:30:53.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:30:53.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.975+0000 7fea1ddcc700 1 -- 172.21.15.2:0/3897762109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180ffc80 msgr2=0x7fea181000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.975+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/3897762109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180ffc80 0x7fea181000d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009990 tx=0x7fea0c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 -- 172.21.15.2:0/3897762109 shutdown_connections 2023-06-23T18:30:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/3897762109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea18100610 0x7fea18106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/3897762109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea180fea90 0x7fea180feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/3897762109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180ffc80 0x7fea181000d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 -- 172.21.15.2:0/3897762109 >> 172.21.15.2:0/3897762109 conn(0x7fea180fa090 msgr2=0x7fea180fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 -- 172.21.15.2:0/3897762109 shutdown_connections 2023-06-23T18:30:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 -- 172.21.15.2:0/3897762109 wait complete. 2023-06-23T18:30:53.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 Processor -- start 2023-06-23T18:30:53.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.976+0000 7fea1ddcc700 1 -- start start 2023-06-23T18:30:53.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea1ddcc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180fea90 0x7fea18069070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea1ddcc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea180ffc80 0x7fea1819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea1ddcc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea18100610 0x7fea1819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea1ddcc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1819d2c0 con 0x7fea180fea90 2023-06-23T18:30:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea1ddcc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1819d430 con 0x7fea180ffc80 2023-06-23T18:30:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea1ddcc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1819d5f0 con 0x7fea18100610 2023-06-23T18:30:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180fea90 0x7fea18069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180fea90 0x7fea18069070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55290/0 (socket says 172.21.15.2:55290) 2023-06-23T18:30:53.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 -- 172.21.15.2:0/2551459542 learned_addr learned my addr 172.21.15.2:0/2551459542 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:53.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea16ffd700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea180ffc80 0x7fea1819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea17fff700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea18100610 0x7fea1819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 -- 172.21.15.2:0/2551459542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea18100610 msgr2=0x7fea1819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea18100610 0x7fea1819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 -- 172.21.15.2:0/2551459542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea180ffc80 msgr2=0x7fea1819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:53.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea180ffc80 0x7fea1819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:53.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 -- 172.21.15.2:0/2551459542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c0095f0 con 0x7fea180fea90 2023-06-23T18:30:53.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea17fff700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea18100610 0x7fea1819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:53.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea16ffd700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea180ffc80 0x7fea1819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:53.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.977+0000 7fea177fe700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180fea90 0x7fea18069070 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fea0800d980 tx=0x7fea0800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.978+0000 7fea14ff9700 1 -- 172.21.15.2:0/2551459542 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08004020 con 0x7fea180fea90 2023-06-23T18:30:53.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.978+0000 7fea14ff9700 1 -- 172.21.15.2:0/2551459542 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea08009730 con 0x7fea180fea90 2023-06-23T18:30:53.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.978+0000 7fea1ddcc700 1 -- 172.21.15.2:0/2551459542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1819d8d0 con 0x7fea180fea90 2023-06-23T18:30:53.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.978+0000 7fea14ff9700 1 -- 172.21.15.2:0/2551459542 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08003a00 con 0x7fea180fea90 2023-06-23T18:30:53.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.978+0000 7fea1ddcc700 1 -- 172.21.15.2:0/2551459542 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1819dde0 con 0x7fea180fea90 2023-06-23T18:30:53.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.980+0000 7fea14ff9700 1 -- 172.21.15.2:0/2551459542 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fea080098a0 con 0x7fea180fea90 2023-06-23T18:30:53.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.981+0000 7fea14ff9700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea000a1150 0x7fea000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.981+0000 7fea14ff9700 1 -- 172.21.15.2:0/2551459542 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea080cc620 con 0x7fea180fea90 2023-06-23T18:30:53.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.981+0000 7fea16ffd700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea000a1150 0x7fea000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea16ffd700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea000a1150 0x7fea000a3600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00b2e0 tx=0x7fea0c004500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:53.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea29f51b80 1 Processor -- start 2023-06-23T18:30:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea29f51b80 1 -- start start 2023-06-23T18:30:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab6c80 0x5580c9ab7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab75d0 0x5580c9ab7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab8000 0x5580c9ab8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea29f51b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c99bbcb0 con 0x5580c9ab8000 2023-06-23T18:30:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea29f51b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c9a0ea40 con 0x5580c9ab6c80 2023-06-23T18:30:53.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.982+0000 7fea29f51b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c9a07ed0 con 0x5580c9ab75d0 2023-06-23T18:30:53.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab75d0 0x5580c9ab7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:53.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab75d0 0x5580c9ab7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55638/0 (socket says 172.21.15.2:55638) 2023-06-23T18:30:53.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 -- 172.21.15.2:0/3731227 learned_addr learned my addr 172.21.15.2:0/3731227 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea177fe700 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab6c80 0x5580c9ab7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea17fff700 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab8000 0x5580c9ab8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 -- 172.21.15.2:0/3731227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab6c80 msgr2=0x5580c9ab7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab6c80 0x5580c9ab7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 -- 172.21.15.2:0/3731227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab8000 msgr2=0x5580c9ab8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab8000 0x5580c9ab8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 -- 172.21.15.2:0/3731227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580c9889630 con 0x5580c9ab75d0 2023-06-23T18:30:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea17fff700 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab8000 0x5580c9ab8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:30:54.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fea16ffd700 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab75d0 0x5580c9ab7a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea0c015000 tx=0x7fea0c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33df3c92ba877400 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fe9fdffb700 1 -- 172.21.15.2:0/3731227 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c02b070 con 0x5580c9ab75d0 2023-06-23T18:30:54.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fe9fdffb700 1 -- 172.21.15.2:0/3731227 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea0c01e040 con 0x5580c9ab75d0 2023-06-23T18:30:54.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.983+0000 7fe9fdffb700 1 -- 172.21.15.2:0/3731227 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c004a10 con 0x5580c9ab75d0 2023-06-23T18:30:54.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 -- 172.21.15.2:0/3731227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab75d0 msgr2=0x5580c9ab7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab75d0 0x5580c9ab7a70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fea0c015000 tx=0x7fea0c020000 comp rx=0 tx=0).stop 2023-06-23T18:30:54.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 -- 172.21.15.2:0/3731227 shutdown_connections 2023-06-23T18:30:54.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab8000 0x5580c9ab8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab6c80 0x5580c9ab7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 --2- 172.21.15.2:0/3731227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab75d0 0x5580c9ab7a70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 -- 172.21.15.2:0/3731227 >> 172.21.15.2:0/3731227 conn(0x5580c9aac3f0 msgr2=0x5580c9ab1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 -- 172.21.15.2:0/3731227 shutdown_connections 2023-06-23T18:30:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.984+0000 7fea29f51b80 1 -- 172.21.15.2:0/3731227 wait complete. 2023-06-23T18:30:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 1 -- start start 2023-06-23T18:30:54.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 1 Processor -- start 2023-06-23T18:30:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 0x5580c9ac0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 0x5580c9ac7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c99a7ac0 con 0x5580c9ab75d0 2023-06-23T18:30:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c99a77c0 con 0x5580c9ab8000 2023-06-23T18:30:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c988b920 con 0x5580c9ab6c80 2023-06-23T18:30:54.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea29f51b80 10 client.0 _ll_register_callbacks cb 0x5580c99ad260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:54.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea16ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea177fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 0x5580c9ac0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea16ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55306/0 (socket says 172.21.15.2:55306) 2023-06-23T18:30:54.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.985+0000 7fea16ffd700 1 -- 172.21.15.2:0/432699051 learned_addr learned my addr 172.21.15.2:0/432699051 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea17fff700 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 0x5580c9ac7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea16ffd700 1 -- 172.21.15.2:0/432699051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 msgr2=0x5580c9ac0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea16ffd700 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 0x5580c9ac0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea16ffd700 1 -- 172.21.15.2:0/432699051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 msgr2=0x5580c9ac7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea16ffd700 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 0x5580c9ac7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea16ffd700 1 -- 172.21.15.2:0/432699051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea18107800 con 0x5580c9ab75d0 2023-06-23T18:30:54.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea16ffd700 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac0a40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea0c006010 tx=0x7fea0c00f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d0f7f9e74ff8b1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fe9f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:54.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/432699051 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c02b070 con 0x5580c9ab75d0 2023-06-23T18:30:54.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/432699051 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea0c00fd90 con 0x5580c9ab75d0 2023-06-23T18:30:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea29f51b80 1 -- 172.21.15.2:0/432699051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5580c9aa9bb0 con 0x5580c9ab75d0 2023-06-23T18:30:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/432699051 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea0c039470 con 0x5580c9ab75d0 2023-06-23T18:30:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea29f51b80 10 client.15909 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.986+0000 7fea29f51b80 1 -- 172.21.15.2:0/432699051 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5580c9835de0 con 0x5580c9ab75d0 2023-06-23T18:30:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/432699051 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5580c9aa9bb0 con 0x5580c9ab75d0 2023-06-23T18:30:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/432699051 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea0c030580 con 0x5580c9ab75d0 2023-06-23T18:30:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fea29f51b80 10 client.15909 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:54.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fea29f51b80 10 client.15909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:30:54.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fea29f51b80 20 client.15909 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:54.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fea29f51b80 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x5580c9acbdc0 0x5580c9ace1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fea29f51b80 4 client.15909 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:30:54.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fea29f51b80 1 -- 172.21.15.2:0/432699051 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5580c9a00eb0 con 0x5580c9acbdc0 2023-06-23T18:30:54.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.987+0000 7fea177fe700 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x5580c9acbdc0 0x5580c9ace1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fe9f6ffd700 10 client.15909 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:30:54.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea177fe700 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x5580c9acbdc0 0x5580c9ace1a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=86d0d70656688f90 server_cookie=46568f8da07c3626 in_seq=0 out_seq=0 2023-06-23T18:30:54.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/432699051 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5580c9a00eb0 con 0x5580c9acbdc0 2023-06-23T18:30:54.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fe9f6ffd700 10 client.15909 handle_command_reply: tid=0 2023-06-23T18:30:54.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 client.15909 shutdown 2023-06-23T18:30:54.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 -- 172.21.15.2:0/432699051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 msgr2=0x5580c9ac0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac0a40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fea0c006010 tx=0x7fea0c00f710 comp rx=0 tx=0).stop 2023-06-23T18:30:54.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 -- 172.21.15.2:0/432699051 shutdown_connections 2023-06-23T18:30:54.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x5580c9acbdc0 0x5580c9ace1a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 0x5580c9ac7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fe9f6ffd700 0 client.15909 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:30:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 0x5580c9ac0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 --2- 172.21.15.2:0/432699051 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac0a40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.988+0000 7fea29f51b80 1 -- 172.21.15.2:0/432699051 >> 172.21.15.2:0/432699051 conn(0x5580c9aac3f0 msgr2=0x5580c9ab0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.989+0000 7fea29f51b80 1 -- 172.21.15.2:0/432699051 shutdown_connections 2023-06-23T18:30:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.989+0000 7fea29f51b80 1 -- 172.21.15.2:0/432699051 wait complete. 2023-06-23T18:30:54.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:53.989+0000 7fea29f51b80 20 client.15909 trim_cache size 0 max 16384 2023-06-23T18:30:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea29f51b80 1 Processor -- start 2023-06-23T18:30:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea29f51b80 1 -- start start 2023-06-23T18:30:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 0x5580c9abf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 0x5580c9ac0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea29f51b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c9ab3010 con 0x5580c9ab75d0 2023-06-23T18:30:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea29f51b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c9aabf70 con 0x5580c9ab8000 2023-06-23T18:30:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea29f51b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c9ac1680 con 0x5580c9ab6c80 2023-06-23T18:30:54.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea16ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea16ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55314/0 (socket says 172.21.15.2:55314) 2023-06-23T18:30:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea16ffd700 1 -- 172.21.15.2:0/715390090 learned_addr learned my addr 172.21.15.2:0/715390090 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea177fe700 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 0x5580c9abf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.003+0000 7fea17fff700 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 0x5580c9ac0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fea16ffd700 1 -- 172.21.15.2:0/715390090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 msgr2=0x5580c9abf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fea16ffd700 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 0x5580c9abf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fea16ffd700 1 -- 172.21.15.2:0/715390090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 msgr2=0x5580c9ac0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fea16ffd700 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 0x5580c9ac0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fea16ffd700 1 -- 172.21.15.2:0/715390090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5580c9ab88f0 con 0x5580c9ab75d0 2023-06-23T18:30:54.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fea16ffd700 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac7a40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009350 tx=0x7fea0c00bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24212e026c22faef server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fe9fe7fc700 1 -- 172.21.15.2:0/715390090 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c02b070 con 0x5580c9ab75d0 2023-06-23T18:30:54.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fe9fe7fc700 1 -- 172.21.15.2:0/715390090 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea0c02c040 con 0x5580c9ab75d0 2023-06-23T18:30:54.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.004+0000 7fe9fe7fc700 1 -- 172.21.15.2:0/715390090 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0398a0 con 0x5580c9ab75d0 2023-06-23T18:30:54.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 -- 172.21.15.2:0/715390090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 msgr2=0x5580c9ac7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac7a40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009350 tx=0x7fea0c00bf40 comp rx=0 tx=0).stop 2023-06-23T18:30:54.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 -- 172.21.15.2:0/715390090 shutdown_connections 2023-06-23T18:30:54.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab8000 0x5580c9ac0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab6c80 0x5580c9abf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 --2- 172.21.15.2:0/715390090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab75d0 0x5580c9ac7a40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 -- 172.21.15.2:0/715390090 >> 172.21.15.2:0/715390090 conn(0x5580c9aac3f0 msgr2=0x5580c9ad2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 -- 172.21.15.2:0/715390090 shutdown_connections 2023-06-23T18:30:54.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 -- 172.21.15.2:0/715390090 wait complete. 2023-06-23T18:30:54.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 -- start start 2023-06-23T18:30:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.005+0000 7fea29f51b80 1 Processor -- start 2023-06-23T18:30:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab6c80 0x5580c9ac3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab75d0 0x5580c9ac3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea29f51b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab8000 0x5580c9ac6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea29f51b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c9ac73d0 con 0x5580c9ab6c80 2023-06-23T18:30:54.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea29f51b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c9ac7540 con 0x5580c9ab75d0 2023-06-23T18:30:54.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea29f51b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580c9ac7700 con 0x5580c9ab8000 2023-06-23T18:30:54.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea29f51b80 10 client.0 _ll_register_callbacks cb 0x5580c99ad260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:54.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab6c80 0x5580c9ac3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab6c80 0x5580c9ac3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55318/0 (socket says 172.21.15.2:55318) 2023-06-23T18:30:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 -- 172.21.15.2:0/1490307485 learned_addr learned my addr 172.21.15.2:0/1490307485 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea17fff700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab8000 0x5580c9ac6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea16ffd700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab75d0 0x5580c9ac3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 -- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab8000 msgr2=0x5580c9ac6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab8000 0x5580c9ac6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 -- 172.21.15.2:0/1490307485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab75d0 msgr2=0x5580c9ac3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab75d0 0x5580c9ac3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 -- 172.21.15.2:0/1490307485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5580c9835de0 con 0x5580c9ab6c80 2023-06-23T18:30:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea17fff700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab8000 0x5580c9ac6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea16ffd700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab75d0 0x5580c9ac3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:30:54.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fea177fe700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab6c80 0x5580c9ac3930 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fea0800d950 tx=0x7fea08099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cebd8882c154a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.006+0000 7fe9f6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/1490307485 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08098f20 con 0x5580c9ab6c80 2023-06-23T18:30:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/1490307485 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea08099080 con 0x5580c9ab6c80 2023-06-23T18:30:54.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 1 -- 172.21.15.2:0/1490307485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea0c01e040 con 0x5580c9ab6c80 2023-06-23T18:30:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/1490307485 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea080a19f0 con 0x5580c9ab6c80 2023-06-23T18:30:54.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 10 client.15915 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 1 -- 172.21.15.2:0/1490307485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5580c981ec60 con 0x5580c9ab6c80 2023-06-23T18:30:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/1490307485 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fea0c01e040 con 0x5580c9ab6c80 2023-06-23T18:30:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/1490307485 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fea080c6060 con 0x5580c9ab6c80 2023-06-23T18:30:54.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 10 client.15915 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:54.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 10 client.15915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:30:54.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 20 client.15915 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:54.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x5580c9b20d00 0x5580c9b23110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 4 client.15915 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:30:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea29f51b80 1 -- 172.21.15.2:0/1490307485 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5580c99c3050 con 0x5580c9b20d00 2023-06-23T18:30:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.007+0000 7fea16ffd700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x5580c9b20d00 0x5580c9b23110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fe9f6ffd700 10 client.15915 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:30:54.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea16ffd700 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x5580c9b20d00 0x5580c9b23110 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca7bbf405aa6aa55 server_cookie=6c8e750947132732 in_seq=0 out_seq=0 2023-06-23T18:30:54.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fe9f6ffd700 1 -- 172.21.15.2:0/1490307485 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5580c99c3050 con 0x5580c9b20d00 2023-06-23T18:30:54.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fe9f6ffd700 10 client.15915 handle_command_reply: tid=0 2023-06-23T18:30:54.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 client.15915 shutdown 2023-06-23T18:30:54.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 -- 172.21.15.2:0/1490307485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab6c80 msgr2=0x5580c9ac3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab6c80 0x5580c9ac3930 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fea0800d950 tx=0x7fea08099bb0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 -- 172.21.15.2:0/1490307485 shutdown_connections 2023-06-23T18:30:54.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x5580c9b20d00 0x5580c9b23110 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5580c9ab8000 0x5580c9ac6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5580c9ab6c80 0x5580c9ac3930 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fe9f6ffd700 0 client.15915 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:30:54.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 --2- 172.21.15.2:0/1490307485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5580c9ab75d0 0x5580c9ac3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 -- 172.21.15.2:0/1490307485 >> 172.21.15.2:0/1490307485 conn(0x5580c9aac3f0 msgr2=0x5580c9abf200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 -- 172.21.15.2:0/1490307485 shutdown_connections 2023-06-23T18:30:54.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.008+0000 7fea29f51b80 1 -- 172.21.15.2:0/1490307485 wait complete. 2023-06-23T18:30:54.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea29f51b80 20 client.15915 trim_cache size 0 max 16384 2023-06-23T18:30:54.039 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:30:54.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 -- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea000a1150 msgr2=0x7fea000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea000a1150 0x7fea000a3600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fea0c00b2e0 tx=0x7fea0c004500 comp rx=0 tx=0).stop 2023-06-23T18:30:54.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 -- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180fea90 msgr2=0x7fea18069070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180fea90 0x7fea18069070 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fea0800d980 tx=0x7fea0800dc90 comp rx=0 tx=0).stop 2023-06-23T18:30:54.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 -- 172.21.15.2:0/2551459542 shutdown_connections 2023-06-23T18:30:54.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea18100610 0x7fea1819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fea180fea90 0x7fea18069070 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fea000a1150 0x7fea000a3600 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 --2- 172.21.15.2:0/2551459542 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea180ffc80 0x7fea1819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 -- 172.21.15.2:0/2551459542 >> 172.21.15.2:0/2551459542 conn(0x7fea180fa090 msgr2=0x7fea180fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 -- 172.21.15.2:0/2551459542 shutdown_connections 2023-06-23T18:30:54.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.009+0000 7fea1ddcc700 1 -- 172.21.15.2:0/2551459542 wait complete. 2023-06-23T18:30:54.045 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:30:54.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 -- 172.21.15.2:0/2109562706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa500fde50 msgr2=0x7faa500fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 --2- 172.21.15.2:0/2109562706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa500fde50 0x7faa500fe260 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faa40009990 tx=0x7faa40009ca0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 -- 172.21.15.2:0/2109562706 shutdown_connections 2023-06-23T18:30:54.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 --2- 172.21.15.2:0/2109562706 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa501017b0 0x7faa50108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 --2- 172.21.15.2:0/2109562706 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa500fde50 0x7faa500fe260 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 --2- 172.21.15.2:0/2109562706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa50100e20 0x7faa50101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 -- 172.21.15.2:0/2109562706 >> 172.21.15.2:0/2109562706 conn(0x7faa500f9850 msgr2=0x7faa500fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 -- 172.21.15.2:0/2109562706 shutdown_connections 2023-06-23T18:30:54.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 -- 172.21.15.2:0/2109562706 wait complete. 2023-06-23T18:30:54.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.186+0000 7faa568a6700 1 Processor -- start 2023-06-23T18:30:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa568a6700 1 -- start start 2023-06-23T18:30:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa568a6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa500fde50 0x7faa50195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa568a6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa50100e20 0x7faa50195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa568a6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa501017b0 0x7faa5019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa568a6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019d2c0 con 0x7faa501017b0 2023-06-23T18:30:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa568a6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019d400 con 0x7faa500fde50 2023-06-23T18:30:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa568a6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019d590 con 0x7faa50100e20 2023-06-23T18:30:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa54e43700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa501017b0 0x7faa5019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa54e43700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa501017b0 0x7faa5019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55332/0 (socket says 172.21.15.2:55332) 2023-06-23T18:30:54.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa54e43700 1 -- 172.21.15.2:0/1913279498 learned_addr learned my addr 172.21.15.2:0/1913279498 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa50100e20 0x7faa50195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa4ffff700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa500fde50 0x7faa50195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa54e43700 1 -- 172.21.15.2:0/1913279498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa50100e20 msgr2=0x7faa50195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa54e43700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa50100e20 0x7faa50195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.187+0000 7faa54e43700 1 -- 172.21.15.2:0/1913279498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa500fde50 msgr2=0x7faa50195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.188+0000 7faa54e43700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa500fde50 0x7faa50195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.188+0000 7faa54e43700 1 -- 172.21.15.2:0/1913279498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400095f0 con 0x7faa501017b0 2023-06-23T18:30:54.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.188+0000 7faa54e43700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa501017b0 0x7faa5019ccf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faa48007ee0 tx=0x7faa4800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.188+0000 7faa4d7fa700 1 -- 172.21.15.2:0/1913279498 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48017070 con 0x7faa501017b0 2023-06-23T18:30:54.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.188+0000 7faa568a6700 1 -- 172.21.15.2:0/1913279498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5019d870 con 0x7faa501017b0 2023-06-23T18:30:54.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.188+0000 7faa4d7fa700 1 -- 172.21.15.2:0/1913279498 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa4800f040 con 0x7faa501017b0 2023-06-23T18:30:54.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.188+0000 7faa4d7fa700 1 -- 172.21.15.2:0/1913279498 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48005570 con 0x7faa501017b0 2023-06-23T18:30:54.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.188+0000 7faa568a6700 1 -- 172.21.15.2:0/1913279498 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5019ddd0 con 0x7faa501017b0 2023-06-23T18:30:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.190+0000 7faa4d7fa700 1 -- 172.21.15.2:0/1913279498 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7faa4800db20 con 0x7faa501017b0 2023-06-23T18:30:54.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.191+0000 7faa4d7fa700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faa380a1150 0x7faa380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.191+0000 7faa4d7fa700 1 -- 172.21.15.2:0/1913279498 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faa480cbfd0 con 0x7faa501017b0 2023-06-23T18:30:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.191+0000 7faa4ffff700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faa380a1150 0x7faa380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa4ffff700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faa380a1150 0x7faa380a3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b5c0 tx=0x7faa40005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa62a2bb80 1 Processor -- start 2023-06-23T18:30:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa62a2bb80 1 -- start start 2023-06-23T18:30:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e1d6cb0 con 0x56364e2d3000 2023-06-23T18:30:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e229a40 con 0x56364e2d1c80 2023-06-23T18:30:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e222ed0 con 0x56364e2d25d0 2023-06-23T18:30:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa54e43700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa54e43700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55340/0 (socket says 172.21.15.2:55340) 2023-06-23T18:30:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa54e43700 1 -- 172.21.15.2:0/1741307652 learned_addr learned my addr 172.21.15.2:0/1741307652 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa4ffff700 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.192+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa54e43700 1 -- 172.21.15.2:0/1741307652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 msgr2=0x56364e2d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa54e43700 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa54e43700 1 -- 172.21.15.2:0/1741307652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 msgr2=0x56364e2d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa54e43700 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa54e43700 1 -- 172.21.15.2:0/1741307652 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56364e0a4630 con 0x56364e2d3000 2023-06-23T18:30:54.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa4ffff700 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa54e43700 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d3470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faa48099410 tx=0x7faa48004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866fac5d817e00f8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa367fc700 1 -- 172.21.15.2:0/1741307652 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48004c40 con 0x56364e2d3000 2023-06-23T18:30:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa367fc700 1 -- 172.21.15.2:0/1741307652 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa48099480 con 0x56364e2d3000 2023-06-23T18:30:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.193+0000 7faa367fc700 1 -- 172.21.15.2:0/1741307652 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4809d170 con 0x56364e2d3000 2023-06-23T18:30:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 -- 172.21.15.2:0/1741307652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 msgr2=0x56364e2d3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d3470 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7faa48099410 tx=0x7faa48004ad0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 -- 172.21.15.2:0/1741307652 shutdown_connections 2023-06-23T18:30:54.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d3470 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/1741307652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 -- 172.21.15.2:0/1741307652 >> 172.21.15.2:0/1741307652 conn(0x56364e2c73f0 msgr2=0x56364e2cc520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 -- 172.21.15.2:0/1741307652 shutdown_connections 2023-06-23T18:30:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 -- 172.21.15.2:0/1741307652 wait complete. 2023-06-23T18:30:54.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.194+0000 7faa62a2bb80 1 -- start start 2023-06-23T18:30:54.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa62a2bb80 1 Processor -- start 2023-06-23T18:30:54.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d1c80 0x56364e2db520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d25d0 0x56364e2dba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d3000 0x56364e2e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e1c2ac0 con 0x56364e2d1c80 2023-06-23T18:30:54.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e1c27c0 con 0x56364e2d25d0 2023-06-23T18:30:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e0a6920 con 0x56364e2d3000 2023-06-23T18:30:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa62a2bb80 10 client.0 _ll_register_callbacks cb 0x56364e1c8260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa54e43700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d3000 0x56364e2e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa54e43700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d3000 0x56364e2e2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55684/0 (socket says 172.21.15.2:55684) 2023-06-23T18:30:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa54e43700 1 -- 172.21.15.2:0/3667781187 learned_addr learned my addr 172.21.15.2:0/3667781187 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa4ffff700 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d1c80 0x56364e2db520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d25d0 0x56364e2dba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa54e43700 1 -- 172.21.15.2:0/3667781187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d25d0 msgr2=0x56364e2dba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa54e43700 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d25d0 0x56364e2dba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa54e43700 1 -- 172.21.15.2:0/3667781187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d1c80 msgr2=0x56364e2db520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa54e43700 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d1c80 0x56364e2db520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.195+0000 7faa54e43700 1 -- 172.21.15.2:0/3667781187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5019ddd0 con 0x56364e2d3000 2023-06-23T18:30:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa54e43700 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d3000 0x56364e2e2aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa480c9bc0 tx=0x7faa48098ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3bee67c85230aa4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:30:54.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa2f7fe700 1 -- 172.21.15.2:0/3667781187 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa480a18d0 con 0x56364e2d3000 2023-06-23T18:30:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa2f7fe700 1 -- 172.21.15.2:0/3667781187 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faa480b0070 con 0x56364e2d3000 2023-06-23T18:30:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3667781187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56364e2c4bb0 con 0x56364e2d3000 2023-06-23T18:30:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa2f7fe700 1 -- 172.21.15.2:0/3667781187 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa480996e0 con 0x56364e2d3000 2023-06-23T18:30:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa62a2bb80 10 client.25678 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3667781187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56364e050de0 con 0x56364e2d3000 2023-06-23T18:30:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa2f7fe700 1 -- 172.21.15.2:0/3667781187 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56364e2c4bb0 con 0x56364e2d3000 2023-06-23T18:30:54.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.196+0000 7faa2f7fe700 1 -- 172.21.15.2:0/3667781187 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faa480a1440 con 0x56364e2d3000 2023-06-23T18:30:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa62a2bb80 10 client.25678 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa62a2bb80 10 client.25678 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:30:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa62a2bb80 20 client.25678 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x56364e2e6e90 0x56364e2e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa62a2bb80 4 client.25678 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:30:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3667781187 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56364e21beb0 con 0x56364e2e6e90 2023-06-23T18:30:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa4ffff700 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x56364e2e6e90 0x56364e2e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa2f7fe700 10 client.25678 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:30:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.197+0000 7faa4ffff700 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x56364e2e6e90 0x56364e2e92a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e545d0e680181371 server_cookie=e17dc61a8abe8c98 in_seq=0 out_seq=0 2023-06-23T18:30:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa2f7fe700 1 -- 172.21.15.2:0/3667781187 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56364e21beb0 con 0x56364e2e6e90 2023-06-23T18:30:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa2f7fe700 10 client.25678 handle_command_reply: tid=0 2023-06-23T18:30:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 client.25678 shutdown 2023-06-23T18:30:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3667781187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d3000 msgr2=0x56364e2e2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d3000 0x56364e2e2aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faa480c9bc0 tx=0x7faa48098ec0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3667781187 shutdown_connections 2023-06-23T18:30:54.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x56364e2e6e90 0x56364e2e92a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d3000 0x56364e2e2aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa2f7fe700 0 client.25678 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:30:54.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d1c80 0x56364e2db520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3667781187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d25d0 0x56364e2dba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3667781187 >> 172.21.15.2:0/3667781187 conn(0x56364e2c73f0 msgr2=0x56364e2cbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3667781187 shutdown_connections 2023-06-23T18:30:54.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3667781187 wait complete. 2023-06-23T18:30:54.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.198+0000 7faa62a2bb80 20 client.25678 trim_cache size 0 max 16384 2023-06-23T18:30:54.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa62a2bb80 1 Processor -- start 2023-06-23T18:30:54.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa62a2bb80 1 -- start start 2023-06-23T18:30:54.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2e3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e2daa10 con 0x56364e2d3000 2023-06-23T18:30:54.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e2cd220 con 0x56364e2d1c80 2023-06-23T18:30:54.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e2cd3f0 con 0x56364e2d25d0 2023-06-23T18:30:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa54e43700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa54e43700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55354/0 (socket says 172.21.15.2:55354) 2023-06-23T18:30:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa54e43700 1 -- 172.21.15.2:0/3171301961 learned_addr learned my addr 172.21.15.2:0/3171301961 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa4ffff700 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2e3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa4f7fe700 1 -- 172.21.15.2:0/3171301961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 msgr2=0x56364e2e3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2e3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa4f7fe700 1 -- 172.21.15.2:0/3171301961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 msgr2=0x56364e2d9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.214+0000 7faa4f7fe700 1 -- 172.21.15.2:0/3171301961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56364e039c60 con 0x56364e2d25d0 2023-06-23T18:30:54.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.215+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2e2a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x56364e2d29e0 tx=0x7faa44017750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84b0495c9081ddf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.215+0000 7faa36ffd700 1 -- 172.21.15.2:0/3171301961 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4401c070 con 0x56364e2d25d0 2023-06-23T18:30:54.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.215+0000 7faa36ffd700 1 -- 172.21.15.2:0/3171301961 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa440044e0 con 0x56364e2d25d0 2023-06-23T18:30:54.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.215+0000 7faa36ffd700 1 -- 172.21.15.2:0/3171301961 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44003aa0 con 0x56364e2d25d0 2023-06-23T18:30:54.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3171301961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 msgr2=0x56364e2e2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2e2a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x56364e2d29e0 tx=0x7faa44017750 comp rx=0 tx=0).stop 2023-06-23T18:30:54.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3171301961 shutdown_connections 2023-06-23T18:30:54.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d3000 0x56364e2d9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d1c80 0x56364e2e3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/3171301961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d25d0 0x56364e2e2a60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3171301961 >> 172.21.15.2:0/3171301961 conn(0x56364e2c73f0 msgr2=0x56364e2da200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3171301961 shutdown_connections 2023-06-23T18:30:54.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 -- 172.21.15.2:0/3171301961 wait complete. 2023-06-23T18:30:54.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.216+0000 7faa62a2bb80 1 -- start start 2023-06-23T18:30:54.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa62a2bb80 1 Processor -- start 2023-06-23T18:30:54.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d1c80 0x56364e2e2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d25d0 0x56364e2df0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa62a2bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d3000 0x56364e2df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e2dfcd0 con 0x56364e2d25d0 2023-06-23T18:30:54.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e2e2c90 con 0x56364e2d3000 2023-06-23T18:30:54.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa62a2bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56364e2e2e20 con 0x56364e2d1c80 2023-06-23T18:30:54.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa62a2bb80 10 client.0 _ll_register_callbacks cb 0x56364e1c8260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:54.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa4f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d25d0 0x56364e2df0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa4f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d25d0 0x56364e2df0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55358/0 (socket says 172.21.15.2:55358) 2023-06-23T18:30:54.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.217+0000 7faa4f7fe700 1 -- 172.21.15.2:0/4083064797 learned_addr learned my addr 172.21.15.2:0/4083064797 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa4ffff700 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d1c80 0x56364e2e2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa4f7fe700 1 -- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d1c80 msgr2=0x56364e2e2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d1c80 0x56364e2e2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa4f7fe700 1 -- 172.21.15.2:0/4083064797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d3000 msgr2=0x56364e2df5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d3000 0x56364e2df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa4f7fe700 1 -- 172.21.15.2:0/4083064797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4400aa60 con 0x56364e2d25d0 2023-06-23T18:30:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa4ffff700 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d1c80 0x56364e2e2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:30:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa4f7fe700 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d25d0 0x56364e2df0b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faa440036f0 tx=0x7faa44017320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d82bdee6f1366c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa2f7fe700 1 -- 172.21.15.2:0/4083064797 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4401c070 con 0x56364e2d25d0 2023-06-23T18:30:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa2f7fe700 1 -- 172.21.15.2:0/4083064797 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faa44003ab0 con 0x56364e2d25d0 2023-06-23T18:30:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa62a2bb80 1 -- 172.21.15.2:0/4083064797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa48099480 con 0x56364e2d25d0 2023-06-23T18:30:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa2f7fe700 1 -- 172.21.15.2:0/4083064797 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa44025070 con 0x56364e2d25d0 2023-06-23T18:30:54.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa62a2bb80 10 client.15933 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:54.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.218+0000 7faa62a2bb80 1 -- 172.21.15.2:0/4083064797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56364e077250 con 0x56364e2d25d0 2023-06-23T18:30:54.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa2f7fe700 1 -- 172.21.15.2:0/4083064797 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa48099480 con 0x56364e2d25d0 2023-06-23T18:30:54.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa2f7fe700 1 -- 172.21.15.2:0/4083064797 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faa44015450 con 0x56364e2d25d0 2023-06-23T18:30:54.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa62a2bb80 10 client.15933 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:54.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa62a2bb80 10 client.15933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:30:54.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa62a2bb80 20 client.15933 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:54.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x56364e33c400 0x56364e33e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa62a2bb80 4 client.15933 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:30:54.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa62a2bb80 1 -- 172.21.15.2:0/4083064797 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56364e1de050 con 0x56364e33c400 2023-06-23T18:30:54.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.219+0000 7faa4ffff700 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x56364e33c400 0x56364e33e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa2f7fe700 10 client.15933 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:30:54.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa4ffff700 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x56364e33c400 0x56364e33e810 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=765ebf51b34c055e server_cookie=2ffc58087ffe807a in_seq=0 out_seq=0 2023-06-23T18:30:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa2f7fe700 1 -- 172.21.15.2:0/4083064797 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56364e1de050 con 0x56364e33c400 2023-06-23T18:30:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa2f7fe700 10 client.15933 handle_command_reply: tid=0 2023-06-23T18:30:54.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 client.15933 shutdown 2023-06-23T18:30:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 -- 172.21.15.2:0/4083064797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d25d0 msgr2=0x56364e2df0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d25d0 0x56364e2df0b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7faa440036f0 tx=0x7faa44017320 comp rx=0 tx=0).stop 2023-06-23T18:30:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 -- 172.21.15.2:0/4083064797 shutdown_connections 2023-06-23T18:30:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x56364e33c400 0x56364e33e810 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56364e2d3000 0x56364e2df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56364e2d1c80 0x56364e2e2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 --2- 172.21.15.2:0/4083064797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56364e2d25d0 0x56364e2df0b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 -- 172.21.15.2:0/4083064797 >> 172.21.15.2:0/4083064797 conn(0x56364e2c73f0 msgr2=0x56364e2d5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa2f7fe700 0 client.15933 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:30:54.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 -- 172.21.15.2:0/4083064797 shutdown_connections 2023-06-23T18:30:54.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 1 -- 172.21.15.2:0/4083064797 wait complete. 2023-06-23T18:30:54.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.220+0000 7faa62a2bb80 20 client.15933 trim_cache size 0 max 16384 2023-06-23T18:30:54.233 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:30:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 -- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faa380a1150 msgr2=0x7faa380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faa380a1150 0x7faa380a3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b5c0 tx=0x7faa40005fb0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 -- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa501017b0 msgr2=0x7faa5019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa501017b0 0x7faa5019ccf0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7faa48007ee0 tx=0x7faa4800d3b0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 -- 172.21.15.2:0/1913279498 shutdown_connections 2023-06-23T18:30:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7faa501017b0 0x7faa5019ccf0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7faa380a1150 0x7faa380a3600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa500fde50 0x7faa50195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 --2- 172.21.15.2:0/1913279498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa50100e20 0x7faa50195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 -- 172.21.15.2:0/1913279498 >> 172.21.15.2:0/1913279498 conn(0x7faa500f9850 msgr2=0x7faa500fba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 -- 172.21.15.2:0/1913279498 shutdown_connections 2023-06-23T18:30:54.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.221+0000 7faa568a6700 1 -- 172.21.15.2:0/1913279498 wait complete. 2023-06-23T18:30:54.239 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:30:54.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.380+0000 7fb76735a700 1 -- 172.21.15.2:0/2235496525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb760100610 msgr2=0x7fb760106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.380+0000 7fb76735a700 1 --2- 172.21.15.2:0/2235496525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb760100610 0x7fb760106ed0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb75800b0a0 tx=0x7fb75800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.380+0000 7fb76735a700 1 -- 172.21.15.2:0/2235496525 shutdown_connections 2023-06-23T18:30:54.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.380+0000 7fb76735a700 1 --2- 172.21.15.2:0/2235496525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb760100610 0x7fb760106ed0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.381+0000 7fb76735a700 1 --2- 172.21.15.2:0/2235496525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7600fea90 0x7fb7600feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.381+0000 7fb76735a700 1 --2- 172.21.15.2:0/2235496525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7600ffc80 0x7fb7601000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.381+0000 7fb76735a700 1 -- 172.21.15.2:0/2235496525 >> 172.21.15.2:0/2235496525 conn(0x7fb7600fa090 msgr2=0x7fb7600fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.381+0000 7fb76735a700 1 -- 172.21.15.2:0/2235496525 shutdown_connections 2023-06-23T18:30:54.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.381+0000 7fb76735a700 1 -- 172.21.15.2:0/2235496525 wait complete. 2023-06-23T18:30:54.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.381+0000 7fb76735a700 1 Processor -- start 2023-06-23T18:30:54.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.381+0000 7fb76735a700 1 -- start start 2023-06-23T18:30:54.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb76735a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7600fea90 0x7fb760069050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb76735a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7600ffc80 0x7fb76019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb76735a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb760100610 0x7fb76019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb76735a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76019d480 con 0x7fb7600fea90 2023-06-23T18:30:54.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb76735a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76019d5f0 con 0x7fb7600ffc80 2023-06-23T18:30:54.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb76735a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb76019d7b0 con 0x7fb760100610 2023-06-23T18:30:54.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7600fea90 0x7fb760069050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7600fea90 0x7fb760069050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55370/0 (socket says 172.21.15.2:55370) 2023-06-23T18:30:54.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 -- 172.21.15.2:0/987901846 learned_addr learned my addr 172.21.15.2:0/987901846 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7648f5700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7600ffc80 0x7fb76019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7658f7700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb760100610 0x7fb76019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 -- 172.21.15.2:0/987901846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb760100610 msgr2=0x7fb76019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb760100610 0x7fb76019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 -- 172.21.15.2:0/987901846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7600ffc80 msgr2=0x7fb76019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7600ffc80 0x7fb76019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 -- 172.21.15.2:0/987901846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb758009d00 con 0x7fb7600fea90 2023-06-23T18:30:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7658f7700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb760100610 0x7fb76019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.382+0000 7fb7650f6700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7600fea90 0x7fb760069050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00d980 tx=0x7fb75c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.383+0000 7fb7567fc700 1 -- 172.21.15.2:0/987901846 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00b950 con 0x7fb7600fea90 2023-06-23T18:30:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.383+0000 7fb7567fc700 1 -- 172.21.15.2:0/987901846 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb75c007620 con 0x7fb7600fea90 2023-06-23T18:30:54.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.383+0000 7fb7567fc700 1 -- 172.21.15.2:0/987901846 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c00f3c0 con 0x7fb7600fea90 2023-06-23T18:30:54.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.383+0000 7fb76735a700 1 -- 172.21.15.2:0/987901846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb76019da90 con 0x7fb7600fea90 2023-06-23T18:30:54.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.383+0000 7fb76735a700 1 -- 172.21.15.2:0/987901846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76019dfa0 con 0x7fb7600fea90 2023-06-23T18:30:54.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.385+0000 7fb7567fc700 1 -- 172.21.15.2:0/987901846 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb75c00bab0 con 0x7fb7600fea90 2023-06-23T18:30:54.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.385+0000 7fb7567fc700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb74c0a11d0 0x7fb74c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.386+0000 7fb7567fc700 1 -- 172.21.15.2:0/987901846 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb75c0cc7d0 con 0x7fb7600fea90 2023-06-23T18:30:54.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.386+0000 7fb7648f5700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb74c0a11d0 0x7fb74c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.386+0000 7fb7648f5700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb74c0a11d0 0x7fb74c0a3680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb760068dc0 tx=0x7fb75000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7734dfb80 1 Processor -- start 2023-06-23T18:30:54.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7734dfb80 1 -- start start 2023-06-23T18:30:54.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a100c80 0x55e63a101090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a1015d0 0x55e63a101a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 0x55e63a102470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a005cb0 con 0x55e63a100c80 2023-06-23T18:30:54.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a058a40 con 0x55e63a1015d0 2023-06-23T18:30:54.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a051ed0 con 0x55e63a102000 2023-06-23T18:30:54.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7650f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a100c80 0x55e63a101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7650f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a100c80 0x55e63a101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55378/0 (socket says 172.21.15.2:55378) 2023-06-23T18:30:54.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7650f6700 1 -- 172.21.15.2:0/1223332797 learned_addr learned my addr 172.21.15.2:0/1223332797 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7658f7700 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 0x55e63a102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.387+0000 7fb7648f5700 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a1015d0 0x55e63a101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb7650f6700 1 -- 172.21.15.2:0/1223332797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 msgr2=0x55e63a102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb7650f6700 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 0x55e63a102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb7650f6700 1 -- 172.21.15.2:0/1223332797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a1015d0 msgr2=0x55e63a101a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb7650f6700 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a1015d0 0x55e63a101a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb7650f6700 1 -- 172.21.15.2:0/1223332797 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e639ed3630 con 0x55e63a100c80 2023-06-23T18:30:54.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb7648f5700 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a1015d0 0x55e63a101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:54.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb7650f6700 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a100c80 0x55e63a101090 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb75c099b70 tx=0x7fb75c099610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65831d306fd7f1e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb74b7fe700 1 -- 172.21.15.2:0/1223332797 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c09dc10 con 0x55e63a100c80 2023-06-23T18:30:54.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb74b7fe700 1 -- 172.21.15.2:0/1223332797 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb75c0a2030 con 0x55e63a100c80 2023-06-23T18:30:54.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.388+0000 7fb74b7fe700 1 -- 172.21.15.2:0/1223332797 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c099e40 con 0x55e63a100c80 2023-06-23T18:30:54.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1223332797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a100c80 msgr2=0x55e63a101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a100c80 0x55e63a101090 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb75c099b70 tx=0x7fb75c099610 comp rx=0 tx=0).stop 2023-06-23T18:30:54.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1223332797 shutdown_connections 2023-06-23T18:30:54.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 0x55e63a102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a100c80 0x55e63a101090 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1223332797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a1015d0 0x55e63a101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1223332797 >> 172.21.15.2:0/1223332797 conn(0x55e63a0f63f0 msgr2=0x55e63a0fb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1223332797 shutdown_connections 2023-06-23T18:30:54.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1223332797 wait complete. 2023-06-23T18:30:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 -- start start 2023-06-23T18:30:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.389+0000 7fb7734dfb80 1 Processor -- start 2023-06-23T18:30:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a10a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a10aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 0x55e63a111b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e639ff1ac0 con 0x55e63a1015d0 2023-06-23T18:30:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e639ff17c0 con 0x55e63a100c80 2023-06-23T18:30:54.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e639ed5920 con 0x55e63a102000 2023-06-23T18:30:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7734dfb80 10 client.0 _ll_register_callbacks cb 0x55e639ff7260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7648f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a10aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7648f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a10aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55386/0 (socket says 172.21.15.2:55386) 2023-06-23T18:30:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7648f5700 1 -- 172.21.15.2:0/344939088 learned_addr learned my addr 172.21.15.2:0/344939088 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7658f7700 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 0x55e63a111b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7650f6700 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a10a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7648f5700 1 -- 172.21.15.2:0/344939088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 msgr2=0x55e63a111b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7648f5700 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 0x55e63a111b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7648f5700 1 -- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 msgr2=0x55e63a10a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7648f5700 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a10a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7648f5700 1 -- 172.21.15.2:0/344939088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb76019dfa0 con 0x55e63a1015d0 2023-06-23T18:30:54.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.390+0000 7fb7650f6700 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a10a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7648f5700 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a10aac0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb750011560 tx=0x7fb7500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8358ae9bd8fe5db8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb748ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:54.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb748ff9700 1 -- 172.21.15.2:0/344939088 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb750028670 con 0x55e63a1015d0 2023-06-23T18:30:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb748ff9700 1 -- 172.21.15.2:0/344939088 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb750017070 con 0x55e63a1015d0 2023-06-23T18:30:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 1 -- 172.21.15.2:0/344939088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e63a0f3bb0 con 0x55e63a1015d0 2023-06-23T18:30:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb748ff9700 1 -- 172.21.15.2:0/344939088 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb750004830 con 0x55e63a1015d0 2023-06-23T18:30:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 10 client.15945 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 1 -- 172.21.15.2:0/344939088 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e639e7fde0 con 0x55e63a1015d0 2023-06-23T18:30:54.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb748ff9700 1 -- 172.21.15.2:0/344939088 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e63a0f3bb0 con 0x55e63a1015d0 2023-06-23T18:30:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb748ff9700 1 -- 172.21.15.2:0/344939088 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb750031080 con 0x55e63a1015d0 2023-06-23T18:30:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 10 client.15945 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 20 client.15945 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:54.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e63a116410 0x55e63a118820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 4 client.15945 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:30:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.391+0000 7fb7734dfb80 1 -- 172.21.15.2:0/344939088 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e63a04aeb0 con 0x55e63a116410 2023-06-23T18:30:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.392+0000 7fb7650f6700 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e63a116410 0x55e63a118820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.392+0000 7fb748ff9700 10 client.15945 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.392+0000 7fb7650f6700 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e63a116410 0x55e63a118820 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efeb04a29d5246b4 server_cookie=b4f20e6e2b5d3c97 in_seq=0 out_seq=0 2023-06-23T18:30:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb748ff9700 1 -- 172.21.15.2:0/344939088 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e63a04aeb0 con 0x55e63a116410 2023-06-23T18:30:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb748ff9700 10 client.15945 handle_command_reply: tid=0 2023-06-23T18:30:54.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 client.15945 shutdown 2023-06-23T18:30:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 -- 172.21.15.2:0/344939088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 msgr2=0x55e63a10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a10aac0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb750011560 tx=0x7fb7500047c0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 -- 172.21.15.2:0/344939088 shutdown_connections 2023-06-23T18:30:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e63a116410 0x55e63a118820 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a102000 0x55e63a111b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a10a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb748ff9700 0 client.15945 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/344939088 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a10aac0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 -- 172.21.15.2:0/344939088 >> 172.21.15.2:0/344939088 conn(0x55e63a0f63f0 msgr2=0x55e63a0fac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 -- 172.21.15.2:0/344939088 shutdown_connections 2023-06-23T18:30:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 1 -- 172.21.15.2:0/344939088 wait complete. 2023-06-23T18:30:54.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.393+0000 7fb7734dfb80 20 client.15945 trim_cache size 0 max 16384 2023-06-23T18:30:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7734dfb80 1 Processor -- start 2023-06-23T18:30:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7734dfb80 1 -- start start 2023-06-23T18:30:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a100c80 0x55e63a0f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a11d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a102000 0x55e63a0fbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a10a120 con 0x55e63a1015d0 2023-06-23T18:30:54.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a0fcfb0 con 0x55e63a102000 2023-06-23T18:30:54.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a111eb0 con 0x55e63a100c80 2023-06-23T18:30:54.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7648f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a11d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7648f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a11d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55392/0 (socket says 172.21.15.2:55392) 2023-06-23T18:30:54.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.411+0000 7fb7648f5700 1 -- 172.21.15.2:0/2560378153 learned_addr learned my addr 172.21.15.2:0/2560378153 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb7650f6700 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a100c80 0x55e63a0f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb7658f7700 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a102000 0x55e63a0fbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb7648f5700 1 -- 172.21.15.2:0/2560378153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a100c80 msgr2=0x55e63a0f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb7648f5700 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a100c80 0x55e63a0f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb7648f5700 1 -- 172.21.15.2:0/2560378153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a102000 msgr2=0x55e63a0fbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb7648f5700 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a102000 0x55e63a0fbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb7648f5700 1 -- 172.21.15.2:0/2560378153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e639e68c60 con 0x55e63a1015d0 2023-06-23T18:30:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb7648f5700 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a11d180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb750011530 tx=0x7fb7500107e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78674f7babc35aa server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb74bfff700 1 -- 172.21.15.2:0/2560378153 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75001fa10 con 0x55e63a1015d0 2023-06-23T18:30:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb74bfff700 1 -- 172.21.15.2:0/2560378153 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb750015070 con 0x55e63a1015d0 2023-06-23T18:30:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.412+0000 7fb74bfff700 1 -- 172.21.15.2:0/2560378153 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7500289d0 con 0x55e63a1015d0 2023-06-23T18:30:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 -- 172.21.15.2:0/2560378153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 msgr2=0x55e63a11d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a11d180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb750011530 tx=0x7fb7500107e0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 -- 172.21.15.2:0/2560378153 shutdown_connections 2023-06-23T18:30:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a102000 0x55e63a0fbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a100c80 0x55e63a0f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/2560378153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a1015d0 0x55e63a11d180 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 -- 172.21.15.2:0/2560378153 >> 172.21.15.2:0/2560378153 conn(0x55e63a0f63f0 msgr2=0x55e63a0fb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 -- 172.21.15.2:0/2560378153 shutdown_connections 2023-06-23T18:30:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 -- 172.21.15.2:0/2560378153 wait complete. 2023-06-23T18:30:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.413+0000 7fb7734dfb80 1 -- start start 2023-06-23T18:30:54.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7734dfb80 1 Processor -- start 2023-06-23T18:30:54.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a1664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a1015d0 0x55e63a166a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7734dfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a102000 0x55e63a16d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a16d950 con 0x55e63a102000 2023-06-23T18:30:54.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a16da90 con 0x55e63a100c80 2023-06-23T18:30:54.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7734dfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63a16dc20 con 0x55e63a1015d0 2023-06-23T18:30:54.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7734dfb80 10 client.0 _ll_register_callbacks cb 0x55e639ff7260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:54.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7658f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a102000 0x55e63a16d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7650f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a1664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7658f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a102000 0x55e63a16d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55400/0 (socket says 172.21.15.2:55400) 2023-06-23T18:30:54.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7650f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a1664f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:45988/0 (socket says 172.21.15.2:45988) 2023-06-23T18:30:54.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7658f7700 1 -- 172.21.15.2:0/1562248047 learned_addr learned my addr 172.21.15.2:0/1562248047 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:54.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.414+0000 7fb7648f5700 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a1015d0 0x55e63a166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7658f7700 1 -- 172.21.15.2:0/1562248047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a1015d0 msgr2=0x55e63a166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7658f7700 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a1015d0 0x55e63a166a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7658f7700 1 -- 172.21.15.2:0/1562248047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 msgr2=0x55e63a1664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:54.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7658f7700 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a1664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7658f7700 1 -- 172.21.15.2:0/1562248047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e639e7fde0 con 0x55e63a102000 2023-06-23T18:30:54.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7648f5700 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a1015d0 0x55e63a166a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:54.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7650f6700 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a1664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:54.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7658f7700 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a102000 0x55e63a16d410 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb75800b0a0 tx=0x7fb758004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edcd5a28d29a26f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:54.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb748ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:54.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb748ff9700 1 -- 172.21.15.2:0/1562248047 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb758021040 con 0x55e63a102000 2023-06-23T18:30:54.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb748ff9700 1 -- 172.21.15.2:0/1562248047 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb758005450 con 0x55e63a102000 2023-06-23T18:30:54.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1562248047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb75c0a2030 con 0x55e63a102000 2023-06-23T18:30:54.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb748ff9700 1 -- 172.21.15.2:0/1562248047 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb75800e040 con 0x55e63a102000 2023-06-23T18:30:54.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7734dfb80 10 client.15951 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:54.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1562248047 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e639ea6250 con 0x55e63a102000 2023-06-23T18:30:54.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.415+0000 7fb748ff9700 1 -- 172.21.15.2:0/1562248047 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb75c0a2030 con 0x55e63a102000 2023-06-23T18:30:54.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb748ff9700 1 -- 172.21.15.2:0/1562248047 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb75800b0d0 con 0x55e63a102000 2023-06-23T18:30:54.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb7734dfb80 10 client.15951 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:54.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb7734dfb80 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:54.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb7734dfb80 20 client.15951 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:54.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e63a16f700 0x55e63a171b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:54.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb7734dfb80 4 client.15951 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:30:54.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1562248047 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e63a00d050 con 0x55e63a16f700 2023-06-23T18:30:54.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb7650f6700 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e63a16f700 0x55e63a171b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:54.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb7650f6700 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e63a16f700 0x55e63a171b10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ecec7d9580ac4af server_cookie=9777b3a009873329 in_seq=0 out_seq=0 2023-06-23T18:30:54.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.416+0000 7fb748ff9700 10 client.15951 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:54.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb748ff9700 1 -- 172.21.15.2:0/1562248047 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e63a00d050 con 0x55e63a16f700 2023-06-23T18:30:54.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb748ff9700 10 client.15951 handle_command_reply: tid=0 2023-06-23T18:30:54.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 client.15951 shutdown 2023-06-23T18:30:54.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1562248047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a102000 msgr2=0x55e63a16d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a102000 0x55e63a16d410 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb75800b0a0 tx=0x7fb758004c60 comp rx=0 tx=0).stop 2023-06-23T18:30:54.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1562248047 shutdown_connections 2023-06-23T18:30:54.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e63a16f700 0x55e63a171b10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e63a102000 0x55e63a16d410 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e63a100c80 0x55e63a1664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb748ff9700 0 client.15951 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:54.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 --2- 172.21.15.2:0/1562248047 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e63a1015d0 0x55e63a166a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1562248047 >> 172.21.15.2:0/1562248047 conn(0x55e63a0f63f0 msgr2=0x55e63a11d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1562248047 shutdown_connections 2023-06-23T18:30:54.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 1 -- 172.21.15.2:0/1562248047 wait complete. 2023-06-23T18:30:54.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.417+0000 7fb7734dfb80 20 client.15951 trim_cache size 0 max 16384 2023-06-23T18:30:54.435 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:30:54.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 -- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb74c0a11d0 msgr2=0x7fb74c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb74c0a11d0 0x7fb74c0a3680 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb760068dc0 tx=0x7fb75000b410 comp rx=0 tx=0).stop 2023-06-23T18:30:54.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 -- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7600fea90 msgr2=0x7fb760069050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:54.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7600fea90 0x7fb760069050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00d980 tx=0x7fb75c00dc90 comp rx=0 tx=0).stop 2023-06-23T18:30:54.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 -- 172.21.15.2:0/987901846 shutdown_connections 2023-06-23T18:30:54.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb760100610 0x7fb76019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7600fea90 0x7fb760069050 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb74c0a11d0 0x7fb74c0a3680 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 --2- 172.21.15.2:0/987901846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7600ffc80 0x7fb76019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:54.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 -- 172.21.15.2:0/987901846 >> 172.21.15.2:0/987901846 conn(0x7fb7600fa090 msgr2=0x7fb760103160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:54.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 -- 172.21.15.2:0/987901846 shutdown_connections 2023-06-23T18:30:54.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:54.418+0000 7fb76735a700 1 -- 172.21.15.2:0/987901846 wait complete. 2023-06-23T18:30:54.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:54 smithi002 ceph-mon[95947]: pgmap v562: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 57 MiB/s wr, 32 op/s 2023-06-23T18:30:54.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:54 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2959450723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:30:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:54 smithi017 ceph-mon[97472]: pgmap v562: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 57 MiB/s wr, 32 op/s 2023-06-23T18:30:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:54 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2959450723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:30:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:54 smithi082 ceph-mon[97889]: pgmap v562: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 57 MiB/s wr, 32 op/s 2023-06-23T18:30:54.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:54 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2959450723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:30:55.439 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:30:55.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:30:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.581+0000 7fbc17907700 1 -- 172.21.15.2:0/2696299018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc10101710 msgr2=0x7fbc10107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.581+0000 7fbc17907700 1 --2- 172.21.15.2:0/2696299018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc10101710 0x7fbc10107fd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800b0a0 tx=0x7fbc0800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.581+0000 7fbc17907700 1 -- 172.21.15.2:0/2696299018 shutdown_connections 2023-06-23T18:30:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.581+0000 7fbc17907700 1 --2- 172.21.15.2:0/2696299018 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc10101710 0x7fbc10107fd0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.581+0000 7fbc17907700 1 --2- 172.21.15.2:0/2696299018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc100ffb90 0x7fbc100fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.582+0000 7fbc17907700 1 --2- 172.21.15.2:0/2696299018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc10100d80 0x7fbc101011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.582+0000 7fbc17907700 1 -- 172.21.15.2:0/2696299018 >> 172.21.15.2:0/2696299018 conn(0x7fbc100fb1b0 msgr2=0x7fbc100fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.582+0000 7fbc17907700 1 -- 172.21.15.2:0/2696299018 shutdown_connections 2023-06-23T18:30:55.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.582+0000 7fbc17907700 1 -- 172.21.15.2:0/2696299018 wait complete. 2023-06-23T18:30:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.582+0000 7fbc17907700 1 Processor -- start 2023-06-23T18:30:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.582+0000 7fbc17907700 1 -- start start 2023-06-23T18:30:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.582+0000 7fbc17907700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc100ffb90 0x7fbc10074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.582+0000 7fbc17907700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc10100d80 0x7fbc100728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc17907700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10101710 0x7fbc10072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc17907700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc100735a0 con 0x7fbc100ffb90 2023-06-23T18:30:55.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc17907700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc10073710 con 0x7fbc10101710 2023-06-23T18:30:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc17907700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1019f8c0 con 0x7fbc10100d80 2023-06-23T18:30:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc100ffb90 0x7fbc10074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc100ffb90 0x7fbc10074280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55410/0 (socket says 172.21.15.2:55410) 2023-06-23T18:30:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 -- 172.21.15.2:0/445861432 learned_addr learned my addr 172.21.15.2:0/445861432 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc10100d80 0x7fbc100728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 -- 172.21.15.2:0/445861432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc10100d80 msgr2=0x7fbc100728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc10100d80 0x7fbc100728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc15ea4700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10101710 0x7fbc10072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 -- 172.21.15.2:0/445861432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10101710 msgr2=0x7fbc10072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10101710 0x7fbc10072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 -- 172.21.15.2:0/445861432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08009d00 con 0x7fbc100ffb90 2023-06-23T18:30:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc15ea4700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10101710 0x7fbc10072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:30:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc156a3700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc100ffb90 0x7fbc10074280 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c990 tx=0x7fbc0c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc067fc700 1 -- 172.21.15.2:0/445861432 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c007b00 con 0x7fbc100ffb90 2023-06-23T18:30:55.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.583+0000 7fbc067fc700 1 -- 172.21.15.2:0/445861432 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc0c007c60 con 0x7fbc100ffb90 2023-06-23T18:30:55.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.584+0000 7fbc067fc700 1 -- 172.21.15.2:0/445861432 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c018660 con 0x7fbc100ffb90 2023-06-23T18:30:55.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.584+0000 7fbc17907700 1 -- 172.21.15.2:0/445861432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1019fba0 con 0x7fbc100ffb90 2023-06-23T18:30:55.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.584+0000 7fbc17907700 1 -- 172.21.15.2:0/445861432 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc101a00b0 con 0x7fbc100ffb90 2023-06-23T18:30:55.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.587+0000 7fbc067fc700 1 -- 172.21.15.2:0/445861432 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbc0c00f450 con 0x7fbc100ffb90 2023-06-23T18:30:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.587+0000 7fbc067fc700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbbfc0a1250 0x7fbbfc0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.587+0000 7fbc067fc700 1 -- 172.21.15.2:0/445861432 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc0c0cc900 con 0x7fbc100ffb90 2023-06-23T18:30:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.587+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbbfc0a1250 0x7fbbfc0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc23a8cb80 1 Processor -- start 2023-06-23T18:30:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc23a8cb80 1 -- start start 2023-06-23T18:30:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbbfc0a1250 0x7fbbfc0a3700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005fd0 tx=0x7fbc00005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280510 0x555ca8280920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280e60 0x555ca8281300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8281d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca817cd30 con 0x555ca8281890 2023-06-23T18:30:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca81cfac0 con 0x555ca8280510 2023-06-23T18:30:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca81c8f50 con 0x555ca8280e60 2023-06-23T18:30:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc156a3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280510 0x555ca8280920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc15ea4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8281d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc14ea2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280e60 0x555ca8281300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc156a3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280510 0x555ca8280920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46010/0 (socket says 172.21.15.2:46010) 2023-06-23T18:30:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.588+0000 7fbc156a3700 1 -- 172.21.15.2:0/2855140210 learned_addr learned my addr 172.21.15.2:0/2855140210 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbc14ea2700 1 -- 172.21.15.2:0/2855140210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280510 msgr2=0x555ca8280920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280510 0x555ca8280920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbc14ea2700 1 -- 172.21.15.2:0/2855140210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 msgr2=0x555ca8281d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8281d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbc14ea2700 1 -- 172.21.15.2:0/2855140210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ca804a6b0 con 0x555ca8280e60 2023-06-23T18:30:55.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbc156a3700 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280510 0x555ca8280920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:55.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbc15ea4700 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8281d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:55.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280e60 0x555ca8281300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005e20 tx=0x7fbc00011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5218aa665a8c326 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbbfb7fe700 1 -- 172.21.15.2:0/2855140210 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc00010ca0 con 0x555ca8280e60 2023-06-23T18:30:55.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbbfb7fe700 1 -- 172.21.15.2:0/2855140210 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc0000ee90 con 0x555ca8280e60 2023-06-23T18:30:55.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.589+0000 7fbbfb7fe700 1 -- 172.21.15.2:0/2855140210 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0000f620 con 0x555ca8280e60 2023-06-23T18:30:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2855140210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280e60 msgr2=0x555ca8281300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280e60 0x555ca8281300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005e20 tx=0x7fbc00011000 comp rx=0 tx=0).stop 2023-06-23T18:30:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2855140210 shutdown_connections 2023-06-23T18:30:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8281d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280510 0x555ca8280920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2855140210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280e60 0x555ca8281300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2855140210 >> 172.21.15.2:0/2855140210 conn(0x555ca826d470 msgr2=0x555ca82725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2855140210 shutdown_connections 2023-06-23T18:30:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.590+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2855140210 wait complete. 2023-06-23T18:30:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 1 -- start start 2023-06-23T18:30:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 1 Processor -- start 2023-06-23T18:30:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca8289da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca828a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8291320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca8168b40 con 0x555ca8281890 2023-06-23T18:30:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca8168840 con 0x555ca8280e60 2023-06-23T18:30:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca804c9a0 con 0x555ca8280510 2023-06-23T18:30:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc23a8cb80 10 client.0 _ll_register_callbacks cb 0x555ca816e2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc156a3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca8289da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc15ea4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8291320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc156a3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca8289da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55762/0 (socket says 172.21.15.2:55762) 2023-06-23T18:30:55.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.591+0000 7fbc156a3700 1 -- 172.21.15.2:0/2195219780 learned_addr learned my addr 172.21.15.2:0/2195219780 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca828a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc156a3700 1 -- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 msgr2=0x555ca828a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc156a3700 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca828a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc156a3700 1 -- 172.21.15.2:0/2195219780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 msgr2=0x555ca8291320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc156a3700 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8291320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc156a3700 1 -- 172.21.15.2:0/2195219780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc10108900 con 0x555ca8280510 2023-06-23T18:30:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca828a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:55.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc156a3700 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca8289da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c000c00 tx=0x7fbc0c099cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cdf18db423643b0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbbf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:30:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/2195219780 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c09c040 con 0x555ca8280510 2023-06-23T18:30:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/2195219780 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc0c0a2120 con 0x555ca8280510 2023-06-23T18:30:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2195219780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ca826ac30 con 0x555ca8280510 2023-06-23T18:30:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/2195219780 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc0c0b22b0 con 0x555ca8280510 2023-06-23T18:30:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc23a8cb80 10 client.25708 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2195219780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fbc0000e890 con 0x555ca8280510 2023-06-23T18:30:55.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.592+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/2195219780 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555ca826ac30 con 0x555ca8280510 2023-06-23T18:30:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.593+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/2195219780 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbc0c099250 con 0x555ca8280510 2023-06-23T18:30:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.593+0000 7fbc23a8cb80 10 client.25708 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.593+0000 7fbc23a8cb80 10 client.25708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.593+0000 7fbc23a8cb80 20 client.25708 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.593+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555ca8295a00 0x555ca8297e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.593+0000 7fbc23a8cb80 4 client.25708 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:30:55.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.593+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2195219780 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ca81c1f30 con 0x555ca8295a00 2023-06-23T18:30:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.593+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555ca8295a00 0x555ca8297e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.594+0000 7fbbf8ff9700 10 client.25708 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.594+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555ca8295a00 0x555ca8297e10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=605438bc71624565 server_cookie=fd7cbff79c4f3bd7 in_seq=0 out_seq=0 2023-06-23T18:30:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.594+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/2195219780 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555ca81c1f30 con 0x555ca8295a00 2023-06-23T18:30:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.594+0000 7fbbf8ff9700 10 client.25708 handle_command_reply: tid=0 2023-06-23T18:30:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.594+0000 7fbc23a8cb80 1 client.25708 shutdown 2023-06-23T18:30:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.594+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2195219780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 msgr2=0x555ca8289da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.594+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca8289da0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c000c00 tx=0x7fbc0c099cd0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.594+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2195219780 shutdown_connections 2023-06-23T18:30:55.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555ca8295a00 0x555ca8297e10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8281890 0x555ca8291320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbbf8ff9700 0 client.25708 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:55.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca8289da0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/2195219780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca828a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2195219780 >> 172.21.15.2:0/2195219780 conn(0x555ca826d470 msgr2=0x555ca8271cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2195219780 shutdown_connections 2023-06-23T18:30:55.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/2195219780 wait complete. 2023-06-23T18:30:55.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.595+0000 7fbc23a8cb80 20 client.25708 trim_cache size 0 max 16384 2023-06-23T18:30:55.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.613+0000 7fbc23a8cb80 1 Processor -- start 2023-06-23T18:30:55.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.613+0000 7fbc23a8cb80 1 -- start start 2023-06-23T18:30:55.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280510 0x555ca828a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca82912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8281890 0x555ca8271440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca81c1f60 con 0x555ca8280510 2023-06-23T18:30:55.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca828afc0 con 0x555ca8280e60 2023-06-23T18:30:55.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca8289250 con 0x555ca8281890 2023-06-23T18:30:55.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280510 0x555ca828a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280510 0x555ca828a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55432/0 (socket says 172.21.15.2:55432) 2023-06-23T18:30:55.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 -- 172.21.15.2:0/3387302090 learned_addr learned my addr 172.21.15.2:0/3387302090 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc15ea4700 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8281890 0x555ca8271440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca82912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 -- 172.21.15.2:0/3387302090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8281890 msgr2=0x555ca8271440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8281890 0x555ca8271440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 -- 172.21.15.2:0/3387302090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 msgr2=0x555ca82912e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca82912e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 -- 172.21.15.2:0/3387302090 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ca7ff6e60 con 0x555ca8280510 2023-06-23T18:30:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca82912e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:30:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc15ea4700 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8281890 0x555ca8271440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbc156a3700 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280510 0x555ca828a260 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0966f0 tx=0x7fbc0c09de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f64131e48cc0eb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.614+0000 7fbbfbfff700 1 -- 172.21.15.2:0/3387302090 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c09c030 con 0x555ca8280510 2023-06-23T18:30:55.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbbfbfff700 1 -- 172.21.15.2:0/3387302090 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc0c099aa0 con 0x555ca8280510 2023-06-23T18:30:55.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbbfbfff700 1 -- 172.21.15.2:0/3387302090 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c099f30 con 0x555ca8280510 2023-06-23T18:30:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/3387302090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280510 msgr2=0x555ca828a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280510 0x555ca828a260 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0966f0 tx=0x7fbc0c09de80 comp rx=0 tx=0).stop 2023-06-23T18:30:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/3387302090 shutdown_connections 2023-06-23T18:30:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8281890 0x555ca8271440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280510 0x555ca828a260 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/3387302090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8280e60 0x555ca82912e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/3387302090 >> 172.21.15.2:0/3387302090 conn(0x555ca826d470 msgr2=0x555ca8289b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/3387302090 shutdown_connections 2023-06-23T18:30:55.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.615+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/3387302090 wait complete. 2023-06-23T18:30:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 1 -- start start 2023-06-23T18:30:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 1 Processor -- start 2023-06-23T18:30:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca828e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280e60 0x555ca828ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8281890 0x555ca828fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca82901d0 con 0x555ca8280e60 2023-06-23T18:30:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca8290340 con 0x555ca8281890 2023-06-23T18:30:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ca8290500 con 0x555ca8280510 2023-06-23T18:30:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc23a8cb80 10 client.0 _ll_register_callbacks cb 0x555ca816e2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc14ea2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280e60 0x555ca828ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.616+0000 7fbc14ea2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280e60 0x555ca828ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55440/0 (socket says 172.21.15.2:55440) 2023-06-23T18:30:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc14ea2700 1 -- 172.21.15.2:0/1663657503 learned_addr learned my addr 172.21.15.2:0/1663657503 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc156a3700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca828e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc15ea4700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8281890 0x555ca828fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc14ea2700 1 -- 172.21.15.2:0/1663657503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 msgr2=0x555ca828e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca828e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc14ea2700 1 -- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8281890 msgr2=0x555ca828fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8281890 0x555ca828fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc14ea2700 1 -- 172.21.15.2:0/1663657503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0000e890 con 0x555ca8280e60 2023-06-23T18:30:55.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc156a3700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca828e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:55.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc15ea4700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8281890 0x555ca828fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:55.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc14ea2700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280e60 0x555ca828ec80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x555ca8281270 tx=0x7fbc0000e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ad22d4c4dfcf08 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbbf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:30:55.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/1663657503 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0002d040 con 0x555ca8280e60 2023-06-23T18:30:55.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/1663657503 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbc000325b0 con 0x555ca8280e60 2023-06-23T18:30:55.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/1663657503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc0000ee90 con 0x555ca8280e60 2023-06-23T18:30:55.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/1663657503 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc00032b50 con 0x555ca8280e60 2023-06-23T18:30:55.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc23a8cb80 10 client.15969 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.617+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/1663657503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x555ca7fdfce0 con 0x555ca8280e60 2023-06-23T18:30:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/1663657503 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbc0000ee90 con 0x555ca8280e60 2023-06-23T18:30:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/1663657503 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbc000047d0 con 0x555ca8280e60 2023-06-23T18:30:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbc23a8cb80 10 client.15969 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbc23a8cb80 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbc23a8cb80 20 client.15969 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555ca82ebca0 0x555ca82ee0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbc23a8cb80 4 client.15969 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:30:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/1663657503 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555ca81840d0 con 0x555ca82ebca0 2023-06-23T18:30:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbc156a3700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555ca82ebca0 0x555ca82ee0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbbf8ff9700 10 client.15969 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.618+0000 7fbc156a3700 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555ca82ebca0 0x555ca82ee0b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85951bd9ea5e37e server_cookie=bd7cde832bf86c12 in_seq=0 out_seq=0 2023-06-23T18:30:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbbf8ff9700 1 -- 172.21.15.2:0/1663657503 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555ca81840d0 con 0x555ca82ebca0 2023-06-23T18:30:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbbf8ff9700 10 client.15969 handle_command_reply: tid=0 2023-06-23T18:30:55.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 client.15969 shutdown 2023-06-23T18:30:55.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/1663657503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280e60 msgr2=0x555ca828ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280e60 0x555ca828ec80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x555ca8281270 tx=0x7fbc0000e6f0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/1663657503 shutdown_connections 2023-06-23T18:30:55.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555ca82ebca0 0x555ca82ee0b0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555ca8281890 0x555ca828fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555ca8280510 0x555ca828e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbbf8ff9700 0 client.15969 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:55.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 --2- 172.21.15.2:0/1663657503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555ca8280e60 0x555ca828ec80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/1663657503 >> 172.21.15.2:0/1663657503 conn(0x555ca826d470 msgr2=0x555ca8271440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/1663657503 shutdown_connections 2023-06-23T18:30:55.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 1 -- 172.21.15.2:0/1663657503 wait complete. 2023-06-23T18:30:55.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.620+0000 7fbc23a8cb80 20 client.15969 trim_cache size 0 max 16384 2023-06-23T18:30:55.634 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:30:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:30:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "2037856c-8a03-4e0b-8268-e72804addc6b", 2023-06-23T18:30:55.635 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:30:55.635 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:30:55.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 -- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbbfc0a1250 msgr2=0x7fbbfc0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbbfc0a1250 0x7fbbfc0a3700 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbc00005fd0 tx=0x7fbc00005db0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 -- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc100ffb90 msgr2=0x7fbc10074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc100ffb90 0x7fbc10074280 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00c990 tx=0x7fbc0c00cd50 comp rx=0 tx=0).stop 2023-06-23T18:30:55.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 -- 172.21.15.2:0/445861432 shutdown_connections 2023-06-23T18:30:55.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbbfc0a1250 0x7fbbfc0a3700 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc10101710 0x7fbc10072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbc100ffb90 0x7fbc10074280 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 --2- 172.21.15.2:0/445861432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc10100d80 0x7fbc100728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 -- 172.21.15.2:0/445861432 >> 172.21.15.2:0/445861432 conn(0x7fbc100fb1b0 msgr2=0x7fbc100fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 -- 172.21.15.2:0/445861432 shutdown_connections 2023-06-23T18:30:55.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.621+0000 7fbc17907700 1 -- 172.21.15.2:0/445861432 wait complete. 2023-06-23T18:30:55.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:30:55.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.793+0000 7fe3e12b6700 1 -- 172.21.15.2:0/237837826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0feaa0 msgr2=0x7fe3dc0feeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.793+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/237837826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0feaa0 0x7fe3dc0feeb0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc009990 tx=0x7fe3cc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.793+0000 7fe3e12b6700 1 -- 172.21.15.2:0/237837826 shutdown_connections 2023-06-23T18:30:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.793+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/237837826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3dc100620 0x7fe3dc106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.793+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/237837826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0feaa0 0x7fe3dc0feeb0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.793+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/237837826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0ffc90 0x7fe3dc1000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.793+0000 7fe3e12b6700 1 -- 172.21.15.2:0/237837826 >> 172.21.15.2:0/237837826 conn(0x7fe3dc0fa0c0 msgr2=0x7fe3dc0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.794+0000 7fe3e12b6700 1 -- 172.21.15.2:0/237837826 shutdown_connections 2023-06-23T18:30:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.794+0000 7fe3e12b6700 1 -- 172.21.15.2:0/237837826 wait complete. 2023-06-23T18:30:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.794+0000 7fe3e12b6700 1 Processor -- start 2023-06-23T18:30:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3e12b6700 1 -- start start 2023-06-23T18:30:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3e12b6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0feaa0 0x7fe3dc069090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3e12b6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0ffc90 0x7fe3dc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3e12b6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3dc100620 0x7fe3dc19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3e12b6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19d300 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3e12b6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19d470 con 0x7fe3dc100620 2023-06-23T18:30:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3e12b6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc19d630 con 0x7fe3dc0feaa0 2023-06-23T18:30:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0ffc90 0x7fe3dc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0ffc90 0x7fe3dc19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55456/0 (socket says 172.21.15.2:55456) 2023-06-23T18:30:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 -- 172.21.15.2:0/835995905 learned_addr learned my addr 172.21.15.2:0/835995905 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3daffd700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0feaa0 0x7fe3dc069090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3dc100620 0x7fe3dc19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 -- 172.21.15.2:0/835995905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0feaa0 msgr2=0x7fe3dc069090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0feaa0 0x7fe3dc069090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 -- 172.21.15.2:0/835995905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3dc100620 msgr2=0x7fe3dc19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3dc100620 0x7fe3dc19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 -- 172.21.15.2:0/835995905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0095f0 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3dc100620 0x7fe3dc19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3daffd700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0feaa0 0x7fe3dc069090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.795+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0ffc90 0x7fe3dc19a4e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000e9a0 tx=0x7fe3d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.796+0000 7fe3c3fff700 1 -- 172.21.15.2:0/835995905 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0009790 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.796+0000 7fe3c3fff700 1 -- 172.21.15.2:0/835995905 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d0010430 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.796+0000 7fe3c3fff700 1 -- 172.21.15.2:0/835995905 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000cea0 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.796+0000 7fe3e12b6700 1 -- 172.21.15.2:0/835995905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc19d910 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.796+0000 7fe3e12b6700 1 -- 172.21.15.2:0/835995905 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc19ddd0 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.799+0000 7fe3c3fff700 1 -- 172.21.15.2:0/835995905 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe3d00105a0 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.799+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3c40a1150 0x7fe3c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.800+0000 7fe3daffd700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3c40a1150 0x7fe3c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.800+0000 7fe3c3fff700 1 -- 172.21.15.2:0/835995905 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3d0014070 con 0x7fe3dc0ffc90 2023-06-23T18:30:55.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.800+0000 7fe3daffd700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3c40a1150 0x7fe3c40a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006010 tx=0x7fe3cc00b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.800+0000 7fe3ed43bb80 1 Processor -- start 2023-06-23T18:30:55.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.800+0000 7fe3ed43bb80 1 -- start start 2023-06-23T18:30:55.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d84090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d84a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d85000 0x55e315d85470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315c88cb0 con 0x55e315d85000 2023-06-23T18:30:55.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315cdba40 con 0x55e315d845d0 2023-06-23T18:30:55.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315cd4ed0 con 0x55e315d83c80 2023-06-23T18:30:55.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d85000 0x55e315d85470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d85000 0x55e315d85470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55464/0 (socket says 172.21.15.2:55464) 2023-06-23T18:30:55.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 -- 172.21.15.2:0/3632134872 learned_addr learned my addr 172.21.15.2:0/3632134872 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d84a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3daffd700 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d84090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 -- 172.21.15.2:0/3632134872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 msgr2=0x55e315d84090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d84090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 -- 172.21.15.2:0/3632134872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 msgr2=0x55e315d84a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d84a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 -- 172.21.15.2:0/3632134872 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e315b56630 con 0x55e315d85000 2023-06-23T18:30:55.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d84a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:55.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.801+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d85000 0x55e315d85470 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4003eb0 tx=0x7fe3d4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53612310cf31c3b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3c17fa700 1 -- 172.21.15.2:0/3632134872 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40092d0 con 0x55e315d85000 2023-06-23T18:30:55.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3c17fa700 1 -- 172.21.15.2:0/3632134872 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d400f040 con 0x55e315d85000 2023-06-23T18:30:55.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3c17fa700 1 -- 172.21.15.2:0/3632134872 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400e750 con 0x55e315d85000 2023-06-23T18:30:55.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/3632134872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d85000 msgr2=0x55e315d85470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d85000 0x55e315d85470 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4003eb0 tx=0x7fe3d4009920 comp rx=0 tx=0).stop 2023-06-23T18:30:55.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/3632134872 shutdown_connections 2023-06-23T18:30:55.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d85000 0x55e315d85470 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d84090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/3632134872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d84a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.802+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/3632134872 >> 172.21.15.2:0/3632134872 conn(0x55e315d793f0 msgr2=0x55e315d7e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/3632134872 shutdown_connections 2023-06-23T18:30:55.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/3632134872 wait complete. 2023-06-23T18:30:55.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 -- start start 2023-06-23T18:30:55.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 Processor -- start 2023-06-23T18:30:55.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d83c80 0x55e315d8d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d8da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d94a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315c74ac0 con 0x55e315d83c80 2023-06-23T18:30:55.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315c747c0 con 0x55e315d845d0 2023-06-23T18:30:55.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315b58920 con 0x55e315d85000 2023-06-23T18:30:55.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.803+0000 7fe3ed43bb80 10 client.0 _ll_register_callbacks cb 0x55e315c7a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:55.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d83c80 0x55e315d8d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d83c80 0x55e315d8d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55466/0 (socket says 172.21.15.2:55466) 2023-06-23T18:30:55.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3daffd700 1 -- 172.21.15.2:0/4035757300 learned_addr learned my addr 172.21.15.2:0/4035757300 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d94a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d8da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3db7fe700 1 -- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 msgr2=0x55e315d8da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d8da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3db7fe700 1 -- 172.21.15.2:0/4035757300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d83c80 msgr2=0x55e315d8d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d83c80 0x55e315d8d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3db7fe700 1 -- 172.21.15.2:0/4035757300 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc19ddd0 con 0x55e315d85000 2023-06-23T18:30:55.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d8da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:30:55.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3daffd700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d83c80 0x55e315d8d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d94a40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40049e0 tx=0x7fe3d4007b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1043ac0d54d8826 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3beff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:30:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.804+0000 7fe3beff5700 1 -- 172.21.15.2:0/4035757300 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4007910 con 0x55e315d85000 2023-06-23T18:30:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/4035757300 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e315d76bb0 con 0x55e315d85000 2023-06-23T18:30:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3beff5700 1 -- 172.21.15.2:0/4035757300 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3d400e4f0 con 0x55e315d85000 2023-06-23T18:30:55.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3beff5700 1 -- 172.21.15.2:0/4035757300 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3d400e6b0 con 0x55e315d85000 2023-06-23T18:30:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 10 client.25726 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/4035757300 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fe3d4003b90 con 0x55e315d85000 2023-06-23T18:30:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3beff5700 1 -- 172.21.15.2:0/4035757300 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e315d76bb0 con 0x55e315d85000 2023-06-23T18:30:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3beff5700 1 -- 172.21.15.2:0/4035757300 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3d40157e0 con 0x55e315d85000 2023-06-23T18:30:55.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 10 client.25726 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:55.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 10 client.25726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:55.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 20 client.25726 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:55.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e315d99160 0x55e315d9b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 4 client.25726 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:30:55.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.805+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/4035757300 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e315ccdeb0 con 0x55e315d99160 2023-06-23T18:30:55.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.806+0000 7fe3daffd700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e315d99160 0x55e315d9b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.806+0000 7fe3beff5700 10 client.25726 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:55.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.806+0000 7fe3daffd700 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e315d99160 0x55e315d9b570 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d18e5c1c6c51ffd server_cookie=3665ba27c7ab02a9 in_seq=0 out_seq=0 2023-06-23T18:30:55.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3beff5700 1 -- 172.21.15.2:0/4035757300 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e315ccdeb0 con 0x55e315d99160 2023-06-23T18:30:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3beff5700 10 client.25726 handle_command_reply: tid=0 2023-06-23T18:30:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 client.25726 shutdown 2023-06-23T18:30:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/4035757300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 msgr2=0x55e315d94a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d94a40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40049e0 tx=0x7fe3d4007b10 comp rx=0 tx=0).stop 2023-06-23T18:30:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/4035757300 shutdown_connections 2023-06-23T18:30:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e315d99160 0x55e315d9b570 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d94a40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d83c80 0x55e315d8d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3beff5700 0 client.25726 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:55.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/4035757300 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d845d0 0x55e315d8da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/4035757300 >> 172.21.15.2:0/4035757300 conn(0x55e315d793f0 msgr2=0x55e315d7dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/4035757300 shutdown_connections 2023-06-23T18:30:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/4035757300 wait complete. 2023-06-23T18:30:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.807+0000 7fe3ed43bb80 20 client.25726 trim_cache size 0 max 16384 2023-06-23T18:30:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.826+0000 7fe3ed43bb80 1 Processor -- start 2023-06-23T18:30:55.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.826+0000 7fe3ed43bb80 1 -- start start 2023-06-23T18:30:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.826+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d83c80 0x55e315d87de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.826+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d94a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.826+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d88320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.826+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315d7f630 con 0x55e315d845d0 2023-06-23T18:30:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.826+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315d8c970 con 0x55e315d83c80 2023-06-23T18:30:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.826+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315d8b080 con 0x55e315d85000 2023-06-23T18:30:55.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d94a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3da7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d94a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55476/0 (socket says 172.21.15.2:55476) 2023-06-23T18:30:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3da7fc700 1 -- 172.21.15.2:0/1981300183 learned_addr learned my addr 172.21.15.2:0/1981300183 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d88320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3daffd700 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d83c80 0x55e315d87de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3db7fe700 1 -- 172.21.15.2:0/1981300183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d83c80 msgr2=0x55e315d87de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d83c80 0x55e315d87de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3db7fe700 1 -- 172.21.15.2:0/1981300183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 msgr2=0x55e315d94a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d94a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3db7fe700 1 -- 172.21.15.2:0/1981300183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e315b02de0 con 0x55e315d85000 2023-06-23T18:30:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d94a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:30:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d88320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009470 tx=0x7fe3d4009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c7c910d5a69470 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3c1ffb700 1 -- 172.21.15.2:0/1981300183 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4007910 con 0x55e315d85000 2023-06-23T18:30:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3c1ffb700 1 -- 172.21.15.2:0/1981300183 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d402e030 con 0x55e315d85000 2023-06-23T18:30:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.827+0000 7fe3c1ffb700 1 -- 172.21.15.2:0/1981300183 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400e920 con 0x55e315d85000 2023-06-23T18:30:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1981300183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 msgr2=0x55e315d88320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d88320 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009470 tx=0x7fe3d4009e70 comp rx=0 tx=0).stop 2023-06-23T18:30:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1981300183 shutdown_connections 2023-06-23T18:30:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d85000 0x55e315d88320 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d83c80 0x55e315d87de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1981300183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d94a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1981300183 >> 172.21.15.2:0/1981300183 conn(0x55e315d793f0 msgr2=0x55e315d9fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1981300183 shutdown_connections 2023-06-23T18:30:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1981300183 wait complete. 2023-06-23T18:30:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 -- start start 2023-06-23T18:30:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.828+0000 7fe3ed43bb80 1 Processor -- start 2023-06-23T18:30:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d90940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d90ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3ed43bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d85000 0x55e315d93bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315d94100 con 0x55e315d845d0 2023-06-23T18:30:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315d94270 con 0x55e315d85000 2023-06-23T18:30:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3ed43bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e315d94430 con 0x55e315d83c80 2023-06-23T18:30:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3ed43bb80 10 client.0 _ll_register_callbacks cb 0x55e315c7a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:30:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d90940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d90940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55818/0 (socket says 172.21.15.2:55818) 2023-06-23T18:30:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 -- 172.21.15.2:0/1876852030 learned_addr learned my addr 172.21.15.2:0/1876852030 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:30:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d90ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d85000 0x55e315d93bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 -- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d85000 msgr2=0x55e315d93bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d85000 0x55e315d93bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 -- 172.21.15.2:0/1876852030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 msgr2=0x55e315d90ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:30:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d90ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 -- 172.21.15.2:0/1876852030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc19d910 con 0x55e315d83c80 2023-06-23T18:30:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3db7fe700 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d85000 0x55e315d93bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:30:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3daffd700 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d90940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc005910 tx=0x7fe3cc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e189fd6f4e96c891 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:30:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.829+0000 7fe3beff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:30:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3beff5700 1 -- 172.21.15.2:0/1876852030 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc02b040 con 0x55e315d83c80 2023-06-23T18:30:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3beff5700 1 -- 172.21.15.2:0/1876852030 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3cc0360b0 con 0x55e315d83c80 2023-06-23T18:30:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1876852030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3d400f040 con 0x55e315d83c80 2023-06-23T18:30:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3beff5700 1 -- 172.21.15.2:0/1876852030 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3cc00beb0 con 0x55e315d83c80 2023-06-23T18:30:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 10 client.25732 fetch_fsmap learned FSMap version 32 2023-06-23T18:30:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1876852030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fe3d4003b90 con 0x55e315d83c80 2023-06-23T18:30:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3beff5700 1 -- 172.21.15.2:0/1876852030 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3d400f040 con 0x55e315d83c80 2023-06-23T18:30:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3beff5700 1 -- 172.21.15.2:0/1876852030 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe3cc03c020 con 0x55e315d83c80 2023-06-23T18:30:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 10 client.25732 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:30:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 10 client.25732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:30:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 20 client.25732 populate_metadata read hostname 'smithi002' 2023-06-23T18:30:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e315df6a50 0x55e315df8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:30:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 4 client.25732 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:30:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1876852030 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e315c90050 con 0x55e315df6a50 2023-06-23T18:30:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.830+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e315df6a50 0x55e315df8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:30:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3beff5700 10 client.25732 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3da7fc700 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e315df6a50 0x55e315df8e60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225147cb833c89d3 server_cookie=2e73f1fec259e203 in_seq=0 out_seq=0 2023-06-23T18:30:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3beff5700 1 -- 172.21.15.2:0/1876852030 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e315c90050 con 0x55e315df6a50 2023-06-23T18:30:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3beff5700 10 client.25732 handle_command_reply: tid=0 2023-06-23T18:30:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 client.25732 shutdown 2023-06-23T18:30:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1876852030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 msgr2=0x55e315d90940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d90940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc005910 tx=0x7fe3cc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1876852030 shutdown_connections 2023-06-23T18:30:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e315df6a50 0x55e315df8e60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e315d85000 0x55e315d93bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3beff5700 0 client.25732 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:30:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e315d83c80 0x55e315d90940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 --2- 172.21.15.2:0/1876852030 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e315d845d0 0x55e315d90ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1876852030 >> 172.21.15.2:0/1876852030 conn(0x55e315d793f0 msgr2=0x55e315d8b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1876852030 shutdown_connections 2023-06-23T18:30:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 1 -- 172.21.15.2:0/1876852030 wait complete. 2023-06-23T18:30:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.831+0000 7fe3ed43bb80 20 client.25732 trim_cache size 0 max 16384 2023-06-23T18:30:55.847 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:30:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:30:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:30:55.848 INFO:teuthology.orchestra.run.smithi002.stdout: "2037856c-8a03-4e0b-8268-e72804addc6b": { 2023-06-23T18:30:55.848 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:30:55.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "2037856c-8a03-4e0b-8268-e72804addc6b", 2023-06-23T18:30:55.848 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:30:55.848 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:30:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:30:55.849 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:30:55.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 -- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3c40a1150 msgr2=0x7fe3c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3c40a1150 0x7fe3c40a3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006010 tx=0x7fe3cc00b2a0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 -- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0ffc90 msgr2=0x7fe3dc19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:30:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0ffc90 0x7fe3dc19a4e0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000e9a0 tx=0x7fe3d000ecb0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 -- 172.21.15.2:0/835995905 shutdown_connections 2023-06-23T18:30:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3c40a1150 0x7fe3c40a3600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3dc100620 0x7fe3dc19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0feaa0 0x7fe3dc069090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 --2- 172.21.15.2:0/835995905 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3dc0ffc90 0x7fe3dc19a4e0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:30:55.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 -- 172.21.15.2:0/835995905 >> 172.21.15.2:0/835995905 conn(0x7fe3dc0fa0c0 msgr2=0x7fe3dc0fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:30:55.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 -- 172.21.15.2:0/835995905 shutdown_connections 2023-06-23T18:30:55.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:30:55.832+0000 7fe3e12b6700 1 -- 172.21.15.2:0/835995905 wait complete. 2023-06-23T18:30:55.853 INFO:tasks.cephfs.filesystem:scrub status for tag:2037856c-8a03-4e0b-8268-e72804addc6b - {'path': '/', 'tag': '2037856c-8a03-4e0b-8268-e72804addc6b', 'options': 'recursive,force'} 2023-06-23T18:30:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:56 smithi017 ceph-mon[97472]: pgmap v563: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 26 op/s 2023-06-23T18:30:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:56 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:30:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:56 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:56.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:56 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:30:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:56 smithi002 ceph-mon[95947]: pgmap v563: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 26 op/s 2023-06-23T18:30:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:56 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:30:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:56 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:56 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:30:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:56 smithi082 ceph-mon[97889]: pgmap v563: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 MiB/s wr, 26 op/s 2023-06-23T18:30:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:56 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:30:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:56 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:56.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:56 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:30:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:57 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:30:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:57 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:30:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:57 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:57.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:57 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:57.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:57 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:57 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:30:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:57 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:30:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:57 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:57 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:57.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:57 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:57.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:57 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:30:57.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:57 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:30:57.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:57 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:57.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:57 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:57.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:57 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:30:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:30:58 smithi017 ceph-mon[97472]: pgmap v564: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:30:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:30:58 smithi002 ceph-mon[95947]: pgmap v564: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:30:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:30:58 smithi082 ceph-mon[97889]: pgmap v564: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:31:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:00 smithi017 ceph-mon[97472]: pgmap v565: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:31:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:00 smithi002 ceph-mon[95947]: pgmap v565: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:31:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:00 smithi082 ceph-mon[97889]: pgmap v565: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:31:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:02 smithi017 ceph-mon[97472]: pgmap v566: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-06-23T18:31:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:02 smithi002 ceph-mon[95947]: pgmap v566: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-06-23T18:31:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:02 smithi082 ceph-mon[97889]: pgmap v566: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 42 MiB/s wr, 23 op/s 2023-06-23T18:31:03.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:03 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:31:03.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:03 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:31:03.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:03 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:31:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:04 smithi017 ceph-mon[97472]: pgmap v567: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:31:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:04 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:31:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:04 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:31:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:04 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:31:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:04 smithi002 ceph-mon[95947]: pgmap v567: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:31:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:04 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:31:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:04 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:31:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:04 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:31:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:04 smithi082 ceph-mon[97889]: pgmap v567: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:31:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:04 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:31:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:04 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:31:04.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:04 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:31:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:06 smithi017 ceph-mon[97472]: pgmap v568: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:31:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:06 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:31:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:06 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:31:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:06 smithi002 ceph-mon[95947]: pgmap v568: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:31:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:06 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:31:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:06 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:31:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:06 smithi082 ceph-mon[97889]: pgmap v568: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 45 MiB/s wr, 24 op/s 2023-06-23T18:31:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:06 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:31:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:06 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:31:07.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:07 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:31:07.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:07 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:31:07.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:07 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:31:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:08 smithi017 ceph-mon[97472]: pgmap v569: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:31:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:08 smithi002 ceph-mon[95947]: pgmap v569: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:31:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:08 smithi082 ceph-mon[97889]: pgmap v569: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 43 MiB/s wr, 24 op/s 2023-06-23T18:31:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:10 smithi017 ceph-mon[97472]: pgmap v570: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:31:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:10 smithi002 ceph-mon[95947]: pgmap v570: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:31:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:10 smithi082 ceph-mon[97889]: pgmap v570: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:31:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:12 smithi017 ceph-mon[97472]: pgmap v571: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:31:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:12 smithi002 ceph-mon[95947]: pgmap v571: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:31:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:12 smithi082 ceph-mon[97889]: pgmap v571: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 43 MiB/s wr, 23 op/s 2023-06-23T18:31:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:14 smithi017 ceph-mon[97472]: pgmap v572: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 55 MiB/s wr, 30 op/s 2023-06-23T18:31:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:14 smithi002 ceph-mon[95947]: pgmap v572: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 55 MiB/s wr, 30 op/s 2023-06-23T18:31:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:14 smithi082 ceph-mon[97889]: pgmap v572: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 55 MiB/s wr, 30 op/s 2023-06-23T18:31:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:16 smithi017 ceph-mon[97472]: pgmap v573: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:31:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:16 smithi002 ceph-mon[95947]: pgmap v573: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:31:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:16 smithi082 ceph-mon[97889]: pgmap v573: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:31:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:18 smithi002 ceph-mon[95947]: pgmap v574: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:31:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:18 smithi082 ceph-mon[97889]: pgmap v574: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:31:18.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:18 smithi017 ceph-mon[97472]: pgmap v574: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:31:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:20 smithi002 ceph-mon[95947]: pgmap v575: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 53 MiB/s wr, 28 op/s 2023-06-23T18:31:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:20 smithi082 ceph-mon[97889]: pgmap v575: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 53 MiB/s wr, 28 op/s 2023-06-23T18:31:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:20 smithi017 ceph-mon[97472]: pgmap v575: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 53 MiB/s wr, 28 op/s 2023-06-23T18:31:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:22 smithi002 ceph-mon[95947]: pgmap v576: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 48 MiB/s wr, 24 op/s 2023-06-23T18:31:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:22 smithi082 ceph-mon[97889]: pgmap v576: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 48 MiB/s wr, 24 op/s 2023-06-23T18:31:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:22 smithi017 ceph-mon[97472]: pgmap v576: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 48 MiB/s wr, 24 op/s 2023-06-23T18:31:24.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:24 smithi002 ceph-mon[95947]: pgmap v577: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 56 MiB/s wr, 29 op/s 2023-06-23T18:31:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:24 smithi082 ceph-mon[97889]: pgmap v577: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 56 MiB/s wr, 29 op/s 2023-06-23T18:31:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:24 smithi017 ceph-mon[97472]: pgmap v577: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 56 MiB/s wr, 29 op/s 2023-06-23T18:31:25.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:31:26.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 -- 172.21.15.2:0/1631443721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40840fea90 msgr2=0x7f40840feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 --2- 172.21.15.2:0/1631443721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40840fea90 0x7f40840feea0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4074009990 tx=0x7f4074009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 -- 172.21.15.2:0/1631443721 shutdown_connections 2023-06-23T18:31:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 --2- 172.21.15.2:0/1631443721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4084100610 0x7f4084106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 --2- 172.21.15.2:0/1631443721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40840fea90 0x7f40840feea0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 --2- 172.21.15.2:0/1631443721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40840ffc80 0x7f40841000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 -- 172.21.15.2:0/1631443721 >> 172.21.15.2:0/1631443721 conn(0x7f40840fa090 msgr2=0x7f40840fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 -- 172.21.15.2:0/1631443721 shutdown_connections 2023-06-23T18:31:26.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.005+0000 7f408a736700 1 -- 172.21.15.2:0/1631443721 wait complete. 2023-06-23T18:31:26.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f408a736700 1 Processor -- start 2023-06-23T18:31:26.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f408a736700 1 -- start start 2023-06-23T18:31:26.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f408a736700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40840fea90 0x7f4084069070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f408a736700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40840ffc80 0x7f408419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f408a736700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084100610 0x7f408419edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f408a736700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408419f4d0 con 0x7f4084100610 2023-06-23T18:31:26.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f408a736700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408419f640 con 0x7f40840fea90 2023-06-23T18:31:26.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f408a736700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408419f800 con 0x7f40840ffc80 2023-06-23T18:31:26.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f4088cd3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084100610 0x7f408419edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f4088cd3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084100610 0x7f408419edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55500/0 (socket says 172.21.15.2:55500) 2023-06-23T18:31:26.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f4088cd3700 1 -- 172.21.15.2:0/2014464765 learned_addr learned my addr 172.21.15.2:0/2014464765 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f4083fff700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40840fea90 0x7f4084069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.006+0000 7f40837fe700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40840ffc80 0x7f408419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f4088cd3700 1 -- 172.21.15.2:0/2014464765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40840ffc80 msgr2=0x7f408419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f4088cd3700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40840ffc80 0x7f408419c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f4088cd3700 1 -- 172.21.15.2:0/2014464765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40840fea90 msgr2=0x7f4084069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f4088cd3700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40840fea90 0x7f4084069070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f4088cd3700 1 -- 172.21.15.2:0/2014464765 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40740095f0 con 0x7f4084100610 2023-06-23T18:31:26.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f4088cd3700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084100610 0x7f408419edf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f407c007d70 tx=0x7f407c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f40817fa700 1 -- 172.21.15.2:0/2014464765 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00ac90 con 0x7f4084100610 2023-06-23T18:31:26.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f408a736700 1 -- 172.21.15.2:0/2014464765 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408419fae0 con 0x7f4084100610 2023-06-23T18:31:26.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f40817fa700 1 -- 172.21.15.2:0/2014464765 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f407c00adf0 con 0x7f4084100610 2023-06-23T18:31:26.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f40817fa700 1 -- 172.21.15.2:0/2014464765 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c0115e0 con 0x7f4084100610 2023-06-23T18:31:26.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.007+0000 7f408a736700 1 -- 172.21.15.2:0/2014464765 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408419fff0 con 0x7f4084100610 2023-06-23T18:31:26.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.010+0000 7f40817fa700 1 -- 172.21.15.2:0/2014464765 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f407c012420 con 0x7f4084100610 2023-06-23T18:31:26.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.010+0000 7f40817fa700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f406c0a1150 0x7f406c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.010+0000 7f40817fa700 1 -- 172.21.15.2:0/2014464765 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f407c0cca30 con 0x7f4084100610 2023-06-23T18:31:26.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.010+0000 7f4083fff700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f406c0a1150 0x7f406c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40968bbb80 1 Processor -- start 2023-06-23T18:31:26.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40968bbb80 1 -- start start 2023-06-23T18:31:26.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b80864090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80864a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80865000 0x556b80865470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40968bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b80768cb0 con 0x556b808645d0 2023-06-23T18:31:26.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40968bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b807bba40 con 0x556b80865000 2023-06-23T18:31:26.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40968bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b807b4ed0 con 0x556b80863c80 2023-06-23T18:31:26.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80864a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80864a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55506/0 (socket says 172.21.15.2:55506) 2023-06-23T18:31:26.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f40837fe700 1 -- 172.21.15.2:0/3452477359 learned_addr learned my addr 172.21.15.2:0/3452477359 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.011+0000 7f4083fff700 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b80864090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f4088cd3700 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80865000 0x556b80865470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f40837fe700 1 -- 172.21.15.2:0/3452477359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 msgr2=0x556b80864090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f40837fe700 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b80864090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f40837fe700 1 -- 172.21.15.2:0/3452477359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80865000 msgr2=0x556b80865470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f40837fe700 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80865000 0x556b80865470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f40837fe700 1 -- 172.21.15.2:0/3452477359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b80636630 con 0x556b808645d0 2023-06-23T18:31:26.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f4083fff700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f406c0a1150 0x7f406c0a3600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f407400c010 tx=0x7f4074005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f40837fe700 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80864a70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4078007d70 tx=0x7f40780106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58adb0b503a314e1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f406a7fc700 1 -- 172.21.15.2:0/3452477359 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078010c10 con 0x556b808645d0 2023-06-23T18:31:26.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f406a7fc700 1 -- 172.21.15.2:0/3452477359 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4078015070 con 0x556b808645d0 2023-06-23T18:31:26.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.012+0000 7f406a7fc700 1 -- 172.21.15.2:0/3452477359 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078003c40 con 0x556b808645d0 2023-06-23T18:31:26.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 -- 172.21.15.2:0/3452477359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 msgr2=0x556b80864a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80864a70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4078007d70 tx=0x7f40780106b0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 -- 172.21.15.2:0/3452477359 shutdown_connections 2023-06-23T18:31:26.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80865000 0x556b80865470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b80864090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3452477359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80864a70 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 -- 172.21.15.2:0/3452477359 >> 172.21.15.2:0/3452477359 conn(0x556b808593f0 msgr2=0x556b8085e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 -- 172.21.15.2:0/3452477359 shutdown_connections 2023-06-23T18:31:26.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 -- 172.21.15.2:0/3452477359 wait complete. 2023-06-23T18:31:26.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.013+0000 7f40968bbb80 1 -- start start 2023-06-23T18:31:26.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40968bbb80 1 Processor -- start 2023-06-23T18:31:26.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 0x556b8086d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b808645d0 0x556b8086daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 0x556b80874ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40968bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b80754ac0 con 0x556b80865000 2023-06-23T18:31:26.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40968bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b807547c0 con 0x556b80863c80 2023-06-23T18:31:26.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40968bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b80638920 con 0x556b808645d0 2023-06-23T18:31:26.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40968bbb80 10 client.0 _ll_register_callbacks cb 0x556b8075a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:26.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40837fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b808645d0 0x556b8086daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40837fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b808645d0 0x556b8086daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55850/0 (socket says 172.21.15.2:55850) 2023-06-23T18:31:26.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f40837fe700 1 -- 172.21.15.2:0/2043194187 learned_addr learned my addr 172.21.15.2:0/2043194187 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f4088cd3700 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 0x556b80874ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.014+0000 7f4083fff700 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 0x556b8086d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40837fe700 1 -- 172.21.15.2:0/2043194187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 msgr2=0x556b8086d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40837fe700 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 0x556b8086d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40837fe700 1 -- 172.21.15.2:0/2043194187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 msgr2=0x556b80874ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40837fe700 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 0x556b80874ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40837fe700 1 -- 172.21.15.2:0/2043194187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40780079a0 con 0x556b808645d0 2023-06-23T18:31:26.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40837fe700 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b808645d0 0x556b8086daa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4078000c00 tx=0x7f4078007e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6be2a353a918a78b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40637fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:31:26.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40637fe700 1 -- 172.21.15.2:0/2043194187 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078020070 con 0x556b808645d0 2023-06-23T18:31:26.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40968bbb80 1 -- 172.21.15.2:0/2043194187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b80856bb0 con 0x556b808645d0 2023-06-23T18:31:26.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40637fe700 1 -- 172.21.15.2:0/2043194187 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f407801a470 con 0x556b808645d0 2023-06-23T18:31:26.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.015+0000 7f40637fe700 1 -- 172.21.15.2:0/2043194187 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f407801a630 con 0x556b808645d0 2023-06-23T18:31:26.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40968bbb80 10 client.25744 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:26.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40968bbb80 1 -- 172.21.15.2:0/2043194187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556b805e2de0 con 0x556b808645d0 2023-06-23T18:31:26.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40637fe700 1 -- 172.21.15.2:0/2043194187 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556b80856bb0 con 0x556b808645d0 2023-06-23T18:31:26.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40637fe700 1 -- 172.21.15.2:0/2043194187 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4078011700 con 0x556b808645d0 2023-06-23T18:31:26.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40968bbb80 10 client.25744 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:26.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40968bbb80 10 client.25744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40968bbb80 20 client.25744 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40968bbb80 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556b80879230 0x556b8087b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40968bbb80 4 client.25744 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f40968bbb80 1 -- 172.21.15.2:0/2043194187 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b807adeb0 con 0x556b80879230 2023-06-23T18:31:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.016+0000 7f4083fff700 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556b80879230 0x556b8087b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.017+0000 7f40637fe700 10 client.25744 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.017+0000 7f4083fff700 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556b80879230 0x556b8087b640 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df1b7d3a0f0a97c1 server_cookie=379de990622b319e in_seq=0 out_seq=0 2023-06-23T18:31:26.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.017+0000 7f40637fe700 1 -- 172.21.15.2:0/2043194187 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556b807adeb0 con 0x556b80879230 2023-06-23T18:31:26.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.017+0000 7f40637fe700 10 client.25744 handle_command_reply: tid=0 2023-06-23T18:31:26.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.017+0000 7f40968bbb80 1 client.25744 shutdown 2023-06-23T18:31:26.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 -- 172.21.15.2:0/2043194187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b808645d0 msgr2=0x556b8086daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b808645d0 0x556b8086daa0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4078000c00 tx=0x7f4078007e30 comp rx=0 tx=0).stop 2023-06-23T18:31:26.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 -- 172.21.15.2:0/2043194187 shutdown_connections 2023-06-23T18:31:26.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556b80879230 0x556b8087b640 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 0x556b80874ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 0x556b8086d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40637fe700 0 client.25744 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:26.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 --2- 172.21.15.2:0/2043194187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b808645d0 0x556b8086daa0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 -- 172.21.15.2:0/2043194187 >> 172.21.15.2:0/2043194187 conn(0x556b808593f0 msgr2=0x556b8085dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 -- 172.21.15.2:0/2043194187 shutdown_connections 2023-06-23T18:31:26.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 1 -- 172.21.15.2:0/2043194187 wait complete. 2023-06-23T18:31:26.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.018+0000 7f40968bbb80 20 client.25744 trim_cache size 0 max 16384 2023-06-23T18:31:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.038+0000 7f40968bbb80 1 Processor -- start 2023-06-23T18:31:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f40968bbb80 1 -- start start 2023-06-23T18:31:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b808753d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b808645d0 0x556b80874a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 0x556b8086d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f40968bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b8086d0d0 con 0x556b80865000 2023-06-23T18:31:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f40968bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b80860010 con 0x556b808645d0 2023-06-23T18:31:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f40968bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b80874f40 con 0x556b80863c80 2023-06-23T18:31:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4083fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b808753d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4083fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b808753d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:55856/0 (socket says 172.21.15.2:55856) 2023-06-23T18:31:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4083fff700 1 -- 172.21.15.2:0/1664777469 learned_addr learned my addr 172.21.15.2:0/1664777469 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f40837fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b808645d0 0x556b80874a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4088cd3700 1 --2- 172.21.15.2:0/1664777469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 0x556b8086d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4083fff700 1 -- 172.21.15.2:0/1664777469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b808645d0 msgr2=0x556b80874a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4083fff700 1 --2- 172.21.15.2:0/1664777469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b808645d0 0x556b80874a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4083fff700 1 -- 172.21.15.2:0/1664777469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 msgr2=0x556b8086d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4083fff700 1 --2- 172.21.15.2:0/1664777469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 0x556b8086d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.039+0000 7f4083fff700 1 -- 172.21.15.2:0/1664777469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b805cbc60 con 0x556b80863c80 2023-06-23T18:31:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.040+0000 7f4083fff700 1 --2- 172.21.15.2:0/1664777469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b808753d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f40840ffae0 tx=0x7f407402b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=878e2a34eeb3675 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.040+0000 7f406affd700 1 -- 172.21.15.2:0/1664777469 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074033c90 con 0x556b80863c80 2023-06-23T18:31:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.040+0000 7f406affd700 1 -- 172.21.15.2:0/1664777469 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f407401e040 con 0x556b80863c80 2023-06-23T18:31:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.040+0000 7f406affd700 1 -- 172.21.15.2:0/1664777469 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4074004840 con 0x556b80863c80 2023-06-23T18:31:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.040+0000 7f40968bbb80 1 -- 172.21.15.2:0/1664777469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 msgr2=0x556b808753d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.040+0000 7f40968bbb80 1 --2- 172.21.15.2:0/1664777469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b808753d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f40840ffae0 tx=0x7f407402b000 comp rx=0 tx=0).stop 2023-06-23T18:31:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 -- 172.21.15.2:0/1664777469 shutdown_connections 2023-06-23T18:31:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 --2- 172.21.15.2:0/1664777469 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b80865000 0x556b8086d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 --2- 172.21.15.2:0/1664777469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80863c80 0x556b808753d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 --2- 172.21.15.2:0/1664777469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b808645d0 0x556b80874a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 -- 172.21.15.2:0/1664777469 >> 172.21.15.2:0/1664777469 conn(0x556b808593f0 msgr2=0x556b8087ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 -- 172.21.15.2:0/1664777469 shutdown_connections 2023-06-23T18:31:26.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 -- 172.21.15.2:0/1664777469 wait complete. 2023-06-23T18:31:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 -- start start 2023-06-23T18:31:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 Processor -- start 2023-06-23T18:31:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 0x556b808746e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.041+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80871180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40968bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80865000 0x556b808716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40968bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b80871c00 con 0x556b808645d0 2023-06-23T18:31:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40968bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b80871d70 con 0x556b80863c80 2023-06-23T18:31:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40968bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b808cd0a0 con 0x556b80865000 2023-06-23T18:31:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40968bbb80 10 client.0 _ll_register_callbacks cb 0x556b8075a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80871180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80871180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55526/0 (socket says 172.21.15.2:55526) 2023-06-23T18:31:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 -- 172.21.15.2:0/3248492316 learned_addr learned my addr 172.21.15.2:0/3248492316 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f4083fff700 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 0x556b808746e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f4088cd3700 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80865000 0x556b808716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 -- 172.21.15.2:0/3248492316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80865000 msgr2=0x556b808716c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80865000 0x556b808716c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 -- 172.21.15.2:0/3248492316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 msgr2=0x556b808746e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 0x556b808746e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 -- 172.21.15.2:0/3248492316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4084107800 con 0x556b808645d0 2023-06-23T18:31:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40837fe700 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80871180 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4078011b10 tx=0x7f407802ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1038c757be6bb815 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40637fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40637fe700 1 -- 172.21.15.2:0/3248492316 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078020070 con 0x556b808645d0 2023-06-23T18:31:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40637fe700 1 -- 172.21.15.2:0/3248492316 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4078000e40 con 0x556b808645d0 2023-06-23T18:31:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.042+0000 7f40968bbb80 1 -- 172.21.15.2:0/3248492316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4078015070 con 0x556b808645d0 2023-06-23T18:31:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40637fe700 1 -- 172.21.15.2:0/3248492316 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4078007cc0 con 0x556b808645d0 2023-06-23T18:31:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40968bbb80 10 client.16005 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40968bbb80 1 -- 172.21.15.2:0/3248492316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x556b80609250 con 0x556b808645d0 2023-06-23T18:31:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40637fe700 1 -- 172.21.15.2:0/3248492316 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4078015070 con 0x556b808645d0 2023-06-23T18:31:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40637fe700 1 -- 172.21.15.2:0/3248492316 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f407802e5a0 con 0x556b808645d0 2023-06-23T18:31:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40968bbb80 10 client.16005 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40968bbb80 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40968bbb80 20 client.16005 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556b80879230 0x556b808d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40968bbb80 4 client.16005 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:31:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f40968bbb80 1 -- 172.21.15.2:0/3248492316 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556b80770050 con 0x556b80879230 2023-06-23T18:31:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.043+0000 7f4083fff700 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556b80879230 0x556b808d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f4083fff700 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556b80879230 0x556b808d0840 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3203075601a25ad server_cookie=2ed17c603b33c712 in_seq=0 out_seq=0 2023-06-23T18:31:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40637fe700 10 client.16005 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40637fe700 1 -- 172.21.15.2:0/3248492316 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556b80770050 con 0x556b80879230 2023-06-23T18:31:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40637fe700 10 client.16005 handle_command_reply: tid=0 2023-06-23T18:31:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 client.16005 shutdown 2023-06-23T18:31:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 -- 172.21.15.2:0/3248492316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 msgr2=0x556b80871180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80871180 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4078011b10 tx=0x7f407802ef70 comp rx=0 tx=0).stop 2023-06-23T18:31:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 -- 172.21.15.2:0/3248492316 shutdown_connections 2023-06-23T18:31:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556b80879230 0x556b808d0840 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b80865000 0x556b808716c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556b80863c80 0x556b808746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40637fe700 0 client.16005 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 --2- 172.21.15.2:0/3248492316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556b808645d0 0x556b80871180 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.044+0000 7f40968bbb80 1 -- 172.21.15.2:0/3248492316 >> 172.21.15.2:0/3248492316 conn(0x556b808593f0 msgr2=0x556b80867f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f40968bbb80 1 -- 172.21.15.2:0/3248492316 shutdown_connections 2023-06-23T18:31:26.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f40968bbb80 1 -- 172.21.15.2:0/3248492316 wait complete. 2023-06-23T18:31:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f40968bbb80 20 client.16005 trim_cache size 0 max 16384 2023-06-23T18:31:26.060 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:31:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:31:26.060 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:31:26.060 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:31:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 -- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f406c0a1150 msgr2=0x7f406c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f406c0a1150 0x7f406c0a3600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f407400c010 tx=0x7f4074005fd0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 -- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084100610 msgr2=0x7f408419edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084100610 0x7f408419edf0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f407c007d70 tx=0x7f407c00a340 comp rx=0 tx=0).stop 2023-06-23T18:31:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 -- 172.21.15.2:0/2014464765 shutdown_connections 2023-06-23T18:31:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4084100610 0x7f408419edf0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f406c0a1150 0x7f406c0a3600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40840fea90 0x7f4084069070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 --2- 172.21.15.2:0/2014464765 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40840ffc80 0x7f408419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 -- 172.21.15.2:0/2014464765 >> 172.21.15.2:0/2014464765 conn(0x7f40840fa090 msgr2=0x7f40840fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.045+0000 7f408a736700 1 -- 172.21.15.2:0/2014464765 shutdown_connections 2023-06-23T18:31:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.046+0000 7f408a736700 1 -- 172.21.15.2:0/2014464765 wait complete. 2023-06-23T18:31:26.064 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:31:26.064 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:31:26.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 -- 172.21.15.2:0/3021555204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b00fea60 msgr2=0x7f40b00fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 --2- 172.21.15.2:0/3021555204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b00fea60 0x7f40b00fee70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009990 tx=0x7f40ac009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 -- 172.21.15.2:0/3021555204 shutdown_connections 2023-06-23T18:31:26.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 --2- 172.21.15.2:0/3021555204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b01005e0 0x7f40b0106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 --2- 172.21.15.2:0/3021555204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b00fea60 0x7f40b00fee70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 --2- 172.21.15.2:0/3021555204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b00ffc50 0x7f40b01000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 -- 172.21.15.2:0/3021555204 >> 172.21.15.2:0/3021555204 conn(0x7f40b00fa0e0 msgr2=0x7f40b00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 -- 172.21.15.2:0/3021555204 shutdown_connections 2023-06-23T18:31:26.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.207+0000 7f40b7b37700 1 -- 172.21.15.2:0/3021555204 wait complete. 2023-06-23T18:31:26.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.208+0000 7f40b7b37700 1 Processor -- start 2023-06-23T18:31:26.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.208+0000 7f40b7b37700 1 -- start start 2023-06-23T18:31:26.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.208+0000 7f40b7b37700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b00fea60 0x7f40b0068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.208+0000 7f40b7b37700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00ffc50 0x7f40b019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b7b37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b01005e0 0x7f40b019cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b7b37700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019d300 con 0x7f40b01005e0 2023-06-23T18:31:26.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b7b37700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019d470 con 0x7f40b00ffc50 2023-06-23T18:31:26.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b7b37700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b019d630 con 0x7f40b00fea60 2023-06-23T18:31:26.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b01005e0 0x7f40b019cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b01005e0 0x7f40b019cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55542/0 (socket says 172.21.15.2:55542) 2023-06-23T18:31:26.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 -- 172.21.15.2:0/1916513481 learned_addr learned my addr 172.21.15.2:0/1916513481 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b50d2700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00ffc50 0x7f40b019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b58d3700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b00fea60 0x7f40b0068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 -- 172.21.15.2:0/1916513481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b00fea60 msgr2=0x7f40b0068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b00fea60 0x7f40b0068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 -- 172.21.15.2:0/1916513481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00ffc50 msgr2=0x7f40b019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00ffc50 0x7f40b019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 -- 172.21.15.2:0/1916513481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac0095f0 con 0x7f40b01005e0 2023-06-23T18:31:26.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b60d4700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b01005e0 0x7f40b019cb60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007ee0 tx=0x7f40a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40a6ffd700 1 -- 172.21.15.2:0/1916513481 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8017070 con 0x7f40b01005e0 2023-06-23T18:31:26.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40a6ffd700 1 -- 172.21.15.2:0/1916513481 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a800f040 con 0x7f40b01005e0 2023-06-23T18:31:26.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b0194350 con 0x7f40b01005e0 2023-06-23T18:31:26.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.209+0000 7f40a6ffd700 1 -- 172.21.15.2:0/1916513481 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8005570 con 0x7f40b01005e0 2023-06-23T18:31:26.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.210+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b0194860 con 0x7f40b01005e0 2023-06-23T18:31:26.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.210+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b010a0e0 con 0x7f40b01005e0 2023-06-23T18:31:26.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.212+0000 7f40a6ffd700 1 -- 172.21.15.2:0/1916513481 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f40a800db20 con 0x7f40b01005e0 2023-06-23T18:31:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.212+0000 7f40a6ffd700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f409c0a1150 0x7f409c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.213+0000 7f40b58d3700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f409c0a1150 0x7f409c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.213+0000 7f40a6ffd700 1 -- 172.21.15.2:0/1916513481 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f40a80cc6a0 con 0x7f40b01005e0 2023-06-23T18:31:26.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.213+0000 7f40b58d3700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f409c0a1150 0x7f409c0a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006010 tx=0x7f40ac00b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.217+0000 7f40a6ffd700 1 -- 172.21.15.2:0/1916513481 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f40a8095df0 con 0x7f40b01005e0 2023-06-23T18:31:26.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.504+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40b0195210 con 0x7f40b01005e0 2023-06-23T18:31:26.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.505+0000 7f40a6ffd700 1 -- 172.21.15.2:0/1916513481 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10780 (secure 0 0 0) 0x7f40a8099410 con 0x7f40b01005e0 2023-06-23T18:31:26.509 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:31:26.509 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:30:26.353535+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:31:26.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f409c0a1150 msgr2=0x7f409c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f409c0a1150 0x7f409c0a3600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006010 tx=0x7f40ac00b2a0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b01005e0 msgr2=0x7f40b019cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b01005e0 0x7f40b019cb60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f40a8007ee0 tx=0x7f40a800d3b0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 shutdown_connections 2023-06-23T18:31:26.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40b01005e0 0x7f40b019cb60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40b00fea60 0x7f40b0068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f409c0a1150 0x7f409c0a3600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 --2- 172.21.15.2:0/1916513481 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f40b00ffc50 0x7f40b019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.507+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 >> 172.21.15.2:0/1916513481 conn(0x7f40b00fa0e0 msgr2=0x7f40b00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.508+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 shutdown_connections 2023-06-23T18:31:26.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.508+0000 7f40b7b37700 1 -- 172.21.15.2:0/1916513481 wait complete. 2023-06-23T18:31:26.514 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 32 2023-06-23T18:31:26.524 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:31:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.662+0000 7f6cac488700 1 -- 172.21.15.2:0/994478278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca40fea00 msgr2=0x7f6ca40fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.662+0000 7f6cac488700 1 --2- 172.21.15.2:0/994478278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca40fea00 0x7f6ca40fee10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009990 tx=0x7f6ca0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.662+0000 7f6cac488700 1 -- 172.21.15.2:0/994478278 shutdown_connections 2023-06-23T18:31:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.662+0000 7f6cac488700 1 --2- 172.21.15.2:0/994478278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ca4100580 0x7f6ca4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.662+0000 7f6cac488700 1 --2- 172.21.15.2:0/994478278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca40fea00 0x7f6ca40fee10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.662+0000 7f6cac488700 1 --2- 172.21.15.2:0/994478278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca40ffbf0 0x7f6ca4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.662+0000 7f6cac488700 1 -- 172.21.15.2:0/994478278 >> 172.21.15.2:0/994478278 conn(0x7f6ca40fa020 msgr2=0x7f6ca40fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.662+0000 7f6cac488700 1 -- 172.21.15.2:0/994478278 shutdown_connections 2023-06-23T18:31:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 -- 172.21.15.2:0/994478278 wait complete. 2023-06-23T18:31:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 Processor -- start 2023-06-23T18:31:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 -- start start 2023-06-23T18:31:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ca40fea00 0x7f6ca41956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca40ffbf0 0x7f6ca4195c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca4100580 0x7f6ca419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca419d240 con 0x7f6ca4100580 2023-06-23T18:31:26.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca419d380 con 0x7f6ca40fea00 2023-06-23T18:31:26.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.663+0000 7f6cac488700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca419d510 con 0x7f6ca40ffbf0 2023-06-23T18:31:26.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca4100580 0x7f6ca419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca4100580 0x7f6ca419cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55556/0 (socket says 172.21.15.2:55556) 2023-06-23T18:31:26.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 -- 172.21.15.2:0/3925737150 learned_addr learned my addr 172.21.15.2:0/3925737150 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca40ffbf0 0x7f6ca4195c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 -- 172.21.15.2:0/3925737150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca40ffbf0 msgr2=0x7f6ca4195c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca40ffbf0 0x7f6ca4195c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 -- 172.21.15.2:0/3925737150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ca40fea00 msgr2=0x7f6ca41956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ca40fea00 0x7f6ca41956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 -- 172.21.15.2:0/3925737150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca00095f0 con 0x7f6ca4100580 2023-06-23T18:31:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.664+0000 7f6caaa25700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca4100580 0x7f6ca419cc70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c007ee0 tx=0x7f6c9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.665+0000 7f6c9b7fe700 1 -- 172.21.15.2:0/3925737150 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c017070 con 0x7f6ca4100580 2023-06-23T18:31:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.665+0000 7f6c9b7fe700 1 -- 172.21.15.2:0/3925737150 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c9c00f040 con 0x7f6ca4100580 2023-06-23T18:31:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.665+0000 7f6cac488700 1 -- 172.21.15.2:0/3925737150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca419d7f0 con 0x7f6ca4100580 2023-06-23T18:31:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.665+0000 7f6c9b7fe700 1 -- 172.21.15.2:0/3925737150 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c005710 con 0x7f6ca4100580 2023-06-23T18:31:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.665+0000 7f6cac488700 1 -- 172.21.15.2:0/3925737150 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca419dd00 con 0x7f6ca4100580 2023-06-23T18:31:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.667+0000 7f6c9b7fe700 1 -- 172.21.15.2:0/3925737150 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6c9c005870 con 0x7f6ca4100580 2023-06-23T18:31:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.668+0000 7f6c9b7fe700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c900a11d0 0x7f6c900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.668+0000 7f6c9b7fe700 1 -- 172.21.15.2:0/3925737150 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c9c0cc150 con 0x7f6ca4100580 2023-06-23T18:31:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.668+0000 7f6caa224700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c900a11d0 0x7f6c900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.669+0000 7f6caa224700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c900a11d0 0x7f6c900a3680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0003f60 tx=0x7f6ca0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.669+0000 7f6cb860db80 1 Processor -- start 2023-06-23T18:31:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6cb860db80 1 -- start start 2023-06-23T18:31:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde4a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde4ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde4b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6cb860db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bdd4acb0 con 0x55f6bde4a090 2023-06-23T18:31:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6cb860db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bdd9da40 con 0x55f6bde4a9e0 2023-06-23T18:31:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6cb860db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bdd96ed0 con 0x55f6bde4b410 2023-06-23T18:31:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6caa224700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde4a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6caa224700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde4a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55560/0 (socket says 172.21.15.2:55560) 2023-06-23T18:31:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6caa224700 1 -- 172.21.15.2:0/4021585446 learned_addr learned my addr 172.21.15.2:0/4021585446 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde4ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.670+0000 7f6caaa25700 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde4b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6ca9a23700 1 -- 172.21.15.2:0/4021585446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 msgr2=0x55f6bde4b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde4b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6ca9a23700 1 -- 172.21.15.2:0/4021585446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 msgr2=0x55f6bde4a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde4a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6ca9a23700 1 -- 172.21.15.2:0/4021585446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6bdc18630 con 0x55f6bde4a9e0 2023-06-23T18:31:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6caa224700 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde4a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde4ae80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009c00 tx=0x7f6c94009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81a5b51e45970533 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6c98dac700 1 -- 172.21.15.2:0/4021585446 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94011ab0 con 0x55f6bde4a9e0 2023-06-23T18:31:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6c98dac700 1 -- 172.21.15.2:0/4021585446 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c94011c10 con 0x55f6bde4a9e0 2023-06-23T18:31:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.671+0000 7f6c98dac700 1 -- 172.21.15.2:0/4021585446 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9400f6a0 con 0x55f6bde4a9e0 2023-06-23T18:31:26.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 -- 172.21.15.2:0/4021585446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 msgr2=0x55f6bde4ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde4ae80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009c00 tx=0x7f6c94009f10 comp rx=0 tx=0).stop 2023-06-23T18:31:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 -- 172.21.15.2:0/4021585446 shutdown_connections 2023-06-23T18:31:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde4b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde4a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 --2- 172.21.15.2:0/4021585446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde4ae80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 -- 172.21.15.2:0/4021585446 >> 172.21.15.2:0/4021585446 conn(0x55f6bde3b3f0 msgr2=0x55f6bde40520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 -- 172.21.15.2:0/4021585446 shutdown_connections 2023-06-23T18:31:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.672+0000 7f6cb860db80 1 -- 172.21.15.2:0/4021585446 wait complete. 2023-06-23T18:31:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 1 -- start start 2023-06-23T18:31:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 1 Processor -- start 2023-06-23T18:31:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a090 0x55f6bde53920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a9e0 0x55f6bde53e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4b410 0x55f6bde5aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bdd36ac0 con 0x55f6bde4b410 2023-06-23T18:31:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bdd367c0 con 0x55f6bde4a090 2023-06-23T18:31:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bdc1a920 con 0x55f6bde4a9e0 2023-06-23T18:31:26.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6cb860db80 10 client.0 _ll_register_callbacks cb 0x55f6bdd3c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6caa224700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a090 0x55f6bde53920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6ca9a23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a9e0 0x55f6bde53e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6caa224700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a090 0x55f6bde53920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46158/0 (socket says 172.21.15.2:46158) 2023-06-23T18:31:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.673+0000 7f6caa224700 1 -- 172.21.15.2:0/2624153798 learned_addr learned my addr 172.21.15.2:0/2624153798 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6ca9a23700 1 -- 172.21.15.2:0/2624153798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a090 msgr2=0x55f6bde53920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a090 0x55f6bde53920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6ca9a23700 1 -- 172.21.15.2:0/2624153798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4b410 msgr2=0x55f6bde5aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4b410 0x55f6bde5aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6ca9a23700 1 -- 172.21.15.2:0/2624153798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca4107770 con 0x55f6bde4a9e0 2023-06-23T18:31:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6caa224700 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a090 0x55f6bde53920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a9e0 0x55f6bde53e60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009bd0 tx=0x7f6c940127b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae03a23a293b53a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6c8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:31:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2624153798 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94005260 con 0x55f6bde4a9e0 2023-06-23T18:31:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2624153798 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c940053c0 con 0x55f6bde4a9e0 2023-06-23T18:31:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6cb860db80 1 -- 172.21.15.2:0/2624153798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6bde38bb0 con 0x55f6bde4a9e0 2023-06-23T18:31:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2624153798 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c94005580 con 0x55f6bde4a9e0 2023-06-23T18:31:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.674+0000 7f6cb860db80 10 client.25768 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6cb860db80 1 -- 172.21.15.2:0/2624153798 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f6c94009830 con 0x55f6bde4a9e0 2023-06-23T18:31:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2624153798 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f6bde38bb0 con 0x55f6bde4a9e0 2023-06-23T18:31:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2624153798 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6c94022070 con 0x55f6bde4a9e0 2023-06-23T18:31:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6cb860db80 10 client.25768 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:26.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6cb860db80 10 client.25768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:31:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6cb860db80 20 client.25768 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55f6bde5f610 0x55f6bde619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6cb860db80 4 client.25768 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6cb860db80 1 -- 172.21.15.2:0/2624153798 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6bdd8feb0 con 0x55f6bde5f610 2023-06-23T18:31:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.675+0000 7f6caa224700 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55f6bde5f610 0x55f6bde619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.676+0000 7f6c8dffb700 10 client.25768 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:31:26.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.676+0000 7f6caa224700 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55f6bde5f610 0x55f6bde619f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff1de3f60a50edd server_cookie=48a40ec8f2fe7236 in_seq=0 out_seq=0 2023-06-23T18:31:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2624153798 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f6bdd8feb0 con 0x55f6bde5f610 2023-06-23T18:31:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6c8dffb700 10 client.25768 handle_command_reply: tid=0 2023-06-23T18:31:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 client.25768 shutdown 2023-06-23T18:31:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 -- 172.21.15.2:0/2624153798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a9e0 msgr2=0x55f6bde53e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a9e0 0x55f6bde53e60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009bd0 tx=0x7f6c940127b0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 -- 172.21.15.2:0/2624153798 shutdown_connections 2023-06-23T18:31:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55f6bde5f610 0x55f6bde619f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4b410 0x55f6bde5aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6c8dffb700 0 client.25768 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:31:26.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a090 0x55f6bde53920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2624153798 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a9e0 0x55f6bde53e60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 -- 172.21.15.2:0/2624153798 >> 172.21.15.2:0/2624153798 conn(0x55f6bde3b3f0 msgr2=0x55f6bde3fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 -- 172.21.15.2:0/2624153798 shutdown_connections 2023-06-23T18:31:26.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 1 -- 172.21.15.2:0/2624153798 wait complete. 2023-06-23T18:31:26.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.677+0000 7f6cb860db80 20 client.25768 trim_cache size 0 max 16384 2023-06-23T18:31:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.693+0000 7f6cb860db80 1 Processor -- start 2023-06-23T18:31:26.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.693+0000 7f6cb860db80 1 -- start start 2023-06-23T18:31:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.693+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a090 0x55f6bde66350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.693+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a9e0 0x55f6bde5ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.693+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4b410 0x55f6bde4e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.693+0000 7f6cb860db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bde3af70 con 0x55f6bde4a9e0 2023-06-23T18:31:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.693+0000 7f6cb860db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bde54aa0 con 0x55f6bde4b410 2023-06-23T18:31:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.693+0000 7f6cb860db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bde515e0 con 0x55f6bde4a090 2023-06-23T18:31:26.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a9e0 0x55f6bde5ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a9e0 0x55f6bde5ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55576/0 (socket says 172.21.15.2:55576) 2023-06-23T18:31:26.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6caa224700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a090 0x55f6bde66350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 -- 172.21.15.2:0/1488271261 learned_addr learned my addr 172.21.15.2:0/1488271261 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6caaa25700 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4b410 0x55f6bde4e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 -- 172.21.15.2:0/1488271261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a090 msgr2=0x55f6bde66350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a090 0x55f6bde66350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 -- 172.21.15.2:0/1488271261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4b410 msgr2=0x55f6bde4e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4b410 0x55f6bde4e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 -- 172.21.15.2:0/1488271261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6bdbc4de0 con 0x55f6bde4a9e0 2023-06-23T18:31:26.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6caaa25700 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4b410 0x55f6bde4e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:26.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a9e0 0x55f6bde5ae60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009800 tx=0x7f6c94024790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a4528365bbc62 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6c995ad700 1 -- 172.21.15.2:0/1488271261 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94011ec0 con 0x55f6bde4a9e0 2023-06-23T18:31:26.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6c995ad700 1 -- 172.21.15.2:0/1488271261 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c94024e90 con 0x55f6bde4a9e0 2023-06-23T18:31:26.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.694+0000 7f6c995ad700 1 -- 172.21.15.2:0/1488271261 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c94023f10 con 0x55f6bde4a9e0 2023-06-23T18:31:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 -- 172.21.15.2:0/1488271261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a9e0 msgr2=0x55f6bde5ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a9e0 0x55f6bde5ae60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009800 tx=0x7f6c94024790 comp rx=0 tx=0).stop 2023-06-23T18:31:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 -- 172.21.15.2:0/1488271261 shutdown_connections 2023-06-23T18:31:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4b410 0x55f6bde4e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4a090 0x55f6bde66350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 --2- 172.21.15.2:0/1488271261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a9e0 0x55f6bde5ae60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 -- 172.21.15.2:0/1488271261 >> 172.21.15.2:0/1488271261 conn(0x55f6bde3b3f0 msgr2=0x55f6bde3c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 -- 172.21.15.2:0/1488271261 shutdown_connections 2023-06-23T18:31:26.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.695+0000 7f6cb860db80 1 -- 172.21.15.2:0/1488271261 wait complete. 2023-06-23T18:31:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 1 -- start start 2023-06-23T18:31:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 1 Processor -- start 2023-06-23T18:31:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde56810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde56d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde59d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bde5a2b0 con 0x55f6bde4a090 2023-06-23T18:31:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bde5a420 con 0x55f6bde4a9e0 2023-06-23T18:31:26.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bde5a5e0 con 0x55f6bde4b410 2023-06-23T18:31:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6cb860db80 10 client.0 _ll_register_callbacks cb 0x55f6bdd3c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caa224700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde56810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caa224700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde56810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55580/0 (socket says 172.21.15.2:55580) 2023-06-23T18:31:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caa224700 1 -- 172.21.15.2:0/2416907310 learned_addr learned my addr 172.21.15.2:0/2416907310 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caaa25700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde59d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde56d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caa224700 1 -- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 msgr2=0x55f6bde59d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caa224700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde59d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caa224700 1 -- 172.21.15.2:0/2416907310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 msgr2=0x55f6bde56d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caa224700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde56d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caa224700 1 -- 172.21.15.2:0/2416907310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca419dd00 con 0x55f6bde4a090 2023-06-23T18:31:26.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6caaa25700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde59d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:31:26.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.696+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde56d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:31:26.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6caa224700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde56810 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55f6bde62300 tx=0x7f6ca0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5afb71b2f816c21 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6c8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:26.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2416907310 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0029040 con 0x55f6bde4a090 2023-06-23T18:31:26.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2416907310 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6ca0003d30 con 0x55f6bde4a090 2023-06-23T18:31:26.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6cb860db80 1 -- 172.21.15.2:0/2416907310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c94011c10 con 0x55f6bde4a090 2023-06-23T18:31:26.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2416907310 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca001d070 con 0x55f6bde4a090 2023-06-23T18:31:26.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6cb860db80 10 client.16026 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:26.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6cb860db80 1 -- 172.21.15.2:0/2416907310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f6c94009830 con 0x55f6bde4a090 2023-06-23T18:31:26.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2416907310 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c94011c10 con 0x55f6bde4a090 2023-06-23T18:31:26.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2416907310 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ca0004860 con 0x55f6bde4a090 2023-06-23T18:31:26.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6cb860db80 10 client.16026 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:26.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6cb860db80 10 client.16026 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:31:26.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.697+0000 7f6cb860db80 20 client.16026 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:26.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55f6bdeb4090 0x55f6bdeb64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6cb860db80 4 client.16026 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:31:26.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6cb860db80 1 -- 172.21.15.2:0/2416907310 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6bdd52050 con 0x55f6bdeb4090 2023-06-23T18:31:26.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55f6bdeb4090 0x55f6bdeb64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6ca9a23700 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55f6bdeb4090 0x55f6bdeb64a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfee44a98805fb45 server_cookie=b70135cb04e33696 in_seq=0 out_seq=0 2023-06-23T18:31:26.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6c8dffb700 10 client.16026 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:31:26.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6c8dffb700 1 -- 172.21.15.2:0/2416907310 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6bdd52050 con 0x55f6bdeb4090 2023-06-23T18:31:26.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6c8dffb700 10 client.16026 handle_command_reply: tid=0 2023-06-23T18:31:26.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.698+0000 7f6cb860db80 1 client.16026 shutdown 2023-06-23T18:31:26.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 -- 172.21.15.2:0/2416907310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 msgr2=0x55f6bde56810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde56810 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x55f6bde62300 tx=0x7f6ca0009310 comp rx=0 tx=0).stop 2023-06-23T18:31:26.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 -- 172.21.15.2:0/2416907310 shutdown_connections 2023-06-23T18:31:26.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55f6bdeb4090 0x55f6bdeb64a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f6bde4b410 0x55f6bde59d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6c8dffb700 0 client.16026 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:31:26.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f6bde4a090 0x55f6bde56810 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 --2- 172.21.15.2:0/2416907310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f6bde4a9e0 0x55f6bde56d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 -- 172.21.15.2:0/2416907310 >> 172.21.15.2:0/2416907310 conn(0x55f6bde3b3f0 msgr2=0x55f6bde66350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 -- 172.21.15.2:0/2416907310 shutdown_connections 2023-06-23T18:31:26.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 1 -- 172.21.15.2:0/2416907310 wait complete. 2023-06-23T18:31:26.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cb860db80 20 client.16026 trim_cache size 0 max 16384 2023-06-23T18:31:26.721 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:31:26.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 -- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c900a11d0 msgr2=0x7f6c900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c900a11d0 0x7f6c900a3680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0003f60 tx=0x7f6ca0005fb0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 -- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca4100580 msgr2=0x7f6ca419cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca4100580 0x7f6ca419cc70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c007ee0 tx=0x7f6c9c00d3b0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 -- 172.21.15.2:0/3925737150 shutdown_connections 2023-06-23T18:31:26.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ca4100580 0x7f6ca419cc70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c900a11d0 0x7f6c900a3680 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6ca40fea00 0x7f6ca41956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 --2- 172.21.15.2:0/3925737150 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ca40ffbf0 0x7f6ca4195c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.699+0000 7f6cac488700 1 -- 172.21.15.2:0/3925737150 >> 172.21.15.2:0/3925737150 conn(0x7f6ca40fa020 msgr2=0x7f6ca4068460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.700+0000 7f6cac488700 1 -- 172.21.15.2:0/3925737150 shutdown_connections 2023-06-23T18:31:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.700+0000 7f6cac488700 1 -- 172.21.15.2:0/3925737150 wait complete. 2023-06-23T18:31:26.726 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:26 smithi002 ceph-mon[95947]: pgmap v578: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:31:26.727 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:31:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:26 smithi082 ceph-mon[97889]: pgmap v578: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:31:26.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 -- 172.21.15.2:0/4039536994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f00fea80 msgr2=0x7f42f00fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/4039536994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f00fea80 0x7f42f00fee90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009990 tx=0x7f42ec009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 -- 172.21.15.2:0/4039536994 shutdown_connections 2023-06-23T18:31:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/4039536994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f0100600 0x7f42f0106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/4039536994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f00fea80 0x7f42f00fee90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/4039536994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00ffc70 0x7f42f01000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 -- 172.21.15.2:0/4039536994 >> 172.21.15.2:0/4039536994 conn(0x7f42f00fa0e0 msgr2=0x7f42f00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 -- 172.21.15.2:0/4039536994 shutdown_connections 2023-06-23T18:31:26.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 -- 172.21.15.2:0/4039536994 wait complete. 2023-06-23T18:31:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.868+0000 7f42f7e2d700 1 Processor -- start 2023-06-23T18:31:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f7e2d700 1 -- start start 2023-06-23T18:31:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f7e2d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f00fea80 0x7f42f0068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f7e2d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00ffc70 0x7f42f019a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f7e2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0100600 0x7f42f019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f7e2d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019d340 con 0x7f42f0100600 2023-06-23T18:31:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f7e2d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019d4b0 con 0x7f42f00ffc70 2023-06-23T18:31:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f7e2d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f019d670 con 0x7f42f00fea80 2023-06-23T18:31:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0100600 0x7f42f019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0100600 0x7f42f019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55598/0 (socket says 172.21.15.2:55598) 2023-06-23T18:31:26.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 -- 172.21.15.2:0/3706835589 learned_addr learned my addr 172.21.15.2:0/3706835589 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f00fea80 0x7f42f0068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f53c8700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00ffc70 0x7f42f019a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 -- 172.21.15.2:0/3706835589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f00fea80 msgr2=0x7f42f0068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f00fea80 0x7f42f0068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 -- 172.21.15.2:0/3706835589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00ffc70 msgr2=0x7f42f019a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00ffc70 0x7f42f019a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 -- 172.21.15.2:0/3706835589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0095f0 con 0x7f42f0100600 2023-06-23T18:31:26.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.869+0000 7f42f63ca700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0100600 0x7f42f019cb70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f42e8007ee0 tx=0x7f42e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.870+0000 7f42e6ffd700 1 -- 172.21.15.2:0/3706835589 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8017070 con 0x7f42f0100600 2023-06-23T18:31:26.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.870+0000 7f42e6ffd700 1 -- 172.21.15.2:0/3706835589 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42e800f040 con 0x7f42f0100600 2023-06-23T18:31:26.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.870+0000 7f42e6ffd700 1 -- 172.21.15.2:0/3706835589 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8005710 con 0x7f42f0100600 2023-06-23T18:31:26.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.870+0000 7f42f7e2d700 1 -- 172.21.15.2:0/3706835589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f019d950 con 0x7f42f0100600 2023-06-23T18:31:26.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.870+0000 7f42f7e2d700 1 -- 172.21.15.2:0/3706835589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f019de10 con 0x7f42f0100600 2023-06-23T18:31:26.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.873+0000 7f42e6ffd700 1 -- 172.21.15.2:0/3706835589 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f42e8005870 con 0x7f42f0100600 2023-06-23T18:31:26.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.873+0000 7f42e6ffd700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42dc0a11d0 0x7f42dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.874+0000 7f42e6ffd700 1 -- 172.21.15.2:0/3706835589 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42e80cc0b0 con 0x7f42f0100600 2023-06-23T18:31:26.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.874+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42dc0a11d0 0x7f42dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.874+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42dc0a11d0 0x7f42dc0a3680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00c010 tx=0x7f42ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.874+0000 7f4303fb2b80 1 Processor -- start 2023-06-23T18:31:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.874+0000 7f4303fb2b80 1 -- start start 2023-06-23T18:31:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.874+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x55731892b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892b9e0 0x55731892be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892c410 0x55731892c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731882bcb0 con 0x55731892b090 2023-06-23T18:31:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731887ea40 con 0x55731892b9e0 2023-06-23T18:31:26.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557318877ed0 con 0x55731892c410 2023-06-23T18:31:26.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x55731892b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x55731892b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55602/0 (socket says 172.21.15.2:55602) 2023-06-23T18:31:26.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 -- 172.21.15.2:0/1792342107 learned_addr learned my addr 172.21.15.2:0/1792342107 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f63ca700 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892c410 0x55731892c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f53c8700 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892b9e0 0x55731892be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 -- 172.21.15.2:0/1792342107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892c410 msgr2=0x55731892c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892c410 0x55731892c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 -- 172.21.15.2:0/1792342107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892b9e0 msgr2=0x55731892be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892b9e0 0x55731892be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 -- 172.21.15.2:0/1792342107 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573186f9630 con 0x55731892b090 2023-06-23T18:31:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f53c8700 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892b9e0 0x55731892be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x55731892b4a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f42f00ffad0 tx=0x7f42ec020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd1375dc1cf7d76 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.875+0000 7f42dbfff700 1 -- 172.21.15.2:0/1792342107 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec02b070 con 0x55731892b090 2023-06-23T18:31:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f42dbfff700 1 -- 172.21.15.2:0/1792342107 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42ec01e040 con 0x55731892b090 2023-06-23T18:31:26.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f42dbfff700 1 -- 172.21.15.2:0/1792342107 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec004e80 con 0x55731892b090 2023-06-23T18:31:26.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f4303fb2b80 1 -- 172.21.15.2:0/1792342107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 msgr2=0x55731892b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x55731892b4a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f42f00ffad0 tx=0x7f42ec020000 comp rx=0 tx=0).stop 2023-06-23T18:31:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f4303fb2b80 1 -- 172.21.15.2:0/1792342107 shutdown_connections 2023-06-23T18:31:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892c410 0x55731892c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x55731892b4a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/1792342107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892b9e0 0x55731892be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.876+0000 7f4303fb2b80 1 -- 172.21.15.2:0/1792342107 >> 172.21.15.2:0/1792342107 conn(0x55731891c3f0 msgr2=0x557318921520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 -- 172.21.15.2:0/1792342107 shutdown_connections 2023-06-23T18:31:26.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 -- 172.21.15.2:0/1792342107 wait complete. 2023-06-23T18:31:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 -- start start 2023-06-23T18:31:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 Processor -- start 2023-06-23T18:31:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 0x5573189348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x557318934e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557318817ac0 con 0x55731892b9e0 2023-06-23T18:31:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573188177c0 con 0x55731892c410 2023-06-23T18:31:26.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.877+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573186fb920 con 0x55731892b090 2023-06-23T18:31:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f4303fb2b80 10 client.0 _ll_register_callbacks cb 0x55731881d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x557318934e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x557318934e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55610/0 (socket says 172.21.15.2:55610) 2023-06-23T18:31:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 -- 172.21.15.2:0/3671827214 learned_addr learned my addr 172.21.15.2:0/3671827214 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 0x5573189348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f63ca700 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 -- 172.21.15.2:0/3671827214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 msgr2=0x5573189348d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 0x5573189348d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 -- 172.21.15.2:0/3671827214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 msgr2=0x55731893be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 -- 172.21.15.2:0/3671827214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f01077f0 con 0x55731892b9e0 2023-06-23T18:31:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42f53c8700 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x557318934e10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f42e000bfa0 tx=0x7f42e00116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30be0f9458ea33b9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42d97fa700 1 -- 172.21.15.2:0/3671827214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0004030 con 0x55731892b9e0 2023-06-23T18:31:26.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f42d97fa700 1 -- 172.21.15.2:0/3671827214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42e0004190 con 0x55731892b9e0 2023-06-23T18:31:26.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.878+0000 7f4303fb2b80 1 -- 172.21.15.2:0/3671827214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557318919bb0 con 0x55731892b9e0 2023-06-23T18:31:26.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f42d97fa700 1 -- 172.21.15.2:0/3671827214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e0004350 con 0x55731892b9e0 2023-06-23T18:31:26.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f4303fb2b80 10 client.16038 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:26.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f4303fb2b80 1 -- 172.21.15.2:0/3671827214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5573186a5de0 con 0x55731892b9e0 2023-06-23T18:31:26.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f42d97fa700 1 -- 172.21.15.2:0/3671827214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557318919bb0 con 0x55731892b9e0 2023-06-23T18:31:26.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f42d97fa700 1 -- 172.21.15.2:0/3671827214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f42e001b590 con 0x55731892b9e0 2023-06-23T18:31:26.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f4303fb2b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:26.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f4303fb2b80 10 client.16038 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:31:26.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f4303fb2b80 20 client.16038 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:26.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5573189407b0 0x557318942bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f4303fb2b80 4 client.16038 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:26.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f4303fb2b80 1 -- 172.21.15.2:0/3671827214 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557318870eb0 con 0x5573189407b0 2023-06-23T18:31:26.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.879+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5573189407b0 0x557318942bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.880+0000 7f42d97fa700 10 client.16038 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:31:26.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.880+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5573189407b0 0x557318942bc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8a942391dd3adcb server_cookie=5b199fa4721ab718 in_seq=0 out_seq=0 2023-06-23T18:31:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.880+0000 7f42d97fa700 1 -- 172.21.15.2:0/3671827214 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557318870eb0 con 0x5573189407b0 2023-06-23T18:31:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f42d97fa700 10 client.16038 handle_command_reply: tid=0 2023-06-23T18:31:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 client.16038 shutdown 2023-06-23T18:31:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 -- 172.21.15.2:0/3671827214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 msgr2=0x557318934e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x557318934e10 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f42e000bfa0 tx=0x7f42e00116c0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 -- 172.21.15.2:0/3671827214 shutdown_connections 2023-06-23T18:31:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5573189407b0 0x557318942bc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f42d97fa700 0 client.16038 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:31:26.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 0x5573189348d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/3671827214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x557318934e10 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 -- 172.21.15.2:0/3671827214 >> 172.21.15.2:0/3671827214 conn(0x55731891c3f0 msgr2=0x557318920c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 -- 172.21.15.2:0/3671827214 shutdown_connections 2023-06-23T18:31:26.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 1 -- 172.21.15.2:0/3671827214 wait complete. 2023-06-23T18:31:26.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.881+0000 7f4303fb2b80 20 client.16038 trim_cache size 0 max 16384 2023-06-23T18:31:26.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.895+0000 7f4303fb2b80 1 Processor -- start 2023-06-23T18:31:26.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f4303fb2b80 1 -- start start 2023-06-23T18:31:26.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 0x5573189348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x55731893be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x557318934e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557318935a50 con 0x55731892b9e0 2023-06-23T18:31:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557318922210 con 0x55731892c410 2023-06-23T18:31:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557318933d80 con 0x55731892b090 2023-06-23T18:31:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f53c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x55731893be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f53c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x55731893be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55618/0 (socket says 172.21.15.2:55618) 2023-06-23T18:31:26.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f53c8700 1 -- 172.21.15.2:0/2268902226 learned_addr learned my addr 172.21.15.2:0/2268902226 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 0x5573189348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f63ca700 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x557318934e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f53c8700 1 -- 172.21.15.2:0/2268902226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 msgr2=0x5573189348d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f53c8700 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 0x5573189348d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f53c8700 1 -- 172.21.15.2:0/2268902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 msgr2=0x557318934e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f53c8700 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x557318934e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f53c8700 1 -- 172.21.15.2:0/2268902226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55731868ec60 con 0x55731892b9e0 2023-06-23T18:31:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.896+0000 7f42f63ca700 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x557318934e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f42f53c8700 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x55731893be10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f42e0005c50 tx=0x7f42e000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efeeab65fcb685bb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f42e4ff9700 1 -- 172.21.15.2:0/2268902226 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e001ba70 con 0x55731892b9e0 2023-06-23T18:31:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f42e4ff9700 1 -- 172.21.15.2:0/2268902226 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42e0021030 con 0x55731892b9e0 2023-06-23T18:31:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f42e4ff9700 1 -- 172.21.15.2:0/2268902226 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0004710 con 0x55731892b9e0 2023-06-23T18:31:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f4303fb2b80 1 -- 172.21.15.2:0/2268902226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 msgr2=0x55731893be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x55731893be10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f42e0005c50 tx=0x7f42e000b6e0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f4303fb2b80 1 -- 172.21.15.2:0/2268902226 shutdown_connections 2023-06-23T18:31:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x557318934e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b090 0x5573189348d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/2268902226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b9e0 0x55731893be10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.897+0000 7f4303fb2b80 1 -- 172.21.15.2:0/2268902226 >> 172.21.15.2:0/2268902226 conn(0x55731891c3f0 msgr2=0x5573189335e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.898+0000 7f4303fb2b80 1 -- 172.21.15.2:0/2268902226 shutdown_connections 2023-06-23T18:31:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.898+0000 7f4303fb2b80 1 -- 172.21.15.2:0/2268902226 wait complete. 2023-06-23T18:31:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.898+0000 7f4303fb2b80 1 -- start start 2023-06-23T18:31:26.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.898+0000 7f4303fb2b80 1 Processor -- start 2023-06-23T18:31:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x557318937e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b9e0 0x5573189383b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f4303fb2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731893b910 con 0x55731892b090 2023-06-23T18:31:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731893ba80 con 0x55731892c410 2023-06-23T18:31:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f4303fb2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55731893bc40 con 0x55731892b9e0 2023-06-23T18:31:26.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f4303fb2b80 10 client.0 _ll_register_callbacks cb 0x55731881d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x557318937e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x557318937e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55622/0 (socket says 172.21.15.2:55622) 2023-06-23T18:31:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 -- 172.21.15.2:0/513124315 learned_addr learned my addr 172.21.15.2:0/513124315 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f53c8700 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b9e0 0x5573189383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f63ca700 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 -- 172.21.15.2:0/513124315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b9e0 msgr2=0x5573189383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b9e0 0x5573189383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 -- 172.21.15.2:0/513124315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 msgr2=0x55731893b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 -- 172.21.15.2:0/513124315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573186a5de0 con 0x55731892b090 2023-06-23T18:31:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f63ca700 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:31:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42f5bc9700 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x557318937e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f42ec005950 tx=0x7f42ec004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ca8d1aa6c495d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42d97fa700 1 -- 172.21.15.2:0/513124315 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec02b040 con 0x55731892b090 2023-06-23T18:31:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f42d97fa700 1 -- 172.21.15.2:0/513124315 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f42ec03cd30 con 0x55731892b090 2023-06-23T18:31:26.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.899+0000 7f4303fb2b80 1 -- 172.21.15.2:0/513124315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42ec01e040 con 0x55731892b090 2023-06-23T18:31:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f42d97fa700 1 -- 172.21.15.2:0/513124315 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42ec03c0b0 con 0x55731892b090 2023-06-23T18:31:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f4303fb2b80 10 client.16044 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f4303fb2b80 1 -- 172.21.15.2:0/513124315 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5573186cc250 con 0x55731892b090 2023-06-23T18:31:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f42d97fa700 1 -- 172.21.15.2:0/513124315 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42ec01e040 con 0x55731892b090 2023-06-23T18:31:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f42d97fa700 1 -- 172.21.15.2:0/513124315 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f42ec003860 con 0x55731892b090 2023-06-23T18:31:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f4303fb2b80 10 client.16044 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f4303fb2b80 10 client.16044 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:31:26.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f4303fb2b80 20 client.16044 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x557318995430 0x557318997840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f4303fb2b80 4 client.16044 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:31:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f4303fb2b80 1 -- 172.21.15.2:0/513124315 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557318833050 con 0x557318995430 2023-06-23T18:31:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.900+0000 7f42f53c8700 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x557318995430 0x557318997840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f42f53c8700 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x557318995430 0x557318997840 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b9f53e763ff0b96 server_cookie=cd8a97415579a0c2 in_seq=0 out_seq=0 2023-06-23T18:31:26.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f42d97fa700 10 client.16044 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:31:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f42d97fa700 1 -- 172.21.15.2:0/513124315 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557318833050 con 0x557318995430 2023-06-23T18:31:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f42d97fa700 10 client.16044 handle_command_reply: tid=0 2023-06-23T18:31:26.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 client.16044 shutdown 2023-06-23T18:31:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 -- 172.21.15.2:0/513124315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 msgr2=0x557318937e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x557318937e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f42ec005950 tx=0x7f42ec004f50 comp rx=0 tx=0).stop 2023-06-23T18:31:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 -- 172.21.15.2:0/513124315 shutdown_connections 2023-06-23T18:31:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x557318995430 0x557318997840 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55731892c410 0x55731893b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55731892b090 0x557318937e70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f42d97fa700 0 client.16044 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:31:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 --2- 172.21.15.2:0/513124315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55731892b9e0 0x5573189383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 -- 172.21.15.2:0/513124315 >> 172.21.15.2:0/513124315 conn(0x55731891c3f0 msgr2=0x55731892f270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 -- 172.21.15.2:0/513124315 shutdown_connections 2023-06-23T18:31:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 1 -- 172.21.15.2:0/513124315 wait complete. 2023-06-23T18:31:26.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.901+0000 7f4303fb2b80 20 client.16044 trim_cache size 0 max 16384 2023-06-23T18:31:26.915 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:31:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 -- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42dc0a11d0 msgr2=0x7f42dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42dc0a11d0 0x7f42dc0a3680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00c010 tx=0x7f42ec005fb0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 -- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0100600 msgr2=0x7f42f019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:26.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0100600 0x7f42f019cb70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f42e8007ee0 tx=0x7f42e800d3b0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 -- 172.21.15.2:0/3706835589 shutdown_connections 2023-06-23T18:31:26.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f42f0100600 0x7f42f019cb70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f42f00fea80 0x7f42f0068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f42dc0a11d0 0x7f42dc0a3680 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 --2- 172.21.15.2:0/3706835589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42f00ffc70 0x7f42f019a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:26.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 -- 172.21.15.2:0/3706835589 >> 172.21.15.2:0/3706835589 conn(0x7f42f00fa0e0 msgr2=0x7f42f01031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 -- 172.21.15.2:0/3706835589 shutdown_connections 2023-06-23T18:31:26.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:26.902+0000 7f42f7e2d700 1 -- 172.21.15.2:0/3706835589 wait complete. 2023-06-23T18:31:26.919 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:31:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:26 smithi017 ceph-mon[97472]: pgmap v578: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:31:27.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 -- 172.21.15.2:0/1596770052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8102f80 msgr2=0x7fb3c8105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 --2- 172.21.15.2:0/1596770052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8102f80 0x7fb3c8105360 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb3b8009990 tx=0x7fb3b8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 -- 172.21.15.2:0/1596770052 shutdown_connections 2023-06-23T18:31:27.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 --2- 172.21.15.2:0/1596770052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c81058a0 0x7fb3c8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 --2- 172.21.15.2:0/1596770052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3c8100660 0x7fb3c8102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 --2- 172.21.15.2:0/1596770052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8102f80 0x7fb3c8105360 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 -- 172.21.15.2:0/1596770052 >> 172.21.15.2:0/1596770052 conn(0x7fb3c80fa0e0 msgr2=0x7fb3c80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:27.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 -- 172.21.15.2:0/1596770052 shutdown_connections 2023-06-23T18:31:27.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.058+0000 7fb3cf986700 1 -- 172.21.15.2:0/1596770052 wait complete. 2023-06-23T18:31:27.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cf986700 1 Processor -- start 2023-06-23T18:31:27.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cf986700 1 -- start start 2023-06-23T18:31:27.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cf986700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8100660 0x7fb3c81957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cf986700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3c8102f80 0x7fb3c8195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cf986700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c81058a0 0x7fb3c819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cf986700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c819d320 con 0x7fb3c8100660 2023-06-23T18:31:27.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cf986700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c819d460 con 0x7fb3c81058a0 2023-06-23T18:31:27.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cf986700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c819d5f0 con 0x7fb3c8102f80 2023-06-23T18:31:27.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cd722700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8100660 0x7fb3c81957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cd722700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8100660 0x7fb3c81957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55636/0 (socket says 172.21.15.2:55636) 2023-06-23T18:31:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cd722700 1 -- 172.21.15.2:0/3840181984 learned_addr learned my addr 172.21.15.2:0/3840181984 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3ccf21700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3c8102f80 0x7fb3c8195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.059+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c81058a0 0x7fb3c819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3cd722700 1 -- 172.21.15.2:0/3840181984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3c8102f80 msgr2=0x7fb3c8195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3cd722700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3c8102f80 0x7fb3c8195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3cd722700 1 -- 172.21.15.2:0/3840181984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c81058a0 msgr2=0x7fb3c819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3cd722700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c81058a0 0x7fb3c819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3cd722700 1 -- 172.21.15.2:0/3840181984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b80095f0 con 0x7fb3c8100660 2023-06-23T18:31:27.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3ccf21700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3c8102f80 0x7fb3c8195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:27.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3cd722700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8100660 0x7fb3c81957a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9a0 tx=0x7fb3c400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:27.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3be7fc700 1 -- 172.21.15.2:0/3840181984 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4009790 con 0x7fb3c8100660 2023-06-23T18:31:27.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3be7fc700 1 -- 172.21.15.2:0/3840181984 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3c400cbd0 con 0x7fb3c8100660 2023-06-23T18:31:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3cf986700 1 -- 172.21.15.2:0/3840181984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c819d8d0 con 0x7fb3c8100660 2023-06-23T18:31:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3be7fc700 1 -- 172.21.15.2:0/3840181984 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4018680 con 0x7fb3c8100660 2023-06-23T18:31:27.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.060+0000 7fb3cf986700 1 -- 172.21.15.2:0/3840181984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c819dde0 con 0x7fb3c8100660 2023-06-23T18:31:27.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.062+0000 7fb3be7fc700 1 -- 172.21.15.2:0/3840181984 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb3c400cd40 con 0x7fb3c8100660 2023-06-23T18:31:27.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.063+0000 7fb3be7fc700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3b40a1150 0x7fb3b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.063+0000 7fb3be7fc700 1 -- 172.21.15.2:0/3840181984 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb3c4014070 con 0x7fb3c8100660 2023-06-23T18:31:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.063+0000 7fb3ccf21700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3b40a1150 0x7fb3b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3ccf21700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3b40a1150 0x7fb3b40a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80059d0 tx=0x7fb3b8005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3dbb0bb80 1 Processor -- start 2023-06-23T18:31:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3dbb0bb80 1 -- start start 2023-06-23T18:31:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b57c80 0x55a704b58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b585d0 0x55a704b58a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b59000 0x55a704b59470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704a5ccb0 con 0x55a704b57c80 2023-06-23T18:31:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704aafa40 con 0x55a704b59000 2023-06-23T18:31:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704aa8ed0 con 0x55a704b585d0 2023-06-23T18:31:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3cd722700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b57c80 0x55a704b58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3cd722700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b57c80 0x55a704b58090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55644/0 (socket says 172.21.15.2:55644) 2023-06-23T18:31:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3cd722700 1 -- 172.21.15.2:0/1355813813 learned_addr learned my addr 172.21.15.2:0/1355813813 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.064+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b59000 0x55a704b59470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3ccf21700 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b585d0 0x55a704b58a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3cd722700 1 -- 172.21.15.2:0/1355813813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b585d0 msgr2=0x55a704b58a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:27.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3cd722700 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b585d0 0x55a704b58a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3cd722700 1 -- 172.21.15.2:0/1355813813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b59000 msgr2=0x55a704b59470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:27.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3cd722700 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b59000 0x55a704b59470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3cd722700 1 -- 172.21.15.2:0/1355813813 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a70492a630 con 0x55a704b57c80 2023-06-23T18:31:27.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3cd722700 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b57c80 0x55a704b58090 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40a1ff0 tx=0x7fb3c4099b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d747f7fc1000d9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:27.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3b37fe700 1 -- 172.21.15.2:0/1355813813 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c409e100 con 0x55a704b57c80 2023-06-23T18:31:27.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3b37fe700 1 -- 172.21.15.2:0/1355813813 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3c40a2520 con 0x55a704b57c80 2023-06-23T18:31:27.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.065+0000 7fb3b37fe700 1 -- 172.21.15.2:0/1355813813 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c40a22f0 con 0x55a704b57c80 2023-06-23T18:31:27.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1355813813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b57c80 msgr2=0x55a704b58090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b57c80 0x55a704b58090 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40a1ff0 tx=0x7fb3c4099b30 comp rx=0 tx=0).stop 2023-06-23T18:31:27.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1355813813 shutdown_connections 2023-06-23T18:31:27.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b59000 0x55a704b59470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b57c80 0x55a704b58090 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1355813813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b585d0 0x55a704b58a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1355813813 >> 172.21.15.2:0/1355813813 conn(0x55a704b4d3f0 msgr2=0x55a704b52520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1355813813 shutdown_connections 2023-06-23T18:31:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1355813813 wait complete. 2023-06-23T18:31:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.066+0000 7fb3dbb0bb80 1 -- start start 2023-06-23T18:31:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3dbb0bb80 1 Processor -- start 2023-06-23T18:31:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b57c80 0x55a704b61500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b585d0 0x55a704b61a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b59000 0x55a704b68a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704a48ac0 con 0x55a704b585d0 2023-06-23T18:31:27.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704a487c0 con 0x55a704b57c80 2023-06-23T18:31:27.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a70492c920 con 0x55a704b59000 2023-06-23T18:31:27.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3dbb0bb80 10 client.0 _ll_register_callbacks cb 0x55a704a4e260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:27.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3ccf21700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b585d0 0x55a704b61a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3cd722700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b57c80 0x55a704b61500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3cdf23700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b59000 0x55a704b68a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3ccf21700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b585d0 0x55a704b61a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55652/0 (socket says 172.21.15.2:55652) 2023-06-23T18:31:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.067+0000 7fb3ccf21700 1 -- 172.21.15.2:0/3870547804 learned_addr learned my addr 172.21.15.2:0/3870547804 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3cdf23700 1 -- 172.21.15.2:0/3870547804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b57c80 msgr2=0x55a704b61500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b57c80 0x55a704b61500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3cdf23700 1 -- 172.21.15.2:0/3870547804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b585d0 msgr2=0x55a704b61a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b585d0 0x55a704b61a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3cdf23700 1 -- 172.21.15.2:0/3870547804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c819dde0 con 0x55a704b59000 2023-06-23T18:31:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3cd722700 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b57c80 0x55a704b61500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b59000 0x55a704b68a80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0007b20 tx=0x7fb3c0007e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d66f6604fd851dc server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:31:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/3870547804 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0012b40 con 0x55a704b59000 2023-06-23T18:31:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/3870547804 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb3c0012ca0 con 0x55a704b59000 2023-06-23T18:31:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/3870547804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a704b4abb0 con 0x55a704b59000 2023-06-23T18:31:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/3870547804 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3c0012e60 con 0x55a704b59000 2023-06-23T18:31:27.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3dbb0bb80 10 client.25801 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:27.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/3870547804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a7048d6de0 con 0x55a704b59000 2023-06-23T18:31:27.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.068+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/3870547804 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a704b4abb0 con 0x55a704b59000 2023-06-23T18:31:27.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.069+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/3870547804 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb3c0014040 con 0x55a704b59000 2023-06-23T18:31:27.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.069+0000 7fb3dbb0bb80 10 client.25801 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:27.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.069+0000 7fb3dbb0bb80 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.069+0000 7fb3dbb0bb80 20 client.25801 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.069+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a704b6d290 0x55a704b6f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.069+0000 7fb3dbb0bb80 4 client.25801 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.069+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/3870547804 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a704aa1eb0 con 0x55a704b6d290 2023-06-23T18:31:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.069+0000 7fb3cd722700 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a704b6d290 0x55a704b6f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3cd722700 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a704b6d290 0x55a704b6f6a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1f9cb5e25499d1a server_cookie=5bdae63791169902 in_seq=0 out_seq=0 2023-06-23T18:31:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3b0ff9700 10 client.25801 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/3870547804 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a704aa1eb0 con 0x55a704b6d290 2023-06-23T18:31:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3b0ff9700 10 client.25801 handle_command_reply: tid=0 2023-06-23T18:31:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 client.25801 shutdown 2023-06-23T18:31:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/3870547804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b59000 msgr2=0x55a704b68a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b59000 0x55a704b68a80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0007b20 tx=0x7fb3c0007e30 comp rx=0 tx=0).stop 2023-06-23T18:31:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/3870547804 shutdown_connections 2023-06-23T18:31:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a704b6d290 0x55a704b6f6a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b59000 0x55a704b68a80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b57c80 0x55a704b61500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3b0ff9700 0 client.25801 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/3870547804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b585d0 0x55a704b61a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.070+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/3870547804 >> 172.21.15.2:0/3870547804 conn(0x55a704b4d3f0 msgr2=0x55a704b51c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.071+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/3870547804 shutdown_connections 2023-06-23T18:31:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.071+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/3870547804 wait complete. 2023-06-23T18:31:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.071+0000 7fb3dbb0bb80 20 client.25801 trim_cache size 0 max 16384 2023-06-23T18:31:27.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.090+0000 7fb3dbb0bb80 1 Processor -- start 2023-06-23T18:31:27.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3dbb0bb80 1 -- start start 2023-06-23T18:31:27.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 0x55a704b69370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 0x55a704b68a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704b609b0 con 0x55a704b59000 2023-06-23T18:31:27.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704b61070 con 0x55a704b585d0 2023-06-23T18:31:27.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704b6a6f0 con 0x55a704b57c80 2023-06-23T18:31:27.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b614c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55662/0 (socket says 172.21.15.2:55662) 2023-06-23T18:31:27.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 -- 172.21.15.2:0/2531063923 learned_addr learned my addr 172.21.15.2:0/2531063923 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:27.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3ccf21700 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 0x55a704b68a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cd722700 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 0x55a704b69370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 -- 172.21.15.2:0/2531063923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 msgr2=0x55a704b69370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 0x55a704b69370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 -- 172.21.15.2:0/2531063923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 msgr2=0x55a704b68a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 0x55a704b68a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 -- 172.21.15.2:0/2531063923 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7048bfc60 con 0x55a704b59000 2023-06-23T18:31:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.091+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b614c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0007fd0 tx=0x7fb3c0012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58872d0cc3220e2a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:27.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3b3fff700 1 -- 172.21.15.2:0/2531063923 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0004c50 con 0x55a704b59000 2023-06-23T18:31:27.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3b3fff700 1 -- 172.21.15.2:0/2531063923 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3c000f040 con 0x55a704b59000 2023-06-23T18:31:27.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3b3fff700 1 -- 172.21.15.2:0/2531063923 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0012d30 con 0x55a704b59000 2023-06-23T18:31:27.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/2531063923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 msgr2=0x55a704b614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b614c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0007fd0 tx=0x7fb3c0012520 comp rx=0 tx=0).stop 2023-06-23T18:31:27.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/2531063923 shutdown_connections 2023-06-23T18:31:27.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b614c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 0x55a704b69370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/2531063923 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 0x55a704b68a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/2531063923 >> 172.21.15.2:0/2531063923 conn(0x55a704b4d3f0 msgr2=0x55a704b74000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:27.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.092+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/2531063923 shutdown_connections 2023-06-23T18:31:27.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.093+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/2531063923 wait complete. 2023-06-23T18:31:27.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.093+0000 7fb3dbb0bb80 1 -- start start 2023-06-23T18:31:27.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.093+0000 7fb3dbb0bb80 1 Processor -- start 2023-06-23T18:31:27.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.093+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 0x55a704b64b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.093+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 0x55a704b650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3dbb0bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b680f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704b68630 con 0x55a704b59000 2023-06-23T18:31:27.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704b687a0 con 0x55a704b585d0 2023-06-23T18:31:27.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3dbb0bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a704b68960 con 0x55a704b57c80 2023-06-23T18:31:27.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3dbb0bb80 10 client.0 _ll_register_callbacks cb 0x55a704a4e260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:27.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b680f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55666/0 (socket says 172.21.15.2:55666) 2023-06-23T18:31:27.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 -- 172.21.15.2:0/1943006540 learned_addr learned my addr 172.21.15.2:0/1943006540 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:27.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cd722700 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 0x55a704b64b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 -- 172.21.15.2:0/1943006540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 msgr2=0x55a704b64b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:27.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 0x55a704b64b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 -- 172.21.15.2:0/1943006540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 msgr2=0x55a704b650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 0x55a704b650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 -- 172.21.15.2:0/1943006540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7048d6de0 con 0x55a704b59000 2023-06-23T18:31:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3cdf23700 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b680f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009690 tx=0x7fb3c0004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb64935aa29e4d1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/1943006540 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0037070 con 0x55a704b59000 2023-06-23T18:31:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/1943006540 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb3c0012cc0 con 0x55a704b59000 2023-06-23T18:31:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.094+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1943006540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3c40a2520 con 0x55a704b59000 2023-06-23T18:31:27.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/1943006540 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3c00060b0 con 0x55a704b59000 2023-06-23T18:31:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3dbb0bb80 10 client.16062 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1943006540 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a7048fd250 con 0x55a704b59000 2023-06-23T18:31:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/1943006540 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3c0042020 con 0x55a704b59000 2023-06-23T18:31:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/1943006540 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb3c003a020 con 0x55a704b59000 2023-06-23T18:31:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3dbb0bb80 10 client.16062 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3dbb0bb80 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3dbb0bb80 20 client.16062 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:27.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a704bc2080 0x55a704bc4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3dbb0bb80 4 client.16062 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:31:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1943006540 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a704a64050 con 0x55a704bc2080 2023-06-23T18:31:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.095+0000 7fb3cd722700 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a704bc2080 0x55a704bc4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3b0ff9700 10 client.16062 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3cd722700 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a704bc2080 0x55a704bc4490 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a295f293baa282c server_cookie=c81a1aceb346c8ef in_seq=0 out_seq=0 2023-06-23T18:31:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3b0ff9700 1 -- 172.21.15.2:0/1943006540 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a704a64050 con 0x55a704bc2080 2023-06-23T18:31:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3b0ff9700 10 client.16062 handle_command_reply: tid=0 2023-06-23T18:31:27.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 client.16062 shutdown 2023-06-23T18:31:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1943006540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 msgr2=0x55a704b680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b680f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009690 tx=0x7fb3c0004ce0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1943006540 shutdown_connections 2023-06-23T18:31:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a704bc2080 0x55a704bc4490 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a704b59000 0x55a704b680f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3b0ff9700 0 client.16062 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:27.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a704b57c80 0x55a704b64b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 --2- 172.21.15.2:0/1943006540 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a704b585d0 0x55a704b650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.096+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1943006540 >> 172.21.15.2:0/1943006540 conn(0x55a704b4d3f0 msgr2=0x55a704b5be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1943006540 shutdown_connections 2023-06-23T18:31:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3dbb0bb80 1 -- 172.21.15.2:0/1943006540 wait complete. 2023-06-23T18:31:27.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3dbb0bb80 20 client.16062 trim_cache size 0 max 16384 2023-06-23T18:31:27.109 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:31:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 -- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3b40a1150 msgr2=0x7fb3b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3b40a1150 0x7fb3b40a3600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80059d0 tx=0x7fb3b8005940 comp rx=0 tx=0).stop 2023-06-23T18:31:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 -- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8100660 msgr2=0x7fb3c81957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:27.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8100660 0x7fb3c81957a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400e9a0 tx=0x7fb3c400ed60 comp rx=0 tx=0).stop 2023-06-23T18:31:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 -- 172.21.15.2:0/3840181984 shutdown_connections 2023-06-23T18:31:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3b40a1150 0x7fb3b40a3600 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3c81058a0 0x7fb3c819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb3c8100660 0x7fb3c81957a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 --2- 172.21.15.2:0/3840181984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3c8102f80 0x7fb3c8195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 -- 172.21.15.2:0/3840181984 >> 172.21.15.2:0/3840181984 conn(0x7fb3c80fa0e0 msgr2=0x7fb3c8101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:27.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 -- 172.21.15.2:0/3840181984 shutdown_connections 2023-06-23T18:31:27.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:27.097+0000 7fb3cf986700 1 -- 172.21.15.2:0/3840181984 wait complete. 2023-06-23T18:31:27.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:27 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1916513481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:31:27.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:27 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1916513481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:31:27.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:27 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1916513481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:31:28.115 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:31:28.116 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:31:28.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 -- 172.21.15.2:0/1274434187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77dc100720 msgr2=0x7f77dc106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 --2- 172.21.15.2:0/1274434187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77dc100720 0x7f77dc106fc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f77d400b0a0 tx=0x7f77d400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 -- 172.21.15.2:0/1274434187 shutdown_connections 2023-06-23T18:31:28.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 --2- 172.21.15.2:0/1274434187 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77dc100720 0x7f77dc106fc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 --2- 172.21.15.2:0/1274434187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc0fefc0 0x7f77dc0ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 --2- 172.21.15.2:0/1274434187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77dc0ffd70 0x7f77dc1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 -- 172.21.15.2:0/1274434187 >> 172.21.15.2:0/1274434187 conn(0x7f77dc0f9820 msgr2=0x7f77dc0fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 -- 172.21.15.2:0/1274434187 shutdown_connections 2023-06-23T18:31:28.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.258+0000 7f77e15a5700 1 -- 172.21.15.2:0/1274434187 wait complete. 2023-06-23T18:31:28.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77e15a5700 1 Processor -- start 2023-06-23T18:31:28.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77e15a5700 1 -- start start 2023-06-23T18:31:28.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77e15a5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77dc0fefc0 0x7f77dc194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77e15a5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77dc0ffd70 0x7f77dc194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77e15a5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc100720 0x7f77dc19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77e15a5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc19c3b0 con 0x7f77dc0fefc0 2023-06-23T18:31:28.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77e15a5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc19c4f0 con 0x7f77dc0ffd70 2023-06-23T18:31:28.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77e15a5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc19c680 con 0x7f77dc100720 2023-06-23T18:31:28.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77da7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77dc0ffd70 0x7f77dc194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc100720 0x7f77dc19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc100720 0x7f77dc19bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56020/0 (socket says 172.21.15.2:56020) 2023-06-23T18:31:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77da7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77dc0ffd70 0x7f77dc194d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46272/0 (socket says 172.21.15.2:46272) 2023-06-23T18:31:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77db7fe700 1 -- 172.21.15.2:0/2160539044 learned_addr learned my addr 172.21.15.2:0/2160539044 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.259+0000 7f77daffd700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77dc0fefc0 0x7f77dc194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77db7fe700 1 -- 172.21.15.2:0/2160539044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77dc0ffd70 msgr2=0x7f77dc194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77db7fe700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77dc0ffd70 0x7f77dc194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77db7fe700 1 -- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77dc0fefc0 msgr2=0x7f77dc194830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77db7fe700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77dc0fefc0 0x7f77dc194830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77db7fe700 1 -- 172.21.15.2:0/2160539044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d4009d00 con 0x7f77dc100720 2023-06-23T18:31:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77db7fe700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc100720 0x7f77dc19bde0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f77d400b780 tx=0x7f77d40078e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77c3fff700 1 -- 172.21.15.2:0/2160539044 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d400e040 con 0x7f77dc100720 2023-06-23T18:31:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77e15a5700 1 -- 172.21.15.2:0/2160539044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc0fdbb0 con 0x7f77dc100720 2023-06-23T18:31:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77c3fff700 1 -- 172.21.15.2:0/2160539044 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77d401e070 con 0x7f77dc100720 2023-06-23T18:31:28.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77c3fff700 1 -- 172.21.15.2:0/2160539044 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4025480 con 0x7f77dc100720 2023-06-23T18:31:28.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.260+0000 7f77e15a5700 1 -- 172.21.15.2:0/2160539044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc0fe0c0 con 0x7f77dc100720 2023-06-23T18:31:28.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.263+0000 7f77c3fff700 1 -- 172.21.15.2:0/2160539044 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f77d401a4c0 con 0x7f77dc100720 2023-06-23T18:31:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.263+0000 7f77c3fff700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77c40a1150 0x7f77c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.263+0000 7f77daffd700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77c40a1150 0x7f77c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.263+0000 7f77c3fff700 1 -- 172.21.15.2:0/2160539044 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f77d4020070 con 0x7f77dc100720 2023-06-23T18:31:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.264+0000 7f77daffd700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77c40a1150 0x7f77c40a3600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f77dc068540 tx=0x7f77cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.264+0000 7f77ed72ab80 1 Processor -- start 2023-06-23T18:31:28.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77ed72ab80 1 -- start start 2023-06-23T18:31:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73aaf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 0x55da73aaff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73ab0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da739afd30 con 0x55da73ab0490 2023-06-23T18:31:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73a02ac0 con 0x55da73aaf110 2023-06-23T18:31:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da739fbf50 con 0x55da73aafa60 2023-06-23T18:31:28.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73ab0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73ab0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55690/0 (socket says 172.21.15.2:55690) 2023-06-23T18:31:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 -- 172.21.15.2:0/3751795573 learned_addr learned my addr 172.21.15.2:0/3751795573 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77daffd700 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73aaf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77da7fc700 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 0x55da73aaff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 -- 172.21.15.2:0/3751795573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 msgr2=0x55da73aaff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 0x55da73aaff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 -- 172.21.15.2:0/3751795573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 msgr2=0x55da73aaf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73aaf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 -- 172.21.15.2:0/3751795573 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da7387d6b0 con 0x55da73ab0490 2023-06-23T18:31:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.265+0000 7f77db7fe700 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73ab0900 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f77d409cb70 tx=0x7f77d400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993a0d5a7f9f9460 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77c17fa700 1 -- 172.21.15.2:0/3751795573 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d40a0ea0 con 0x55da73ab0490 2023-06-23T18:31:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77c17fa700 1 -- 172.21.15.2:0/3751795573 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77d40a5230 con 0x55da73ab0490 2023-06-23T18:31:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77c17fa700 1 -- 172.21.15.2:0/3751795573 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d409c360 con 0x55da73ab0490 2023-06-23T18:31:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3751795573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 msgr2=0x55da73ab0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73ab0900 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f77d409cb70 tx=0x7f77d400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3751795573 shutdown_connections 2023-06-23T18:31:28.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73ab0900 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73aaf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3751795573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 0x55da73aaff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.266+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3751795573 >> 172.21.15.2:0/3751795573 conn(0x55da73aa0470 msgr2=0x55da73aa55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3751795573 shutdown_connections 2023-06-23T18:31:28.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3751795573 wait complete. 2023-06-23T18:31:28.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 -- start start 2023-06-23T18:31:28.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 Processor -- start 2023-06-23T18:31:28.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aaf110 0x55da73ab87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aafa60 0x55da73ab8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73abfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7399bb40 con 0x55da73ab0490 2023-06-23T18:31:28.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7399b840 con 0x55da73aafa60 2023-06-23T18:31:28.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da7387f9a0 con 0x55da73aaf110 2023-06-23T18:31:28.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77ed72ab80 10 client.0 _ll_register_callbacks cb 0x55da739a12e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:28.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aaf110 0x55da73ab87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73abfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aaf110 0x55da73ab87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56030/0 (socket says 172.21.15.2:56030) 2023-06-23T18:31:28.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77daffd700 1 -- 172.21.15.2:0/3081436657 learned_addr learned my addr 172.21.15.2:0/3081436657 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.267+0000 7f77da7fc700 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aafa60 0x55da73ab8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77db7fe700 1 -- 172.21.15.2:0/3081436657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aaf110 msgr2=0x55da73ab87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77db7fe700 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aaf110 0x55da73ab87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77db7fe700 1 -- 172.21.15.2:0/3081436657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aafa60 msgr2=0x55da73ab8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77db7fe700 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aafa60 0x55da73ab8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77db7fe700 1 -- 172.21.15.2:0/3081436657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc0fe0c0 con 0x55da73ab0490 2023-06-23T18:31:28.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77db7fe700 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73abfd60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f77d4099370 tx=0x7f77d401a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7d64f4faec0287 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77beff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:28.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77beff5700 1 -- 172.21.15.2:0/3081436657 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d40a0ea0 con 0x55da73ab0490 2023-06-23T18:31:28.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77beff5700 1 -- 172.21.15.2:0/3081436657 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f77d40bb180 con 0x55da73ab0490 2023-06-23T18:31:28.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3081436657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da73a9dc30 con 0x55da73ab0490 2023-06-23T18:31:28.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77beff5700 1 -- 172.21.15.2:0/3081436657 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d40a4fb0 con 0x55da73ab0490 2023-06-23T18:31:28.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77ed72ab80 10 client.16074 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:28.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3081436657 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55da73829e60 con 0x55da73ab0490 2023-06-23T18:31:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.268+0000 7f77beff5700 1 -- 172.21.15.2:0/3081436657 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77d4066020 con 0x55da73ab0490 2023-06-23T18:31:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.269+0000 7f77beff5700 1 -- 172.21.15.2:0/3081436657 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f77d4019070 con 0x55da73ab0490 2023-06-23T18:31:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.269+0000 7f77ed72ab80 10 client.16074 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.269+0000 7f77ed72ab80 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.269+0000 7f77ed72ab80 20 client.16074 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.269+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55da73ac4090 0x55da73ac64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.269+0000 7f77ed72ab80 4 client.16074 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:28.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.269+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3081436657 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da739f4f30 con 0x55da73ac4090 2023-06-23T18:31:28.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.269+0000 7f77daffd700 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55da73ac4090 0x55da73ac64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.270+0000 7f77daffd700 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55da73ac4090 0x55da73ac64a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5201600ee88ecc0d server_cookie=3c8a0eee258d1dd7 in_seq=0 out_seq=0 2023-06-23T18:31:28.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.270+0000 7f77beff5700 10 client.16074 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:28.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.270+0000 7f77beff5700 1 -- 172.21.15.2:0/3081436657 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da739f4f30 con 0x55da73ac4090 2023-06-23T18:31:28.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.270+0000 7f77beff5700 10 client.16074 handle_command_reply: tid=0 2023-06-23T18:31:28.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.270+0000 7f77ed72ab80 1 client.16074 shutdown 2023-06-23T18:31:28.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.270+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3081436657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 msgr2=0x55da73abfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.270+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73abfd60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f77d4099370 tx=0x7f77d401a980 comp rx=0 tx=0).stop 2023-06-23T18:31:28.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.270+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3081436657 shutdown_connections 2023-06-23T18:31:28.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55da73ac4090 0x55da73ac64a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73abfd60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77beff5700 0 client.16074 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:28.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aaf110 0x55da73ab87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3081436657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aafa60 0x55da73ab8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3081436657 >> 172.21.15.2:0/3081436657 conn(0x55da73aa0470 msgr2=0x55da73aa4cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3081436657 shutdown_connections 2023-06-23T18:31:28.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3081436657 wait complete. 2023-06-23T18:31:28.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.271+0000 7f77ed72ab80 20 client.16074 trim_cache size 0 max 16384 2023-06-23T18:31:28.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77ed72ab80 1 Processor -- start 2023-06-23T18:31:28.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77ed72ab80 1 -- start start 2023-06-23T18:31:28.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73ab87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73aafa60 0x55da73abfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73ab0490 0x55da73ab8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73ab8350 con 0x55da73aafa60 2023-06-23T18:31:28.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73a9fff0 con 0x55da73aaf110 2023-06-23T18:31:28.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73aa7030 con 0x55da73ab0490 2023-06-23T18:31:28.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77da7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73aafa60 0x55da73abfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77da7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73aafa60 0x55da73abfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55702/0 (socket says 172.21.15.2:55702) 2023-06-23T18:31:28.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77da7fc700 1 -- 172.21.15.2:0/3274612884 learned_addr learned my addr 172.21.15.2:0/3274612884 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77daffd700 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73ab87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77db7fe700 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73ab0490 0x55da73ab8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77da7fc700 1 -- 172.21.15.2:0/3274612884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73ab0490 msgr2=0x55da73ab8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77da7fc700 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73ab0490 0x55da73ab8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77da7fc700 1 -- 172.21.15.2:0/3274612884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 msgr2=0x55da73ab87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77da7fc700 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73ab87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.290+0000 7f77da7fc700 1 -- 172.21.15.2:0/3274612884 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da73aa1c10 con 0x55da73aafa60 2023-06-23T18:31:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.291+0000 7f77da7fc700 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73aafa60 0x55da73abfd20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009dd0 tx=0x7f77d0010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694ea65e2fdfe56f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.291+0000 7f77c1ffb700 1 -- 172.21.15.2:0/3274612884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0010d10 con 0x55da73aafa60 2023-06-23T18:31:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.291+0000 7f77c1ffb700 1 -- 172.21.15.2:0/3274612884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77d0018040 con 0x55da73aafa60 2023-06-23T18:31:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.291+0000 7f77c1ffb700 1 -- 172.21.15.2:0/3274612884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0003c70 con 0x55da73aafa60 2023-06-23T18:31:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3274612884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73aafa60 msgr2=0x55da73abfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73aafa60 0x55da73abfd20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009dd0 tx=0x7f77d0010710 comp rx=0 tx=0).stop 2023-06-23T18:31:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3274612884 shutdown_connections 2023-06-23T18:31:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73ab0490 0x55da73ab8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73ab87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/3274612884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73aafa60 0x55da73abfd20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3274612884 >> 172.21.15.2:0/3274612884 conn(0x55da73aa0470 msgr2=0x55da73ac0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3274612884 shutdown_connections 2023-06-23T18:31:28.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 -- 172.21.15.2:0/3274612884 wait complete. 2023-06-23T18:31:28.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 -- start start 2023-06-23T18:31:28.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.292+0000 7f77ed72ab80 1 Processor -- start 2023-06-23T18:31:28.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73abc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 0x55da73abcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77ed72ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73b17f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73abfec0 con 0x55da73ab0490 2023-06-23T18:31:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73ac0030 con 0x55da73aaf110 2023-06-23T18:31:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77ed72ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da73b18440 con 0x55da73aafa60 2023-06-23T18:31:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77ed72ab80 10 client.0 _ll_register_callbacks cb 0x55da739a12e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73b17f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73b17f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55710/0 (socket says 172.21.15.2:55710) 2023-06-23T18:31:28.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 -- 172.21.15.2:0/1546546755 learned_addr learned my addr 172.21.15.2:0/1546546755 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77daffd700 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73abc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77da7fc700 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 0x55da73abcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 -- 172.21.15.2:0/1546546755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 msgr2=0x55da73abcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 0x55da73abcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 -- 172.21.15.2:0/1546546755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 msgr2=0x55da73abc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73abc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 -- 172.21.15.2:0/1546546755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d0009a80 con 0x55da73ab0490 2023-06-23T18:31:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77db7fe700 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73b17f00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f77d4099320 tx=0x7f77d409b210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccc4777fba0c888 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77beff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77beff5700 1 -- 172.21.15.2:0/1546546755 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d409b070 con 0x55da73ab0490 2023-06-23T18:31:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.293+0000 7f77beff5700 1 -- 172.21.15.2:0/1546546755 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f77d4067080 con 0x55da73ab0490 2023-06-23T18:31:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 1 -- 172.21.15.2:0/1546546755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77d40a5230 con 0x55da73ab0490 2023-06-23T18:31:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77beff5700 1 -- 172.21.15.2:0/1546546755 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d409b280 con 0x55da73ab0490 2023-06-23T18:31:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 10 client.16080 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:28.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 1 -- 172.21.15.2:0/1546546755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55da73812ce0 con 0x55da73ab0490 2023-06-23T18:31:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77beff5700 1 -- 172.21.15.2:0/1546546755 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77d40a5230 con 0x55da73ab0490 2023-06-23T18:31:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77beff5700 1 -- 172.21.15.2:0/1546546755 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f77d4019070 con 0x55da73ab0490 2023-06-23T18:31:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 10 client.16080 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 20 client.16080 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55da73b197c0 0x55da73b1bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 4 client.16080 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:31:28.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.294+0000 7f77ed72ab80 1 -- 172.21.15.2:0/1546546755 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55da739b70d0 con 0x55da73b197c0 2023-06-23T18:31:28.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.295+0000 7f77daffd700 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55da73b197c0 0x55da73b1bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.295+0000 7f77beff5700 10 client.16080 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.295+0000 7f77daffd700 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55da73b197c0 0x55da73b1bba0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d37fb72980472f server_cookie=3e027d5732dcc4b3 in_seq=0 out_seq=0 2023-06-23T18:31:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.296+0000 7f77beff5700 1 -- 172.21.15.2:0/1546546755 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55da739b70d0 con 0x55da73b197c0 2023-06-23T18:31:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.296+0000 7f77beff5700 10 client.16080 handle_command_reply: tid=0 2023-06-23T18:31:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 client.16080 shutdown 2023-06-23T18:31:28.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 -- 172.21.15.2:0/1546546755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 msgr2=0x55da73b17f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73b17f00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f77d4099320 tx=0x7f77d409b210 comp rx=0 tx=0).stop 2023-06-23T18:31:28.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 -- 172.21.15.2:0/1546546755 shutdown_connections 2023-06-23T18:31:28.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55da73b197c0 0x55da73b1bba0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55da73ab0490 0x55da73b17f00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55da73aaf110 0x55da73abc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 --2- 172.21.15.2:0/1546546755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55da73aafa60 0x55da73abcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.297+0000 7f77ed72ab80 1 -- 172.21.15.2:0/1546546755 >> 172.21.15.2:0/1546546755 conn(0x55da73aa0470 msgr2=0x55da73ab32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.298+0000 7f77beff5700 0 client.16080 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:28.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.298+0000 7f77ed72ab80 1 -- 172.21.15.2:0/1546546755 shutdown_connections 2023-06-23T18:31:28.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.298+0000 7f77ed72ab80 1 -- 172.21.15.2:0/1546546755 wait complete. 2023-06-23T18:31:28.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77ed72ab80 20 client.16080 trim_cache size 0 max 16384 2023-06-23T18:31:28.309 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:31:28.309 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:31:28.310 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "fb12b61a-10f6-47c6-b369-5f276b191670", 2023-06-23T18:31:28.310 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:31:28.310 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:31:28.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 -- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77c40a1150 msgr2=0x7f77c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77c40a1150 0x7f77c40a3600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f77dc068540 tx=0x7f77cc009260 comp rx=0 tx=0).stop 2023-06-23T18:31:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 -- 172.21.15.2:0/2160539044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc100720 msgr2=0x7f77dc19bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc100720 0x7f77dc19bde0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f77d400b780 tx=0x7f77d40078e0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 -- 172.21.15.2:0/2160539044 shutdown_connections 2023-06-23T18:31:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f77dc100720 0x7f77dc19bde0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77dc0fefc0 0x7f77dc194830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77c40a1150 0x7f77c40a3600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 --2- 172.21.15.2:0/2160539044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77dc0ffd70 0x7f77dc194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 -- 172.21.15.2:0/2160539044 >> 172.21.15.2:0/2160539044 conn(0x7f77dc0f9820 msgr2=0x7f77dc0fbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 -- 172.21.15.2:0/2160539044 shutdown_connections 2023-06-23T18:31:28.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.299+0000 7f77e15a5700 1 -- 172.21.15.2:0/2160539044 wait complete. 2023-06-23T18:31:28.320 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:31:28.457 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:28 smithi002 ceph-mon[95947]: pgmap v579: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 49 MiB/s wr, 25 op/s 2023-06-23T18:31:28.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 -- 172.21.15.2:0/2937958260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0638100660 msgr2=0x7f0638102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 --2- 172.21.15.2:0/2937958260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0638100660 0x7f0638102a40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f062800b0a0 tx=0x7f062800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 -- 172.21.15.2:0/2937958260 shutdown_connections 2023-06-23T18:31:28.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 --2- 172.21.15.2:0/2937958260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06381058a0 0x7f0638107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 --2- 172.21.15.2:0/2937958260 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0638100660 0x7f0638102a40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 --2- 172.21.15.2:0/2937958260 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0638102f80 0x7f0638105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 -- 172.21.15.2:0/2937958260 >> 172.21.15.2:0/2937958260 conn(0x7f06380fa0e0 msgr2=0x7f06380fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 -- 172.21.15.2:0/2937958260 shutdown_connections 2023-06-23T18:31:28.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.476+0000 7f064006c700 1 -- 172.21.15.2:0/2937958260 wait complete. 2023-06-23T18:31:28.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.477+0000 7f064006c700 1 Processor -- start 2023-06-23T18:31:28.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.477+0000 7f064006c700 1 -- start start 2023-06-23T18:31:28.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.477+0000 7f064006c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0638100660 0x7f0638195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.477+0000 7f064006c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638102f80 0x7f0638195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.477+0000 7f064006c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06381058a0 0x7f063819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.477+0000 7f064006c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063819d4c0 con 0x7f06381058a0 2023-06-23T18:31:28.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.477+0000 7f064006c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063819d600 con 0x7f0638100660 2023-06-23T18:31:28.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.477+0000 7f064006c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063819d790 con 0x7f0638102f80 2023-06-23T18:31:28.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06381058a0 0x7f063819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063de08700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0638100660 0x7f0638195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06381058a0 0x7f063819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55722/0 (socket says 172.21.15.2:55722) 2023-06-23T18:31:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 -- 172.21.15.2:0/2285748722 learned_addr learned my addr 172.21.15.2:0/2285748722 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063d607700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638102f80 0x7f0638195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 -- 172.21.15.2:0/2285748722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638102f80 msgr2=0x7f0638195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638102f80 0x7f0638195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 -- 172.21.15.2:0/2285748722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0638100660 msgr2=0x7f0638195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0638100660 0x7f0638195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 -- 172.21.15.2:0/2285748722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630009520 con 0x7f06381058a0 2023-06-23T18:31:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063de08700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0638100660 0x7f0638195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f063e609700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06381058a0 0x7f063819cef0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f063000ea80 tx=0x7f063000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f0636ffd700 1 -- 172.21.15.2:0/2285748722 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300098a0 con 0x7f06381058a0 2023-06-23T18:31:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f0636ffd700 1 -- 172.21.15.2:0/2285748722 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f063000c950 con 0x7f06381058a0 2023-06-23T18:31:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f064006c700 1 -- 172.21.15.2:0/2285748722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0628009d00 con 0x7f06381058a0 2023-06-23T18:31:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f0636ffd700 1 -- 172.21.15.2:0/2285748722 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630003c50 con 0x7f06381058a0 2023-06-23T18:31:28.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.478+0000 7f064006c700 1 -- 172.21.15.2:0/2285748722 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063819ddf0 con 0x7f06381058a0 2023-06-23T18:31:28.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.481+0000 7f0636ffd700 1 -- 172.21.15.2:0/2285748722 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0630004220 con 0x7f06381058a0 2023-06-23T18:31:28.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.481+0000 7f0636ffd700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06200a11d0 0x7f06200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.482+0000 7f0636ffd700 1 -- 172.21.15.2:0/2285748722 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0630014070 con 0x7f06381058a0 2023-06-23T18:31:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.482+0000 7f063de08700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06200a11d0 0x7f06200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.482+0000 7f064c1f1b80 1 Processor -- start 2023-06-23T18:31:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.482+0000 7f063de08700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06200a11d0 0x7f06200a3680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0628009fd0 tx=0x7f062800b740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.482+0000 7f064c1f1b80 1 -- start start 2023-06-23T18:31:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71acc80 0x557fb71ad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ad5d0 0x557fb71ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 0x557fb71ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb70b1cb0 con 0x557fb71ad5d0 2023-06-23T18:31:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb7104a40 con 0x557fb71ae000 2023-06-23T18:31:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb70fded0 con 0x557fb71acc80 2023-06-23T18:31:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ad5d0 0x557fb71ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ad5d0 0x557fb71ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55728/0 (socket says 172.21.15.2:55728) 2023-06-23T18:31:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 -- 172.21.15.2:0/2575097369 learned_addr learned my addr 172.21.15.2:0/2575097369 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063e609700 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 0x557fb71ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063de08700 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71acc80 0x557fb71ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 -- 172.21.15.2:0/2575097369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71acc80 msgr2=0x557fb71ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71acc80 0x557fb71ad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 -- 172.21.15.2:0/2575097369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 msgr2=0x557fb71ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 0x557fb71ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 -- 172.21.15.2:0/2575097369 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fb6f7f630 con 0x557fb71ad5d0 2023-06-23T18:31:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063de08700 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71acc80 0x557fb71ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063e609700 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 0x557fb71ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:28.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.483+0000 7f063d607700 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ad5d0 0x557fb71ada70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f062c00bd70 tx=0x7f062c0106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80c340f122e1706 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.484+0000 7f061ffff700 1 -- 172.21.15.2:0/2575097369 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c010ec0 con 0x557fb71ad5d0 2023-06-23T18:31:28.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.484+0000 7f061ffff700 1 -- 172.21.15.2:0/2575097369 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f062c015070 con 0x557fb71ad5d0 2023-06-23T18:31:28.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.484+0000 7f061ffff700 1 -- 172.21.15.2:0/2575097369 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0116a0 con 0x557fb71ad5d0 2023-06-23T18:31:28.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.484+0000 7f064c1f1b80 1 -- 172.21.15.2:0/2575097369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ad5d0 msgr2=0x557fb71ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.484+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ad5d0 0x557fb71ada70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f062c00bd70 tx=0x7f062c0106b0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 -- 172.21.15.2:0/2575097369 shutdown_connections 2023-06-23T18:31:28.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 0x557fb71ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71acc80 0x557fb71ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/2575097369 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ad5d0 0x557fb71ada70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 -- 172.21.15.2:0/2575097369 >> 172.21.15.2:0/2575097369 conn(0x557fb71a23f0 msgr2=0x557fb71a7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 -- 172.21.15.2:0/2575097369 shutdown_connections 2023-06-23T18:31:28.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 -- 172.21.15.2:0/2575097369 wait complete. 2023-06-23T18:31:28.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 -- start start 2023-06-23T18:31:28.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.485+0000 7f064c1f1b80 1 Processor -- start 2023-06-23T18:31:28.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 0x557fb71b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb709dac0 con 0x557fb71ae000 2023-06-23T18:31:28.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb709d7c0 con 0x557fb71acc80 2023-06-23T18:31:28.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb6f81920 con 0x557fb71ad5d0 2023-06-23T18:31:28.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f064c1f1b80 10 client.0 _ll_register_callbacks cb 0x557fb70a3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:28.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063e609700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71bdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063e609700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71bdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55736/0 (socket says 172.21.15.2:55736) 2023-06-23T18:31:28.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063e609700 1 -- 172.21.15.2:0/4075043739 learned_addr learned my addr 172.21.15.2:0/4075043739 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063d607700 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063de08700 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 0x557fb71b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063d607700 1 -- 172.21.15.2:0/4075043739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 msgr2=0x557fb71b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063d607700 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 0x557fb71b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063d607700 1 -- 172.21.15.2:0/4075043739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 msgr2=0x557fb71bdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063d607700 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71bdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063d607700 1 -- 172.21.15.2:0/4075043739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c00b9a0 con 0x557fb71ad5d0 2023-06-23T18:31:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.486+0000 7f063e609700 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71bdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f063d607700 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71b6a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f062c00bd40 tx=0x7f062c010f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa2884590b90167 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f061d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:31:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f061d7fa700 1 -- 172.21.15.2:0/4075043739 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c01c890 con 0x557fb71ad5d0 2023-06-23T18:31:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 1 -- 172.21.15.2:0/4075043739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fb719fbb0 con 0x557fb71ad5d0 2023-06-23T18:31:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f061d7fa700 1 -- 172.21.15.2:0/4075043739 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f062c017070 con 0x557fb71ad5d0 2023-06-23T18:31:28.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f061d7fa700 1 -- 172.21.15.2:0/4075043739 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062c01c9f0 con 0x557fb71ad5d0 2023-06-23T18:31:28.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 10 client.25837 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:28.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 1 -- 172.21.15.2:0/4075043739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557fb6f2bde0 con 0x557fb71ad5d0 2023-06-23T18:31:28.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f061d7fa700 1 -- 172.21.15.2:0/4075043739 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557fb719fbb0 con 0x557fb71ad5d0 2023-06-23T18:31:28.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f061d7fa700 1 -- 172.21.15.2:0/4075043739 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f062c033500 con 0x557fb71ad5d0 2023-06-23T18:31:28.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 10 client.25837 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:28.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 10 client.25837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 20 client.25837 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557fb71c21a0 0x557fb71c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 4 client.25837 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.487+0000 7f064c1f1b80 1 -- 172.21.15.2:0/4075043739 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fb70f6eb0 con 0x557fb71c21a0 2023-06-23T18:31:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.488+0000 7f063de08700 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557fb71c21a0 0x557fb71c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.488+0000 7f061d7fa700 10 client.25837 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.488+0000 7f063de08700 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557fb71c21a0 0x557fb71c45b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9adea47a3288a550 server_cookie=35720f5150cfdc2e in_seq=0 out_seq=0 2023-06-23T18:31:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f061d7fa700 1 -- 172.21.15.2:0/4075043739 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557fb70f6eb0 con 0x557fb71c21a0 2023-06-23T18:31:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f061d7fa700 10 client.25837 handle_command_reply: tid=0 2023-06-23T18:31:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 client.25837 shutdown 2023-06-23T18:31:28.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 -- 172.21.15.2:0/4075043739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 msgr2=0x557fb71b6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71b6a70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f062c00bd40 tx=0x7f062c010f70 comp rx=0 tx=0).stop 2023-06-23T18:31:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 -- 172.21.15.2:0/4075043739 shutdown_connections 2023-06-23T18:31:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557fb71c21a0 0x557fb71c45b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71bdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f061d7fa700 0 client.25837 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:28.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 0x557fb71b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/4075043739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71b6a70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 -- 172.21.15.2:0/4075043739 >> 172.21.15.2:0/4075043739 conn(0x557fb71a23f0 msgr2=0x557fb71a6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 -- 172.21.15.2:0/4075043739 shutdown_connections 2023-06-23T18:31:28.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 1 -- 172.21.15.2:0/4075043739 wait complete. 2023-06-23T18:31:28.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.489+0000 7f064c1f1b80 20 client.25837 trim_cache size 0 max 16384 2023-06-23T18:31:28.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.508+0000 7f064c1f1b80 1 Processor -- start 2023-06-23T18:31:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.508+0000 7f064c1f1b80 1 -- start start 2023-06-23T18:31:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71acc80 0x557fb71b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 0x557fb71b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb71b60a0 con 0x557fb71acc80 2023-06-23T18:31:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb71a1f70 con 0x557fb71ae000 2023-06-23T18:31:28.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb71b41e0 con 0x557fb71ad5d0 2023-06-23T18:31:28.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f063de08700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71acc80 0x557fb71b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f063de08700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71acc80 0x557fb71b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55740/0 (socket says 172.21.15.2:55740) 2023-06-23T18:31:28.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f063de08700 1 -- 172.21.15.2:0/212884193 learned_addr learned my addr 172.21.15.2:0/212884193 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f063de08700 1 -- 172.21.15.2:0/212884193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 msgr2=0x557fb71bda70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:31:28.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f063de08700 1 --2- 172.21.15.2:0/212884193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71bda70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f063de08700 1 -- 172.21.15.2:0/212884193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 msgr2=0x557fb71b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f063de08700 1 --2- 172.21.15.2:0/212884193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 0x557fb71b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.509+0000 7f063de08700 1 -- 172.21.15.2:0/212884193 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fb6f14c60 con 0x557fb71acc80 2023-06-23T18:31:28.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.510+0000 7f063de08700 1 --2- 172.21.15.2:0/212884193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71acc80 0x557fb71b0f60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0628009470 tx=0x7f0628020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae0cd8dce2c78dd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.510+0000 7f0634ff9700 1 -- 172.21.15.2:0/212884193 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800e040 con 0x557fb71acc80 2023-06-23T18:31:28.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.510+0000 7f0634ff9700 1 -- 172.21.15.2:0/212884193 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0628003e90 con 0x557fb71acc80 2023-06-23T18:31:28.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.510+0000 7f0634ff9700 1 -- 172.21.15.2:0/212884193 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06280050d0 con 0x557fb71acc80 2023-06-23T18:31:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 -- 172.21.15.2:0/212884193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71acc80 msgr2=0x557fb71b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/212884193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71acc80 0x557fb71b0f60 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0628009470 tx=0x7f0628020000 comp rx=0 tx=0).stop 2023-06-23T18:31:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 -- 172.21.15.2:0/212884193 shutdown_connections 2023-06-23T18:31:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/212884193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71ae000 0x557fb71b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/212884193 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71acc80 0x557fb71b0f60 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/212884193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 -- 172.21.15.2:0/212884193 >> 172.21.15.2:0/212884193 conn(0x557fb71a23f0 msgr2=0x557fb71b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 -- 172.21.15.2:0/212884193 shutdown_connections 2023-06-23T18:31:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 -- 172.21.15.2:0/212884193 wait complete. 2023-06-23T18:31:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 -- start start 2023-06-23T18:31:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.511+0000 7f064c1f1b80 1 Processor -- start 2023-06-23T18:31:28.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 0x557fb71bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71ba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f064c1f1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb71bac60 con 0x557fb71ae000 2023-06-23T18:31:28.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb71bdc20 con 0x557fb71acc80 2023-06-23T18:31:28.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f064c1f1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb71bddb0 con 0x557fb71ad5d0 2023-06-23T18:31:28.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f064c1f1b80 10 client.0 _ll_register_callbacks cb 0x557fb70a3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:28.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063e609700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063e609700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71ba580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55750/0 (socket says 172.21.15.2:55750) 2023-06-23T18:31:28.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063e609700 1 -- 172.21.15.2:0/3620531635 learned_addr learned my addr 172.21.15.2:0/3620531635 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:28.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063d607700 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71ba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063de08700 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 0x557fb71bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063d607700 1 -- 172.21.15.2:0/3620531635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 msgr2=0x557fb71bd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063d607700 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 0x557fb71bd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063d607700 1 -- 172.21.15.2:0/3620531635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 msgr2=0x557fb71ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:28.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063d607700 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71ba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063d607700 1 -- 172.21.15.2:0/3620531635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0638108890 con 0x557fb71ad5d0 2023-06-23T18:31:28.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.512+0000 7f063e609700 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:28.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.513+0000 7f063d607700 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71ba040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f062c001eb0 tx=0x7f062c033ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9aad99360b9755d0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:28.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.513+0000 7f061d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:31:28.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.513+0000 7f061d7fa700 1 -- 172.21.15.2:0/3620531635 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c00b410 con 0x557fb71ad5d0 2023-06-23T18:31:28.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.513+0000 7f061d7fa700 1 -- 172.21.15.2:0/3620531635 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f062c017040 con 0x557fb71ad5d0 2023-06-23T18:31:28.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.513+0000 7f064c1f1b80 1 -- 172.21.15.2:0/3620531635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f062c015070 con 0x557fb71ad5d0 2023-06-23T18:31:28.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.513+0000 7f061d7fa700 1 -- 172.21.15.2:0/3620531635 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f062c033b80 con 0x557fb71ad5d0 2023-06-23T18:31:28.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.513+0000 7f064c1f1b80 10 client.25840 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:28.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.513+0000 7f064c1f1b80 1 -- 172.21.15.2:0/3620531635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x557fb6f52250 con 0x557fb71ad5d0 2023-06-23T18:31:28.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f061d7fa700 1 -- 172.21.15.2:0/3620531635 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f062c015070 con 0x557fb71ad5d0 2023-06-23T18:31:28.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f061d7fa700 1 -- 172.21.15.2:0/3620531635 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f062c01c690 con 0x557fb71ad5d0 2023-06-23T18:31:28.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f064c1f1b80 10 client.25840 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:28.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f064c1f1b80 10 client.25840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:28.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f064c1f1b80 20 client.25840 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:28.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557fb721fd40 0x557fb7222150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:28.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f064c1f1b80 4 client.25840 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:31:28.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f064c1f1b80 1 -- 172.21.15.2:0/3620531635 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557fb70b9050 con 0x557fb721fd40 2023-06-23T18:31:28.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.514+0000 7f063de08700 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557fb721fd40 0x557fb7222150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:28.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.515+0000 7f061d7fa700 10 client.25840 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:28.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.515+0000 7f063de08700 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557fb721fd40 0x557fb7222150 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8946015600edbd36 server_cookie=bc5627a50733b312 in_seq=0 out_seq=0 2023-06-23T18:31:28.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f061d7fa700 1 -- 172.21.15.2:0/3620531635 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557fb70b9050 con 0x557fb721fd40 2023-06-23T18:31:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f061d7fa700 10 client.25840 handle_command_reply: tid=0 2023-06-23T18:31:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 client.25840 shutdown 2023-06-23T18:31:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 -- 172.21.15.2:0/3620531635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 msgr2=0x557fb71ba040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71ba040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f062c001eb0 tx=0x7f062c033ab0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 -- 172.21.15.2:0/3620531635 shutdown_connections 2023-06-23T18:31:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557fb721fd40 0x557fb7222150 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557fb71ae000 0x557fb71ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557fb71acc80 0x557fb71bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 --2- 172.21.15.2:0/3620531635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557fb71ad5d0 0x557fb71ba040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 -- 172.21.15.2:0/3620531635 >> 172.21.15.2:0/3620531635 conn(0x557fb71a23f0 msgr2=0x557fb71b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f061d7fa700 0 client.25840 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:28.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 -- 172.21.15.2:0/3620531635 shutdown_connections 2023-06-23T18:31:28.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 1 -- 172.21.15.2:0/3620531635 wait complete. 2023-06-23T18:31:28.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.516+0000 7f064c1f1b80 20 client.25840 trim_cache size 0 max 16384 2023-06-23T18:31:28.529 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:31:28.529 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:31:28.529 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:31:28.529 INFO:teuthology.orchestra.run.smithi002.stdout: "fb12b61a-10f6-47c6-b369-5f276b191670": { 2023-06-23T18:31:28.529 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:31:28.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "fb12b61a-10f6-47c6-b369-5f276b191670", 2023-06-23T18:31:28.529 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:31:28.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:31:28.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:31:28.530 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:31:28.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 -- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06200a11d0 msgr2=0x7f06200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06200a11d0 0x7f06200a3680 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0628009fd0 tx=0x7f062800b740 comp rx=0 tx=0).stop 2023-06-23T18:31:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 -- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06381058a0 msgr2=0x7f063819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06381058a0 0x7f063819cef0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f063000ea80 tx=0x7f063000ed90 comp rx=0 tx=0).stop 2023-06-23T18:31:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 -- 172.21.15.2:0/2285748722 shutdown_connections 2023-06-23T18:31:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f06381058a0 0x7f063819cef0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f06200a11d0 0x7f06200a3680 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0638100660 0x7f0638195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 --2- 172.21.15.2:0/2285748722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0638102f80 0x7f0638195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.517+0000 7f064006c700 1 -- 172.21.15.2:0/2285748722 >> 172.21.15.2:0/2285748722 conn(0x7f06380fa0e0 msgr2=0x7f0638101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.518+0000 7f064006c700 1 -- 172.21.15.2:0/2285748722 shutdown_connections 2023-06-23T18:31:28.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:28.518+0000 7f064006c700 1 -- 172.21.15.2:0/2285748722 wait complete. 2023-06-23T18:31:28.536 INFO:tasks.cephfs.filesystem:scrub status for tag:fb12b61a-10f6-47c6-b369-5f276b191670 - {'path': '/', 'tag': 'fb12b61a-10f6-47c6-b369-5f276b191670', 'options': 'recursive,force'} 2023-06-23T18:31:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:28 smithi082 ceph-mon[97889]: pgmap v579: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 49 MiB/s wr, 25 op/s 2023-06-23T18:31:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:28 smithi017 ceph-mon[97472]: pgmap v579: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 49 MiB/s wr, 25 op/s 2023-06-23T18:31:29.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:29 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:31:29.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:29 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:29 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:31:29.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:29 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:31:29.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:29 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:31:29.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:29 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:29 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:29 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:29 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:31:29.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:29 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:29 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:31:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:29 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:31:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:29 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:31:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:29 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:29 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:29 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:29 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:31:29.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:29 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:29 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:31:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:29 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:31:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:29 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:31:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:29 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:29 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:29.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:29 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:31:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:30 smithi002 ceph-mon[95947]: pgmap v580: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 51 MiB/s wr, 27 op/s 2023-06-23T18:31:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:30 smithi082 ceph-mon[97889]: pgmap v580: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 51 MiB/s wr, 27 op/s 2023-06-23T18:31:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:30 smithi017 ceph-mon[97472]: pgmap v580: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 51 MiB/s wr, 27 op/s 2023-06-23T18:31:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:32 smithi002 ceph-mon[95947]: pgmap v581: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:31:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:32 smithi082 ceph-mon[97889]: pgmap v581: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:31:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:32 smithi017 ceph-mon[97472]: pgmap v581: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:31:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:34 smithi002 ceph-mon[95947]: pgmap v582: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 54 MiB/s wr, 30 op/s 2023-06-23T18:31:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:34 smithi082 ceph-mon[97889]: pgmap v582: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 54 MiB/s wr, 30 op/s 2023-06-23T18:31:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:34 smithi017 ceph-mon[97472]: pgmap v582: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 54 MiB/s wr, 30 op/s 2023-06-23T18:31:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:36 smithi002 ceph-mon[95947]: pgmap v583: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:31:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:36 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:31:36.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:36 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:31:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:36 smithi082 ceph-mon[97889]: pgmap v583: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:31:36.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:36 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:31:36.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:36 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:31:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:36 smithi017 ceph-mon[97472]: pgmap v583: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:31:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:36 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:31:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:36 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:31:37.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:37 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:31:37.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:37 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:31:37.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:37 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:31:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:38 smithi002 ceph-mon[95947]: pgmap v584: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 45 MiB/s wr, 23 op/s 2023-06-23T18:31:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:38 smithi082 ceph-mon[97889]: pgmap v584: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 45 MiB/s wr, 23 op/s 2023-06-23T18:31:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:38 smithi017 ceph-mon[97472]: pgmap v584: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 45 MiB/s wr, 23 op/s 2023-06-23T18:31:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:40 smithi002 ceph-mon[95947]: pgmap v585: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:31:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:40 smithi082 ceph-mon[97889]: pgmap v585: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:31:40.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:40 smithi017 ceph-mon[97472]: pgmap v585: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 49 MiB/s wr, 27 op/s 2023-06-23T18:31:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:42 smithi002 ceph-mon[95947]: pgmap v586: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 MiB/s wr, 26 op/s 2023-06-23T18:31:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:42 smithi082 ceph-mon[97889]: pgmap v586: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 MiB/s wr, 26 op/s 2023-06-23T18:31:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:42 smithi017 ceph-mon[97472]: pgmap v586: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 MiB/s wr, 26 op/s 2023-06-23T18:31:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:44 smithi002 ceph-mon[95947]: pgmap v587: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 MiB/s wr, 30 op/s 2023-06-23T18:31:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:44 smithi082 ceph-mon[97889]: pgmap v587: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 MiB/s wr, 30 op/s 2023-06-23T18:31:44.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:44 smithi017 ceph-mon[97472]: pgmap v587: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 MiB/s wr, 30 op/s 2023-06-23T18:31:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:46 smithi002 ceph-mon[95947]: pgmap v588: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:31:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:46 smithi082 ceph-mon[97889]: pgmap v588: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:31:46.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:46 smithi017 ceph-mon[97472]: pgmap v588: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 48 MiB/s wr, 27 op/s 2023-06-23T18:31:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:48 smithi002 ceph-mon[95947]: pgmap v589: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 MiB/s wr, 28 op/s 2023-06-23T18:31:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:48 smithi082 ceph-mon[97889]: pgmap v589: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 MiB/s wr, 28 op/s 2023-06-23T18:31:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:48 smithi017 ceph-mon[97472]: pgmap v589: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 MiB/s wr, 28 op/s 2023-06-23T18:31:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:50 smithi002 ceph-mon[95947]: pgmap v590: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 MiB/s wr, 30 op/s 2023-06-23T18:31:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:50 smithi082 ceph-mon[97889]: pgmap v590: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 MiB/s wr, 30 op/s 2023-06-23T18:31:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:50 smithi017 ceph-mon[97472]: pgmap v590: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 51 MiB/s wr, 30 op/s 2023-06-23T18:31:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:52 smithi002 ceph-mon[95947]: pgmap v591: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 MiB/s wr, 26 op/s 2023-06-23T18:31:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:52 smithi082 ceph-mon[97889]: pgmap v591: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 MiB/s wr, 26 op/s 2023-06-23T18:31:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:52 smithi017 ceph-mon[97472]: pgmap v591: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 46 MiB/s wr, 26 op/s 2023-06-23T18:31:54.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:54 smithi002 ceph-mon[95947]: pgmap v592: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 53 MiB/s wr, 30 op/s 2023-06-23T18:31:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:54 smithi082 ceph-mon[97889]: pgmap v592: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 53 MiB/s wr, 30 op/s 2023-06-23T18:31:54.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:54 smithi017 ceph-mon[97472]: pgmap v592: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 53 MiB/s wr, 30 op/s 2023-06-23T18:31:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:56 smithi002 ceph-mon[95947]: pgmap v593: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:31:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:56 smithi082 ceph-mon[97889]: pgmap v593: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:31:56.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:56 smithi017 ceph-mon[97472]: pgmap v593: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 48 MiB/s wr, 25 op/s 2023-06-23T18:31:58.537 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:31:58.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.692+0000 7fb8ed147700 1 -- 172.21.15.2:0/205537307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 msgr2=0x7fb8e80fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.692+0000 7fb8ed147700 1 --2- 172.21.15.2:0/205537307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 0x7fb8e80fee90 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009990 tx=0x7fb8d8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.693+0000 7fb8ed147700 1 -- 172.21.15.2:0/205537307 shutdown_connections 2023-06-23T18:31:58.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.693+0000 7fb8ed147700 1 --2- 172.21.15.2:0/205537307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8100600 0x7fb8e8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.693+0000 7fb8ed147700 1 --2- 172.21.15.2:0/205537307 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 0x7fb8e80fee90 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.693+0000 7fb8ed147700 1 --2- 172.21.15.2:0/205537307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e80ffc70 0x7fb8e81000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.693+0000 7fb8ed147700 1 -- 172.21.15.2:0/205537307 >> 172.21.15.2:0/205537307 conn(0x7fb8e80fa0e0 msgr2=0x7fb8e80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:58.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.693+0000 7fb8ed147700 1 -- 172.21.15.2:0/205537307 shutdown_connections 2023-06-23T18:31:58.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.693+0000 7fb8ed147700 1 -- 172.21.15.2:0/205537307 wait complete. 2023-06-23T18:31:58.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.693+0000 7fb8ed147700 1 Processor -- start 2023-06-23T18:31:58.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8ed147700 1 -- start start 2023-06-23T18:31:58.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8ed147700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 0x7fb8e8068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8ed147700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e80ffc70 0x7fb8e819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8ed147700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8100600 0x7fb8e819cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8ed147700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819d2b0 con 0x7fb8e80fea80 2023-06-23T18:31:58.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8ed147700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819d420 con 0x7fb8e8100600 2023-06-23T18:31:58.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8ed147700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e819d5e0 con 0x7fb8e80ffc70 2023-06-23T18:31:58.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 0x7fb8e8068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 0x7fb8e8068310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55770/0 (socket says 172.21.15.2:55770) 2023-06-23T18:31:58.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/2841612935 learned_addr learned my addr 172.21.15.2:0/2841612935 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:58.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e659c700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e80ffc70 0x7fb8e819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e759e700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8100600 0x7fb8e819cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/2841612935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e80ffc70 msgr2=0x7fb8e819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e80ffc70 0x7fb8e819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/2841612935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8100600 msgr2=0x7fb8e819cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8100600 0x7fb8e819cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/2841612935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d80095f0 con 0x7fb8e80fea80 2023-06-23T18:31:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.694+0000 7fb8e659c700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e80ffc70 0x7fb8e819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.695+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 0x7fb8e8068310 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80098e0 tx=0x7fb8d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.695+0000 7fb8cffff700 1 -- 172.21.15.2:0/2841612935 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d801d070 con 0x7fb8e80fea80 2023-06-23T18:31:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.695+0000 7fb8cffff700 1 -- 172.21.15.2:0/2841612935 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8d800bd50 con 0x7fb8e80fea80 2023-06-23T18:31:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.695+0000 7fb8cffff700 1 -- 172.21.15.2:0/2841612935 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d8022cf0 con 0x7fb8e80fea80 2023-06-23T18:31:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.695+0000 7fb8ed147700 1 -- 172.21.15.2:0/2841612935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e819d860 con 0x7fb8e80fea80 2023-06-23T18:31:58.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.695+0000 7fb8ed147700 1 -- 172.21.15.2:0/2841612935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e819dc80 con 0x7fb8e80fea80 2023-06-23T18:31:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.697+0000 7fb8cffff700 1 -- 172.21.15.2:0/2841612935 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb8d8022470 con 0x7fb8e80fea80 2023-06-23T18:31:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.698+0000 7fb8cffff700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb8d00a11d0 0x7fb8d00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.698+0000 7fb8e659c700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb8d00a11d0 0x7fb8d00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.698+0000 7fb8cffff700 1 -- 172.21.15.2:0/2841612935 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb8d80cded0 con 0x7fb8e80fea80 2023-06-23T18:31:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.699+0000 7fb8e659c700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb8d00a11d0 0x7fb8d00a3680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc011490 tx=0x7fb8dc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:58.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.699+0000 7fb8f92ccb80 1 Processor -- start 2023-06-23T18:31:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.699+0000 7fb8f92ccb80 1 -- start start 2023-06-23T18:31:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d68090 0x5592e0d684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d68e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d69880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0c68cb0 con 0x5592e0d69410 2023-06-23T18:31:58.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0cbba40 con 0x5592e0d689e0 2023-06-23T18:31:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0cb4ed0 con 0x5592e0d68090 2023-06-23T18:31:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d69880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d69880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55780/0 (socket says 172.21.15.2:55780) 2023-06-23T18:31:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 -- 172.21.15.2:0/1160708625 learned_addr learned my addr 172.21.15.2:0/1160708625 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d68090 0x5592e0d684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e659c700 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d68e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 -- 172.21.15.2:0/1160708625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d68090 msgr2=0x5592e0d684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:58.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d68090 0x5592e0d684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 -- 172.21.15.2:0/1160708625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 msgr2=0x5592e0d68e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d68e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 -- 172.21.15.2:0/1160708625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592e0b36630 con 0x5592e0d69410 2023-06-23T18:31:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e659c700 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d68e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8e759e700 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d69880 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0011680 tx=0x7fb8e0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9020f4e286d9105b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.700+0000 7fb8cd7fa700 1 -- 172.21.15.2:0/1160708625 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000e040 con 0x5592e0d69410 2023-06-23T18:31:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8cd7fa700 1 -- 172.21.15.2:0/1160708625 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8e000f040 con 0x5592e0d69410 2023-06-23T18:31:58.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8cd7fa700 1 -- 172.21.15.2:0/1160708625 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0004750 con 0x5592e0d69410 2023-06-23T18:31:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/1160708625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 msgr2=0x5592e0d69880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d69880 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0011680 tx=0x7fb8e0009920 comp rx=0 tx=0).stop 2023-06-23T18:31:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/1160708625 shutdown_connections 2023-06-23T18:31:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d69880 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d68090 0x5592e0d684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/1160708625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d68e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/1160708625 >> 172.21.15.2:0/1160708625 conn(0x5592e0d593f0 msgr2=0x5592e0d5e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/1160708625 shutdown_connections 2023-06-23T18:31:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.701+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/1160708625 wait complete. 2023-06-23T18:31:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 1 -- start start 2023-06-23T18:31:58.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 1 Processor -- start 2023-06-23T18:31:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d68090 0x5592e0d718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d71e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d78e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0c54ac0 con 0x5592e0d69410 2023-06-23T18:31:58.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0c547c0 con 0x5592e0d68090 2023-06-23T18:31:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0b38920 con 0x5592e0d689e0 2023-06-23T18:31:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8f92ccb80 10 client.0 _ll_register_callbacks cb 0x5592e0c5a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8e659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d71e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8e659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d71e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56124/0 (socket says 172.21.15.2:56124) 2023-06-23T18:31:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8e659c700 1 -- 172.21.15.2:0/73115531 learned_addr learned my addr 172.21.15.2:0/73115531 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.702+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d68090 0x5592e0d718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8e759e700 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d78e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8e659c700 1 -- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d68090 msgr2=0x5592e0d718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8e659c700 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d68090 0x5592e0d718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8e659c700 1 -- 172.21.15.2:0/73115531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 msgr2=0x5592e0d78e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8e659c700 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d78e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8e659c700 1 -- 172.21.15.2:0/73115531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e0011330 con 0x5592e0d689e0 2023-06-23T18:31:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d68090 0x5592e0d718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8e659c700 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d71e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0039d0 tx=0x7fb8dc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c5711207f097e4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:58.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8caff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:31:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8caff5700 1 -- 172.21.15.2:0/73115531 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00e970 con 0x5592e0d689e0 2023-06-23T18:31:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8caff5700 1 -- 172.21.15.2:0/73115531 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb8dc00ead0 con 0x5592e0d689e0 2023-06-23T18:31:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/73115531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592e0d56bb0 con 0x5592e0d689e0 2023-06-23T18:31:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8caff5700 1 -- 172.21.15.2:0/73115531 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8dc00ec90 con 0x5592e0d689e0 2023-06-23T18:31:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8f92ccb80 10 client.25852 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.703+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/73115531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5592e0ae2de0 con 0x5592e0d689e0 2023-06-23T18:31:58.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8caff5700 1 -- 172.21.15.2:0/73115531 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5592e0d56bb0 con 0x5592e0d689e0 2023-06-23T18:31:58.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8caff5700 1 -- 172.21.15.2:0/73115531 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb8dc033400 con 0x5592e0d689e0 2023-06-23T18:31:58.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8f92ccb80 10 client.25852 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:58.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8f92ccb80 10 client.25852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:58.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8f92ccb80 20 client.25852 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:58.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5592e0d7d750 0x5592e0d7fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8f92ccb80 4 client.25852 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:58.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/73115531 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592e0cadeb0 con 0x5592e0d7d750 2023-06-23T18:31:58.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.704+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5592e0d7d750 0x5592e0d7fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.705+0000 7fb8caff5700 10 client.25852 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:58.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.705+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5592e0d7d750 0x5592e0d7fb60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac7d49a1f92057a server_cookie=8fcb097bf68d90e2 in_seq=0 out_seq=0 2023-06-23T18:31:58.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.705+0000 7fb8caff5700 1 -- 172.21.15.2:0/73115531 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5592e0cadeb0 con 0x5592e0d7d750 2023-06-23T18:31:58.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.705+0000 7fb8caff5700 10 client.25852 handle_command_reply: tid=0 2023-06-23T18:31:58.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.705+0000 7fb8f92ccb80 1 client.25852 shutdown 2023-06-23T18:31:58.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/73115531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 msgr2=0x5592e0d71e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d71e30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0039d0 tx=0x7fb8dc0047c0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/73115531 shutdown_connections 2023-06-23T18:31:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5592e0d7d750 0x5592e0d7fb60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d69410 0x5592e0d78e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8caff5700 0 client.25852 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d68090 0x5592e0d718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/73115531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d71e30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/73115531 >> 172.21.15.2:0/73115531 conn(0x5592e0d593f0 msgr2=0x5592e0d5dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:58.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/73115531 shutdown_connections 2023-06-23T18:31:58.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/73115531 wait complete. 2023-06-23T18:31:58.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.706+0000 7fb8f92ccb80 20 client.25852 trim_cache size 0 max 16384 2023-06-23T18:31:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8f92ccb80 1 Processor -- start 2023-06-23T18:31:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8f92ccb80 1 -- start start 2023-06-23T18:31:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d69410 0x5592e0d6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0d727f0 con 0x5592e0d68090 2023-06-23T18:31:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0d70da0 con 0x5592e0d689e0 2023-06-23T18:31:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0d5fd50 con 0x5592e0d69410 2023-06-23T18:31:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d844c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55792/0 (socket says 172.21.15.2:55792) 2023-06-23T18:31:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/3823039368 learned_addr learned my addr 172.21.15.2:0/3823039368 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8e659c700 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.726+0000 7fb8e759e700 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d69410 0x5592e0d6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/3823039368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d69410 msgr2=0x5592e0d6c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d69410 0x5592e0d6c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/3823039368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 msgr2=0x5592e0d78d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d78d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/3823039368 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592e0acbc60 con 0x5592e0d68090 2023-06-23T18:31:58.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d844c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8097870 tx=0x7fb8d80cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85338c2de1cfba1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:58.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8cdffb700 1 -- 172.21.15.2:0/3823039368 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d809f3a0 con 0x5592e0d68090 2023-06-23T18:31:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8cdffb700 1 -- 172.21.15.2:0/3823039368 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8d80cbe90 con 0x5592e0d68090 2023-06-23T18:31:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.727+0000 7fb8cdffb700 1 -- 172.21.15.2:0/3823039368 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d809a860 con 0x5592e0d68090 2023-06-23T18:31:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/3823039368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 msgr2=0x5592e0d844c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d844c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8097870 tx=0x7fb8d80cd040 comp rx=0 tx=0).stop 2023-06-23T18:31:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/3823039368 shutdown_connections 2023-06-23T18:31:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d69410 0x5592e0d6c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d844c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/3823039368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d689e0 0x5592e0d78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/3823039368 >> 172.21.15.2:0/3823039368 conn(0x5592e0d593f0 msgr2=0x5592e0d5a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/3823039368 shutdown_connections 2023-06-23T18:31:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/3823039368 wait complete. 2023-06-23T18:31:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 -- start start 2023-06-23T18:31:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.728+0000 7fb8f92ccb80 1 Processor -- start 2023-06-23T18:31:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d74bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8f92ccb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d69410 0x5592e0d78110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0d78650 con 0x5592e0d68090 2023-06-23T18:31:58.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0d787c0 con 0x5592e0d69410 2023-06-23T18:31:58.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8f92ccb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592e0d78980 con 0x5592e0d689e0 2023-06-23T18:31:58.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8f92ccb80 10 client.0 _ll_register_callbacks cb 0x5592e0c5a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:58.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d74bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d74bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55798/0 (socket says 172.21.15.2:55798) 2023-06-23T18:31:58.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/236819472 learned_addr learned my addr 172.21.15.2:0/236819472 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:58.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e659c700 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e759e700 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d69410 0x5592e0d78110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/236819472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 msgr2=0x5592e0d750f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:58.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d750f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/236819472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d69410 msgr2=0x5592e0d78110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:58.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d69410 0x5592e0d78110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 -- 172.21.15.2:0/236819472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e819dc80 con 0x5592e0d68090 2023-06-23T18:31:58.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8e6d9d700 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d74bb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80cd010 tx=0x7fb8d8004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0bf7709118a04 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:58.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8caff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:58.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.729+0000 7fb8caff5700 1 -- 172.21.15.2:0/236819472 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d809aa60 con 0x5592e0d68090 2023-06-23T18:31:58.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8caff5700 1 -- 172.21.15.2:0/236819472 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb8d80a3730 con 0x5592e0d68090 2023-06-23T18:31:58.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/236819472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8e000f040 con 0x5592e0d68090 2023-06-23T18:31:58.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8caff5700 1 -- 172.21.15.2:0/236819472 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8d80a32a0 con 0x5592e0d68090 2023-06-23T18:31:58.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 10 client.16116 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:58.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/236819472 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5592e0b09250 con 0x5592e0d68090 2023-06-23T18:31:58.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8caff5700 1 -- 172.21.15.2:0/236819472 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8e000f040 con 0x5592e0d68090 2023-06-23T18:31:58.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8caff5700 1 -- 172.21.15.2:0/236819472 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb8d809a2d0 con 0x5592e0d68090 2023-06-23T18:31:58.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 10 client.16116 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:58.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 10 client.16116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:58.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 20 client.16116 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:58.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5592e0dd21f0 0x5592e0dd45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 4 client.16116 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:31:58.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/236819472 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5592e0c70050 con 0x5592e0dd21f0 2023-06-23T18:31:58.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.730+0000 7fb8e659c700 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5592e0dd21f0 0x5592e0dd45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8caff5700 10 client.16116 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:58.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8e659c700 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5592e0dd21f0 0x5592e0dd45d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afbf8c5ad233e4c5 server_cookie=9447d4a39e958567 in_seq=0 out_seq=0 2023-06-23T18:31:58.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8caff5700 1 -- 172.21.15.2:0/236819472 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5592e0c70050 con 0x5592e0dd21f0 2023-06-23T18:31:58.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8caff5700 10 client.16116 handle_command_reply: tid=0 2023-06-23T18:31:58.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 client.16116 shutdown 2023-06-23T18:31:58.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/236819472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 msgr2=0x5592e0d74bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d74bb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80cd010 tx=0x7fb8d8004320 comp rx=0 tx=0).stop 2023-06-23T18:31:58.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/236819472 shutdown_connections 2023-06-23T18:31:58.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5592e0dd21f0 0x5592e0dd45d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5592e0d69410 0x5592e0d78110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5592e0d68090 0x5592e0d74bb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8caff5700 0 client.16116 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:58.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 --2- 172.21.15.2:0/236819472 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5592e0d689e0 0x5592e0d750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/236819472 >> 172.21.15.2:0/236819472 conn(0x5592e0d593f0 msgr2=0x5592e0d844c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:58.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/236819472 shutdown_connections 2023-06-23T18:31:58.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.731+0000 7fb8f92ccb80 1 -- 172.21.15.2:0/236819472 wait complete. 2023-06-23T18:31:58.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8f92ccb80 20 client.16116 trim_cache size 0 max 16384 2023-06-23T18:31:58.746 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:31:58.746 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:31:58.747 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:31:58.747 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:31:58.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 -- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb8d00a11d0 msgr2=0x7fb8d00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb8d00a11d0 0x7fb8d00a3680 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc011490 tx=0x7fb8dc009260 comp rx=0 tx=0).stop 2023-06-23T18:31:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 -- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 msgr2=0x7fb8e8068310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 0x7fb8e8068310 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb8d80098e0 tx=0x7fb8d8004730 comp rx=0 tx=0).stop 2023-06-23T18:31:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 -- 172.21.15.2:0/2841612935 shutdown_connections 2023-06-23T18:31:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb8d00a11d0 0x7fb8d00a3680 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb8e8100600 0x7fb8e819cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb8e80fea80 0x7fb8e8068310 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 --2- 172.21.15.2:0/2841612935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e80ffc70 0x7fb8e819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 -- 172.21.15.2:0/2841612935 >> 172.21.15.2:0/2841612935 conn(0x7fb8e80fa0e0 msgr2=0x7fb8e81031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 -- 172.21.15.2:0/2841612935 shutdown_connections 2023-06-23T18:31:58.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.732+0000 7fb8ed147700 1 -- 172.21.15.2:0/2841612935 wait complete. 2023-06-23T18:31:58.752 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:31:58 smithi002 ceph-mon[95947]: pgmap v594: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:31:58.753 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:31:58.753 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:31:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:31:58 smithi082 ceph-mon[97889]: pgmap v594: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:31:58.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.893+0000 7f3425e40700 1 -- 172.21.15.2:0/2329568897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200fff80 msgr2=0x7f3420100390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 --2- 172.21.15.2:0/2329568897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200fff80 0x7f3420100390 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f3410009990 tx=0x7f3410009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 -- 172.21.15.2:0/2329568897 shutdown_connections 2023-06-23T18:31:58.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 --2- 172.21.15.2:0/2329568897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34200ff580 0x7f3420106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 --2- 172.21.15.2:0/2329568897 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200fff80 0x7f3420100390 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 --2- 172.21.15.2:0/2329568897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34200febd0 0x7f34200ff040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 -- 172.21.15.2:0/2329568897 >> 172.21.15.2:0/2329568897 conn(0x7f34200f9770 msgr2=0x7f34200fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:58.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 -- 172.21.15.2:0/2329568897 shutdown_connections 2023-06-23T18:31:58.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 -- 172.21.15.2:0/2329568897 wait complete. 2023-06-23T18:31:58.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.894+0000 7f3425e40700 1 Processor -- start 2023-06-23T18:31:58.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f3425e40700 1 -- start start 2023-06-23T18:31:58.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f3425e40700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200febd0 0x7f34201956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f3425e40700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34200ff580 0x7f3420195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f3425e40700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34200fff80 0x7f342019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f3425e40700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342019d230 con 0x7f34200febd0 2023-06-23T18:31:58.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f3425e40700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342019d370 con 0x7f34200fff80 2023-06-23T18:31:58.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f3425e40700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342019d500 con 0x7f34200ff580 2023-06-23T18:31:58.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200febd0 0x7f34201956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200febd0 0x7f34201956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55808/0 (socket says 172.21.15.2:55808) 2023-06-23T18:31:58.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 -- 172.21.15.2:0/4087795295 learned_addr learned my addr 172.21.15.2:0/4087795295 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:58.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341effd700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34200ff580 0x7f3420195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341ffff700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34200fff80 0x7f342019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 -- 172.21.15.2:0/4087795295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34200ff580 msgr2=0x7f3420195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34200ff580 0x7f3420195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 -- 172.21.15.2:0/4087795295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34200fff80 msgr2=0x7f342019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34200fff80 0x7f342019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 -- 172.21.15.2:0/4087795295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34100095f0 con 0x7f34200febd0 2023-06-23T18:31:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.895+0000 7f341f7fe700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200febd0 0x7f34201956b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f34100059d0 tx=0x7f3410004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:58.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.896+0000 7f341cff9700 1 -- 172.21.15.2:0/4087795295 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341001d070 con 0x7f34200febd0 2023-06-23T18:31:58.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.896+0000 7f341cff9700 1 -- 172.21.15.2:0/4087795295 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f341000f460 con 0x7f34200febd0 2023-06-23T18:31:58.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.896+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342019d780 con 0x7f34200febd0 2023-06-23T18:31:58.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.896+0000 7f341cff9700 1 -- 172.21.15.2:0/4087795295 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410021670 con 0x7f34200febd0 2023-06-23T18:31:58.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.896+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342019dc30 con 0x7f34200febd0 2023-06-23T18:31:58.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.897+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3420106bd0 con 0x7f34200febd0 2023-06-23T18:31:58.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.898+0000 7f341cff9700 1 -- 172.21.15.2:0/4087795295 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f341000f5d0 con 0x7f34200febd0 2023-06-23T18:31:58.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.899+0000 7f341cff9700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34080a1150 0x7f34080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:58.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.899+0000 7f341effd700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34080a1150 0x7f34080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:58.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.899+0000 7f341cff9700 1 -- 172.21.15.2:0/4087795295 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f34100cdf50 con 0x7f34200febd0 2023-06-23T18:31:58.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.900+0000 7f341effd700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34080a1150 0x7f34080a3600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3414009c60 tx=0x7f3414009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:58.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:58.902+0000 7f341cff9700 1 -- 172.21.15.2:0/4087795295 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f3410097730 con 0x7f34200febd0 2023-06-23T18:31:58.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:31:58 smithi017 ceph-mon[97472]: pgmap v594: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:31:59.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.229+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f342002c880 con 0x7f34200febd0 2023-06-23T18:31:59.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.230+0000 7f341cff9700 1 -- 172.21.15.2:0/4087795295 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10780 (secure 0 0 0) 0x7f341009ad50 con 0x7f34200febd0 2023-06-23T18:31:59.236 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:31:59.236 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:30:26.353535+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:31:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34080a1150 msgr2=0x7f34080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34080a1150 0x7f34080a3600 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3414009c60 tx=0x7f3414009260 comp rx=0 tx=0).stop 2023-06-23T18:31:59.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200febd0 msgr2=0x7f34201956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200febd0 0x7f34201956b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f34100059d0 tx=0x7f3410004790 comp rx=0 tx=0).stop 2023-06-23T18:31:59.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 shutdown_connections 2023-06-23T18:31:59.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34080a1150 0x7f34080a3600 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34200fff80 0x7f342019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f34200febd0 0x7f34201956b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 --2- 172.21.15.2:0/4087795295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f34200ff580 0x7f3420195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 >> 172.21.15.2:0/4087795295 conn(0x7f34200f9770 msgr2=0x7f34200fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 shutdown_connections 2023-06-23T18:31:59.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.232+0000 7f3425e40700 1 -- 172.21.15.2:0/4087795295 wait complete. 2023-06-23T18:31:59.242 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 32 2023-06-23T18:31:59.251 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:31:59.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.388+0000 7fca92994700 1 -- 172.21.15.2:0/3238729479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0ffc70 msgr2=0x7fca8c1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.388+0000 7fca92994700 1 --2- 172.21.15.2:0/3238729479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0ffc70 0x7fca8c1000c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fca80009990 tx=0x7fca80009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.388+0000 7fca92994700 1 -- 172.21.15.2:0/3238729479 shutdown_connections 2023-06-23T18:31:59.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.388+0000 7fca92994700 1 --2- 172.21.15.2:0/3238729479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca8c100600 0x7fca8c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.388+0000 7fca92994700 1 --2- 172.21.15.2:0/3238729479 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca8c0fea80 0x7fca8c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.388+0000 7fca92994700 1 --2- 172.21.15.2:0/3238729479 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0ffc70 0x7fca8c1000c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.388+0000 7fca92994700 1 -- 172.21.15.2:0/3238729479 >> 172.21.15.2:0/3238729479 conn(0x7fca8c0fa0e0 msgr2=0x7fca8c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.388+0000 7fca92994700 1 -- 172.21.15.2:0/3238729479 shutdown_connections 2023-06-23T18:31:59.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.389+0000 7fca92994700 1 -- 172.21.15.2:0/3238729479 wait complete. 2023-06-23T18:31:59.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.389+0000 7fca92994700 1 Processor -- start 2023-06-23T18:31:59.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.389+0000 7fca92994700 1 -- start start 2023-06-23T18:31:59.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca92994700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0fea80 0x7fca8c068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca92994700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca8c0ffc70 0x7fca8c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca92994700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca8c100600 0x7fca8c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca92994700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19d300 con 0x7fca8c0fea80 2023-06-23T18:31:59.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca92994700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19d470 con 0x7fca8c100600 2023-06-23T18:31:59.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca92994700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c19d630 con 0x7fca8c0ffc70 2023-06-23T18:31:59.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0fea80 0x7fca8c068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0fea80 0x7fca8c068370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55822/0 (socket says 172.21.15.2:55822) 2023-06-23T18:31:59.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 -- 172.21.15.2:0/2562183888 learned_addr learned my addr 172.21.15.2:0/2562183888 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca90f31700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca8c100600 0x7fca8c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca8c0ffc70 0x7fca8c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 -- 172.21.15.2:0/2562183888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca8c0ffc70 msgr2=0x7fca8c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca8c0ffc70 0x7fca8c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 -- 172.21.15.2:0/2562183888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca8c100600 msgr2=0x7fca8c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca8c100600 0x7fca8c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 -- 172.21.15.2:0/2562183888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca800095f0 con 0x7fca8c0fea80 2023-06-23T18:31:59.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.390+0000 7fca8bfff700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0fea80 0x7fca8c068370 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d980 tx=0x7fca7c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.391+0000 7fca897fa700 1 -- 172.21.15.2:0/2562183888 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00f890 con 0x7fca8c0fea80 2023-06-23T18:31:59.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.391+0000 7fca897fa700 1 -- 172.21.15.2:0/2562183888 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca7c00fe80 con 0x7fca8c0fea80 2023-06-23T18:31:59.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.391+0000 7fca92994700 1 -- 172.21.15.2:0/2562183888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca8c19d910 con 0x7fca8c0fea80 2023-06-23T18:31:59.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.391+0000 7fca897fa700 1 -- 172.21.15.2:0/2562183888 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c019360 con 0x7fca8c0fea80 2023-06-23T18:31:59.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.391+0000 7fca92994700 1 -- 172.21.15.2:0/2562183888 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca8c19de20 con 0x7fca8c0fea80 2023-06-23T18:31:59.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.394+0000 7fca9eb19b80 1 Processor -- start 2023-06-23T18:31:59.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.394+0000 7fca9eb19b80 1 -- start start 2023-06-23T18:31:59.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e43a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e44470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992d47cb0 con 0x563992e42c80 2023-06-23T18:31:59.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992d9aa40 con 0x563992e435d0 2023-06-23T18:31:59.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992d93ed0 con 0x563992e44000 2023-06-23T18:31:59.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e43090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55828/0 (socket says 172.21.15.2:55828) 2023-06-23T18:31:59.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 -- 172.21.15.2:0/886127451 learned_addr learned my addr 172.21.15.2:0/886127451 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca8bfff700 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e43a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e44470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 -- 172.21.15.2:0/886127451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 msgr2=0x563992e44470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e44470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 -- 172.21.15.2:0/886127451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 msgr2=0x563992e43a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e43a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 -- 172.21.15.2:0/886127451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563992c15630 con 0x563992e42c80 2023-06-23T18:31:59.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca90f31700 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e43090 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fca84003fa0 tx=0x7fca84009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda844ad6dcc78fa server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.395+0000 7fca727fc700 1 -- 172.21.15.2:0/886127451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84019070 con 0x563992e42c80 2023-06-23T18:31:59.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca727fc700 1 -- 172.21.15.2:0/886127451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca8400f040 con 0x563992e42c80 2023-06-23T18:31:59.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca727fc700 1 -- 172.21.15.2:0/886127451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca84007a30 con 0x563992e42c80 2023-06-23T18:31:59.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca9eb19b80 1 -- 172.21.15.2:0/886127451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 msgr2=0x563992e43090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e43090 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fca84003fa0 tx=0x7fca84009ba0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca9eb19b80 1 -- 172.21.15.2:0/886127451 shutdown_connections 2023-06-23T18:31:59.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e44470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e43090 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/886127451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e43a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.396+0000 7fca9eb19b80 1 -- 172.21.15.2:0/886127451 >> 172.21.15.2:0/886127451 conn(0x563992e383f0 msgr2=0x563992e3d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.397+0000 7fca897fa700 1 -- 172.21.15.2:0/2562183888 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fca7c00f9f0 con 0x7fca8c0fea80 2023-06-23T18:31:59.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.397+0000 7fca9eb19b80 1 -- 172.21.15.2:0/886127451 shutdown_connections 2023-06-23T18:31:59.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.397+0000 7fca9eb19b80 1 -- 172.21.15.2:0/886127451 wait complete. 2023-06-23T18:31:59.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.397+0000 7fca9eb19b80 1 -- start start 2023-06-23T18:31:59.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.397+0000 7fca9eb19b80 1 Processor -- start 2023-06-23T18:31:59.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca897fa700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fca740a1060 0x7fca740a3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca90f31700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fca740a1060 0x7fca740a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e42c80 0x563992e4c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e4ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca897fa700 1 -- 172.21.15.2:0/2562183888 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca7c0d0b90 con 0x7fca8c0fea80 2023-06-23T18:31:59.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e44000 0x563992e53a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992d33ac0 con 0x563992e44000 2023-06-23T18:31:59.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992d337c0 con 0x563992e435d0 2023-06-23T18:31:59.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992c17920 con 0x563992e42c80 2023-06-23T18:31:59.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca9eb19b80 10 client.0 _ll_register_callbacks cb 0x563992d39260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:59.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca90f31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e44000 0x563992e53a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca90f31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e44000 0x563992e53a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55838/0 (socket says 172.21.15.2:55838) 2023-06-23T18:31:59.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca90f31700 1 -- 172.21.15.2:0/1208314525 learned_addr learned my addr 172.21.15.2:0/1208314525 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca8bfff700 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e42c80 0x563992e4c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e4ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca8bfff700 1 -- 172.21.15.2:0/1208314525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 msgr2=0x563992e4ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca8bfff700 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e4ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca90f31700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fca740a1060 0x7fca740a3510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fca840046d0 tx=0x7fca84004700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca8bfff700 1 -- 172.21.15.2:0/1208314525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e44000 msgr2=0x563992e53a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca8bfff700 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e44000 0x563992e53a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca8bfff700 1 -- 172.21.15.2:0/1208314525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca84003bd0 con 0x563992e42c80 2023-06-23T18:31:59.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.398+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e4ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:59.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca8bfff700 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e42c80 0x563992e4c500 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0ce690 tx=0x7fca7c0106e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55a300552a4b37a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:31:59.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1208314525 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c09d6d0 con 0x563992e42c80 2023-06-23T18:31:59.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1208314525 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca7c00fca0 con 0x563992e42c80 2023-06-23T18:31:59.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 1 -- 172.21.15.2:0/1208314525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563992e35bb0 con 0x563992e42c80 2023-06-23T18:31:59.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1208314525 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca7c09dfe0 con 0x563992e42c80 2023-06-23T18:31:59.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 10 client.25876 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:59.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 1 -- 172.21.15.2:0/1208314525 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563992bc1de0 con 0x563992e42c80 2023-06-23T18:31:59.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1208314525 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563992e35bb0 con 0x563992e42c80 2023-06-23T18:31:59.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1208314525 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca7c0a5fa0 con 0x563992e42c80 2023-06-23T18:31:59.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 10 client.25876 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:59.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 10 client.25876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:31:59.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 20 client.25876 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:59.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x563992e58140 0x563992e5a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 4 client.25876 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:59.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.399+0000 7fca9eb19b80 1 -- 172.21.15.2:0/1208314525 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563992d8ceb0 con 0x563992e58140 2023-06-23T18:31:59.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.400+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x563992e58140 0x563992e5a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.400+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x563992e58140 0x563992e5a550 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca444b53ea4d223f server_cookie=516e33ab6e4d0c33 in_seq=0 out_seq=0 2023-06-23T18:31:59.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.400+0000 7fca6b7fe700 10 client.25876 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:31:59.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca6b7fe700 1 -- 172.21.15.2:0/1208314525 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563992d8ceb0 con 0x563992e58140 2023-06-23T18:31:59.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca6b7fe700 10 client.25876 handle_command_reply: tid=0 2023-06-23T18:31:59.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 client.25876 shutdown 2023-06-23T18:31:59.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 -- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e42c80 msgr2=0x563992e4c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e42c80 0x563992e4c500 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fca7c0ce690 tx=0x7fca7c0106e0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 -- 172.21.15.2:0/1208314525 shutdown_connections 2023-06-23T18:31:59.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x563992e58140 0x563992e5a550 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e44000 0x563992e53a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e42c80 0x563992e4c500 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca6b7fe700 0 client.25876 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:31:59.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/1208314525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e435d0 0x563992e4ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 -- 172.21.15.2:0/1208314525 >> 172.21.15.2:0/1208314525 conn(0x563992e383f0 msgr2=0x563992e3cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 -- 172.21.15.2:0/1208314525 shutdown_connections 2023-06-23T18:31:59.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 1 -- 172.21.15.2:0/1208314525 wait complete. 2023-06-23T18:31:59.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.401+0000 7fca9eb19b80 20 client.25876 trim_cache size 0 max 16384 2023-06-23T18:31:59.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca9eb19b80 1 Processor -- start 2023-06-23T18:31:59.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca9eb19b80 1 -- start start 2023-06-23T18:31:59.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e4a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e435d0 0x563992e4a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e44000 0x563992e53a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992e4b9b0 con 0x563992e42c80 2023-06-23T18:31:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992e3e210 con 0x563992e44000 2023-06-23T18:31:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992e479b0 con 0x563992e435d0 2023-06-23T18:31:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e4a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca8bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e4a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55844/0 (socket says 172.21.15.2:55844) 2023-06-23T18:31:59.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca8bfff700 1 -- 172.21.15.2:0/3144673889 learned_addr learned my addr 172.21.15.2:0/3144673889 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e435d0 0x563992e4a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.416+0000 7fca90f31700 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e44000 0x563992e53a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca8bfff700 1 -- 172.21.15.2:0/3144673889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e435d0 msgr2=0x563992e4a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca8bfff700 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e435d0 0x563992e4a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca8bfff700 1 -- 172.21.15.2:0/3144673889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e44000 msgr2=0x563992e53a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca8bfff700 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e44000 0x563992e53a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca8bfff700 1 -- 172.21.15.2:0/3144673889 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563992baac60 con 0x563992e42c80 2023-06-23T18:31:59.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca90f31700 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e44000 0x563992e53a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:59.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca8bfff700 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e4a310 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca7c09aa50 tx=0x7fca7c09c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1be5c072a5ef2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca72ffd700 1 -- 172.21.15.2:0/3144673889 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c014070 con 0x563992e42c80 2023-06-23T18:31:59.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca72ffd700 1 -- 172.21.15.2:0/3144673889 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca7c099e20 con 0x563992e42c80 2023-06-23T18:31:59.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.417+0000 7fca72ffd700 1 -- 172.21.15.2:0/3144673889 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c0a5fa0 con 0x563992e42c80 2023-06-23T18:31:59.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3144673889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 msgr2=0x563992e4a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e4a310 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fca7c09aa50 tx=0x7fca7c09c590 comp rx=0 tx=0).stop 2023-06-23T18:31:59.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3144673889 shutdown_connections 2023-06-23T18:31:59.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e44000 0x563992e53a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e42c80 0x563992e4a310 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3144673889 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e435d0 0x563992e4a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3144673889 >> 172.21.15.2:0/3144673889 conn(0x563992e383f0 msgr2=0x563992e5eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3144673889 shutdown_connections 2023-06-23T18:31:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3144673889 wait complete. 2023-06-23T18:31:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 -- start start 2023-06-23T18:31:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.418+0000 7fca9eb19b80 1 Processor -- start 2023-06-23T18:31:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e42c80 0x563992e4f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e435d0 0x563992e4fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca9eb19b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e52ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992e533e0 con 0x563992e435d0 2023-06-23T18:31:59.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992e53550 con 0x563992e42c80 2023-06-23T18:31:59.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca9eb19b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563992e53710 con 0x563992e44000 2023-06-23T18:31:59.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca9eb19b80 10 client.0 _ll_register_callbacks cb 0x563992d39260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:59.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e435d0 0x563992e4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e435d0 0x563992e4fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55852/0 (socket says 172.21.15.2:55852) 2023-06-23T18:31:59.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 -- 172.21.15.2:0/3832173006 learned_addr learned my addr 172.21.15.2:0/3832173006 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8bfff700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e42c80 0x563992e4f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 -- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 msgr2=0x563992e52ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca90f31700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e52ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e52ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 -- 172.21.15.2:0/3832173006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e42c80 msgr2=0x563992e4f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e42c80 0x563992e4f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 -- 172.21.15.2:0/3832173006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563992bc1de0 con 0x563992e435d0 2023-06-23T18:31:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8bfff700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e42c80 0x563992e4f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:31:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca90f31700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e52ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:31:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca8b7fe700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e435d0 0x563992e4fe80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fca800098e0 tx=0x7fca80005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee963a3a573c8b47 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.419+0000 7fca6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca6b7fe700 1 -- 172.21.15.2:0/3832173006 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8001d030 con 0x563992e435d0 2023-06-23T18:31:59.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca6b7fe700 1 -- 172.21.15.2:0/3832173006 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca800280b0 con 0x563992e435d0 2023-06-23T18:31:59.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3832173006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca8400f040 con 0x563992e435d0 2023-06-23T18:31:59.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca6b7fe700 1 -- 172.21.15.2:0/3832173006 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca8000bd60 con 0x563992e435d0 2023-06-23T18:31:59.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 10 client.16140 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:59.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3832173006 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563992be8250 con 0x563992e435d0 2023-06-23T18:31:59.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca6b7fe700 1 -- 172.21.15.2:0/3832173006 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca8400f040 con 0x563992e435d0 2023-06-23T18:31:59.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca6b7fe700 1 -- 172.21.15.2:0/3832173006 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca80004500 con 0x563992e435d0 2023-06-23T18:31:59.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:59.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 10 client.16140 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:31:59.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 20 client.16140 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:59.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x563992e58140 0x563992eaf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 4 client.16140 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:31:59.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.420+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3832173006 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563992d4f050 con 0x563992e58140 2023-06-23T18:31:59.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.421+0000 7fca8bfff700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x563992e58140 0x563992eaf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.421+0000 7fca6b7fe700 10 client.16140 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:31:59.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.421+0000 7fca8bfff700 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x563992e58140 0x563992eaf210 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d1d17f86dadc8e8 server_cookie=67ef0fd56036872 in_seq=0 out_seq=0 2023-06-23T18:31:59.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.421+0000 7fca6b7fe700 1 -- 172.21.15.2:0/3832173006 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563992d4f050 con 0x563992e58140 2023-06-23T18:31:59.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.421+0000 7fca6b7fe700 10 client.16140 handle_command_reply: tid=0 2023-06-23T18:31:59.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.421+0000 7fca9eb19b80 1 client.16140 shutdown 2023-06-23T18:31:59.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.421+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3832173006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e435d0 msgr2=0x563992e4fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.421+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e435d0 0x563992e4fe80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fca800098e0 tx=0x7fca80005fb0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3832173006 shutdown_connections 2023-06-23T18:31:59.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x563992e58140 0x563992eaf210 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563992e44000 0x563992e52ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563992e42c80 0x563992e4f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca6b7fe700 0 client.16140 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:31:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 1 --2- 172.21.15.2:0/3832173006 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563992e435d0 0x563992e4fe80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3832173006 >> 172.21.15.2:0/3832173006 conn(0x563992e383f0 msgr2=0x563992e46e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3832173006 shutdown_connections 2023-06-23T18:31:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 1 -- 172.21.15.2:0/3832173006 wait complete. 2023-06-23T18:31:59.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca9eb19b80 20 client.16140 trim_cache size 0 max 16384 2023-06-23T18:31:59.438 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:31:59.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 -- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fca740a1060 msgr2=0x7fca740a3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fca740a1060 0x7fca740a3510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fca840046d0 tx=0x7fca84004700 comp rx=0 tx=0).stop 2023-06-23T18:31:59.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 -- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0fea80 msgr2=0x7fca8c068370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0fea80 0x7fca8c068370 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00d980 tx=0x7fca7c00dc90 comp rx=0 tx=0).stop 2023-06-23T18:31:59.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 -- 172.21.15.2:0/2562183888 shutdown_connections 2023-06-23T18:31:59.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fca740a1060 0x7fca740a3510 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fca8c100600 0x7fca8c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fca8c0fea80 0x7fca8c068370 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 --2- 172.21.15.2:0/2562183888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca8c0ffc70 0x7fca8c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.422+0000 7fca92994700 1 -- 172.21.15.2:0/2562183888 >> 172.21.15.2:0/2562183888 conn(0x7fca8c0fa0e0 msgr2=0x7fca8c103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.423+0000 7fca92994700 1 -- 172.21.15.2:0/2562183888 shutdown_connections 2023-06-23T18:31:59.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.423+0000 7fca92994700 1 -- 172.21.15.2:0/2562183888 wait complete. 2023-06-23T18:31:59.442 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:31:59.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 -- 172.21.15.2:0/39269432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600fe210 msgr2=0x7f49600fe620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 --2- 172.21.15.2:0/39269432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600fe210 0x7f49600fe620 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4950009990 tx=0x7f4950009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 -- 172.21.15.2:0/39269432 shutdown_connections 2023-06-23T18:31:59.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 --2- 172.21.15.2:0/39269432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49600ffd90 0x7f4960106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 --2- 172.21.15.2:0/39269432 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600fe210 0x7f49600fe620 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 --2- 172.21.15.2:0/39269432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600ff400 0x7f49600ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 -- 172.21.15.2:0/39269432 >> 172.21.15.2:0/39269432 conn(0x7f49600f9850 msgr2=0x7f49600fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 -- 172.21.15.2:0/39269432 shutdown_connections 2023-06-23T18:31:59.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.584+0000 7f4966489700 1 -- 172.21.15.2:0/39269432 wait complete. 2023-06-23T18:31:59.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.585+0000 7f4966489700 1 Processor -- start 2023-06-23T18:31:59.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.585+0000 7f4966489700 1 -- start start 2023-06-23T18:31:59.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.585+0000 7f4966489700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600fe210 0x7f4960194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.585+0000 7f4966489700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49600ff400 0x7f4960194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.585+0000 7f4966489700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600ffd90 0x7f496019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f4966489700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496019c2c0 con 0x7f49600ffd90 2023-06-23T18:31:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f4966489700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496019c400 con 0x7f49600ff400 2023-06-23T18:31:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f4966489700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496019c590 con 0x7f49600fe210 2023-06-23T18:31:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f4964a26700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600ffd90 0x7f496019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600fe210 0x7f4960194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49600ff400 0x7f4960194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f4964a26700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600ffd90 0x7f496019bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55868/0 (socket says 172.21.15.2:55868) 2023-06-23T18:31:59.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f4964a26700 1 -- 172.21.15.2:0/2283141543 learned_addr learned my addr 172.21.15.2:0/2283141543 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495ffff700 1 -- 172.21.15.2:0/2283141543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49600ff400 msgr2=0x7f4960194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495ffff700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49600ff400 0x7f4960194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495ffff700 1 -- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600ffd90 msgr2=0x7f496019bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495ffff700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600ffd90 0x7f496019bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495ffff700 1 -- 172.21.15.2:0/2283141543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49500095f0 con 0x7f49600fe210 2023-06-23T18:31:59.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f4964a26700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600ffd90 0x7f496019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:59.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495ffff700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600fe210 0x7f4960194740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4950003f60 tx=0x7f4950003e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.586+0000 7f495d7fa700 1 -- 172.21.15.2:0/2283141543 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495000bb40 con 0x7f49600fe210 2023-06-23T18:31:59.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.587+0000 7f4966489700 1 -- 172.21.15.2:0/2283141543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496010a2f0 con 0x7f49600fe210 2023-06-23T18:31:59.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.587+0000 7f495d7fa700 1 -- 172.21.15.2:0/2283141543 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f495000bca0 con 0x7f49600fe210 2023-06-23T18:31:59.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.587+0000 7f495d7fa700 1 -- 172.21.15.2:0/2283141543 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500206a0 con 0x7f49600fe210 2023-06-23T18:31:59.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.587+0000 7f4966489700 1 -- 172.21.15.2:0/2283141543 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496010a7c0 con 0x7f49600fe210 2023-06-23T18:31:59.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.589+0000 7f495d7fa700 1 -- 172.21.15.2:0/2283141543 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f495000f460 con 0x7f49600fe210 2023-06-23T18:31:59.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.590+0000 7f495d7fa700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f49480a1150 0x7f49480a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.590+0000 7f495d7fa700 1 -- 172.21.15.2:0/2283141543 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f49500d05d0 con 0x7f49600fe210 2023-06-23T18:31:59.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.590+0000 7f495f7fe700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f49480a1150 0x7f49480a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.590+0000 7f495f7fe700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f49480a1150 0x7f49480a3600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4954009c60 tx=0x7f4954009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f497260eb80 1 Processor -- start 2023-06-23T18:31:59.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f497260eb80 1 -- start start 2023-06-23T18:31:59.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f497260eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b159e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f497260eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b159ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f497260eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b159f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f497260eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b14a2cb0 con 0x55d7b159dc80 2023-06-23T18:31:59.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f497260eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b14f5a40 con 0x55d7b159f000 2023-06-23T18:31:59.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f497260eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b14eeed0 con 0x55d7b159e5d0 2023-06-23T18:31:59.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b159e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b159e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55872/0 (socket says 172.21.15.2:55872) 2023-06-23T18:31:59.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495ffff700 1 -- 172.21.15.2:0/1836413592 learned_addr learned my addr 172.21.15.2:0/1836413592 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b159ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f4964a26700 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b159f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495ffff700 1 -- 172.21.15.2:0/1836413592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 msgr2=0x55d7b159ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495ffff700 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b159ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495ffff700 1 -- 172.21.15.2:0/1836413592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 msgr2=0x55d7b159f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495ffff700 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b159f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495ffff700 1 -- 172.21.15.2:0/1836413592 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7b1370630 con 0x55d7b159dc80 2023-06-23T18:31:59.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f4964a26700 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b159f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:31:59.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.591+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b159ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:59.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.592+0000 7f495ffff700 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b159e090 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4950099f70 tx=0x7f4950009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4516926685cc16 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.592+0000 7f49467fc700 1 -- 172.21.15.2:0/1836413592 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495009da70 con 0x55d7b159dc80 2023-06-23T18:31:59.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.592+0000 7f49467fc700 1 -- 172.21.15.2:0/1836413592 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49500a1aa0 con 0x55d7b159dc80 2023-06-23T18:31:59.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.592+0000 7f49467fc700 1 -- 172.21.15.2:0/1836413592 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495009d1c0 con 0x55d7b159dc80 2023-06-23T18:31:59.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.592+0000 7f497260eb80 1 -- 172.21.15.2:0/1836413592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 msgr2=0x55d7b159e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.592+0000 7f497260eb80 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b159e090 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4950099f70 tx=0x7f4950009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 -- 172.21.15.2:0/1836413592 shutdown_connections 2023-06-23T18:31:59.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b159f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b159e090 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 --2- 172.21.15.2:0/1836413592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b159ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 -- 172.21.15.2:0/1836413592 >> 172.21.15.2:0/1836413592 conn(0x55d7b15933f0 msgr2=0x55d7b1598520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 -- 172.21.15.2:0/1836413592 shutdown_connections 2023-06-23T18:31:59.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 -- 172.21.15.2:0/1836413592 wait complete. 2023-06-23T18:31:59.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 -- start start 2023-06-23T18:31:59.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.593+0000 7f497260eb80 1 Processor -- start 2023-06-23T18:31:59.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f497260eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b15a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f497260eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b15a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f497260eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b15ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f497260eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b148eac0 con 0x55d7b159e5d0 2023-06-23T18:31:59.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f497260eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b148e7c0 con 0x55d7b159f000 2023-06-23T18:31:59.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f497260eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b1372920 con 0x55d7b159dc80 2023-06-23T18:31:59.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f497260eb80 10 client.0 _ll_register_callbacks cb 0x55d7b1494260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:59.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b15a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b15a7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55880/0 (socket says 172.21.15.2:55880) 2023-06-23T18:31:59.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 -- 172.21.15.2:0/1143359951 learned_addr learned my addr 172.21.15.2:0/1143359951 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495ffff700 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b15a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f4964a26700 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b15ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 -- 172.21.15.2:0/1143359951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 msgr2=0x55d7b15a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b15a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 -- 172.21.15.2:0/1143359951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 msgr2=0x55d7b15ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b15ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 -- 172.21.15.2:0/1143359951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496010a7c0 con 0x55d7b159e5d0 2023-06-23T18:31:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b15a7730 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4954003980 tx=0x7f49540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6909b04151ffe9c6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f493f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f493f7fe700 1 -- 172.21.15.2:0/1143359951 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495401e8b0 con 0x55d7b159e5d0 2023-06-23T18:31:59.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.594+0000 7f493f7fe700 1 -- 172.21.15.2:0/1143359951 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f495401ea10 con 0x55d7b159e5d0 2023-06-23T18:31:59.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 1 -- 172.21.15.2:0/1143359951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7b1590bb0 con 0x55d7b159e5d0 2023-06-23T18:31:59.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f493f7fe700 1 -- 172.21.15.2:0/1143359951 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f495401ebd0 con 0x55d7b159e5d0 2023-06-23T18:31:59.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 10 client.16152 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:59.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 1 -- 172.21.15.2:0/1143359951 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d7b131cde0 con 0x55d7b159e5d0 2023-06-23T18:31:59.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f493f7fe700 1 -- 172.21.15.2:0/1143359951 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d7b1590bb0 con 0x55d7b159e5d0 2023-06-23T18:31:59.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f493f7fe700 1 -- 172.21.15.2:0/1143359951 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4954031400 con 0x55d7b159e5d0 2023-06-23T18:31:59.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 10 client.16152 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:59.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 10 client.16152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:31:59.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 20 client.16152 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:59.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55d7b15b2f90 0x55d7b15b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 4 client.16152 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:59.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f497260eb80 1 -- 172.21.15.2:0/1143359951 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7b14e7eb0 con 0x55d7b15b2f90 2023-06-23T18:31:59.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.595+0000 7f495ffff700 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55d7b15b2f90 0x55d7b15b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.596+0000 7f493f7fe700 10 client.16152 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:31:59.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.596+0000 7f495ffff700 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55d7b15b2f90 0x55d7b15b53a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95495fd7b345f00 server_cookie=86d1fbe6ec9c5b9c in_seq=0 out_seq=0 2023-06-23T18:31:59.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.596+0000 7f493f7fe700 1 -- 172.21.15.2:0/1143359951 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d7b14e7eb0 con 0x55d7b15b2f90 2023-06-23T18:31:59.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.596+0000 7f493f7fe700 10 client.16152 handle_command_reply: tid=0 2023-06-23T18:31:59.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.596+0000 7f497260eb80 1 client.16152 shutdown 2023-06-23T18:31:59.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 -- 172.21.15.2:0/1143359951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 msgr2=0x55d7b15a7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b15a7730 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4954003980 tx=0x7f49540047c0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 -- 172.21.15.2:0/1143359951 shutdown_connections 2023-06-23T18:31:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55d7b15b2f90 0x55d7b15b53a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b15ae770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b15a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 --2- 172.21.15.2:0/1143359951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b15a7730 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 -- 172.21.15.2:0/1143359951 >> 172.21.15.2:0/1143359951 conn(0x55d7b15933f0 msgr2=0x55d7b1597c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f493f7fe700 0 client.16152 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:31:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 -- 172.21.15.2:0/1143359951 shutdown_connections 2023-06-23T18:31:59.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 1 -- 172.21.15.2:0/1143359951 wait complete. 2023-06-23T18:31:59.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.597+0000 7f497260eb80 20 client.16152 trim_cache size 0 max 16384 2023-06-23T18:31:59.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.611+0000 7f497260eb80 1 Processor -- start 2023-06-23T18:31:59.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.611+0000 7f497260eb80 1 -- start start 2023-06-23T18:31:59.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.611+0000 7f497260eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b15a5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.611+0000 7f497260eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b15a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.611+0000 7f497260eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b15a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.611+0000 7f497260eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b15a66a0 con 0x55d7b159dc80 2023-06-23T18:31:59.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.611+0000 7f497260eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b15aeb20 con 0x55d7b159f000 2023-06-23T18:31:59.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.611+0000 7f497260eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b15a6d90 con 0x55d7b159e5d0 2023-06-23T18:31:59.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b15a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b15a5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55886/0 (socket says 172.21.15.2:55886) 2023-06-23T18:31:59.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 -- 172.21.15.2:0/1249358525 learned_addr learned my addr 172.21.15.2:0/1249358525 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f4964a26700 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b15a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b15a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 -- 172.21.15.2:0/1249358525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 msgr2=0x55d7b15a71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b15a71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 -- 172.21.15.2:0/1249358525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 msgr2=0x55d7b15a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b15a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 -- 172.21.15.2:0/1249358525 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7b159f8f0 con 0x55d7b159dc80 2023-06-23T18:31:59.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f495ffff700 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b15a5350 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4950000c00 tx=0x7f49500a3040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1be46d35f41c936 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f4946ffd700 1 -- 172.21.15.2:0/1249358525 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500b6070 con 0x55d7b159dc80 2023-06-23T18:31:59.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f4946ffd700 1 -- 172.21.15.2:0/1249358525 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49500b61d0 con 0x55d7b159dc80 2023-06-23T18:31:59.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.612+0000 7f4946ffd700 1 -- 172.21.15.2:0/1249358525 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49500bbdd0 con 0x55d7b159dc80 2023-06-23T18:31:59.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 -- 172.21.15.2:0/1249358525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 msgr2=0x55d7b15a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b15a5350 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4950000c00 tx=0x7f49500a3040 comp rx=0 tx=0).stop 2023-06-23T18:31:59.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 -- 172.21.15.2:0/1249358525 shutdown_connections 2023-06-23T18:31:59.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b15a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159dc80 0x55d7b15a5350 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 --2- 172.21.15.2:0/1249358525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159e5d0 0x55d7b15a71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 -- 172.21.15.2:0/1249358525 >> 172.21.15.2:0/1249358525 conn(0x55d7b15933f0 msgr2=0x55d7b15b9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 -- 172.21.15.2:0/1249358525 shutdown_connections 2023-06-23T18:31:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 -- 172.21.15.2:0/1249358525 wait complete. 2023-06-23T18:31:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.613+0000 7f497260eb80 1 -- start start 2023-06-23T18:31:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f497260eb80 1 Processor -- start 2023-06-23T18:31:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f497260eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b1604e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f497260eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b1607380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f497260eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b16098d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f497260eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b1609e10 con 0x55d7b159e5d0 2023-06-23T18:31:59.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f497260eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b1609f50 con 0x55d7b159f000 2023-06-23T18:31:59.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f497260eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7b160a0c0 con 0x55d7b159dc80 2023-06-23T18:31:59.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f497260eb80 10 client.0 _ll_register_callbacks cb 0x55d7b1494260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:59.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f495f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b1607380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f495f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b1607380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55894/0 (socket says 172.21.15.2:55894) 2023-06-23T18:31:59.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f495f7fe700 1 -- 172.21.15.2:0/1954390499 learned_addr learned my addr 172.21.15.2:0/1954390499 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f495ffff700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b1604e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f4964a26700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b16098d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f495f7fe700 1 -- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 msgr2=0x55d7b1604e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b1604e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f495f7fe700 1 -- 172.21.15.2:0/1954390499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 msgr2=0x55d7b16098d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.614+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b16098d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f495f7fe700 1 -- 172.21.15.2:0/1954390499 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7b1370630 con 0x55d7b159e5d0 2023-06-23T18:31:59.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f495ffff700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b1604e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:31:59.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f4964a26700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b16098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:31:59.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f495f7fe700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b1607380 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4954011b30 tx=0x7f4954031530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3fee406d69a290 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f493f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:59.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f493f7fe700 1 -- 172.21.15.2:0/1954390499 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954031b40 con 0x55d7b159e5d0 2023-06-23T18:31:59.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f493f7fe700 1 -- 172.21.15.2:0/1954390499 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f495402d030 con 0x55d7b159e5d0 2023-06-23T18:31:59.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f497260eb80 1 -- 172.21.15.2:0/1954390499 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49500a1aa0 con 0x55d7b159e5d0 2023-06-23T18:31:59.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f493f7fe700 1 -- 172.21.15.2:0/1954390499 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f495401e6b0 con 0x55d7b159e5d0 2023-06-23T18:31:59.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f497260eb80 10 client.16158 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:59.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f497260eb80 1 -- 172.21.15.2:0/1954390499 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55d7b1305c60 con 0x55d7b159e5d0 2023-06-23T18:31:59.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.615+0000 7f493f7fe700 1 -- 172.21.15.2:0/1954390499 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49500a1aa0 con 0x55d7b159e5d0 2023-06-23T18:31:59.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f493f7fe700 1 -- 172.21.15.2:0/1954390499 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f495403a7d0 con 0x55d7b159e5d0 2023-06-23T18:31:59.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f497260eb80 10 client.16158 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:59.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f497260eb80 10 client.16158 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:31:59.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f497260eb80 20 client.16158 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:59.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f497260eb80 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55d7b160bac0 0x55d7b160ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f497260eb80 4 client.16158 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:31:59.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f497260eb80 1 -- 172.21.15.2:0/1954390499 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d7b14aa050 con 0x55d7b160bac0 2023-06-23T18:31:59.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f495ffff700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55d7b160bac0 0x55d7b160ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f493f7fe700 10 client.16158 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:31:59.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.616+0000 7f495ffff700 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55d7b160bac0 0x55d7b160ded0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48afe5e2057c6a7c server_cookie=47c08af9c4c728d1 in_seq=0 out_seq=0 2023-06-23T18:31:59.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f493f7fe700 1 -- 172.21.15.2:0/1954390499 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d7b14aa050 con 0x55d7b160bac0 2023-06-23T18:31:59.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f493f7fe700 10 client.16158 handle_command_reply: tid=0 2023-06-23T18:31:59.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 client.16158 shutdown 2023-06-23T18:31:59.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 -- 172.21.15.2:0/1954390499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 msgr2=0x55d7b1607380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b1607380 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4954011b30 tx=0x7f4954031530 comp rx=0 tx=0).stop 2023-06-23T18:31:59.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 -- 172.21.15.2:0/1954390499 shutdown_connections 2023-06-23T18:31:59.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55d7b160bac0 0x55d7b160ded0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d7b159f000 0x55d7b16098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f493f7fe700 0 client.16158 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:31:59.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d7b159dc80 0x55d7b1604e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 --2- 172.21.15.2:0/1954390499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d7b159e5d0 0x55d7b1607380 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 -- 172.21.15.2:0/1954390499 >> 172.21.15.2:0/1954390499 conn(0x55d7b15933f0 msgr2=0x55d7b15a5350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 -- 172.21.15.2:0/1954390499 shutdown_connections 2023-06-23T18:31:59.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 1 -- 172.21.15.2:0/1954390499 wait complete. 2023-06-23T18:31:59.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.617+0000 7f497260eb80 20 client.16158 trim_cache size 0 max 16384 2023-06-23T18:31:59.635 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:31:59.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 -- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f49480a1150 msgr2=0x7f49480a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f49480a1150 0x7f49480a3600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4954009c60 tx=0x7f4954009260 comp rx=0 tx=0).stop 2023-06-23T18:31:59.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 -- 172.21.15.2:0/2283141543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600fe210 msgr2=0x7f4960194740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600fe210 0x7f4960194740 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4950003f60 tx=0x7f4950003e50 comp rx=0 tx=0).stop 2023-06-23T18:31:59.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 -- 172.21.15.2:0/2283141543 shutdown_connections 2023-06-23T18:31:59.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f49600ffd90 0x7f496019bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49600fe210 0x7f4960194740 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f49480a1150 0x7f49480a3600 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 --2- 172.21.15.2:0/2283141543 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49600ff400 0x7f4960194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 -- 172.21.15.2:0/2283141543 >> 172.21.15.2:0/2283141543 conn(0x7f49600f9850 msgr2=0x7f49600fb240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 -- 172.21.15.2:0/2283141543 shutdown_connections 2023-06-23T18:31:59.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.618+0000 7f4966489700 1 -- 172.21.15.2:0/2283141543 wait complete. 2023-06-23T18:31:59.641 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:31:59.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 -- 172.21.15.2:0/3989003408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec102f40 msgr2=0x7f76ec105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/3989003408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec102f40 0x7f76ec105320 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009990 tx=0x7f76d8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 -- 172.21.15.2:0/3989003408 shutdown_connections 2023-06-23T18:31:59.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/3989003408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76ec105860 0x7f76ec107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/3989003408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec100620 0x7f76ec102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/3989003408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec102f40 0x7f76ec105320 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 -- 172.21.15.2:0/3989003408 >> 172.21.15.2:0/3989003408 conn(0x7f76ec0fa0c0 msgr2=0x7f76ec0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 -- 172.21.15.2:0/3989003408 shutdown_connections 2023-06-23T18:31:59.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.782+0000 7f76f2ce3700 1 -- 172.21.15.2:0/3989003408 wait complete. 2023-06-23T18:31:59.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f2ce3700 1 Processor -- start 2023-06-23T18:31:59.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f2ce3700 1 -- start start 2023-06-23T18:31:59.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f2ce3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec100620 0x7f76ec197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f2ce3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec102f40 0x7f76ec197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f2ce3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76ec105860 0x7f76ec19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f2ce3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec19f590 con 0x7f76ec100620 2023-06-23T18:31:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f2ce3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec19f6d0 con 0x7f76ec105860 2023-06-23T18:31:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f2ce3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ec19f860 con 0x7f76ec102f40 2023-06-23T18:31:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.783+0000 7f76f0a7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec100620 0x7f76ec197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f0a7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec100620 0x7f76ec197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55906/0 (socket says 172.21.15.2:55906) 2023-06-23T18:31:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f0a7f700 1 -- 172.21.15.2:0/679687803 learned_addr learned my addr 172.21.15.2:0/679687803 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76ebfff700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec102f40 0x7f76ec197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f0a7f700 1 -- 172.21.15.2:0/679687803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec102f40 msgr2=0x7f76ec197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f1280700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76ec105860 0x7f76ec19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec102f40 0x7f76ec197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f0a7f700 1 -- 172.21.15.2:0/679687803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76ec105860 msgr2=0x7f76ec19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76ec105860 0x7f76ec19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f0a7f700 1 -- 172.21.15.2:0/679687803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80095f0 con 0x7f76ec100620 2023-06-23T18:31:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec100620 0x7f76ec197a10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f76e000e9a0 tx=0x7f76e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76e9ffb700 1 -- 172.21.15.2:0/679687803 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0009790 con 0x7f76ec100620 2023-06-23T18:31:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76e9ffb700 1 -- 172.21.15.2:0/679687803 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76e000cbd0 con 0x7f76ec100620 2023-06-23T18:31:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f2ce3700 1 -- 172.21.15.2:0/679687803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ec19fb40 con 0x7f76ec100620 2023-06-23T18:31:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76e9ffb700 1 -- 172.21.15.2:0/679687803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0018680 con 0x7f76ec100620 2023-06-23T18:31:59.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.784+0000 7f76f2ce3700 1 -- 172.21.15.2:0/679687803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec1a00a0 con 0x7f76ec100620 2023-06-23T18:31:59.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.787+0000 7f76e9ffb700 1 -- 172.21.15.2:0/679687803 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f76e000cd40 con 0x7f76ec100620 2023-06-23T18:31:59.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.787+0000 7f76e9ffb700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76dc0a11d0 0x7f76dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.787+0000 7f76e9ffb700 1 -- 172.21.15.2:0/679687803 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76e0014070 con 0x7f76ec100620 2023-06-23T18:31:59.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.788+0000 7f76ebfff700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76dc0a11d0 0x7f76dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.788+0000 7f76ebfff700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76dc0a11d0 0x7f76dc0a3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f76d8006010 tx=0x7f76d800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.788+0000 7f76fee68b80 1 Processor -- start 2023-06-23T18:31:59.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.788+0000 7f76fee68b80 1 -- start start 2023-06-23T18:31:59.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e080090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 0x56181e080a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e081470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76fee68b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181df84cb0 con 0x56181e081000 2023-06-23T18:31:59.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76fee68b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181dfd7a40 con 0x56181e0805d0 2023-06-23T18:31:59.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76fee68b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181dfd0ed0 con 0x56181e07fc80 2023-06-23T18:31:59.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e081470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e081470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55918/0 (socket says 172.21.15.2:55918) 2023-06-23T18:31:59.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 -- 172.21.15.2:0/345286868 learned_addr learned my addr 172.21.15.2:0/345286868 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76ebfff700 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 0x56181e080a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e080090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 -- 172.21.15.2:0/345286868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 msgr2=0x56181e080090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e080090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 -- 172.21.15.2:0/345286868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 msgr2=0x56181e080a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 0x56181e080a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 -- 172.21.15.2:0/345286868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56181de52630 con 0x56181e081000 2023-06-23T18:31:59.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.789+0000 7f76f1280700 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e081470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f76e40060b0 tx=0x7f76e4007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7c5cd753777a7e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76d6ffd700 1 -- 172.21.15.2:0/345286868 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003a00 con 0x56181e081000 2023-06-23T18:31:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76d6ffd700 1 -- 172.21.15.2:0/345286868 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76e4004020 con 0x56181e081000 2023-06-23T18:31:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76d6ffd700 1 -- 172.21.15.2:0/345286868 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4014660 con 0x56181e081000 2023-06-23T18:31:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76fee68b80 1 -- 172.21.15.2:0/345286868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 msgr2=0x56181e081470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76fee68b80 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e081470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f76e40060b0 tx=0x7f76e4007500 comp rx=0 tx=0).stop 2023-06-23T18:31:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76fee68b80 1 -- 172.21.15.2:0/345286868 shutdown_connections 2023-06-23T18:31:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76fee68b80 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e081470 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76fee68b80 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e080090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76fee68b80 1 --2- 172.21.15.2:0/345286868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 0x56181e080a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76fee68b80 1 -- 172.21.15.2:0/345286868 >> 172.21.15.2:0/345286868 conn(0x56181e0753f0 msgr2=0x56181e07a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.790+0000 7f76fee68b80 1 -- 172.21.15.2:0/345286868 shutdown_connections 2023-06-23T18:31:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 -- 172.21.15.2:0/345286868 wait complete. 2023-06-23T18:31:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 -- start start 2023-06-23T18:31:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 Processor -- start 2023-06-23T18:31:59.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e089500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 0x56181e089a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e090a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181df70ac0 con 0x56181e081000 2023-06-23T18:31:59.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181df707c0 con 0x56181e0805d0 2023-06-23T18:31:59.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181de54920 con 0x56181e07fc80 2023-06-23T18:31:59.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.791+0000 7f76fee68b80 10 client.0 _ll_register_callbacks cb 0x56181df76260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:59.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e090a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f0a7f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e089500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e090a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55924/0 (socket says 172.21.15.2:55924) 2023-06-23T18:31:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f0a7f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e089500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56258/0 (socket says 172.21.15.2:56258) 2023-06-23T18:31:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 -- 172.21.15.2:0/1264439848 learned_addr learned my addr 172.21.15.2:0/1264439848 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76ebfff700 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 0x56181e089a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 -- 172.21.15.2:0/1264439848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 msgr2=0x56181e089500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e089500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 -- 172.21.15.2:0/1264439848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 msgr2=0x56181e089a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 0x56181e089a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 -- 172.21.15.2:0/1264439848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e4009920 con 0x56181e081000 2023-06-23T18:31:59.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e089500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:31:59.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76f1280700 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e090a80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098f0 tx=0x7f76e4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1495845676229b2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:59.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76cbfff700 1 -- 172.21.15.2:0/1264439848 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4003e90 con 0x56181e081000 2023-06-23T18:31:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76cbfff700 1 -- 172.21.15.2:0/1264439848 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76e401e460 con 0x56181e081000 2023-06-23T18:31:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.792+0000 7f76fee68b80 1 -- 172.21.15.2:0/1264439848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56181e072bb0 con 0x56181e081000 2023-06-23T18:31:59.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76cbfff700 1 -- 172.21.15.2:0/1264439848 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e401e620 con 0x56181e081000 2023-06-23T18:31:59.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76fee68b80 10 client.16170 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:59.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76fee68b80 1 -- 172.21.15.2:0/1264439848 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56181ddfede0 con 0x56181e081000 2023-06-23T18:31:59.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76cbfff700 1 -- 172.21.15.2:0/1264439848 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56181e072bb0 con 0x56181e081000 2023-06-23T18:31:59.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76cbfff700 1 -- 172.21.15.2:0/1264439848 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f76e401c040 con 0x56181e081000 2023-06-23T18:31:59.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76fee68b80 10 client.16170 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:59.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76fee68b80 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:59.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76fee68b80 20 client.16170 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:59.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56181e095380 0x56181e097790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76fee68b80 4 client.16170 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:31:59.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76fee68b80 1 -- 172.21.15.2:0/1264439848 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56181dfc9eb0 con 0x56181e095380 2023-06-23T18:31:59.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.793+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56181e095380 0x56181e097790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.794+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56181e095380 0x56181e097790 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d4c317495559af0 server_cookie=872f1bf6aeef8ef6 in_seq=0 out_seq=0 2023-06-23T18:31:59.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.794+0000 7f76cbfff700 10 client.16170 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:59.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.794+0000 7f76cbfff700 1 -- 172.21.15.2:0/1264439848 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56181dfc9eb0 con 0x56181e095380 2023-06-23T18:31:59.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.794+0000 7f76cbfff700 10 client.16170 handle_command_reply: tid=0 2023-06-23T18:31:59.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.794+0000 7f76fee68b80 1 client.16170 shutdown 2023-06-23T18:31:59.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 -- 172.21.15.2:0/1264439848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 msgr2=0x56181e090a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e090a80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098f0 tx=0x7f76e4004610 comp rx=0 tx=0).stop 2023-06-23T18:31:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 -- 172.21.15.2:0/1264439848 shutdown_connections 2023-06-23T18:31:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56181e095380 0x56181e097790 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e090a80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76cbfff700 0 client.16170 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e07fc80 0x56181e089500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1264439848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e0805d0 0x56181e089a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 -- 172.21.15.2:0/1264439848 >> 172.21.15.2:0/1264439848 conn(0x56181e0753f0 msgr2=0x56181e079c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 -- 172.21.15.2:0/1264439848 shutdown_connections 2023-06-23T18:31:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 1 -- 172.21.15.2:0/1264439848 wait complete. 2023-06-23T18:31:59.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.795+0000 7f76fee68b80 20 client.16170 trim_cache size 0 max 16384 2023-06-23T18:31:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76fee68b80 1 Processor -- start 2023-06-23T18:31:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76fee68b80 1 -- start start 2023-06-23T18:31:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e07fc80 0x56181e09c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 0x56181e090a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e081000 0x56181e0872e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76fee68b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e0926f0 con 0x56181e07fc80 2023-06-23T18:31:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76fee68b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e087090 con 0x56181e081000 2023-06-23T18:31:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76fee68b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e084980 con 0x56181e0805d0 2023-06-23T18:31:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76f0a7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e07fc80 0x56181e09c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76f0a7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e07fc80 0x56181e09c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55928/0 (socket says 172.21.15.2:55928) 2023-06-23T18:31:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.811+0000 7f76f0a7f700 1 -- 172.21.15.2:0/2350466922 learned_addr learned my addr 172.21.15.2:0/2350466922 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76ebfff700 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 0x56181e090a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76f1280700 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e081000 0x56181e0872e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76f0a7f700 1 -- 172.21.15.2:0/2350466922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 msgr2=0x56181e090a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 0x56181e090a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76f0a7f700 1 -- 172.21.15.2:0/2350466922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e081000 msgr2=0x56181e0872e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e081000 0x56181e0872e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76f0a7f700 1 -- 172.21.15.2:0/2350466922 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56181dde7c60 con 0x56181e07fc80 2023-06-23T18:31:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76f1280700 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e081000 0x56181e0872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:31:59.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e07fc80 0x56181e09c0f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f76e0096a00 tx=0x7f76e009d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cd9773bff039c1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76d77fe700 1 -- 172.21.15.2:0/2350466922 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0098120 con 0x56181e07fc80 2023-06-23T18:31:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76d77fe700 1 -- 172.21.15.2:0/2350466922 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76e0098380 con 0x56181e07fc80 2023-06-23T18:31:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.812+0000 7f76d77fe700 1 -- 172.21.15.2:0/2350466922 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e00cab90 con 0x56181e07fc80 2023-06-23T18:31:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 -- 172.21.15.2:0/2350466922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e07fc80 msgr2=0x56181e09c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e07fc80 0x56181e09c0f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f76e0096a00 tx=0x7f76e009d040 comp rx=0 tx=0).stop 2023-06-23T18:31:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 -- 172.21.15.2:0/2350466922 shutdown_connections 2023-06-23T18:31:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e081000 0x56181e0872e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e07fc80 0x56181e09c0f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 --2- 172.21.15.2:0/2350466922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 0x56181e090a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 -- 172.21.15.2:0/2350466922 >> 172.21.15.2:0/2350466922 conn(0x56181e0753f0 msgr2=0x56181e0768c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 -- 172.21.15.2:0/2350466922 shutdown_connections 2023-06-23T18:31:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 -- 172.21.15.2:0/2350466922 wait complete. 2023-06-23T18:31:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.813+0000 7f76fee68b80 1 -- start start 2023-06-23T18:31:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76fee68b80 1 Processor -- start 2023-06-23T18:31:59.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e07fc80 0x56181e08abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 0x56181e08b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76fee68b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e08f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76fee68b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e08ff20 con 0x56181e081000 2023-06-23T18:31:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76fee68b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e090090 con 0x56181e07fc80 2023-06-23T18:31:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76fee68b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181e090250 con 0x56181e0805d0 2023-06-23T18:31:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76fee68b80 10 client.0 _ll_register_callbacks cb 0x56181df76260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:31:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76f1280700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e08f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76f1280700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e08f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55938/0 (socket says 172.21.15.2:55938) 2023-06-23T18:31:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76f1280700 1 -- 172.21.15.2:0/1000772936 learned_addr learned my addr 172.21.15.2:0/1000772936 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:31:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.814+0000 7f76ebfff700 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 0x56181e08b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76f1280700 1 -- 172.21.15.2:0/1000772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 msgr2=0x56181e08b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76f1280700 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 0x56181e08b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76f1280700 1 -- 172.21.15.2:0/1000772936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e07fc80 msgr2=0x56181e08abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:31:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76f1280700 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e07fc80 0x56181e08abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76f1280700 1 -- 172.21.15.2:0/1000772936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec1a00a0 con 0x56181e081000 2023-06-23T18:31:59.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76f1280700 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e08f9e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x56181e0774b0 tx=0x7f76e40097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a62eedca1328d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:31:59.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:31:59.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76cbfff700 1 -- 172.21.15.2:0/1000772936 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e4027a00 con 0x56181e081000 2023-06-23T18:31:59.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76cbfff700 1 -- 172.21.15.2:0/1000772936 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f76e40133a0 con 0x56181e081000 2023-06-23T18:31:59.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76fee68b80 1 -- 172.21.15.2:0/1000772936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76e4004020 con 0x56181e081000 2023-06-23T18:31:59.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76cbfff700 1 -- 172.21.15.2:0/1000772936 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e4004930 con 0x56181e081000 2023-06-23T18:31:59.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76fee68b80 10 client.16176 fetch_fsmap learned FSMap version 32 2023-06-23T18:31:59.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76fee68b80 1 -- 172.21.15.2:0/1000772936 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x56181de25250 con 0x56181e081000 2023-06-23T18:31:59.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76cbfff700 1 -- 172.21.15.2:0/1000772936 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76e4004020 con 0x56181e081000 2023-06-23T18:31:59.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.815+0000 7f76cbfff700 1 -- 172.21.15.2:0/1000772936 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f76e4007570 con 0x56181e081000 2023-06-23T18:31:59.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.816+0000 7f76fee68b80 10 client.16176 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:31:59.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.816+0000 7f76fee68b80 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:31:59.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.816+0000 7f76fee68b80 20 client.16176 populate_metadata read hostname 'smithi002' 2023-06-23T18:31:59.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.816+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56181e0faa70 0x56181e0fce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:31:59.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.816+0000 7f76fee68b80 4 client.16176 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:31:59.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.816+0000 7f76fee68b80 1 -- 172.21.15.2:0/1000772936 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56181df8c050 con 0x56181e0faa70 2023-06-23T18:31:59.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.816+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56181e0faa70 0x56181e0fce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:31:59.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.817+0000 7f76cbfff700 10 client.16176 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:59.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.817+0000 7f76f0a7f700 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56181e0faa70 0x56181e0fce80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925656b5fb569442 server_cookie=2395ffcb8b41bb20 in_seq=0 out_seq=0 2023-06-23T18:31:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76cbfff700 1 -- 172.21.15.2:0/1000772936 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56181df8c050 con 0x56181e0faa70 2023-06-23T18:31:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76cbfff700 10 client.16176 handle_command_reply: tid=0 2023-06-23T18:31:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 client.16176 shutdown 2023-06-23T18:31:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 -- 172.21.15.2:0/1000772936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 msgr2=0x56181e08f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e08f9e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x56181e0774b0 tx=0x7f76e40097e0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 -- 172.21.15.2:0/1000772936 shutdown_connections 2023-06-23T18:31:59.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56181e0faa70 0x56181e0fce80 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56181e081000 0x56181e08f9e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76cbfff700 0 client.16176 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:31:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56181e07fc80 0x56181e08abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 --2- 172.21.15.2:0/1000772936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56181e0805d0 0x56181e08b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 -- 172.21.15.2:0/1000772936 >> 172.21.15.2:0/1000772936 conn(0x56181e0753f0 msgr2=0x56181e09c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 -- 172.21.15.2:0/1000772936 shutdown_connections 2023-06-23T18:31:59.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 1 -- 172.21.15.2:0/1000772936 wait complete. 2023-06-23T18:31:59.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76fee68b80 20 client.16176 trim_cache size 0 max 16384 2023-06-23T18:31:59.833 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:31:59.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76f2ce3700 1 -- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76dc0a11d0 msgr2=0x7f76dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.818+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76dc0a11d0 0x7f76dc0a3680 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f76d8006010 tx=0x7f76d800b260 comp rx=0 tx=0).stop 2023-06-23T18:31:59.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 -- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec100620 msgr2=0x7f76ec197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:31:59.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec100620 0x7f76ec197a10 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f76e000e9a0 tx=0x7f76e000ed60 comp rx=0 tx=0).stop 2023-06-23T18:31:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 -- 172.21.15.2:0/679687803 shutdown_connections 2023-06-23T18:31:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76dc0a11d0 0x7f76dc0a3680 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76ec105860 0x7f76ec19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76ec100620 0x7f76ec197a10 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 --2- 172.21.15.2:0/679687803 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76ec102f40 0x7f76ec197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:31:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 -- 172.21.15.2:0/679687803 >> 172.21.15.2:0/679687803 conn(0x7f76ec0fa0c0 msgr2=0x7f76ec104880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:31:59.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 -- 172.21.15.2:0/679687803 shutdown_connections 2023-06-23T18:31:59.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:31:59.819+0000 7f76f2ce3700 1 -- 172.21.15.2:0/679687803 wait complete. 2023-06-23T18:32:00.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:00 smithi002 ceph-mon[95947]: pgmap v595: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 27 op/s 2023-06-23T18:32:00.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:00 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/4087795295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:32:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:00 smithi082 ceph-mon[97889]: pgmap v595: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 27 op/s 2023-06-23T18:32:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:00 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/4087795295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:32:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:00 smithi017 ceph-mon[97472]: pgmap v595: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 53 MiB/s wr, 27 op/s 2023-06-23T18:32:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:00 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/4087795295' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:32:00.840 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:32:00.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:32:00.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 -- 172.21.15.2:0/16189885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6690100600 msgr2=0x7f66901029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:00.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 --2- 172.21.15.2:0/16189885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6690100600 0x7f66901029e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6684009990 tx=0x7f6684009ca0 comp rx=0 tx=0).stop 2023-06-23T18:32:00.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 -- 172.21.15.2:0/16189885 shutdown_connections 2023-06-23T18:32:00.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 --2- 172.21.15.2:0/16189885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6690105840 0x7f6690107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:00.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 --2- 172.21.15.2:0/16189885 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6690100600 0x7f66901029e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 --2- 172.21.15.2:0/16189885 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690102f20 0x7f6690105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 -- 172.21.15.2:0/16189885 >> 172.21.15.2:0/16189885 conn(0x7f66900fa0e0 msgr2=0x7f66900fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 -- 172.21.15.2:0/16189885 shutdown_connections 2023-06-23T18:32:00.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.988+0000 7f66971cd700 1 -- 172.21.15.2:0/16189885 wait complete. 2023-06-23T18:32:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.989+0000 7f66971cd700 1 Processor -- start 2023-06-23T18:32:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.989+0000 7f66971cd700 1 -- start start 2023-06-23T18:32:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.989+0000 7f66971cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6690100600 0x7f6690195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:00.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.989+0000 7f66971cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690102f20 0x7f6690195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.989+0000 7f66971cd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6690105840 0x7f669019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.989+0000 7f66971cd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669019d380 con 0x7f6690100600 2023-06-23T18:32:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.989+0000 7f66971cd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669019d4c0 con 0x7f6690102f20 2023-06-23T18:32:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.989+0000 7f66971cd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66900fe3f0 con 0x7f6690105840 2023-06-23T18:32:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690102f20 0x7f6690195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:00.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f669576a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6690105840 0x7f669019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690102f20 0x7f6690195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46542/0 (socket says 172.21.15.2:46542) 2023-06-23T18:32:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 -- 172.21.15.2:0/1818337541 learned_addr learned my addr 172.21.15.2:0/1818337541 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 -- 172.21.15.2:0/1818337541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6690105840 msgr2=0x7f669019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6690105840 0x7f669019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 -- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6690100600 msgr2=0x7f6690195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6690100600 0x7f6690195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:00.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 -- 172.21.15.2:0/1818337541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840095f0 con 0x7f6690102f20 2023-06-23T18:32:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f669576a700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6690105840 0x7f669019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.990+0000 7f668ffff700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690102f20 0x7f6690195d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f667c00d950 tx=0x7f667c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.991+0000 7f668dffb700 1 -- 172.21.15.2:0/1818337541 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c009940 con 0x7f6690102f20 2023-06-23T18:32:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.991+0000 7f66971cd700 1 -- 172.21.15.2:0/1818337541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66900fe6d0 con 0x7f6690102f20 2023-06-23T18:32:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.991+0000 7f66971cd700 1 -- 172.21.15.2:0/1818337541 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66900fecb0 con 0x7f6690102f20 2023-06-23T18:32:01.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.991+0000 7f668dffb700 1 -- 172.21.15.2:0/1818337541 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f667c010460 con 0x7f6690102f20 2023-06-23T18:32:01.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.991+0000 7f668dffb700 1 -- 172.21.15.2:0/1818337541 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00f620 con 0x7f6690102f20 2023-06-23T18:32:01.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.993+0000 7f668dffb700 1 -- 172.21.15.2:0/1818337541 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f667c009aa0 con 0x7f6690102f20 2023-06-23T18:32:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.994+0000 7f668dffb700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66800a11d0 0x7f66800a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.995+0000 7f668dffb700 1 -- 172.21.15.2:0/1818337541 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f667c0cc9d0 con 0x7f6690102f20 2023-06-23T18:32:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.995+0000 7f6694f69700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66800a11d0 0x7f66800a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.995+0000 7f6694f69700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66800a11d0 0x7f66800a3680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f66840098e0 tx=0x7f6684000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f66a3352b80 1 Processor -- start 2023-06-23T18:32:01.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f66a3352b80 1 -- start start 2023-06-23T18:32:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b5b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b5baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 0x561696b5c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f66a3352b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696a5fd30 con 0x561696b5c080 2023-06-23T18:32:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f66a3352b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696ab2ac0 con 0x561696b5ad00 2023-06-23T18:32:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f66a3352b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696aabf50 con 0x561696b5b650 2023-06-23T18:32:01.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f669576a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 0x561696b5c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f668ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b5baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f6694f69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b5b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f668ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b5baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56296/0 (socket says 172.21.15.2:56296) 2023-06-23T18:32:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f669576a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 0x561696b5c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55960/0 (socket says 172.21.15.2:55960) 2023-06-23T18:32:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f668ffff700 1 -- 172.21.15.2:0/2170244800 learned_addr learned my addr 172.21.15.2:0/2170244800 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f668ffff700 1 -- 172.21.15.2:0/2170244800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 msgr2=0x561696b5b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f668ffff700 1 --2- 172.21.15.2:0/2170244800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b5b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f668ffff700 1 -- 172.21.15.2:0/2170244800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 msgr2=0x561696b5c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.996+0000 7f668ffff700 1 --2- 172.21.15.2:0/2170244800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 0x561696b5c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.997+0000 7f668ffff700 1 -- 172.21.15.2:0/2170244800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56169692d6b0 con 0x561696b5b650 2023-06-23T18:32:01.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.997+0000 7f668ffff700 1 --2- 172.21.15.2:0/2170244800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b5baf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f667c0a1d00 tx=0x7f667c099e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f53bb66c71c3f56 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.997+0000 7f667affd700 1 -- 172.21.15.2:0/2170244800 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0a2230 con 0x561696b5b650 2023-06-23T18:32:01.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.997+0000 7f667affd700 1 -- 172.21.15.2:0/2170244800 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f667c098180 con 0x561696b5b650 2023-06-23T18:32:01.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.997+0000 7f667affd700 1 -- 172.21.15.2:0/2170244800 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c099530 con 0x561696b5b650 2023-06-23T18:32:01.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 -- 172.21.15.2:0/2170244800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 msgr2=0x561696b5baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2170244800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b5baf0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f667c0a1d00 tx=0x7f667c099e60 comp rx=0 tx=0).stop 2023-06-23T18:32:01.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 -- 172.21.15.2:0/2170244800 shutdown_connections 2023-06-23T18:32:01.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2170244800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 0x561696b5c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2170244800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b5b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2170244800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b5baf0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 -- 172.21.15.2:0/2170244800 >> 172.21.15.2:0/2170244800 conn(0x561696b50470 msgr2=0x561696b555a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 -- 172.21.15.2:0/2170244800 shutdown_connections 2023-06-23T18:32:01.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 -- 172.21.15.2:0/2170244800 wait complete. 2023-06-23T18:32:01.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 -- start start 2023-06-23T18:32:01.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.998+0000 7f66a3352b80 1 Processor -- start 2023-06-23T18:32:01.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b64a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 0x561696b6bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f66a3352b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696a4bb40 con 0x561696b5ad00 2023-06-23T18:32:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f66a3352b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696a4b840 con 0x561696b5c080 2023-06-23T18:32:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f66a3352b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56169692f9a0 con 0x561696b5b650 2023-06-23T18:32:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f66a3352b80 10 client.0 _ll_register_callbacks cb 0x561696a512e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f6694f69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f6694f69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55962/0 (socket says 172.21.15.2:55962) 2023-06-23T18:32:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f6694f69700 1 -- 172.21.15.2:0/1744752116 learned_addr learned my addr 172.21.15.2:0/1744752116 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f668ffff700 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b64a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f669576a700 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 0x561696b6bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f6694f69700 1 -- 172.21.15.2:0/1744752116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 msgr2=0x561696b64a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f6694f69700 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b64a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f6694f69700 1 -- 172.21.15.2:0/1744752116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 msgr2=0x561696b6bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f6694f69700 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 0x561696b6bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:00.999+0000 7f6694f69700 1 -- 172.21.15.2:0/1744752116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66900fecb0 con 0x561696b5ad00 2023-06-23T18:32:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f6694f69700 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6684015000 tx=0x7f668400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170c57defa88dcc1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f666ffff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:01.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f666ffff700 1 -- 172.21.15.2:0/1744752116 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668402b040 con 0x561696b5ad00 2023-06-23T18:32:01.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f66a3352b80 1 -- 172.21.15.2:0/1744752116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561696b4dc30 con 0x561696b5ad00 2023-06-23T18:32:01.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f666ffff700 1 -- 172.21.15.2:0/1744752116 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66840041a0 con 0x561696b5ad00 2023-06-23T18:32:01.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f666ffff700 1 -- 172.21.15.2:0/1744752116 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f668401d070 con 0x561696b5ad00 2023-06-23T18:32:01.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f66a3352b80 10 client.16185 fetch_fsmap learned FSMap version 32 2023-06-23T18:32:01.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f66a3352b80 1 -- 172.21.15.2:0/1744752116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5616968d9e60 con 0x561696b5ad00 2023-06-23T18:32:01.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f666ffff700 1 -- 172.21.15.2:0/1744752116 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561696b4dc30 con 0x561696b5ad00 2023-06-23T18:32:01.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.000+0000 7f666ffff700 1 -- 172.21.15.2:0/1744752116 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6684025420 con 0x561696b5ad00 2023-06-23T18:32:01.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f66a3352b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:32:01.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f66a3352b80 10 client.16185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:01.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f66a3352b80 20 client.16185 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:01.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f66a3352b80 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561696b703e0 0x561696b727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f66a3352b80 4 client.16185 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:32:01.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f66a3352b80 1 -- 172.21.15.2:0/1744752116 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561696aa4f30 con 0x561696b703e0 2023-06-23T18:32:01.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f668ffff700 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561696b703e0 0x561696b727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f666ffff700 10 client.16185 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:01.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.001+0000 7f668ffff700 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561696b703e0 0x561696b727f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e41a175b4b4d56 server_cookie=98164aaedeab8b22 in_seq=0 out_seq=0 2023-06-23T18:32:01.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f666ffff700 1 -- 172.21.15.2:0/1744752116 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561696aa4f30 con 0x561696b703e0 2023-06-23T18:32:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f666ffff700 10 client.16185 handle_command_reply: tid=0 2023-06-23T18:32:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 client.16185 shutdown 2023-06-23T18:32:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 -- 172.21.15.2:0/1744752116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 msgr2=0x561696b64550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6684015000 tx=0x7f668400bfd0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 -- 172.21.15.2:0/1744752116 shutdown_connections 2023-06-23T18:32:01.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561696b703e0 0x561696b727f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 0x561696b6bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64550 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 --2- 172.21.15.2:0/1744752116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b64a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f666ffff700 0 client.16185 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 -- 172.21.15.2:0/1744752116 >> 172.21.15.2:0/1744752116 conn(0x561696b50470 msgr2=0x561696b54cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 -- 172.21.15.2:0/1744752116 shutdown_connections 2023-06-23T18:32:01.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.002+0000 7f66a3352b80 1 -- 172.21.15.2:0/1744752116 wait complete. 2023-06-23T18:32:01.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.003+0000 7f66a3352b80 20 client.16185 trim_cache size 0 max 16384 2023-06-23T18:32:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f66a3352b80 1 Processor -- start 2023-06-23T18:32:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f66a3352b80 1 -- start start 2023-06-23T18:32:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b6ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 0x561696b64840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f66a3352b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696b56890 con 0x561696b5ad00 2023-06-23T18:32:01.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f66a3352b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696b620e0 con 0x561696b5c080 2023-06-23T18:32:01.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f66a3352b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696b5f9d0 con 0x561696b5b650 2023-06-23T18:32:01.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f6694f69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f6694f69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55970/0 (socket says 172.21.15.2:55970) 2023-06-23T18:32:01.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f6694f69700 1 -- 172.21.15.2:0/3465508596 learned_addr learned my addr 172.21.15.2:0/3465508596 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f668ffff700 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b6ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.021+0000 7f669576a700 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 0x561696b64840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f6694f69700 1 -- 172.21.15.2:0/3465508596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 msgr2=0x561696b6ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f6694f69700 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b6ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f6694f69700 1 -- 172.21.15.2:0/3465508596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 msgr2=0x561696b64840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f6694f69700 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 0x561696b64840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f6694f69700 1 -- 172.21.15.2:0/3465508596 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561696b635e0 con 0x561696b5ad00 2023-06-23T18:32:01.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f6694f69700 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64300 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f668401f040 tx=0x7f668400bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1e8cd562ec097 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f667b7fe700 1 -- 172.21.15.2:0/3465508596 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668402b070 con 0x561696b5ad00 2023-06-23T18:32:01.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f667b7fe700 1 -- 172.21.15.2:0/3465508596 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6684005590 con 0x561696b5ad00 2023-06-23T18:32:01.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.022+0000 7f667b7fe700 1 -- 172.21.15.2:0/3465508596 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684043b50 con 0x561696b5ad00 2023-06-23T18:32:01.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 -- 172.21.15.2:0/3465508596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 msgr2=0x561696b64300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64300 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f668401f040 tx=0x7f668400bdb0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 -- 172.21.15.2:0/3465508596 shutdown_connections 2023-06-23T18:32:01.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5c080 0x561696b64840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5ad00 0x561696b64300 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 --2- 172.21.15.2:0/3465508596 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b6ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 -- 172.21.15.2:0/3465508596 >> 172.21.15.2:0/3465508596 conn(0x561696b50470 msgr2=0x561696b5ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 -- 172.21.15.2:0/3465508596 shutdown_connections 2023-06-23T18:32:01.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 -- 172.21.15.2:0/3465508596 wait complete. 2023-06-23T18:32:01.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 -- start start 2023-06-23T18:32:01.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.023+0000 7f66a3352b80 1 Processor -- start 2023-06-23T18:32:01.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b68790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b68cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f66a3352b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 0x561696bc4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f66a3352b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696b69250 con 0x561696b5c080 2023-06-23T18:32:01.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f66a3352b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696b693c0 con 0x561696b5ad00 2023-06-23T18:32:01.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f66a3352b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561696bc4800 con 0x561696b5b650 2023-06-23T18:32:01.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f66a3352b80 10 client.0 _ll_register_callbacks cb 0x561696a512e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:01.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f668ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b68cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f6694f69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b68790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f668ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b68cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56316/0 (socket says 172.21.15.2:56316) 2023-06-23T18:32:01.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f6694f69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b68790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46568/0 (socket says 172.21.15.2:46568) 2023-06-23T18:32:01.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.024+0000 7f668ffff700 1 -- 172.21.15.2:0/2492266624 learned_addr learned my addr 172.21.15.2:0/2492266624 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f668ffff700 1 -- 172.21.15.2:0/2492266624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 msgr2=0x561696b68790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f668ffff700 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b68790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f668ffff700 1 -- 172.21.15.2:0/2492266624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 msgr2=0x561696bc4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f668ffff700 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 0x561696bc4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f668ffff700 1 -- 172.21.15.2:0/2492266624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616968d9e60 con 0x561696b5b650 2023-06-23T18:32:01.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f6694f69700 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b68790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:01.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f668ffff700 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b68cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f667c099a90 tx=0x7f667c005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23d515ebe5d080a0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f666ffff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:32:01.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f666ffff700 1 -- 172.21.15.2:0/2492266624 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0b1040 con 0x561696b5b650 2023-06-23T18:32:01.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f66a3352b80 1 -- 172.21.15.2:0/2492266624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f667c098180 con 0x561696b5b650 2023-06-23T18:32:01.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f666ffff700 1 -- 172.21.15.2:0/2492266624 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f667c09a160 con 0x561696b5b650 2023-06-23T18:32:01.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f666ffff700 1 -- 172.21.15.2:0/2492266624 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f667c0ca1e0 con 0x561696b5b650 2023-06-23T18:32:01.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f66a3352b80 10 client.25927 fetch_fsmap learned FSMap version 32 2023-06-23T18:32:01.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f66a3352b80 1 -- 172.21.15.2:0/2492266624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x561696b5c720 con 0x561696b5b650 2023-06-23T18:32:01.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f666ffff700 1 -- 172.21.15.2:0/2492266624 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f667c098180 con 0x561696b5b650 2023-06-23T18:32:01.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.025+0000 7f666ffff700 1 -- 172.21.15.2:0/2492266624 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f667c0b8020 con 0x561696b5b650 2023-06-23T18:32:01.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f66a3352b80 10 client.25927 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:32:01.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f66a3352b80 10 client.25927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:01.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f66a3352b80 20 client.25927 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:01.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561696bc5fc0 0x561696bc83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f66a3352b80 4 client.25927 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:32:01.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f66a3352b80 1 -- 172.21.15.2:0/2492266624 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561696a670d0 con 0x561696bc5fc0 2023-06-23T18:32:01.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f6694f69700 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561696bc5fc0 0x561696bc83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f666ffff700 10 client.25927 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:01.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.026+0000 7f6694f69700 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561696bc5fc0 0x561696bc83d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86ec4b526adb534a server_cookie=ed99b48c55d5dbf8 in_seq=0 out_seq=0 2023-06-23T18:32:01.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f666ffff700 1 -- 172.21.15.2:0/2492266624 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561696a670d0 con 0x561696bc5fc0 2023-06-23T18:32:01.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f666ffff700 10 client.25927 handle_command_reply: tid=0 2023-06-23T18:32:01.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 client.25927 shutdown 2023-06-23T18:32:01.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 -- 172.21.15.2:0/2492266624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 msgr2=0x561696b68cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b68cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f667c099a90 tx=0x7f667c005dc0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 -- 172.21.15.2:0/2492266624 shutdown_connections 2023-06-23T18:32:01.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561696bc5fc0 0x561696bc83d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561696b5c080 0x561696bc4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561696b5ad00 0x561696b68790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f666ffff700 0 client.25927 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:01.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 --2- 172.21.15.2:0/2492266624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561696b5b650 0x561696b68cd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 -- 172.21.15.2:0/2492266624 >> 172.21.15.2:0/2492266624 conn(0x561696b50470 msgr2=0x561696b64300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 -- 172.21.15.2:0/2492266624 shutdown_connections 2023-06-23T18:32:01.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 1 -- 172.21.15.2:0/2492266624 wait complete. 2023-06-23T18:32:01.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.028+0000 7f66a3352b80 20 client.25927 trim_cache size 0 max 16384 2023-06-23T18:32:01.050 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:32:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:32:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "ba47bb3d-6257-47b5-9618-16bc24b94191", 2023-06-23T18:32:01.050 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:32:01.050 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:32:01.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 -- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66800a11d0 msgr2=0x7f66800a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66800a11d0 0x7f66800a3680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f66840098e0 tx=0x7f6684000bc0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 -- 172.21.15.2:0/1818337541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690102f20 msgr2=0x7f6690195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690102f20 0x7f6690195d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f667c00d950 tx=0x7f667c00dc60 comp rx=0 tx=0).stop 2023-06-23T18:32:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 -- 172.21.15.2:0/1818337541 shutdown_connections 2023-06-23T18:32:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6690105840 0x7f669019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6690100600 0x7f6690195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f66800a11d0 0x7f66800a3680 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 --2- 172.21.15.2:0/1818337541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6690102f20 0x7f6690195d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 -- 172.21.15.2:0/1818337541 >> 172.21.15.2:0/1818337541 conn(0x7f66900fa0e0 msgr2=0x7f6690101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 -- 172.21.15.2:0/1818337541 shutdown_connections 2023-06-23T18:32:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.029+0000 7f66971cd700 1 -- 172.21.15.2:0/1818337541 wait complete. 2023-06-23T18:32:01.055 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:32:01.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 -- 172.21.15.2:0/424751364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 msgr2=0x7fa308106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 --2- 172.21.15.2:0/424751364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 0x7fa308106ea0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa30000b0a0 tx=0x7fa30000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 -- 172.21.15.2:0/424751364 shutdown_connections 2023-06-23T18:32:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 --2- 172.21.15.2:0/424751364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 0x7fa308106ea0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 --2- 172.21.15.2:0/424751364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3080fea60 0x7fa3080fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 --2- 172.21.15.2:0/424751364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3080ffc50 0x7fa3081000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 -- 172.21.15.2:0/424751364 >> 172.21.15.2:0/424751364 conn(0x7fa3080fa060 msgr2=0x7fa3080fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 -- 172.21.15.2:0/424751364 shutdown_connections 2023-06-23T18:32:01.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 -- 172.21.15.2:0/424751364 wait complete. 2023-06-23T18:32:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.213+0000 7fa30dcf5700 1 Processor -- start 2023-06-23T18:32:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa30dcf5700 1 -- start start 2023-06-23T18:32:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa30dcf5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3080fea60 0x7fa3080694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa30dcf5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3080ffc50 0x7fa30819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa30dcf5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 0x7fa30819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa30dcf5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30819d3f0 con 0x7fa3081005e0 2023-06-23T18:32:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa30dcf5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30819d560 con 0x7fa3080ffc50 2023-06-23T18:32:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa30dcf5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30819d720 con 0x7fa3080fea60 2023-06-23T18:32:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 0x7fa30819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 0x7fa30819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55994/0 (socket says 172.21.15.2:55994) 2023-06-23T18:32:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 -- 172.21.15.2:0/1582242757 learned_addr learned my addr 172.21.15.2:0/1582242757 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa3077fe700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3080fea60 0x7fa3080694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa306ffd700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3080ffc50 0x7fa30819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 -- 172.21.15.2:0/1582242757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3080fea60 msgr2=0x7fa3080694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3080fea60 0x7fa3080694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 -- 172.21.15.2:0/1582242757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3080ffc50 msgr2=0x7fa30819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3080ffc50 0x7fa30819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 -- 172.21.15.2:0/1582242757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa300009d00 con 0x7fa3081005e0 2023-06-23T18:32:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.214+0000 7fa307fff700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 0x7fa30819cc50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa30000bad0 tx=0x7fa30000bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.215+0000 7fa304ff9700 1 -- 172.21.15.2:0/1582242757 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30000e040 con 0x7fa3081005e0 2023-06-23T18:32:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.215+0000 7fa304ff9700 1 -- 172.21.15.2:0/1582242757 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa300004e40 con 0x7fa3081005e0 2023-06-23T18:32:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.215+0000 7fa30dcf5700 1 -- 172.21.15.2:0/1582242757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30819d9a0 con 0x7fa3081005e0 2023-06-23T18:32:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.215+0000 7fa304ff9700 1 -- 172.21.15.2:0/1582242757 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000052a0 con 0x7fa3081005e0 2023-06-23T18:32:01.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.215+0000 7fa30dcf5700 1 -- 172.21.15.2:0/1582242757 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30819de20 con 0x7fa3081005e0 2023-06-23T18:32:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.217+0000 7fa304ff9700 1 -- 172.21.15.2:0/1582242757 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa30001d990 con 0x7fa3081005e0 2023-06-23T18:32:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.218+0000 7fa304ff9700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa2f00a1150 0x7fa2f00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.218+0000 7fa304ff9700 1 -- 172.21.15.2:0/1582242757 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3000ce200 con 0x7fa3081005e0 2023-06-23T18:32:01.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.218+0000 7fa3077fe700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa2f00a1150 0x7fa2f00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.218+0000 7fa3077fe700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa2f00a1150 0x7fa2f00a3600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa3080ffab0 tx=0x7fa2f800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa319e7ab80 1 Processor -- start 2023-06-23T18:32:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa319e7ab80 1 -- start start 2023-06-23T18:32:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682243070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x562682241a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x5626822423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682146cb0 con 0x562682242c60 2023-06-23T18:32:01.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682199a40 con 0x5626822435b0 2023-06-23T18:32:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682192ed0 con 0x562682241f40 2023-06-23T18:32:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa3077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682243070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa3077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682243070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:55998/0 (socket says 172.21.15.2:55998) 2023-06-23T18:32:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa3077fe700 1 -- 172.21.15.2:0/3284343710 learned_addr learned my addr 172.21.15.2:0/3284343710 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa307fff700 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x5626822423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.219+0000 7fa306ffd700 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x562682241a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa307fff700 1 -- 172.21.15.2:0/3284343710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 msgr2=0x562682241a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa307fff700 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x562682241a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa307fff700 1 -- 172.21.15.2:0/3284343710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 msgr2=0x562682243070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa307fff700 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682243070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa307fff700 1 -- 172.21.15.2:0/3284343710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562682014630 con 0x562682241f40 2023-06-23T18:32:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa3077fe700 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682243070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa307fff700 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x5626822423e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa3000062a0 tx=0x7fa3000973e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c65563070681713 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa30dcf5700 1 -- 172.21.15.2:0/3284343710 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000bb0c0 con 0x562682241f40 2023-06-23T18:32:01.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa30dcf5700 1 -- 172.21.15.2:0/3284343710 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa30009b6a0 con 0x562682241f40 2023-06-23T18:32:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.220+0000 7fa30dcf5700 1 -- 172.21.15.2:0/3284343710 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30009b900 con 0x562682241f40 2023-06-23T18:32:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3284343710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 msgr2=0x5626822423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x5626822423e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa3000062a0 tx=0x7fa3000973e0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3284343710 shutdown_connections 2023-06-23T18:32:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x5626822423e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682243070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/3284343710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x562682241a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3284343710 >> 172.21.15.2:0/3284343710 conn(0x5626822373f0 msgr2=0x56268223c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3284343710 shutdown_connections 2023-06-23T18:32:01.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3284343710 wait complete. 2023-06-23T18:32:01.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.221+0000 7fa319e7ab80 1 -- start start 2023-06-23T18:32:01.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa319e7ab80 1 Processor -- start 2023-06-23T18:32:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x56268224b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682242c60 0x56268224b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626822435b0 0x562682252880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682132ac0 con 0x5626822435b0 2023-06-23T18:32:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626821327c0 con 0x562682242c60 2023-06-23T18:32:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562682016920 con 0x562682241f40 2023-06-23T18:32:01.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa319e7ab80 10 client.0 _ll_register_callbacks cb 0x562682138260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa307fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626822435b0 0x562682252880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa307fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626822435b0 0x562682252880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56006/0 (socket says 172.21.15.2:56006) 2023-06-23T18:32:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa307fff700 1 -- 172.21.15.2:0/367991705 learned_addr learned my addr 172.21.15.2:0/367991705 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa3077fe700 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x56268224b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.222+0000 7fa306ffd700 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682242c60 0x56268224b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa307fff700 1 -- 172.21.15.2:0/367991705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 msgr2=0x56268224b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa307fff700 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x56268224b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa307fff700 1 -- 172.21.15.2:0/367991705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682242c60 msgr2=0x56268224b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa307fff700 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682242c60 0x56268224b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa307fff700 1 -- 172.21.15.2:0/367991705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f8010c50 con 0x5626822435b0 2023-06-23T18:32:01.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa307fff700 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626822435b0 0x562682252880 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa300098110 tx=0x7fa30009b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e895030f7bb9852 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa2e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:01.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/367991705 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30009f6d0 con 0x5626822435b0 2023-06-23T18:32:01.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/367991705 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3000c5e40 con 0x5626822435b0 2023-06-23T18:32:01.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa319e7ab80 1 -- 172.21.15.2:0/367991705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562682234bb0 con 0x5626822435b0 2023-06-23T18:32:01.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/367991705 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3000bb080 con 0x5626822435b0 2023-06-23T18:32:01.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa319e7ab80 10 client.16200 fetch_fsmap learned FSMap version 32 2023-06-23T18:32:01.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa319e7ab80 1 -- 172.21.15.2:0/367991705 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7fa30819de20 con 0x5626822435b0 2023-06-23T18:32:01.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.223+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/367991705 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa300065080 con 0x5626822435b0 2023-06-23T18:32:01.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/367991705 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa30001d470 con 0x5626822435b0 2023-06-23T18:32:01.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa319e7ab80 10 client.16200 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:32:01.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa319e7ab80 10 client.16200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:01.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa319e7ab80 20 client.16200 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562682256b60 0x562682258f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa319e7ab80 4 client.16200 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:32:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa319e7ab80 1 -- 172.21.15.2:0/367991705 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56268218beb0 con 0x562682256b60 2023-06-23T18:32:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa3077fe700 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562682256b60 0x562682258f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa3077fe700 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562682256b60 0x562682258f40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd74d2320b5769a0 server_cookie=f48261c2e5bad84f in_seq=0 out_seq=0 2023-06-23T18:32:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.224+0000 7fa2e6ffd700 10 client.16200 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/367991705 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56268218beb0 con 0x562682256b60 2023-06-23T18:32:01.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa2e6ffd700 10 client.16200 handle_command_reply: tid=0 2023-06-23T18:32:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 client.16200 shutdown 2023-06-23T18:32:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 -- 172.21.15.2:0/367991705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626822435b0 msgr2=0x562682252880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626822435b0 0x562682252880 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa300098110 tx=0x7fa30009b0c0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 -- 172.21.15.2:0/367991705 shutdown_connections 2023-06-23T18:32:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562682256b60 0x562682258f40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5626822435b0 0x562682252880 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682241f40 0x56268224b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa2e6ffd700 0 client.16200 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/367991705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682242c60 0x56268224b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.225+0000 7fa319e7ab80 1 -- 172.21.15.2:0/367991705 >> 172.21.15.2:0/367991705 conn(0x5626822373f0 msgr2=0x562682236c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.226+0000 7fa319e7ab80 1 -- 172.21.15.2:0/367991705 shutdown_connections 2023-06-23T18:32:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.226+0000 7fa319e7ab80 1 -- 172.21.15.2:0/367991705 wait complete. 2023-06-23T18:32:01.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.226+0000 7fa319e7ab80 20 client.16200 trim_cache size 0 max 16384 2023-06-23T18:32:01.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa319e7ab80 1 Processor -- start 2023-06-23T18:32:01.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa319e7ab80 1 -- start start 2023-06-23T18:32:01.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682241f40 0x562682241a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682252810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626822435b0 0x56268223c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268224ae70 con 0x562682242c60 2023-06-23T18:32:01.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626822388c0 con 0x562682241f40 2023-06-23T18:32:01.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268223e1f0 con 0x5626822435b0 2023-06-23T18:32:01.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa306ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682252810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.250+0000 7fa306ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682252810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56014/0 (socket says 172.21.15.2:56014) 2023-06-23T18:32:01.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa306ffd700 1 -- 172.21.15.2:0/3194478107 learned_addr learned my addr 172.21.15.2:0/3194478107 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa307fff700 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626822435b0 0x56268223c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa3077fe700 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682241f40 0x562682241a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa306ffd700 1 -- 172.21.15.2:0/3194478107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626822435b0 msgr2=0x56268223c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa306ffd700 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626822435b0 0x56268223c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa306ffd700 1 -- 172.21.15.2:0/3194478107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682241f40 msgr2=0x562682241a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa306ffd700 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682241f40 0x562682241a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa306ffd700 1 -- 172.21.15.2:0/3194478107 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562681fc0de0 con 0x562682242c60 2023-06-23T18:32:01.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa306ffd700 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682252810 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009f00 tx=0x7fa2fc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1380a237f631e1d8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa2edffb700 1 -- 172.21.15.2:0/3194478107 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc020070 con 0x562682242c60 2023-06-23T18:32:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa2edffb700 1 -- 172.21.15.2:0/3194478107 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2fc018040 con 0x562682242c60 2023-06-23T18:32:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.251+0000 7fa2edffb700 1 -- 172.21.15.2:0/3194478107 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc017050 con 0x562682242c60 2023-06-23T18:32:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3194478107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 msgr2=0x562682252810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682252810 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009f00 tx=0x7fa2fc009260 comp rx=0 tx=0).stop 2023-06-23T18:32:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3194478107 shutdown_connections 2023-06-23T18:32:01.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626822435b0 0x56268223c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562682241f40 0x562682241a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/3194478107 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682242c60 0x562682252810 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3194478107 >> 172.21.15.2:0/3194478107 conn(0x5626822373f0 msgr2=0x56268225d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3194478107 shutdown_connections 2023-06-23T18:32:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.252+0000 7fa319e7ab80 1 -- 172.21.15.2:0/3194478107 wait complete. 2023-06-23T18:32:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 1 -- start start 2023-06-23T18:32:01.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 1 Processor -- start 2023-06-23T18:32:01.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682241f40 0x5626822522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682242c60 0x562682252830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x56268224efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268224f6b0 con 0x562682241f40 2023-06-23T18:32:01.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268224f820 con 0x5626822435b0 2023-06-23T18:32:01.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56268224f9e0 con 0x562682242c60 2023-06-23T18:32:01.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa319e7ab80 10 client.0 _ll_register_callbacks cb 0x562682138260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:01.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa3077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682241f40 0x5626822522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa3077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682241f40 0x5626822522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56018/0 (socket says 172.21.15.2:56018) 2023-06-23T18:32:01.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa3077fe700 1 -- 172.21.15.2:0/2185921172 learned_addr learned my addr 172.21.15.2:0/2185921172 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:01.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa306ffd700 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682242c60 0x562682252830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.253+0000 7fa307fff700 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x56268224efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa3077fe700 1 -- 172.21.15.2:0/2185921172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682242c60 msgr2=0x562682252830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa3077fe700 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682242c60 0x562682252830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa3077fe700 1 -- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 msgr2=0x56268224efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:01.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa3077fe700 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x56268224efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa3077fe700 1 -- 172.21.15.2:0/2185921172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc009bb0 con 0x562682241f40 2023-06-23T18:32:01.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa307fff700 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x56268224efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:32:01.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa3077fe700 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682241f40 0x5626822522f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800bb40 tx=0x7fa2f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e198f4b4fb7314dd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:01.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa2e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:01.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/2185921172 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8024040 con 0x562682241f40 2023-06-23T18:32:01.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/2185921172 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa2f8017070 con 0x562682241f40 2023-06-23T18:32:01.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa319e7ab80 1 -- 172.21.15.2:0/2185921172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa30009b6a0 con 0x562682241f40 2023-06-23T18:32:01.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/2185921172 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2f80295a0 con 0x562682241f40 2023-06-23T18:32:01.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa319e7ab80 10 client.16206 fetch_fsmap learned FSMap version 32 2023-06-23T18:32:01.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa319e7ab80 1 -- 172.21.15.2:0/2185921172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x562681fa9c60 con 0x562682241f40 2023-06-23T18:32:01.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.254+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/2185921172 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa30009b6a0 con 0x562682241f40 2023-06-23T18:32:01.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/2185921172 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 32) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2f8003710 con 0x562682241f40 2023-06-23T18:32:01.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa319e7ab80 10 client.16206 fetch_fsmap finished waiting for FSMap version 32 2023-06-23T18:32:01.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa319e7ab80 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:01.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa319e7ab80 20 client.16206 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:01.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5626822ac250 0x5626822ae660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:01.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa319e7ab80 4 client.16206 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:32:01.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa319e7ab80 1 -- 172.21.15.2:0/2185921172 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56268214e050 con 0x5626822ac250 2023-06-23T18:32:01.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa306ffd700 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5626822ac250 0x5626822ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:01.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa306ffd700 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5626822ac250 0x5626822ae660 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441cc332e834908f server_cookie=ab4bbedcc692be22 in_seq=0 out_seq=0 2023-06-23T18:32:01.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.255+0000 7fa2e6ffd700 10 client.16206 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:01.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa2e6ffd700 1 -- 172.21.15.2:0/2185921172 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56268214e050 con 0x5626822ac250 2023-06-23T18:32:01.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa2e6ffd700 10 client.16206 handle_command_reply: tid=0 2023-06-23T18:32:01.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 client.16206 shutdown 2023-06-23T18:32:01.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 -- 172.21.15.2:0/2185921172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682241f40 msgr2=0x5626822522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682241f40 0x5626822522f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fa2f800bb40 tx=0x7fa2f80047c0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 -- 172.21.15.2:0/2185921172 shutdown_connections 2023-06-23T18:32:01.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5626822ac250 0x5626822ae660 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5626822435b0 0x56268224efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562682241f40 0x5626822522f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 --2- 172.21.15.2:0/2185921172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562682242c60 0x562682252830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa2e6ffd700 0 client.16206 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:01.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 -- 172.21.15.2:0/2185921172 >> 172.21.15.2:0/2185921172 conn(0x5626822373f0 msgr2=0x56268223c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 -- 172.21.15.2:0/2185921172 shutdown_connections 2023-06-23T18:32:01.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 1 -- 172.21.15.2:0/2185921172 wait complete. 2023-06-23T18:32:01.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.256+0000 7fa319e7ab80 20 client.16206 trim_cache size 0 max 16384 2023-06-23T18:32:01.271 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:32:01.271 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:32:01.271 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:32:01.271 INFO:teuthology.orchestra.run.smithi002.stdout: "ba47bb3d-6257-47b5-9618-16bc24b94191": { 2023-06-23T18:32:01.271 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:32:01.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "ba47bb3d-6257-47b5-9618-16bc24b94191", 2023-06-23T18:32:01.272 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:32:01.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:32:01.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:32:01.272 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:32:01.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 -- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa2f00a1150 msgr2=0x7fa2f00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa2f00a1150 0x7fa2f00a3600 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa3080ffab0 tx=0x7fa2f800b410 comp rx=0 tx=0).stop 2023-06-23T18:32:01.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 -- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 msgr2=0x7fa30819cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:01.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 0x7fa30819cc50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa30000bad0 tx=0x7fa30000bb00 comp rx=0 tx=0).stop 2023-06-23T18:32:01.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 -- 172.21.15.2:0/1582242757 shutdown_connections 2023-06-23T18:32:01.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa3081005e0 0x7fa30819cc50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3080fea60 0x7fa3080694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa2f00a1150 0x7fa2f00a3600 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 --2- 172.21.15.2:0/1582242757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3080ffc50 0x7fa30819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:01.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 -- 172.21.15.2:0/1582242757 >> 172.21.15.2:0/1582242757 conn(0x7fa3080fa060 msgr2=0x7fa3080fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:01.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 -- 172.21.15.2:0/1582242757 shutdown_connections 2023-06-23T18:32:01.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:01.257+0000 7fa2ee7fc700 1 -- 172.21.15.2:0/1582242757 wait complete. 2023-06-23T18:32:01.276 INFO:tasks.cephfs.filesystem:scrub status for tag:ba47bb3d-6257-47b5-9618-16bc24b94191 - {'path': '/', 'tag': 'ba47bb3d-6257-47b5-9618-16bc24b94191', 'options': 'recursive,force'} 2023-06-23T18:32:02.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: pgmap v596: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:32:02.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:32:02.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:02.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:02.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:02.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:02 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: pgmap v596: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:32:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:32:02.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:02.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:02.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:02.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:02 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: pgmap v596: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 MiB/s wr, 24 op/s 2023-06-23T18:32:02.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:32:02.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:02.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:02.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:02.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:02.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:02 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:04.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:04 smithi002 ceph-mon[95947]: pgmap v597: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 52 MiB/s wr, 28 op/s 2023-06-23T18:32:04.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:04 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:32:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:04 smithi082 ceph-mon[97889]: pgmap v597: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 52 MiB/s wr, 28 op/s 2023-06-23T18:32:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:04 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:32:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:04 smithi017 ceph-mon[97472]: pgmap v597: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 52 MiB/s wr, 28 op/s 2023-06-23T18:32:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:04 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:32:05.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:32:05.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:32:05.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:32:05.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:32:05.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:32:05.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:32:05.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:32:05.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:32:05.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:32:06.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:06 smithi017 ceph-mon[97472]: pgmap v598: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 50 MiB/s wr, 25 op/s 2023-06-23T18:32:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:06 smithi002 ceph-mon[95947]: pgmap v598: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 50 MiB/s wr, 25 op/s 2023-06-23T18:32:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:06 smithi082 ceph-mon[97889]: pgmap v598: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 50 MiB/s wr, 25 op/s 2023-06-23T18:32:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:08 smithi017 ceph-mon[97472]: pgmap v599: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:32:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:08 smithi002 ceph-mon[95947]: pgmap v599: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:32:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:08 smithi082 ceph-mon[97889]: pgmap v599: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 47 MiB/s wr, 26 op/s 2023-06-23T18:32:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:10 smithi017 ceph-mon[97472]: pgmap v600: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:32:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:10 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:32:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:10 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:32:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:10 smithi002 ceph-mon[95947]: pgmap v600: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:32:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:10 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:32:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:10 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:32:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:10 smithi082 ceph-mon[97889]: pgmap v600: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:32:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:10 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:32:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:10 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:32:11.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:11 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:32:12.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:11 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:32:12.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:11 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:32:12.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:12 smithi017 ceph-mon[97472]: pgmap v601: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 46 MiB/s wr, 23 op/s 2023-06-23T18:32:13.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:12 smithi002 ceph-mon[95947]: pgmap v601: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 46 MiB/s wr, 23 op/s 2023-06-23T18:32:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:12 smithi082 ceph-mon[97889]: pgmap v601: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 46 MiB/s wr, 23 op/s 2023-06-23T18:32:15.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:14 smithi002 ceph-mon[95947]: pgmap v602: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:32:15.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:14 smithi082 ceph-mon[97889]: pgmap v602: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:32:15.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:14 smithi017 ceph-mon[97472]: pgmap v602: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 53 MiB/s wr, 29 op/s 2023-06-23T18:32:17.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:16 smithi002 ceph-mon[95947]: pgmap v603: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 50 MiB/s wr, 26 op/s 2023-06-23T18:32:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:16 smithi082 ceph-mon[97889]: pgmap v603: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 50 MiB/s wr, 26 op/s 2023-06-23T18:32:17.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:16 smithi017 ceph-mon[97472]: pgmap v603: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 50 MiB/s wr, 26 op/s 2023-06-23T18:32:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:18 smithi002 ceph-mon[95947]: pgmap v604: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:18 smithi082 ceph-mon[97889]: pgmap v604: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:18 smithi017 ceph-mon[97472]: pgmap v604: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:21.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:20 smithi002 ceph-mon[95947]: pgmap v605: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:32:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:20 smithi082 ceph-mon[97889]: pgmap v605: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:32:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:20 smithi017 ceph-mon[97472]: pgmap v605: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 MiB/s wr, 26 op/s 2023-06-23T18:32:23.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:22 smithi002 ceph-mon[95947]: pgmap v606: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:32:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:22 smithi082 ceph-mon[97889]: pgmap v606: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:32:23.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:22 smithi017 ceph-mon[97472]: pgmap v606: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 47 MiB/s wr, 25 op/s 2023-06-23T18:32:25.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:24 smithi002 ceph-mon[95947]: pgmap v607: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 50 MiB/s wr, 29 op/s 2023-06-23T18:32:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:24 smithi082 ceph-mon[97889]: pgmap v607: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 50 MiB/s wr, 29 op/s 2023-06-23T18:32:25.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:24 smithi017 ceph-mon[97472]: pgmap v607: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 50 MiB/s wr, 29 op/s 2023-06-23T18:32:27.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:26 smithi002 ceph-mon[95947]: pgmap v608: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:27.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:26 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:26 smithi082 ceph-mon[97889]: pgmap v608: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:26 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:26 smithi017 ceph-mon[97472]: pgmap v608: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:26 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:29.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:28 smithi002 ceph-mon[95947]: pgmap v609: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:28 smithi082 ceph-mon[97889]: pgmap v609: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:29.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:28 smithi017 ceph-mon[97472]: pgmap v609: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 47 MiB/s wr, 24 op/s 2023-06-23T18:32:29.975 INFO:teuthology.orchestra.run.smithi002.stderr:set primary keys... 2023-06-23T18:32:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:30 smithi002 ceph-mon[95947]: pgmap v610: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 50 MiB/s wr, 27 op/s 2023-06-23T18:32:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:30 smithi082 ceph-mon[97889]: pgmap v610: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 50 MiB/s wr, 27 op/s 2023-06-23T18:32:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:30 smithi017 ceph-mon[97472]: pgmap v610: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 50 MiB/s wr, 27 op/s 2023-06-23T18:32:31.277 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:32:31.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 -- 172.21.15.2:0/1764656474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 msgr2=0x7fd0e8107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/1764656474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 0x7fd0e8107c20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000b0a0 tx=0x7fd0e000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 -- 172.21.15.2:0/1764656474 shutdown_connections 2023-06-23T18:32:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/1764656474 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 0x7fd0e8107c20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/1764656474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e8100600 0x7fd0e81029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/1764656474 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e8102f20 0x7fd0e8105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 -- 172.21.15.2:0/1764656474 >> 172.21.15.2:0/1764656474 conn(0x7fd0e80fa000 msgr2=0x7fd0e80fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 -- 172.21.15.2:0/1764656474 shutdown_connections 2023-06-23T18:32:31.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 -- 172.21.15.2:0/1764656474 wait complete. 2023-06-23T18:32:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.431+0000 7fd0edfd4700 1 Processor -- start 2023-06-23T18:32:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0edfd4700 1 -- start start 2023-06-23T18:32:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0edfd4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e8100600 0x7fd0e8195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0edfd4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e8102f20 0x7fd0e8195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0edfd4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 0x7fd0e819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0edfd4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e819d380 con 0x7fd0e8105840 2023-06-23T18:32:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0edfd4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e819d4c0 con 0x7fd0e8100600 2023-06-23T18:32:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0edfd4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e819d650 con 0x7fd0e8102f20 2023-06-23T18:32:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 0x7fd0e819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 0x7fd0e819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56038/0 (socket says 172.21.15.2:56038) 2023-06-23T18:32:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e7fff700 1 -- 172.21.15.2:0/2524878350 learned_addr learned my addr 172.21.15.2:0/2524878350 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:31.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e8100600 0x7fd0e8195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e8102f20 0x7fd0e8195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e7fff700 1 -- 172.21.15.2:0/2524878350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e8102f20 msgr2=0x7fd0e8195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e8102f20 0x7fd0e8195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e7fff700 1 -- 172.21.15.2:0/2524878350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e8100600 msgr2=0x7fd0e8195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e8100600 0x7fd0e8195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e7fff700 1 -- 172.21.15.2:0/2524878350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e0009d00 con 0x7fd0e8105840 2023-06-23T18:32:31.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.432+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e8102f20 0x7fd0e8195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:32:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.433+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 0x7fd0e819cdb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000b070 tx=0x7fd0e0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.433+0000 7fd0e4ff9700 1 -- 172.21.15.2:0/2524878350 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e000e040 con 0x7fd0e8105840 2023-06-23T18:32:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.433+0000 7fd0e4ff9700 1 -- 172.21.15.2:0/2524878350 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e0003e90 con 0x7fd0e8105840 2023-06-23T18:32:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.433+0000 7fd0edfd4700 1 -- 172.21.15.2:0/2524878350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e819d8d0 con 0x7fd0e8105840 2023-06-23T18:32:31.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.433+0000 7fd0e4ff9700 1 -- 172.21.15.2:0/2524878350 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0004020 con 0x7fd0e8105840 2023-06-23T18:32:31.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.433+0000 7fd0edfd4700 1 -- 172.21.15.2:0/2524878350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e819dd50 con 0x7fd0e8105840 2023-06-23T18:32:31.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.435+0000 7fd0e4ff9700 1 -- 172.21.15.2:0/2524878350 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd0e0003a00 con 0x7fd0e8105840 2023-06-23T18:32:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.436+0000 7fd0e4ff9700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0d00a1150 0x7fd0d00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.436+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0d00a1150 0x7fd0d00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.436+0000 7fd0e4ff9700 1 -- 172.21.15.2:0/2524878350 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0e00cdff0 con 0x7fd0e8105840 2023-06-23T18:32:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.436+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0d00a1150 0x7fd0d00a3600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8005950 tx=0x7fd0d800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0fa159b80 1 Processor -- start 2023-06-23T18:32:31.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0fa159b80 1 -- start start 2023-06-23T18:32:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0640f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee0640fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee06410470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee06313cb0 con 0x55ee0640ec80 2023-06-23T18:32:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee06366a40 con 0x55ee0640f5d0 2023-06-23T18:32:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0635fed0 con 0x55ee06410000 2023-06-23T18:32:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0e77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0640f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0e77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0640f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56042/0 (socket says 172.21.15.2:56042) 2023-06-23T18:32:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0e77fe700 1 -- 172.21.15.2:0/4020598823 learned_addr learned my addr 172.21.15.2:0/4020598823 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee06410470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0e77fe700 1 -- 172.21.15.2:0/4020598823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 msgr2=0x55ee06410470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee0640fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.437+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee06410470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0e77fe700 1 -- 172.21.15.2:0/4020598823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 msgr2=0x55ee0640fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:31.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee0640fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0e77fe700 1 -- 172.21.15.2:0/4020598823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee061e1630 con 0x55ee0640ec80 2023-06-23T18:32:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee06410470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:32:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee0640fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:32:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0640f090 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800fdb0 tx=0x7fd0d8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ac1552deaa166c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0cdffb700 1 -- 172.21.15.2:0/4020598823 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8010ac0 con 0x55ee0640ec80 2023-06-23T18:32:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0cdffb700 1 -- 172.21.15.2:0/4020598823 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0d8020070 con 0x55ee0640ec80 2023-06-23T18:32:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.438+0000 7fd0cdffb700 1 -- 172.21.15.2:0/4020598823 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8025e50 con 0x55ee0640ec80 2023-06-23T18:32:31.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 -- 172.21.15.2:0/4020598823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 msgr2=0x55ee0640f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0640f090 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800fdb0 tx=0x7fd0d8012000 comp rx=0 tx=0).stop 2023-06-23T18:32:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 -- 172.21.15.2:0/4020598823 shutdown_connections 2023-06-23T18:32:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee06410470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0640f090 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/4020598823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee0640fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 -- 172.21.15.2:0/4020598823 >> 172.21.15.2:0/4020598823 conn(0x55ee064043f0 msgr2=0x55ee06409520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:31.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 -- 172.21.15.2:0/4020598823 shutdown_connections 2023-06-23T18:32:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 -- 172.21.15.2:0/4020598823 wait complete. 2023-06-23T18:32:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.439+0000 7fd0fa159b80 1 -- start start 2023-06-23T18:32:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0fa159b80 1 Processor -- start 2023-06-23T18:32:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee064184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee06418a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee0641fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee062ffac0 con 0x55ee0640ec80 2023-06-23T18:32:31.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee062ff7c0 con 0x55ee0640f5d0 2023-06-23T18:32:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee061e3920 con 0x55ee06410000 2023-06-23T18:32:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0fa159b80 10 client.0 _ll_register_callbacks cb 0x55ee06305260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee064184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee064184d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56048/0 (socket says 172.21.15.2:56048) 2023-06-23T18:32:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e77fe700 1 -- 172.21.15.2:0/3775765506 learned_addr learned my addr 172.21.15.2:0/3775765506 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee06418a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee0641fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e77fe700 1 -- 172.21.15.2:0/3775765506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 msgr2=0x55ee0641fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:31.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee0641fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e77fe700 1 -- 172.21.15.2:0/3775765506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 msgr2=0x55ee06418a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:31.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee06418a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e77fe700 1 -- 172.21.15.2:0/3775765506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d800fa60 con 0x55ee0640ec80 2023-06-23T18:32:31.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.440+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee0641fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:32:31.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee064184d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80256d0 tx=0x7fd0d8025700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279bd1082f05c76 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:31.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:31.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/3775765506 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8029070 con 0x55ee0640ec80 2023-06-23T18:32:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3775765506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee06401bb0 con 0x55ee0640ec80 2023-06-23T18:32:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/3775765506 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0d8010b50 con 0x55ee0640ec80 2023-06-23T18:32:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/3775765506 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0d8010d10 con 0x55ee0640ec80 2023-06-23T18:32:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0fa159b80 10 client.16218 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3775765506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ee0618dde0 con 0x55ee0640ec80 2023-06-23T18:32:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.441+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/3775765506 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ee06401bb0 con 0x55ee0640ec80 2023-06-23T18:32:31.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.442+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/3775765506 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd0d8004b50 con 0x55ee0640ec80 2023-06-23T18:32:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.442+0000 7fd0fa159b80 10 client.16218 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.442+0000 7fd0fa159b80 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.442+0000 7fd0fa159b80 20 client.16218 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.442+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ee06424090 0x55ee064264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.442+0000 7fd0fa159b80 4 client.16218 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:32:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.442+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3775765506 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee06358eb0 con 0x55ee06424090 2023-06-23T18:32:31.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.443+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ee06424090 0x55ee064264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.444+0000 7fd0c6ffd700 10 client.16218 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:31.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.444+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ee06424090 0x55ee064264a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188ac9519d2340e2 server_cookie=bb5dbf0b21dee935 in_seq=0 out_seq=0 2023-06-23T18:32:31.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/3775765506 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ee06358eb0 con 0x55ee06424090 2023-06-23T18:32:31.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0c6ffd700 10 client.16218 handle_command_reply: tid=0 2023-06-23T18:32:31.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 client.16218 shutdown 2023-06-23T18:32:31.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3775765506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 msgr2=0x55ee064184d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee064184d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80256d0 tx=0x7fd0d8025700 comp rx=0 tx=0).stop 2023-06-23T18:32:31.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3775765506 shutdown_connections 2023-06-23T18:32:31.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ee06424090 0x55ee064264a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee06410000 0x55ee0641fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee064184d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3775765506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640f5d0 0x55ee06418a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3775765506 >> 172.21.15.2:0/3775765506 conn(0x55ee064043f0 msgr2=0x55ee06408c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:31.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0c6ffd700 0 client.16218 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:31.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3775765506 shutdown_connections 2023-06-23T18:32:31.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.445+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3775765506 wait complete. 2023-06-23T18:32:31.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.446+0000 7fd0fa159b80 20 client.16218 trim_cache size 0 max 16384 2023-06-23T18:32:31.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0fa159b80 1 Processor -- start 2023-06-23T18:32:31.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0fa159b80 1 -- start start 2023-06-23T18:32:31.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640ec80 0x55ee0642ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 0x55ee0641fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee06410000 0x55ee06416360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee064180a0 con 0x55ee06410000 2023-06-23T18:32:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee06417980 con 0x55ee0640ec80 2023-06-23T18:32:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee06416110 con 0x55ee0640f5d0 2023-06-23T18:32:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee06410000 0x55ee06416360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.472 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee06410000 0x55ee06416360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56058/0 (socket says 172.21.15.2:56058) 2023-06-23T18:32:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e7fff700 1 -- 172.21.15.2:0/3064903055 learned_addr learned my addr 172.21.15.2:0/3064903055 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640ec80 0x55ee0642ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 0x55ee0641fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e7fff700 1 -- 172.21.15.2:0/3064903055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 msgr2=0x55ee0641fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:31.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 0x55ee0641fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e7fff700 1 -- 172.21.15.2:0/3064903055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640ec80 msgr2=0x55ee0642ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640ec80 0x55ee0642ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.467+0000 7fd0e7fff700 1 -- 172.21.15.2:0/3064903055 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee06176c60 con 0x55ee06410000 2023-06-23T18:32:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.468+0000 7fd0e7fff700 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee06410000 0x55ee06416360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0097990 tx=0x7fd0e000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102d1146bcde641f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.468+0000 7fd0ce7fc700 1 -- 172.21.15.2:0/3064903055 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e009afd0 con 0x55ee06410000 2023-06-23T18:32:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.468+0000 7fd0ce7fc700 1 -- 172.21.15.2:0/3064903055 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0e009adf0 con 0x55ee06410000 2023-06-23T18:32:31.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.468+0000 7fd0ce7fc700 1 -- 172.21.15.2:0/3064903055 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00a3850 con 0x55ee06410000 2023-06-23T18:32:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3064903055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee06410000 msgr2=0x55ee06416360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee06410000 0x55ee06416360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0097990 tx=0x7fd0e000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3064903055 shutdown_connections 2023-06-23T18:32:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee06410000 0x55ee06416360 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee0640ec80 0x55ee0642ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/3064903055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 0x55ee0641fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3064903055 >> 172.21.15.2:0/3064903055 conn(0x55ee064043f0 msgr2=0x55ee064058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3064903055 shutdown_connections 2023-06-23T18:32:31.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 -- 172.21.15.2:0/3064903055 wait complete. 2023-06-23T18:32:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 -- start start 2023-06-23T18:32:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.469+0000 7fd0fa159b80 1 Processor -- start 2023-06-23T18:32:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0641b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 0x55ee0641ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0fa159b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee06410000 0x55ee0641ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0641ef80 con 0x55ee0640ec80 2023-06-23T18:32:31.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0641f0f0 con 0x55ee06410000 2023-06-23T18:32:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0fa159b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee0641f2b0 con 0x55ee0640f5d0 2023-06-23T18:32:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0fa159b80 10 client.0 _ll_register_callbacks cb 0x55ee06305260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0641b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0641b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56060/0 (socket says 172.21.15.2:56060) 2023-06-23T18:32:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 -- 172.21.15.2:0/2797210804 learned_addr learned my addr 172.21.15.2:0/2797210804 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 0x55ee0641ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 -- 172.21.15.2:0/2797210804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 msgr2=0x55ee0641ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 0x55ee0641ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 -- 172.21.15.2:0/2797210804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee06410000 msgr2=0x55ee0641ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee06410000 0x55ee0641ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 -- 172.21.15.2:0/2797210804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e819dd50 con 0x55ee0640ec80 2023-06-23T18:32:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0e77fe700 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0641b4e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8010010 tx=0x7fd0d801b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651a34aa2287e717 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.470+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/2797210804 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8029020 con 0x55ee0640ec80 2023-06-23T18:32:31.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/2797210804 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd0d8010d50 con 0x55ee0640ec80 2023-06-23T18:32:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 1 -- 172.21.15.2:0/2797210804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0d8020070 con 0x55ee0640ec80 2023-06-23T18:32:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/2797210804 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0d80049c0 con 0x55ee0640ec80 2023-06-23T18:32:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 10 client.16224 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 1 -- 172.21.15.2:0/2797210804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x55ee061b4250 con 0x55ee0640ec80 2023-06-23T18:32:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/2797210804 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd0d8048020 con 0x55ee0640ec80 2023-06-23T18:32:31.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/2797210804 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd0d8004fe0 con 0x55ee0640ec80 2023-06-23T18:32:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 10 client.16224 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 20 client.16224 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ee06478a40 0x55ee0647ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 4 client.16224 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:32:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0fa159b80 1 -- 172.21.15.2:0/2797210804 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ee0631b050 con 0x55ee06478a40 2023-06-23T18:32:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.471+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ee06478a40 0x55ee0647ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0c6ffd700 10 client.16224 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0e6ffd700 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ee06478a40 0x55ee0647ae50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba5c4f739b5a1fe2 server_cookie=c07d0a74a081f91c in_seq=0 out_seq=0 2023-06-23T18:32:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0c6ffd700 1 -- 172.21.15.2:0/2797210804 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ee0631b050 con 0x55ee06478a40 2023-06-23T18:32:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0c6ffd700 10 client.16224 handle_command_reply: tid=0 2023-06-23T18:32:31.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 client.16224 shutdown 2023-06-23T18:32:31.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 -- 172.21.15.2:0/2797210804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 msgr2=0x55ee0641b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0641b4e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8010010 tx=0x7fd0d801b560 comp rx=0 tx=0).stop 2023-06-23T18:32:31.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 -- 172.21.15.2:0/2797210804 shutdown_connections 2023-06-23T18:32:31.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ee06478a40 0x55ee0647ae50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ee06410000 0x55ee0641ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ee0640ec80 0x55ee0641b4e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 --2- 172.21.15.2:0/2797210804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ee0640f5d0 0x55ee0641ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0c6ffd700 0 client.16224 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:31.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.472+0000 7fd0fa159b80 1 -- 172.21.15.2:0/2797210804 >> 172.21.15.2:0/2797210804 conn(0x55ee064043f0 msgr2=0x55ee0642ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:31.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0fa159b80 1 -- 172.21.15.2:0/2797210804 shutdown_connections 2023-06-23T18:32:31.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0fa159b80 1 -- 172.21.15.2:0/2797210804 wait complete. 2023-06-23T18:32:31.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0fa159b80 20 client.16224 trim_cache size 0 max 16384 2023-06-23T18:32:31.487 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:32:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:32:31.488 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:32:31.488 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:32:31.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 -- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0d00a1150 msgr2=0x7fd0d00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0d00a1150 0x7fd0d00a3600 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8005950 tx=0x7fd0d800a300 comp rx=0 tx=0).stop 2023-06-23T18:32:31.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 -- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 msgr2=0x7fd0e819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 0x7fd0e819cdb0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000b070 tx=0x7fd0e0004c80 comp rx=0 tx=0).stop 2023-06-23T18:32:31.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 -- 172.21.15.2:0/2524878350 shutdown_connections 2023-06-23T18:32:31.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd0e8105840 0x7fd0e819cdb0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd0d00a1150 0x7fd0d00a3600 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd0e8100600 0x7fd0e8195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 --2- 172.21.15.2:0/2524878350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0e8102f20 0x7fd0e8195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 -- 172.21.15.2:0/2524878350 >> 172.21.15.2:0/2524878350 conn(0x7fd0e80fa000 msgr2=0x7fd0e8101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:31.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.473+0000 7fd0edfd4700 1 -- 172.21.15.2:0/2524878350 shutdown_connections 2023-06-23T18:32:31.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.474+0000 7fd0edfd4700 1 -- 172.21.15.2:0/2524878350 wait complete. 2023-06-23T18:32:31.491 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:32:31.492 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:32:31.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2022325482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e58100de0 msgr2=0x7f8e58101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2022325482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e58100de0 0x7f8e58101250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009990 tx=0x7f8e48009ca0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2022325482 shutdown_connections 2023-06-23T18:32:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2022325482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e58101790 0x7f8e58108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2022325482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e580ffb50 0x7f8e580fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2022325482 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e58100de0 0x7f8e58101250 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2022325482 >> 172.21.15.2:0/2022325482 conn(0x7f8e580fb150 msgr2=0x7f8e580fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2022325482 shutdown_connections 2023-06-23T18:32:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.623+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2022325482 wait complete. 2023-06-23T18:32:31.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e5d65e700 1 Processor -- start 2023-06-23T18:32:31.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e5d65e700 1 -- start start 2023-06-23T18:32:31.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e5d65e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e580ffb50 0x7f8e581957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e5d65e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e58100de0 0x7f8e58195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e5d65e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e58101790 0x7f8e5819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e5d65e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5819d360 con 0x7f8e580ffb50 2023-06-23T18:32:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e5d65e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5819d4a0 con 0x7f8e58100de0 2023-06-23T18:32:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e5d65e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5819d630 con 0x7f8e58101790 2023-06-23T18:32:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e56ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e580ffb50 0x7f8e581957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e56ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e580ffb50 0x7f8e581957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56074/0 (socket says 172.21.15.2:56074) 2023-06-23T18:32:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e56ffd700 1 -- 172.21.15.2:0/2292173448 learned_addr learned my addr 172.21.15.2:0/2292173448 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e577fe700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e58101790 0x7f8e5819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.624+0000 7f8e567fc700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e58100de0 0x7f8e58195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e56ffd700 1 -- 172.21.15.2:0/2292173448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e58101790 msgr2=0x7f8e5819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e56ffd700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e58101790 0x7f8e5819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e56ffd700 1 -- 172.21.15.2:0/2292173448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e58100de0 msgr2=0x7f8e58195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e56ffd700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e58100de0 0x7f8e58195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e56ffd700 1 -- 172.21.15.2:0/2292173448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480095f0 con 0x7f8e580ffb50 2023-06-23T18:32:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e56ffd700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e580ffb50 0x7f8e581957e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000b530 tx=0x7f8e4000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e4ffff700 1 -- 172.21.15.2:0/2292173448 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40011830 con 0x7f8e580ffb50 2023-06-23T18:32:31.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e4ffff700 1 -- 172.21.15.2:0/2292173448 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e40011e20 con 0x7f8e580ffb50 2023-06-23T18:32:31.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e4ffff700 1 -- 172.21.15.2:0/2292173448 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000f2e0 con 0x7f8e580ffb50 2023-06-23T18:32:31.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5819d910 con 0x7f8e580ffb50 2023-06-23T18:32:31.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.625+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5819de50 con 0x7f8e580ffb50 2023-06-23T18:32:31.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.626+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e58106b70 con 0x7f8e580ffb50 2023-06-23T18:32:31.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.627+0000 7f8e4ffff700 1 -- 172.21.15.2:0/2292173448 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8e40011990 con 0x7f8e580ffb50 2023-06-23T18:32:31.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.628+0000 7f8e4ffff700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8e440a1150 0x7f8e440a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:31.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.628+0000 7f8e4ffff700 1 -- 172.21.15.2:0/2292173448 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8e400cc850 con 0x7f8e580ffb50 2023-06-23T18:32:31.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.628+0000 7f8e567fc700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8e440a1150 0x7f8e440a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:31.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.628+0000 7f8e567fc700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8e440a1150 0x7f8e440a3600 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800b5c0 tx=0x7f8e48005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:31.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.631+0000 7f8e4ffff700 1 -- 172.21.15.2:0/2292173448 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8e40095fa0 con 0x7f8e580ffb50 2023-06-23T18:32:31.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.923+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e5802c810 con 0x7f8e580ffb50 2023-06-23T18:32:31.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.924+0000 7f8e4ffff700 1 -- 172.21.15.2:0/2292173448 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+10778 (secure 0 0 0) 0x7f8e400995c0 con 0x7f8e580ffb50 2023-06-23T18:32:31.928 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:32:31.928 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:32:25.697663+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:32:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8e440a1150 msgr2=0x7f8e440a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8e440a1150 0x7f8e440a3600 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800b5c0 tx=0x7f8e48005fb0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e580ffb50 msgr2=0x7f8e581957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:31.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e580ffb50 0x7f8e581957e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000b530 tx=0x7f8e4000b840 comp rx=0 tx=0).stop 2023-06-23T18:32:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 shutdown_connections 2023-06-23T18:32:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e58101790 0x7f8e5819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8e580ffb50 0x7f8e581957e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8e440a1150 0x7f8e440a3600 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 --2- 172.21.15.2:0/2292173448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e58100de0 0x7f8e58195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 >> 172.21.15.2:0/2292173448 conn(0x7f8e580fb150 msgr2=0x7f8e580fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 shutdown_connections 2023-06-23T18:32:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:31.926+0000 7f8e5d65e700 1 -- 172.21.15.2:0/2292173448 wait complete. 2023-06-23T18:32:31.932 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 33 2023-06-23T18:32:31.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:32:32.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 -- 172.21.15.2:0/677617239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c100640 msgr2=0x7f0d9c102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/677617239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c100640 0x7f0d9c102a20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009990 tx=0x7f0d8c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 -- 172.21.15.2:0/677617239 shutdown_connections 2023-06-23T18:32:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/677617239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c105880 0x7f0d9c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/677617239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c100640 0x7f0d9c102a20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/677617239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d9c102f60 0x7f0d9c105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 -- 172.21.15.2:0/677617239 >> 172.21.15.2:0/677617239 conn(0x7f0d9c0fa0e0 msgr2=0x7f0d9c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 -- 172.21.15.2:0/677617239 shutdown_connections 2023-06-23T18:32:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.108+0000 7f0da1a4a700 1 -- 172.21.15.2:0/677617239 wait complete. 2023-06-23T18:32:32.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0da1a4a700 1 Processor -- start 2023-06-23T18:32:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0da1a4a700 1 -- start start 2023-06-23T18:32:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0da1a4a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d9c100640 0x7f0d9c195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0da1a4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c102f60 0x7f0d9c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0da1a4a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c105880 0x7f0d9c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0da1a4a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c19d3c0 con 0x7f0d9c102f60 2023-06-23T18:32:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0da1a4a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c19d500 con 0x7f0d9c100640 2023-06-23T18:32:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0da1a4a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c19d690 con 0x7f0d9c105880 2023-06-23T18:32:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0d9a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c102f60 0x7f0d9c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0d9a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c102f60 0x7f0d9c195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56090/0 (socket says 172.21.15.2:56090) 2023-06-23T18:32:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0d9a7fc700 1 -- 172.21.15.2:0/1677303759 learned_addr learned my addr 172.21.15.2:0/1677303759 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c105880 0x7f0d9c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.109+0000 7f0d9affd700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d9c100640 0x7f0d9c195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0d9a7fc700 1 -- 172.21.15.2:0/1677303759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c105880 msgr2=0x7f0d9c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c105880 0x7f0d9c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0d9a7fc700 1 -- 172.21.15.2:0/1677303759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d9c100640 msgr2=0x7f0d9c195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d9c100640 0x7f0d9c195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0d9a7fc700 1 -- 172.21.15.2:0/1677303759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c0095f0 con 0x7f0d9c102f60 2023-06-23T18:32:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c102f60 0x7f0d9c195d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000d950 tx=0x7f0d9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0da0a48700 1 -- 172.21.15.2:0/1677303759 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d90009940 con 0x7f0d9c102f60 2023-06-23T18:32:32.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0da0a48700 1 -- 172.21.15.2:0/1677303759 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d90010460 con 0x7f0d9c102f60 2023-06-23T18:32:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0da1a4a700 1 -- 172.21.15.2:0/1677303759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9c19d970 con 0x7f0d9c102f60 2023-06-23T18:32:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0da0a48700 1 -- 172.21.15.2:0/1677303759 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9000f620 con 0x7f0d9c102f60 2023-06-23T18:32:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.110+0000 7f0da1a4a700 1 -- 172.21.15.2:0/1677303759 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c19de80 con 0x7f0d9c102f60 2023-06-23T18:32:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.113+0000 7f0da0a48700 1 -- 172.21.15.2:0/1677303759 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0d900105d0 con 0x7f0d9c102f60 2023-06-23T18:32:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0dadbcfb80 1 Processor -- start 2023-06-23T18:32:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0dadbcfb80 1 -- start start 2023-06-23T18:32:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0da0a48700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0d840a1150 0x7f0d840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dba74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dba7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dba8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0da0a48700 1 -- 172.21.15.2:0/1677303759 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0d900cc7a0 con 0x7f0d9c102f60 2023-06-23T18:32:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1daa7cb0 con 0x562d1dba7090 2023-06-23T18:32:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1dafaa40 con 0x562d1dba8410 2023-06-23T18:32:32.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1daf3ed0 con 0x562d1dba79e0 2023-06-23T18:32:32.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0d840a1150 0x7f0d840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0d9affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dba74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0d9affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dba74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56096/0 (socket says 172.21.15.2:56096) 2023-06-23T18:32:32.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0d9affd700 1 -- 172.21.15.2:0/3120987610 learned_addr learned my addr 172.21.15.2:0/3120987610 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dba7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.114+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dba8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d9affd700 1 -- 172.21.15.2:0/3120987610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 msgr2=0x562d1dba7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d9affd700 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dba7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d9affd700 1 -- 172.21.15.2:0/3120987610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 msgr2=0x562d1dba8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d9affd700 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dba8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d9affd700 1 -- 172.21.15.2:0/3120987610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d1d975630 con 0x562d1dba7090 2023-06-23T18:32:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dba7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0d840a1150 0x7f0d840a3600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400f870 tx=0x7f0d9400fc30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d9affd700 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dba74a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c006010 tx=0x7f0d8c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a6a7189db03dae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d81ffb700 1 -- 172.21.15.2:0/3120987610 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c005300 con 0x562d1dba7090 2023-06-23T18:32:32.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d81ffb700 1 -- 172.21.15.2:0/3120987610 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d8c01e040 con 0x562d1dba7090 2023-06-23T18:32:32.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.115+0000 7f0d81ffb700 1 -- 172.21.15.2:0/3120987610 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00b970 con 0x562d1dba7090 2023-06-23T18:32:32.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3120987610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 msgr2=0x562d1dba74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dba74a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c006010 tx=0x7f0d8c005f00 comp rx=0 tx=0).stop 2023-06-23T18:32:32.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3120987610 shutdown_connections 2023-06-23T18:32:32.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dba8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dba74a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/3120987610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dba7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3120987610 >> 172.21.15.2:0/3120987610 conn(0x562d1db983f0 msgr2=0x562d1db9d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3120987610 shutdown_connections 2023-06-23T18:32:32.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3120987610 wait complete. 2023-06-23T18:32:32.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.116+0000 7f0dadbcfb80 1 -- start start 2023-06-23T18:32:32.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0dadbcfb80 1 Processor -- start 2023-06-23T18:32:32.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba7090 0x562d1dbb0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba79e0 0x562d1dbb0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbb7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1da93ac0 con 0x562d1dba8410 2023-06-23T18:32:32.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1da937c0 con 0x562d1dba79e0 2023-06-23T18:32:32.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1d977920 con 0x562d1dba7090 2023-06-23T18:32:32.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0dadbcfb80 10 client.0 _ll_register_callbacks cb 0x562d1da99260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:32.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0d9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbb7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0d9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbb7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56106/0 (socket says 172.21.15.2:56106) 2023-06-23T18:32:32.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0d9b7fe700 1 -- 172.21.15.2:0/2238283976 learned_addr learned my addr 172.21.15.2:0/2238283976 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0d9affd700 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba7090 0x562d1dbb0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba79e0 0x562d1dbb0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0d9b7fe700 1 -- 172.21.15.2:0/2238283976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba7090 msgr2=0x562d1dbb0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.117+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba7090 0x562d1dbb0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d9b7fe700 1 -- 172.21.15.2:0/2238283976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba79e0 msgr2=0x562d1dbb0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba79e0 0x562d1dbb0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d9b7fe700 1 -- 172.21.15.2:0/2238283976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c19de80 con 0x562d1dba8410 2023-06-23T18:32:32.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba79e0 0x562d1dbb0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:32:32.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbb7eb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d94018b00 tx=0x7f0d94004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99886fa13b01 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d7affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:32.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d7affd700 1 -- 172.21.15.2:0/2238283976 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400e040 con 0x562d1dba8410 2023-06-23T18:32:32.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/2238283976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562d1db95bb0 con 0x562d1dba8410 2023-06-23T18:32:32.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d7affd700 1 -- 172.21.15.2:0/2238283976 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0d94008db0 con 0x562d1dba8410 2023-06-23T18:32:32.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d7affd700 1 -- 172.21.15.2:0/2238283976 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d94025470 con 0x562d1dba8410 2023-06-23T18:32:32.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0dadbcfb80 10 client.16242 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:32.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/2238283976 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f0d9c108890 con 0x562d1dba8410 2023-06-23T18:32:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.118+0000 7f0d7affd700 1 -- 172.21.15.2:0/2238283976 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d94034020 con 0x562d1dba8410 2023-06-23T18:32:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0d7affd700 1 -- 172.21.15.2:0/2238283976 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0d9401a440 con 0x562d1dba8410 2023-06-23T18:32:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0dadbcfb80 10 client.16242 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0dadbcfb80 10 client.16242 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:32:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0dadbcfb80 20 client.16242 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x562d1dbbc800 0x562d1dbbec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0dadbcfb80 4 client.16242 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:32:32.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/2238283976 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d1daeceb0 con 0x562d1dbbc800 2023-06-23T18:32:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0d9affd700 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x562d1dbbc800 0x562d1dbbec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0d9affd700 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x562d1dbbc800 0x562d1dbbec10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c25111cba793851 server_cookie=e5a39fda9c1cc259 in_seq=0 out_seq=0 2023-06-23T18:32:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.119+0000 7f0d7affd700 10 client.16242 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:32:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0d7affd700 1 -- 172.21.15.2:0/2238283976 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562d1daeceb0 con 0x562d1dbbc800 2023-06-23T18:32:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0d7affd700 10 client.16242 handle_command_reply: tid=0 2023-06-23T18:32:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 client.16242 shutdown 2023-06-23T18:32:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/2238283976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 msgr2=0x562d1dbb7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbb7eb0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0d94018b00 tx=0x7f0d94004b40 comp rx=0 tx=0).stop 2023-06-23T18:32:32.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/2238283976 shutdown_connections 2023-06-23T18:32:32.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x562d1dbbc800 0x562d1dbbec10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbb7eb0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba7090 0x562d1dbb0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/2238283976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba79e0 0x562d1dbb0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0d7affd700 0 client.16242 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:32:32.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.120+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/2238283976 >> 172.21.15.2:0/2238283976 conn(0x562d1db983f0 msgr2=0x562d1db9cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.121+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/2238283976 shutdown_connections 2023-06-23T18:32:32.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.121+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/2238283976 wait complete. 2023-06-23T18:32:32.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.121+0000 7f0dadbcfb80 20 client.16242 trim_cache size 0 max 16384 2023-06-23T18:32:32.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.135+0000 7f0dadbcfb80 1 Processor -- start 2023-06-23T18:32:32.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.135+0000 7f0dadbcfb80 1 -- start start 2023-06-23T18:32:32.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba7090 0x562d1dbaf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1dbb0500 con 0x562d1dba8410 2023-06-23T18:32:32.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1dbb1b50 con 0x562d1dba7090 2023-06-23T18:32:32.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1dbb1830 con 0x562d1dba79e0 2023-06-23T18:32:32.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbab2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56114/0 (socket says 172.21.15.2:56114) 2023-06-23T18:32:32.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 -- 172.21.15.2:0/3350552887 learned_addr learned my addr 172.21.15.2:0/3350552887 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9affd700 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba7090 0x562d1dbaf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 -- 172.21.15.2:0/3350552887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 msgr2=0x562d1dbb7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 -- 172.21.15.2:0/3350552887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba7090 msgr2=0x562d1dbaf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba7090 0x562d1dbaf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 -- 172.21.15.2:0/3350552887 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d1dba8ab0 con 0x562d1dba8410 2023-06-23T18:32:32.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:32:32.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbab2d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0d9401a7c0 tx=0x7f0d9400f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0bd40928ee92db server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.136+0000 7f0d827fc700 1 -- 172.21.15.2:0/3350552887 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9400e030 con 0x562d1dba8410 2023-06-23T18:32:32.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0d827fc700 1 -- 172.21.15.2:0/3350552887 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d94034050 con 0x562d1dba8410 2023-06-23T18:32:32.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0d827fc700 1 -- 172.21.15.2:0/3350552887 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94008e10 con 0x562d1dba8410 2023-06-23T18:32:32.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3350552887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 msgr2=0x562d1dbab2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbab2d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0d9401a7c0 tx=0x7f0d9400f370 comp rx=0 tx=0).stop 2023-06-23T18:32:32.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3350552887 shutdown_connections 2023-06-23T18:32:32.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba8410 0x562d1dbab2d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba7090 0x562d1dbaf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/3350552887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3350552887 >> 172.21.15.2:0/3350552887 conn(0x562d1db983f0 msgr2=0x562d1dbc3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3350552887 shutdown_connections 2023-06-23T18:32:32.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.137+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/3350552887 wait complete. 2023-06-23T18:32:32.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 1 -- start start 2023-06-23T18:32:32.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 1 Processor -- start 2023-06-23T18:32:32.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dbb3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dbb7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1dbb7570 con 0x562d1dba7090 2023-06-23T18:32:32.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1dbb76e0 con 0x562d1dba8410 2023-06-23T18:32:32.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d1dbb78a0 con 0x562d1dba79e0 2023-06-23T18:32:32.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0dadbcfb80 10 client.0 _ll_register_callbacks cb 0x562d1da99260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:32.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.138+0000 7f0d9affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dbb3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dbb3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56116/0 (socket says 172.21.15.2:56116) 2023-06-23T18:32:32.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9affd700 1 -- 172.21.15.2:0/1338845690 learned_addr learned my addr 172.21.15.2:0/1338845690 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dbb7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9affd700 1 -- 172.21.15.2:0/1338845690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 msgr2=0x562d1dbb4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9affd700 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9affd700 1 -- 172.21.15.2:0/1338845690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 msgr2=0x562d1dbb7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9affd700 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dbb7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9affd700 1 -- 172.21.15.2:0/1338845690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c108890 con 0x562d1dba7090 2023-06-23T18:32:32.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9b7fe700 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dbb7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:32:32.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d9affd700 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dbb3ad0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009990 tx=0x7f0d8c005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d52b5cdcd29805 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d7affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:32.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d7affd700 1 -- 172.21.15.2:0/1338845690 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c005440 con 0x562d1dba7090 2023-06-23T18:32:32.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d7affd700 1 -- 172.21.15.2:0/1338845690 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0d8c0300c0 con 0x562d1dba7090 2023-06-23T18:32:32.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/1338845690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0d8c01e040 con 0x562d1dba7090 2023-06-23T18:32:32.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0d7affd700 1 -- 172.21.15.2:0/1338845690 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0d8c030280 con 0x562d1dba7090 2023-06-23T18:32:32.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0dadbcfb80 10 client.16248 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:32.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.139+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/1338845690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x562d1d921de0 con 0x562d1dba7090 2023-06-23T18:32:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0d7affd700 1 -- 172.21.15.2:0/1338845690 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d8c01e040 con 0x562d1dba7090 2023-06-23T18:32:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0d7affd700 1 -- 172.21.15.2:0/1338845690 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0d8c00b710 con 0x562d1dba7090 2023-06-23T18:32:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0dadbcfb80 10 client.16248 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0dadbcfb80 10 client.16248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:32:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0dadbcfb80 20 client.16248 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x562d1dc11080 0x562d1dc13460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0dadbcfb80 4 client.16248 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:32:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/1338845690 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562d1daaf050 con 0x562d1dc11080 2023-06-23T18:32:32.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x562d1dc11080 0x562d1dc13460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0d9a7fc700 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x562d1dc11080 0x562d1dc13460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=116e53c5fec2dcc4 server_cookie=dc142d05f0e5188 in_seq=0 out_seq=0 2023-06-23T18:32:32.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.140+0000 7f0d7affd700 10 client.16248 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:32:32.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0d7affd700 1 -- 172.21.15.2:0/1338845690 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562d1daaf050 con 0x562d1dc11080 2023-06-23T18:32:32.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0d7affd700 10 client.16248 handle_command_reply: tid=0 2023-06-23T18:32:32.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 client.16248 shutdown 2023-06-23T18:32:32.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/1338845690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 msgr2=0x562d1dbb3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dbb3ad0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009990 tx=0x7f0d8c005100 comp rx=0 tx=0).stop 2023-06-23T18:32:32.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/1338845690 shutdown_connections 2023-06-23T18:32:32.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x562d1dc11080 0x562d1dc13460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562d1dba8410 0x562d1dbb7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562d1dba7090 0x562d1dbb3ad0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 --2- 172.21.15.2:0/1338845690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562d1dba79e0 0x562d1dbb4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/1338845690 >> 172.21.15.2:0/1338845690 conn(0x562d1db983f0 msgr2=0x562d1dbaf610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0d7affd700 0 client.16248 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:32:32.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/1338845690 shutdown_connections 2023-06-23T18:32:32.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 1 -- 172.21.15.2:0/1338845690 wait complete. 2023-06-23T18:32:32.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.141+0000 7f0dadbcfb80 20 client.16248 trim_cache size 0 max 16384 2023-06-23T18:32:32.155 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:32:32.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 -- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0d840a1150 msgr2=0x7f0d840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0d840a1150 0x7f0d840a3600 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400f870 tx=0x7f0d9400fc30 comp rx=0 tx=0).stop 2023-06-23T18:32:32.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 -- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c102f60 msgr2=0x7f0d9c195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c102f60 0x7f0d9c195d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000d950 tx=0x7f0d9000dc60 comp rx=0 tx=0).stop 2023-06-23T18:32:32.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 -- 172.21.15.2:0/1677303759 shutdown_connections 2023-06-23T18:32:32.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9c105880 0x7f0d9c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f0d840a1150 0x7f0d840a3600 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d9c100640 0x7f0d9c195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 --2- 172.21.15.2:0/1677303759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0d9c102f60 0x7f0d9c195d80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 -- 172.21.15.2:0/1677303759 >> 172.21.15.2:0/1677303759 conn(0x7f0d9c0fa0e0 msgr2=0x7f0d9c101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 -- 172.21.15.2:0/1677303759 shutdown_connections 2023-06-23T18:32:32.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.142+0000 7f0da1a4a700 1 -- 172.21.15.2:0/1677303759 wait complete. 2023-06-23T18:32:32.160 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:32:32.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 -- 172.21.15.2:0/3682982159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 msgr2=0x7efd980fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/3682982159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 0x7efd980fee90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efd88009990 tx=0x7efd88009ca0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 -- 172.21.15.2:0/3682982159 shutdown_connections 2023-06-23T18:32:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/3682982159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd98100600 0x7efd98106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/3682982159 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 0x7efd980fee90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/3682982159 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd980ffc70 0x7efd981000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 -- 172.21.15.2:0/3682982159 >> 172.21.15.2:0/3682982159 conn(0x7efd980fa0e0 msgr2=0x7efd980fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 -- 172.21.15.2:0/3682982159 shutdown_connections 2023-06-23T18:32:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.302+0000 7efd9d5b4700 1 -- 172.21.15.2:0/3682982159 wait complete. 2023-06-23T18:32:32.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.303+0000 7efd9d5b4700 1 Processor -- start 2023-06-23T18:32:32.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.303+0000 7efd9d5b4700 1 -- start start 2023-06-23T18:32:32.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.303+0000 7efd9d5b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 0x7efd98068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.303+0000 7efd9d5b4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd980ffc70 0x7efd9819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.303+0000 7efd9d5b4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd98100600 0x7efd9819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.303+0000 7efd9d5b4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9819d2f0 con 0x7efd980fea80 2023-06-23T18:32:32.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.303+0000 7efd9d5b4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9819d460 con 0x7efd980ffc70 2023-06-23T18:32:32.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.303+0000 7efd9d5b4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9819d620 con 0x7efd98100600 2023-06-23T18:32:32.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 0x7efd98068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 0x7efd98068310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56130/0 (socket says 172.21.15.2:56130) 2023-06-23T18:32:32.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 -- 172.21.15.2:0/2912250167 learned_addr learned my addr 172.21.15.2:0/2912250167 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd967fc700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd980ffc70 0x7efd9819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd977fe700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd98100600 0x7efd9819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 -- 172.21.15.2:0/2912250167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd98100600 msgr2=0x7efd9819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd98100600 0x7efd9819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 -- 172.21.15.2:0/2912250167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd980ffc70 msgr2=0x7efd9819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd980ffc70 0x7efd9819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 -- 172.21.15.2:0/2912250167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880095f0 con 0x7efd980fea80 2023-06-23T18:32:32.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd96ffd700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 0x7efd98068310 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efd88000c00 tx=0x7efd88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd7ffff700 1 -- 172.21.15.2:0/2912250167 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8801d070 con 0x7efd980fea80 2023-06-23T18:32:32.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd9d5b4700 1 -- 172.21.15.2:0/2912250167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9819d8a0 con 0x7efd980fea80 2023-06-23T18:32:32.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd7ffff700 1 -- 172.21.15.2:0/2912250167 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efd8800bd50 con 0x7efd980fea80 2023-06-23T18:32:32.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd7ffff700 1 -- 172.21.15.2:0/2912250167 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88022cd0 con 0x7efd980fea80 2023-06-23T18:32:32.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.304+0000 7efd9d5b4700 1 -- 172.21.15.2:0/2912250167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9819dcc0 con 0x7efd980fea80 2023-06-23T18:32:32.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.307+0000 7efd7ffff700 1 -- 172.21.15.2:0/2912250167 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7efd88022470 con 0x7efd980fea80 2023-06-23T18:32:32.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.307+0000 7efd7ffff700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efd800a1150 0x7efd800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.307+0000 7efd7ffff700 1 -- 172.21.15.2:0/2912250167 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efd880cddc0 con 0x7efd980fea80 2023-06-23T18:32:32.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.307+0000 7efd967fc700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efd800a1150 0x7efd800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.308+0000 7efd967fc700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efd800a1150 0x7efd800a3600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009c60 tx=0x7efd8c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.308+0000 7efda9739b80 1 Processor -- start 2023-06-23T18:32:32.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.308+0000 7efda9739b80 1 -- start start 2023-06-23T18:32:32.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.308+0000 7efda9739b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 0x5559f51274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.308+0000 7efda9739b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f51279e0 0x5559f5127e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efda9739b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f5128410 0x5559f5128880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efda9739b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f5027cb0 con 0x5559f51279e0 2023-06-23T18:32:32.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efda9739b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f507aa40 con 0x5559f5128410 2023-06-23T18:32:32.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efda9739b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f5073ed0 con 0x5559f5127090 2023-06-23T18:32:32.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f51279e0 0x5559f5127e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f51279e0 0x5559f5127e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56138/0 (socket says 172.21.15.2:56138) 2023-06-23T18:32:32.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 -- 172.21.15.2:0/1008927695 learned_addr learned my addr 172.21.15.2:0/1008927695 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd96ffd700 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 0x5559f51274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd977fe700 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f5128410 0x5559f5128880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 -- 172.21.15.2:0/1008927695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 msgr2=0x5559f51274a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 0x5559f51274a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 -- 172.21.15.2:0/1008927695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f5128410 msgr2=0x5559f5128880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f5128410 0x5559f5128880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 -- 172.21.15.2:0/1008927695 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559f4ef5630 con 0x5559f51279e0 2023-06-23T18:32:32.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd967fc700 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f51279e0 0x5559f5127e80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00e4e0 tx=0x7efd8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afc13d98b06c92c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd7d7fa700 1 -- 172.21.15.2:0/1008927695 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c00ee00 con 0x5559f51279e0 2023-06-23T18:32:32.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd7d7fa700 1 -- 172.21.15.2:0/1008927695 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efd8c021070 con 0x5559f51279e0 2023-06-23T18:32:32.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.309+0000 7efd7d7fa700 1 -- 172.21.15.2:0/1008927695 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c01d740 con 0x5559f51279e0 2023-06-23T18:32:32.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 -- 172.21.15.2:0/1008927695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f51279e0 msgr2=0x5559f5127e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f51279e0 0x5559f5127e80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00e4e0 tx=0x7efd8c014000 comp rx=0 tx=0).stop 2023-06-23T18:32:32.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 -- 172.21.15.2:0/1008927695 shutdown_connections 2023-06-23T18:32:32.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f5128410 0x5559f5128880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 0x5559f51274a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 --2- 172.21.15.2:0/1008927695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f51279e0 0x5559f5127e80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 -- 172.21.15.2:0/1008927695 >> 172.21.15.2:0/1008927695 conn(0x5559f51183f0 msgr2=0x5559f511d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 -- 172.21.15.2:0/1008927695 shutdown_connections 2023-06-23T18:32:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.310+0000 7efda9739b80 1 -- 172.21.15.2:0/1008927695 wait complete. 2023-06-23T18:32:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 1 -- start start 2023-06-23T18:32:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 1 Processor -- start 2023-06-23T18:32:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 0x5559f5130950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f5130e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5128410 0x5559f5137ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f5013ac0 con 0x5559f5128410 2023-06-23T18:32:32.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f50137c0 con 0x5559f51279e0 2023-06-23T18:32:32.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f4ef7920 con 0x5559f5127090 2023-06-23T18:32:32.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efda9739b80 10 client.0 _ll_register_callbacks cb 0x5559f5019260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:32.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd977fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5128410 0x5559f5137ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd977fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5128410 0x5559f5137ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56148/0 (socket says 172.21.15.2:56148) 2023-06-23T18:32:32.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd977fe700 1 -- 172.21.15.2:0/3689135075 learned_addr learned my addr 172.21.15.2:0/3689135075 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd967fc700 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f5130e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd96ffd700 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 0x5559f5130950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd977fe700 1 -- 172.21.15.2:0/3689135075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 msgr2=0x5559f5130950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd977fe700 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 0x5559f5130950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd977fe700 1 -- 172.21.15.2:0/3689135075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 msgr2=0x5559f5130e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd977fe700 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f5130e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.311+0000 7efd977fe700 1 -- 172.21.15.2:0/3689135075 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c017070 con 0x5559f5128410 2023-06-23T18:32:32.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efd977fe700 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5128410 0x5559f5137ed0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efd90011f60 tx=0x7efd90014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76845847fb8b90 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efd7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efd7aff5700 1 -- 172.21.15.2:0/3689135075 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9000e040 con 0x5559f5128410 2023-06-23T18:32:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efd7aff5700 1 -- 172.21.15.2:0/3689135075 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efd900090d0 con 0x5559f5128410 2023-06-23T18:32:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 1 -- 172.21.15.2:0/3689135075 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559f5115bb0 con 0x5559f5128410 2023-06-23T18:32:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efd7aff5700 1 -- 172.21.15.2:0/3689135075 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd90009290 con 0x5559f5128410 2023-06-23T18:32:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 10 client.16260 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 1 -- 172.21.15.2:0/3689135075 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5559f4ea1de0 con 0x5559f5128410 2023-06-23T18:32:32.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efd7aff5700 1 -- 172.21.15.2:0/3689135075 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5559f5115bb0 con 0x5559f5128410 2023-06-23T18:32:32.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efd7aff5700 1 -- 172.21.15.2:0/3689135075 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7efd9001c020 con 0x5559f5128410 2023-06-23T18:32:32.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 10 client.16260 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:32.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 10 client.16260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:32:32.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 20 client.16260 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:32.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5559f513c830 0x5559f513ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 4 client.16260 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:32:32.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.312+0000 7efda9739b80 1 -- 172.21.15.2:0/3689135075 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559f506ceb0 con 0x5559f513c830 2023-06-23T18:32:32.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.313+0000 7efd96ffd700 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5559f513c830 0x5559f513ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.313+0000 7efd7aff5700 10 client.16260 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:32:32.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.313+0000 7efd96ffd700 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5559f513c830 0x5559f513ec40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2baee2f51edecc1 server_cookie=8f96a793194f0852 in_seq=0 out_seq=0 2023-06-23T18:32:32.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efd7aff5700 1 -- 172.21.15.2:0/3689135075 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5559f506ceb0 con 0x5559f513c830 2023-06-23T18:32:32.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efd7aff5700 10 client.16260 handle_command_reply: tid=0 2023-06-23T18:32:32.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 client.16260 shutdown 2023-06-23T18:32:32.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 -- 172.21.15.2:0/3689135075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5128410 msgr2=0x5559f5137ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5128410 0x5559f5137ed0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7efd90011f60 tx=0x7efd90014600 comp rx=0 tx=0).stop 2023-06-23T18:32:32.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 -- 172.21.15.2:0/3689135075 shutdown_connections 2023-06-23T18:32:32.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5559f513c830 0x5559f513ec40 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5128410 0x5559f5137ed0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efd7aff5700 0 client.16260 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:32:32.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5127090 0x5559f5130950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 --2- 172.21.15.2:0/3689135075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f5130e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 -- 172.21.15.2:0/3689135075 >> 172.21.15.2:0/3689135075 conn(0x5559f51183f0 msgr2=0x5559f511cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 -- 172.21.15.2:0/3689135075 shutdown_connections 2023-06-23T18:32:32.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 1 -- 172.21.15.2:0/3689135075 wait complete. 2023-06-23T18:32:32.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.314+0000 7efda9739b80 20 client.16260 trim_cache size 0 max 16384 2023-06-23T18:32:32.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.327+0000 7efda9739b80 1 Processor -- start 2023-06-23T18:32:32.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.327+0000 7efda9739b80 1 -- start start 2023-06-23T18:32:32.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efda9739b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f512b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efda9739b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f512b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efda9739b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f5137df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efda9739b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f51304c0 con 0x5559f5127090 2023-06-23T18:32:32.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efda9739b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f5131ad0 con 0x5559f51279e0 2023-06-23T18:32:32.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efda9739b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f511efb0 con 0x5559f5128410 2023-06-23T18:32:32.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f512b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f512b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56152/0 (socket says 172.21.15.2:56152) 2023-06-23T18:32:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 -- 172.21.15.2:0/4030156613 learned_addr learned my addr 172.21.15.2:0/4030156613 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd967fc700 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f512b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd977fe700 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f5137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 -- 172.21.15.2:0/4030156613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 msgr2=0x5559f5137df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f5137df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 -- 172.21.15.2:0/4030156613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 msgr2=0x5559f512b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f512b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 -- 172.21.15.2:0/4030156613 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559f4e8ac60 con 0x5559f5127090 2023-06-23T18:32:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd977fe700 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f5137df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:32:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd967fc700 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f512b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:32:32.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd96ffd700 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f512b2d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efd8800b2e0 tx=0x7efd88086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f39699707a17e63 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.328+0000 7efd7dffb700 1 -- 172.21.15.2:0/4030156613 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd880a3620 con 0x5559f5127090 2023-06-23T18:32:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efd7dffb700 1 -- 172.21.15.2:0/4030156613 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efd880a10b0 con 0x5559f5127090 2023-06-23T18:32:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efd7dffb700 1 -- 172.21.15.2:0/4030156613 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8809a9b0 con 0x5559f5127090 2023-06-23T18:32:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 -- 172.21.15.2:0/4030156613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 msgr2=0x5559f512b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f512b2d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7efd8800b2e0 tx=0x7efd88086040 comp rx=0 tx=0).stop 2023-06-23T18:32:32.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 -- 172.21.15.2:0/4030156613 shutdown_connections 2023-06-23T18:32:32.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f5137df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f512b2d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 --2- 172.21.15.2:0/4030156613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f512b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 -- 172.21.15.2:0/4030156613 >> 172.21.15.2:0/4030156613 conn(0x5559f51183f0 msgr2=0x5559f512f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 -- 172.21.15.2:0/4030156613 shutdown_connections 2023-06-23T18:32:32.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.329+0000 7efda9739b80 1 -- 172.21.15.2:0/4030156613 wait complete. 2023-06-23T18:32:32.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 1 -- start start 2023-06-23T18:32:32.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 1 Processor -- start 2023-06-23T18:32:32.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f5137630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f5137b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f51340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f51346a0 con 0x5559f5127090 2023-06-23T18:32:32.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f51347e0 con 0x5559f51279e0 2023-06-23T18:32:32.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559f51349a0 con 0x5559f5128410 2023-06-23T18:32:32.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efda9739b80 10 client.0 _ll_register_callbacks cb 0x5559f5019260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:32.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd96ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f5137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd96ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f5137630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56158/0 (socket says 172.21.15.2:56158) 2023-06-23T18:32:32.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd96ffd700 1 -- 172.21.15.2:0/1660158584 learned_addr learned my addr 172.21.15.2:0/1660158584 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd977fe700 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f51340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd96ffd700 1 -- 172.21.15.2:0/1660158584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 msgr2=0x5559f51340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd967fc700 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f5137b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd96ffd700 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f51340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd96ffd700 1 -- 172.21.15.2:0/1660158584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 msgr2=0x5559f5137b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd96ffd700 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f5137b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.330+0000 7efd96ffd700 1 -- 172.21.15.2:0/1660158584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd88095e70 con 0x5559f5127090 2023-06-23T18:32:32.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efd96ffd700 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f5137630 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efd880a30f0 tx=0x7efd88004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b4302690baebb3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efd7aff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:32.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efd7aff5700 1 -- 172.21.15.2:0/1660158584 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd880993a0 con 0x5559f5127090 2023-06-23T18:32:32.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efd7aff5700 1 -- 172.21.15.2:0/1660158584 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efd880a3620 con 0x5559f5127090 2023-06-23T18:32:32.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 1 -- 172.21.15.2:0/1660158584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd8c021070 con 0x5559f5127090 2023-06-23T18:32:32.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efd7aff5700 1 -- 172.21.15.2:0/1660158584 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd880a3190 con 0x5559f5127090 2023-06-23T18:32:32.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 10 client.16266 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:32.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 1 -- 172.21.15.2:0/1660158584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x5559f4ec8250 con 0x5559f5127090 2023-06-23T18:32:32.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efd7aff5700 1 -- 172.21.15.2:0/1660158584 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efd8c021070 con 0x5559f5127090 2023-06-23T18:32:32.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efd7aff5700 1 -- 172.21.15.2:0/1660158584 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7efd880b93b0 con 0x5559f5127090 2023-06-23T18:32:32.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 10 client.16266 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:32.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 10 client.16266 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:32:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 20 client.16266 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5559f51917b0 0x5559f5193bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 4 client.16266 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:32:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.331+0000 7efda9739b80 1 -- 172.21.15.2:0/1660158584 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5559f502f050 con 0x5559f51917b0 2023-06-23T18:32:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.332+0000 7efd967fc700 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5559f51917b0 0x5559f5193bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.332+0000 7efd7aff5700 10 client.16266 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:32:32.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.332+0000 7efd967fc700 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5559f51917b0 0x5559f5193bc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7e0e0a919fa253e server_cookie=9861b5890c25fc73 in_seq=0 out_seq=0 2023-06-23T18:32:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.332+0000 7efd7aff5700 1 -- 172.21.15.2:0/1660158584 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5559f502f050 con 0x5559f51917b0 2023-06-23T18:32:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.332+0000 7efd7aff5700 10 client.16266 handle_command_reply: tid=0 2023-06-23T18:32:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.332+0000 7efda9739b80 1 client.16266 shutdown 2023-06-23T18:32:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 -- 172.21.15.2:0/1660158584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 msgr2=0x5559f5137630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f5137630 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7efd880a30f0 tx=0x7efd88004320 comp rx=0 tx=0).stop 2023-06-23T18:32:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 -- 172.21.15.2:0/1660158584 shutdown_connections 2023-06-23T18:32:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x5559f51917b0 0x5559f5193bc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5559f5128410 0x5559f51340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5559f5127090 0x5559f5137630 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 --2- 172.21.15.2:0/1660158584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5559f51279e0 0x5559f5137b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd7aff5700 0 client.16266 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:32:32.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 -- 172.21.15.2:0/1660158584 >> 172.21.15.2:0/1660158584 conn(0x5559f51183f0 msgr2=0x5559f512b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 -- 172.21.15.2:0/1660158584 shutdown_connections 2023-06-23T18:32:32.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 1 -- 172.21.15.2:0/1660158584 wait complete. 2023-06-23T18:32:32.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efda9739b80 20 client.16266 trim_cache size 0 max 16384 2023-06-23T18:32:32.347 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:32:32.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 -- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efd800a1150 msgr2=0x7efd800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efd800a1150 0x7efd800a3600 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009c60 tx=0x7efd8c009260 comp rx=0 tx=0).stop 2023-06-23T18:32:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 -- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 msgr2=0x7efd98068310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 0x7efd98068310 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7efd88000c00 tx=0x7efd88004730 comp rx=0 tx=0).stop 2023-06-23T18:32:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 -- 172.21.15.2:0/2912250167 shutdown_connections 2023-06-23T18:32:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd98100600 0x7efd9819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efd980fea80 0x7efd98068310 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efd800a1150 0x7efd800a3600 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 --2- 172.21.15.2:0/2912250167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd980ffc70 0x7efd9819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.333+0000 7efd9d5b4700 1 -- 172.21.15.2:0/2912250167 >> 172.21.15.2:0/2912250167 conn(0x7efd980fa0e0 msgr2=0x7efd980fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.334+0000 7efd9d5b4700 1 -- 172.21.15.2:0/2912250167 shutdown_connections 2023-06-23T18:32:32.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.334+0000 7efd9d5b4700 1 -- 172.21.15.2:0/2912250167 wait complete. 2023-06-23T18:32:32.350 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:32:32.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 -- 172.21.15.2:0/1413008267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 msgr2=0x7f1f64105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1413008267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 0x7f1f64105340 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009990 tx=0x7f1f58009ca0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 -- 172.21.15.2:0/1413008267 shutdown_connections 2023-06-23T18:32:32.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1413008267 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f64105880 0x7f1f64107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1413008267 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f64100640 0x7f1f64102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1413008267 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 0x7f1f64105340 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 -- 172.21.15.2:0/1413008267 >> 172.21.15.2:0/1413008267 conn(0x7f1f640fa0e0 msgr2=0x7f1f640fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 -- 172.21.15.2:0/1413008267 shutdown_connections 2023-06-23T18:32:32.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.476+0000 7f1f6a501700 1 -- 172.21.15.2:0/1413008267 wait complete. 2023-06-23T18:32:32.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.477+0000 7f1f6a501700 1 Processor -- start 2023-06-23T18:32:32.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.477+0000 7f1f6a501700 1 -- start start 2023-06-23T18:32:32.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.477+0000 7f1f6a501700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f64100640 0x7f1f640fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.477+0000 7f1f6a501700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 0x7f1f640fe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.477+0000 7f1f6a501700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f64105880 0x7f1f640feb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.477+0000 7f1f6a501700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640ff2d0 con 0x7f1f64102f60 2023-06-23T18:32:32.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.477+0000 7f1f6a501700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f640ff440 con 0x7f1f64105880 2023-06-23T18:32:32.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.477+0000 7f1f6a501700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6419f860 con 0x7f1f64100640 2023-06-23T18:32:32.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 0x7f1f640fe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 0x7f1f640fe5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56174/0 (socket says 172.21.15.2:56174) 2023-06-23T18:32:32.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 -- 172.21.15.2:0/1122005278 learned_addr learned my addr 172.21.15.2:0/1122005278 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f63fff700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f64100640 0x7f1f640fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f64105880 0x7f1f640feb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 -- 172.21.15.2:0/1122005278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f64100640 msgr2=0x7f1f640fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f64100640 0x7f1f640fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 -- 172.21.15.2:0/1122005278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f64105880 msgr2=0x7f1f640feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f64105880 0x7f1f640feb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 -- 172.21.15.2:0/1122005278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f580095f0 con 0x7f1f64102f60 2023-06-23T18:32:32.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f64105880 0x7f1f640feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:32:32.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f63fff700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f64100640 0x7f1f640fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f637fe700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 0x7f1f640fe5f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1f580098e0 tx=0x7f1f58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f617fa700 1 -- 172.21.15.2:0/1122005278 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801d070 con 0x7f1f64102f60 2023-06-23T18:32:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f6a501700 1 -- 172.21.15.2:0/1122005278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6419fae0 con 0x7f1f64102f60 2023-06-23T18:32:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f617fa700 1 -- 172.21.15.2:0/1122005278 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5800be30 con 0x7f1f64102f60 2023-06-23T18:32:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f617fa700 1 -- 172.21.15.2:0/1122005278 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58022e00 con 0x7f1f64102f60 2023-06-23T18:32:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.478+0000 7f1f6a501700 1 -- 172.21.15.2:0/1122005278 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6419ff00 con 0x7f1f64102f60 2023-06-23T18:32:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.480+0000 7f1f617fa700 1 -- 172.21.15.2:0/1122005278 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1f5800b9a0 con 0x7f1f64102f60 2023-06-23T18:32:32.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.481+0000 7f1f617fa700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1f4c0a1150 0x7f1f4c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.481+0000 7f1f63fff700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1f4c0a1150 0x7f1f4c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.481+0000 7f1f617fa700 1 -- 172.21.15.2:0/1122005278 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1f580cdeb0 con 0x7f1f64102f60 2023-06-23T18:32:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.482+0000 7f1f63fff700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1f4c0a1150 0x7f1f4c0a3600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1f54011490 tx=0x7f1f54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.482+0000 7f1f76686b80 1 Processor -- start 2023-06-23T18:32:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.482+0000 7f1f76686b80 1 -- start start 2023-06-23T18:32:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.482+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d3885090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.482+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d38855d0 0x5630d3885a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.482+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3886000 0x5630d3886470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.482+0000 7f1f76686b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d3789cb0 con 0x5630d3886000 2023-06-23T18:32:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f76686b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d37dca40 con 0x5630d38855d0 2023-06-23T18:32:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f76686b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d37d5ed0 con 0x5630d3884c80 2023-06-23T18:32:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3886000 0x5630d3886470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3886000 0x5630d3886470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56184/0 (socket says 172.21.15.2:56184) 2023-06-23T18:32:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 -- 172.21.15.2:0/2640566980 learned_addr learned my addr 172.21.15.2:0/2640566980 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f637fe700 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d38855d0 0x5630d3885a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f63fff700 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d3885090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 -- 172.21.15.2:0/2640566980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 msgr2=0x5630d3885090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d3885090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 -- 172.21.15.2:0/2640566980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d38855d0 msgr2=0x5630d3885a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d38855d0 0x5630d3885a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 -- 172.21.15.2:0/2640566980 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630d3657630 con 0x5630d3886000 2023-06-23T18:32:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f637fe700 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d38855d0 0x5630d3885a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3886000 0x5630d3886470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009e00 tx=0x7f1f5c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2e9e4abe7d460a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f4a7fc700 1 -- 172.21.15.2:0/2640566980 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c014610 con 0x5630d3886000 2023-06-23T18:32:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f4a7fc700 1 -- 172.21.15.2:0/2640566980 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5c00f050 con 0x5630d3886000 2023-06-23T18:32:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.483+0000 7f1f4a7fc700 1 -- 172.21.15.2:0/2640566980 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0138d0 con 0x5630d3886000 2023-06-23T18:32:32.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 -- 172.21.15.2:0/2640566980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3886000 msgr2=0x5630d3886470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3886000 0x5630d3886470 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009e00 tx=0x7f1f5c0077e0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 -- 172.21.15.2:0/2640566980 shutdown_connections 2023-06-23T18:32:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3886000 0x5630d3886470 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d3885090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 --2- 172.21.15.2:0/2640566980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d38855d0 0x5630d3885a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 -- 172.21.15.2:0/2640566980 >> 172.21.15.2:0/2640566980 conn(0x5630d387a3f0 msgr2=0x5630d387f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 -- 172.21.15.2:0/2640566980 shutdown_connections 2023-06-23T18:32:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.484+0000 7f1f76686b80 1 -- 172.21.15.2:0/2640566980 wait complete. 2023-06-23T18:32:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 1 -- start start 2023-06-23T18:32:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 1 Processor -- start 2023-06-23T18:32:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3884c80 0x5630d388e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3886000 0x5630d3895a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d3775ac0 con 0x5630d38855d0 2023-06-23T18:32:32.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d37757c0 con 0x5630d3884c80 2023-06-23T18:32:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d3659920 con 0x5630d3886000 2023-06-23T18:32:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f76686b80 10 client.0 _ll_register_callbacks cb 0x5630d377b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f63fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3884c80 0x5630d388e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56188/0 (socket says 172.21.15.2:56188) 2023-06-23T18:32:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3886000 0x5630d3895a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.485+0000 7f1f637fe700 1 -- 172.21.15.2:0/786808015 learned_addr learned my addr 172.21.15.2:0/786808015 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f68a9e700 1 -- 172.21.15.2:0/786808015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3884c80 msgr2=0x5630d388e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3884c80 0x5630d388e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f68a9e700 1 -- 172.21.15.2:0/786808015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 msgr2=0x5630d388ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f68a9e700 1 -- 172.21.15.2:0/786808015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6419ff00 con 0x5630d3886000 2023-06-23T18:32:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f637fe700 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3886000 0x5630d3895a40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000f80 tx=0x7f1f5c0138a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a475ac07410e04e8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f437fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:32:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f437fe700 1 -- 172.21.15.2:0/786808015 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c013ee0 con 0x5630d3886000 2023-06-23T18:32:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f437fe700 1 -- 172.21.15.2:0/786808015 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1f5c003a00 con 0x5630d3886000 2023-06-23T18:32:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f76686b80 1 -- 172.21.15.2:0/786808015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630d3877bb0 con 0x5630d3886000 2023-06-23T18:32:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f437fe700 1 -- 172.21.15.2:0/786808015 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f5c003bc0 con 0x5630d3886000 2023-06-23T18:32:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f76686b80 10 client.26014 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.486+0000 7f1f76686b80 1 -- 172.21.15.2:0/786808015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f1f5c009a60 con 0x5630d3886000 2023-06-23T18:32:32.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f437fe700 1 -- 172.21.15.2:0/786808015 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5630d3877bb0 con 0x5630d3886000 2023-06-23T18:32:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f437fe700 1 -- 172.21.15.2:0/786808015 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1f5c014410 con 0x5630d3886000 2023-06-23T18:32:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f76686b80 10 client.26014 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f76686b80 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f76686b80 20 client.26014 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f76686b80 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630d389a200 0x5630d389c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f76686b80 4 client.26014 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:32:32.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f76686b80 1 -- 172.21.15.2:0/786808015 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630d37ceeb0 con 0x5630d389a200 2023-06-23T18:32:32.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.487+0000 7f1f63fff700 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630d389a200 0x5630d389c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.488+0000 7f1f437fe700 10 client.26014 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:32.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.488+0000 7f1f63fff700 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630d389a200 0x5630d389c610 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdeb020ca8394b9 server_cookie=5bbdf3b0b89d68de in_seq=0 out_seq=0 2023-06-23T18:32:32.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.488+0000 7f1f437fe700 1 -- 172.21.15.2:0/786808015 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5630d37ceeb0 con 0x5630d389a200 2023-06-23T18:32:32.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.488+0000 7f1f437fe700 10 client.26014 handle_command_reply: tid=0 2023-06-23T18:32:32.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.488+0000 7f1f76686b80 1 client.26014 shutdown 2023-06-23T18:32:32.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.488+0000 7f1f76686b80 1 -- 172.21.15.2:0/786808015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3886000 msgr2=0x5630d3895a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.488+0000 7f1f76686b80 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3886000 0x5630d3895a40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000f80 tx=0x7f1f5c0138a0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 1 -- 172.21.15.2:0/786808015 shutdown_connections 2023-06-23T18:32:32.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630d389a200 0x5630d389c610 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3886000 0x5630d3895a40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f437fe700 0 client.26014 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:32.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3884c80 0x5630d388e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 1 --2- 172.21.15.2:0/786808015 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 1 -- 172.21.15.2:0/786808015 >> 172.21.15.2:0/786808015 conn(0x5630d387a3f0 msgr2=0x5630d387ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 1 -- 172.21.15.2:0/786808015 shutdown_connections 2023-06-23T18:32:32.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 1 -- 172.21.15.2:0/786808015 wait complete. 2023-06-23T18:32:32.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.489+0000 7f1f76686b80 20 client.26014 trim_cache size 0 max 16384 2023-06-23T18:32:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f76686b80 1 Processor -- start 2023-06-23T18:32:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f76686b80 1 -- start start 2023-06-23T18:32:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d388c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 0x5630d3895a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f76686b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d388d970 con 0x5630d38855d0 2023-06-23T18:32:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f76686b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d388e030 con 0x5630d3886000 2023-06-23T18:32:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f76686b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d3880d50 con 0x5630d3884c80 2023-06-23T18:32:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f63fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d388c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56196/0 (socket says 172.21.15.2:56196) 2023-06-23T18:32:32.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f63fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d388c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56532/0 (socket says 172.21.15.2:56532) 2023-06-23T18:32:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.502+0000 7f1f637fe700 1 -- 172.21.15.2:0/217400576 learned_addr learned my addr 172.21.15.2:0/217400576 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 0x5630d3895a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f637fe700 1 -- 172.21.15.2:0/217400576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 msgr2=0x5630d388c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f637fe700 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d388c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f637fe700 1 -- 172.21.15.2:0/217400576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 msgr2=0x5630d3895a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f637fe700 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 0x5630d3895a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f637fe700 1 -- 172.21.15.2:0/217400576 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630d3603de0 con 0x5630d38855d0 2023-06-23T18:32:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f63fff700 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d388c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f637fe700 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388c810 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1f58006010 tx=0x7f1f580cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d860b9150aaead82 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f4affd700 1 -- 172.21.15.2:0/217400576 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5809add0 con 0x5630d38855d0 2023-06-23T18:32:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f4affd700 1 -- 172.21.15.2:0/217400576 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5809b470 con 0x5630d38855d0 2023-06-23T18:32:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.503+0000 7f1f4affd700 1 -- 172.21.15.2:0/217400576 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580a33a0 con 0x5630d38855d0 2023-06-23T18:32:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 -- 172.21.15.2:0/217400576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 msgr2=0x5630d388c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388c810 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1f58006010 tx=0x7f1f580cd040 comp rx=0 tx=0).stop 2023-06-23T18:32:32.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 -- 172.21.15.2:0/217400576 shutdown_connections 2023-06-23T18:32:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 0x5630d3895a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d3884c80 0x5630d388c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 --2- 172.21.15.2:0/217400576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d38855d0 0x5630d388c810 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 -- 172.21.15.2:0/217400576 >> 172.21.15.2:0/217400576 conn(0x5630d387a3f0 msgr2=0x5630d38a0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 -- 172.21.15.2:0/217400576 shutdown_connections 2023-06-23T18:32:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 -- 172.21.15.2:0/217400576 wait complete. 2023-06-23T18:32:32.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.504+0000 7f1f76686b80 1 -- start start 2023-06-23T18:32:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f76686b80 1 Processor -- start 2023-06-23T18:32:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3884c80 0x5630d38912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d38855d0 0x5630d3891830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f76686b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 0x5630d3894850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f76686b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d3894d90 con 0x5630d3884c80 2023-06-23T18:32:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f76686b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d3894f00 con 0x5630d3886000 2023-06-23T18:32:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f76686b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630d38950c0 con 0x5630d38855d0 2023-06-23T18:32:32.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f76686b80 10 client.0 _ll_register_callbacks cb 0x5630d377b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f63fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3884c80 0x5630d38912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f63fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3884c80 0x5630d38912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56200/0 (socket says 172.21.15.2:56200) 2023-06-23T18:32:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f63fff700 1 -- 172.21.15.2:0/3640825089 learned_addr learned my addr 172.21.15.2:0/3640825089 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f68a9e700 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 0x5630d3894850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f637fe700 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d38855d0 0x5630d3891830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f63fff700 1 -- 172.21.15.2:0/3640825089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d38855d0 msgr2=0x5630d3891830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f63fff700 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d38855d0 0x5630d3891830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f63fff700 1 -- 172.21.15.2:0/3640825089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 msgr2=0x5630d3894850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f63fff700 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 0x5630d3894850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.505+0000 7f1f63fff700 1 -- 172.21.15.2:0/3640825089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f54017070 con 0x5630d3884c80 2023-06-23T18:32:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f63fff700 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3884c80 0x5630d38912f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5630d389cef0 tx=0x7f1f54013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7a5e7ab9b953f4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f437fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f437fe700 1 -- 172.21.15.2:0/3640825089 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54024040 con 0x5630d3884c80 2023-06-23T18:32:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f437fe700 1 -- 172.21.15.2:0/3640825089 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1f54003680 con 0x5630d3884c80 2023-06-23T18:32:32.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 1 -- 172.21.15.2:0/3640825089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f5c00f050 con 0x5630d3884c80 2023-06-23T18:32:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f437fe700 1 -- 172.21.15.2:0/3640825089 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f54003840 con 0x5630d3884c80 2023-06-23T18:32:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 10 client.16284 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 1 -- 172.21.15.2:0/3640825089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7f1f580cbe70 con 0x5630d3884c80 2023-06-23T18:32:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f437fe700 1 -- 172.21.15.2:0/3640825089 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1f5c00f050 con 0x5630d3884c80 2023-06-23T18:32:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f437fe700 1 -- 172.21.15.2:0/3640825089 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f1f5401f7e0 con 0x5630d3884c80 2023-06-23T18:32:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 10 client.16284 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:32.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 20 client.16284 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630d38eeb50 0x5630d38f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 4 client.16284 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:32:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.506+0000 7f1f76686b80 1 -- 172.21.15.2:0/3640825089 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5630d3791050 con 0x5630d38eeb50 2023-06-23T18:32:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f637fe700 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630d38eeb50 0x5630d38f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f637fe700 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630d38eeb50 0x5630d38f0f60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=472d7f2b5b3b1309 server_cookie=be06f2f17c569e6b in_seq=0 out_seq=0 2023-06-23T18:32:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f437fe700 10 client.16284 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:32.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f437fe700 1 -- 172.21.15.2:0/3640825089 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5630d3791050 con 0x5630d38eeb50 2023-06-23T18:32:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f437fe700 10 client.16284 handle_command_reply: tid=0 2023-06-23T18:32:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 client.16284 shutdown 2023-06-23T18:32:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 -- 172.21.15.2:0/3640825089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3884c80 msgr2=0x5630d38912f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3884c80 0x5630d38912f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5630d389cef0 tx=0x7f1f54013040 comp rx=0 tx=0).stop 2023-06-23T18:32:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 -- 172.21.15.2:0/3640825089 shutdown_connections 2023-06-23T18:32:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630d38eeb50 0x5630d38f0f60 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630d3886000 0x5630d3894850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f437fe700 0 client.16284 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:32.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630d3884c80 0x5630d38912f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 --2- 172.21.15.2:0/3640825089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630d38855d0 0x5630d3891830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 -- 172.21.15.2:0/3640825089 >> 172.21.15.2:0/3640825089 conn(0x5630d387a3f0 msgr2=0x5630d3888de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.507+0000 7f1f76686b80 1 -- 172.21.15.2:0/3640825089 shutdown_connections 2023-06-23T18:32:32.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f76686b80 1 -- 172.21.15.2:0/3640825089 wait complete. 2023-06-23T18:32:32.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f76686b80 20 client.16284 trim_cache size 0 max 16384 2023-06-23T18:32:32.533 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:32:32.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 -- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1f4c0a1150 msgr2=0x7f1f4c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1f4c0a1150 0x7f1f4c0a3600 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1f54011490 tx=0x7f1f54009260 comp rx=0 tx=0).stop 2023-06-23T18:32:32.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 -- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 msgr2=0x7f1f640fe5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:32.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 0x7f1f640fe5f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f1f580098e0 tx=0x7f1f58004730 comp rx=0 tx=0).stop 2023-06-23T18:32:32.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 -- 172.21.15.2:0/1122005278 shutdown_connections 2023-06-23T18:32:32.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1f4c0a1150 0x7f1f4c0a3600 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f64105880 0x7f1f640feb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f64100640 0x7f1f640fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 --2- 172.21.15.2:0/1122005278 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f64102f60 0x7f1f640fe5f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:32.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 -- 172.21.15.2:0/1122005278 >> 172.21.15.2:0/1122005278 conn(0x7f1f640fa0e0 msgr2=0x7f1f64101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:32.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 -- 172.21.15.2:0/1122005278 shutdown_connections 2023-06-23T18:32:32.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:32.508+0000 7f1f6a501700 1 -- 172.21.15.2:0/1122005278 wait complete. 2023-06-23T18:32:32.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:32 smithi002 ceph-mon[95947]: pgmap v611: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 48 MiB/s wr, 37 op/s 2023-06-23T18:32:32.714 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:32 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2292173448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:32:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:32 smithi082 ceph-mon[97889]: pgmap v611: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 48 MiB/s wr, 37 op/s 2023-06-23T18:32:33.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:32 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2292173448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:32:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:32 smithi017 ceph-mon[97472]: pgmap v611: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 48 MiB/s wr, 37 op/s 2023-06-23T18:32:33.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:32 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2292173448' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:32:33.537 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:32:33.537 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:32:33.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 -- 172.21.15.2:0/3189473089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828102f20 msgr2=0x7fe828105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 --2- 172.21.15.2:0/3189473089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828102f20 0x7fe828105300 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009990 tx=0x7fe81c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 -- 172.21.15.2:0/3189473089 shutdown_connections 2023-06-23T18:32:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 --2- 172.21.15.2:0/3189473089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe828105840 0x7fe828107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 --2- 172.21.15.2:0/3189473089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe828100600 0x7fe8281029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 --2- 172.21.15.2:0/3189473089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828102f20 0x7fe828105300 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 -- 172.21.15.2:0/3189473089 >> 172.21.15.2:0/3189473089 conn(0x7fe8280fa0e0 msgr2=0x7fe8280fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 -- 172.21.15.2:0/3189473089 shutdown_connections 2023-06-23T18:32:33.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.682+0000 7fe82cc26700 1 -- 172.21.15.2:0/3189473089 wait complete. 2023-06-23T18:32:33.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82cc26700 1 Processor -- start 2023-06-23T18:32:33.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82cc26700 1 -- start start 2023-06-23T18:32:33.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82cc26700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828100600 0x7fe828197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82cc26700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe828102f20 0x7fe828197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82cc26700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe828105840 0x7fe82819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82cc26700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82819f510 con 0x7fe828100600 2023-06-23T18:32:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82cc26700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82819f650 con 0x7fe828102f20 2023-06-23T18:32:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82cc26700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82819f7e0 con 0x7fe828105840 2023-06-23T18:32:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828100600 0x7fe828197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828100600 0x7fe828197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56214/0 (socket says 172.21.15.2:56214) 2023-06-23T18:32:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82659c700 1 -- 172.21.15.2:0/2079293335 learned_addr learned my addr 172.21.15.2:0/2079293335 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe825d9b700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe828102f20 0x7fe828197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe826d9d700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe828105840 0x7fe82819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82659c700 1 -- 172.21.15.2:0/2079293335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe828105840 msgr2=0x7fe82819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82659c700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe828105840 0x7fe82819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82659c700 1 -- 172.21.15.2:0/2079293335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe828102f20 msgr2=0x7fe828197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82659c700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe828102f20 0x7fe828197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.683+0000 7fe82659c700 1 -- 172.21.15.2:0/2079293335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0095f0 con 0x7fe828100600 2023-06-23T18:32:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.684+0000 7fe82659c700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828100600 0x7fe828197990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe81800d980 tx=0x7fe81800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.684+0000 7fe8177fe700 1 -- 172.21.15.2:0/2079293335 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818004220 con 0x7fe828100600 2023-06-23T18:32:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.684+0000 7fe82cc26700 1 -- 172.21.15.2:0/2079293335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82819fac0 con 0x7fe828100600 2023-06-23T18:32:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.684+0000 7fe8177fe700 1 -- 172.21.15.2:0/2079293335 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe818007640 con 0x7fe828100600 2023-06-23T18:32:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.684+0000 7fe8177fe700 1 -- 172.21.15.2:0/2079293335 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8180055e0 con 0x7fe828100600 2023-06-23T18:32:33.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.684+0000 7fe82cc26700 1 -- 172.21.15.2:0/2079293335 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8281a0020 con 0x7fe828100600 2023-06-23T18:32:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.686+0000 7fe8177fe700 1 -- 172.21.15.2:0/2079293335 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe818004750 con 0x7fe828100600 2023-06-23T18:32:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.687+0000 7fe8177fe700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe8100a1150 0x7fe8100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.687+0000 7fe825d9b700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe8100a1150 0x7fe8100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.687+0000 7fe8177fe700 1 -- 172.21.15.2:0/2079293335 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe8180cc7a0 con 0x7fe828100600 2023-06-23T18:32:33.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.688+0000 7fe825d9b700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe8100a1150 0x7fe8100a3600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00b5c0 tx=0x7fe81c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.688+0000 7fe838dabb80 1 Processor -- start 2023-06-23T18:32:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe838dabb80 1 -- start start 2023-06-23T18:32:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f512110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f512650 0x555e4f512af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f5134f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe838dabb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f416d30 con 0x555e4f513080 2023-06-23T18:32:33.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe838dabb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f469ac0 con 0x555e4f512650 2023-06-23T18:32:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe838dabb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f462f50 con 0x555e4f511d00 2023-06-23T18:32:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe826d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f5134f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe826d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f5134f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56226/0 (socket says 172.21.15.2:56226) 2023-06-23T18:32:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe826d9d700 1 -- 172.21.15.2:0/4259851382 learned_addr learned my addr 172.21.15.2:0/4259851382 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe825d9b700 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f512650 0x555e4f512af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe82659c700 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f512110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe826d9d700 1 -- 172.21.15.2:0/4259851382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 msgr2=0x555e4f512110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe826d9d700 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f512110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe826d9d700 1 -- 172.21.15.2:0/4259851382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f512650 msgr2=0x555e4f512af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe826d9d700 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f512650 0x555e4f512af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe826d9d700 1 -- 172.21.15.2:0/4259851382 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4f2e46b0 con 0x555e4f513080 2023-06-23T18:32:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.689+0000 7fe825d9b700 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f512650 0x555e4f512af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.690+0000 7fe826d9d700 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f5134f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe820009d20 tx=0x7fe8200077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc501f6bef9173 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.690+0000 7fe814ff9700 1 -- 172.21.15.2:0/4259851382 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820018070 con 0x555e4f513080 2023-06-23T18:32:33.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.690+0000 7fe814ff9700 1 -- 172.21.15.2:0/4259851382 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe82000f050 con 0x555e4f513080 2023-06-23T18:32:33.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.690+0000 7fe814ff9700 1 -- 172.21.15.2:0/4259851382 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820005940 con 0x555e4f513080 2023-06-23T18:32:33.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 -- 172.21.15.2:0/4259851382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 msgr2=0x555e4f5134f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f5134f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe820009d20 tx=0x7fe8200077e0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 -- 172.21.15.2:0/4259851382 shutdown_connections 2023-06-23T18:32:33.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f5134f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f512110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 --2- 172.21.15.2:0/4259851382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f512650 0x555e4f512af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 -- 172.21.15.2:0/4259851382 >> 172.21.15.2:0/4259851382 conn(0x555e4f507470 msgr2=0x555e4f50c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 -- 172.21.15.2:0/4259851382 shutdown_connections 2023-06-23T18:32:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 -- 172.21.15.2:0/4259851382 wait complete. 2023-06-23T18:32:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 -- start start 2023-06-23T18:32:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.691+0000 7fe838dabb80 1 Processor -- start 2023-06-23T18:32:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f51b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f512650 0x555e4f51bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f522b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe838dabb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f402b40 con 0x555e4f512650 2023-06-23T18:32:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe838dabb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f402840 con 0x555e4f513080 2023-06-23T18:32:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe838dabb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f2e69a0 con 0x555e4f511d00 2023-06-23T18:32:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe838dabb80 10 client.0 _ll_register_callbacks cb 0x555e4f4082e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:33.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f512650 0x555e4f51bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f512650 0x555e4f51bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56230/0 (socket says 172.21.15.2:56230) 2023-06-23T18:32:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 -- 172.21.15.2:0/1952883868 learned_addr learned my addr 172.21.15.2:0/1952883868 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe82659c700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f51b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe826d9d700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f522b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 -- 172.21.15.2:0/1952883868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 msgr2=0x555e4f51b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f51b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 -- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 msgr2=0x555e4f522b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f522b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 -- 172.21.15.2:0/1952883868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe828108890 con 0x555e4f512650 2023-06-23T18:32:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe82659c700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f51b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe826d9d700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f522b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe825d9b700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f512650 0x555e4f51bac0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009990 tx=0x7fe81c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99f0a830a5de50d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe80dffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.692+0000 7fe80dffb700 1 -- 172.21.15.2:0/1952883868 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c02b050 con 0x555e4f512650 2023-06-23T18:32:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe80dffb700 1 -- 172.21.15.2:0/1952883868 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe81c004cf0 con 0x555e4f512650 2023-06-23T18:32:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 1 -- 172.21.15.2:0/1952883868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e4f504c30 con 0x555e4f512650 2023-06-23T18:32:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe80dffb700 1 -- 172.21.15.2:0/1952883868 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe81c003820 con 0x555e4f512650 2023-06-23T18:32:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 10 client.16296 fetch_fsmap learned FSMap version 33 2023-06-23T18:32:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 1 -- 172.21.15.2:0/1952883868 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) v3 -- 0x7fe820009980 con 0x555e4f512650 2023-06-23T18:32:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe80dffb700 1 -- 172.21.15.2:0/1952883868 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe81c03f020 con 0x555e4f512650 2023-06-23T18:32:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe80dffb700 1 -- 172.21.15.2:0/1952883868 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 33) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fe81c00f580 con 0x555e4f512650 2023-06-23T18:32:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 10 client.16296 fetch_fsmap finished waiting for FSMap version 33 2023-06-23T18:32:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 20 client.16296 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555e4f5273d0 0x555e4f5297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 4 client.16296 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:32:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe838dabb80 1 -- 172.21.15.2:0/1952883868 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e4f45bf30 con 0x555e4f5273d0 2023-06-23T18:32:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.693+0000 7fe82659c700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555e4f5273d0 0x555e4f5297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.694+0000 7fe80dffb700 10 client.16296 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.694+0000 7fe82659c700 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555e4f5273d0 0x555e4f5297e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3d62b51341651c8 server_cookie=c3548dd4bb0750e in_seq=0 out_seq=0 2023-06-23T18:32:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe80dffb700 1 -- 172.21.15.2:0/1952883868 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e4f45bf30 con 0x555e4f5273d0 2023-06-23T18:32:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe80dffb700 10 client.16296 handle_command_reply: tid=0 2023-06-23T18:32:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 client.16296 shutdown 2023-06-23T18:32:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 -- 172.21.15.2:0/1952883868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f512650 msgr2=0x555e4f51bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f512650 0x555e4f51bac0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009990 tx=0x7fe81c01a050 comp rx=0 tx=0).stop 2023-06-23T18:32:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 -- 172.21.15.2:0/1952883868 shutdown_connections 2023-06-23T18:32:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555e4f5273d0 0x555e4f5297e0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f522b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe80dffb700 0 client.16296 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f511d00 0x555e4f51b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 --2- 172.21.15.2:0/1952883868 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f512650 0x555e4f51bac0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 -- 172.21.15.2:0/1952883868 >> 172.21.15.2:0/1952883868 conn(0x555e4f507470 msgr2=0x555e4f50bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 -- 172.21.15.2:0/1952883868 shutdown_connections 2023-06-23T18:32:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 1 -- 172.21.15.2:0/1952883868 wait complete. 2023-06-23T18:32:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.695+0000 7fe838dabb80 20 client.16296 trim_cache size 0 max 16384 2023-06-23T18:32:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.717+0000 7fe838dabb80 1 Processor -- start 2023-06-23T18:32:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.717+0000 7fe838dabb80 1 -- start start 2023-06-23T18:32:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.717+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f511d00 0x555e4f51b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.717+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f522ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.717+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 0x555e4f50b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.717+0000 7fe838dabb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f51aa30 con 0x555e4f513080 2023-06-23T18:32:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.717+0000 7fe838dabb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f51b0f0 con 0x555e4f511d00 2023-06-23T18:32:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.717+0000 7fe838dabb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f51c230 con 0x555e4f51b870 2023-06-23T18:32:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f511d00 0x555e4f51b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe825d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f522ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f511d00 0x555e4f51b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46828/0 (socket says 172.21.15.2:46828) 2023-06-23T18:32:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 -- 172.21.15.2:0/4247645771 learned_addr learned my addr 172.21.15.2:0/4247645771 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe826d9d700 1 --2- 172.21.15.2:0/4247645771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 0x555e4f50b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 -- 172.21.15.2:0/4247645771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 msgr2=0x555e4f50b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 --2- 172.21.15.2:0/4247645771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 0x555e4f50b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 -- 172.21.15.2:0/4247645771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 msgr2=0x555e4f522ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 --2- 172.21.15.2:0/4247645771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f522ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 -- 172.21.15.2:0/4247645771 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4f51a5e0 con 0x555e4f511d00 2023-06-23T18:32:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe82659c700 1 --2- 172.21.15.2:0/4247645771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f511d00 0x555e4f51b330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8180998b0 tx=0x7fe818096020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771e6d6c80cbe94b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe8157fa700 1 -- 172.21.15.2:0/4247645771 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818099bb0 con 0x555e4f511d00 2023-06-23T18:32:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe8157fa700 1 -- 172.21.15.2:0/4247645771 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe81809dbe0 con 0x555e4f511d00 2023-06-23T18:32:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.718+0000 7fe8157fa700 1 -- 172.21.15.2:0/4247645771 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818098070 con 0x555e4f511d00 2023-06-23T18:32:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.719+0000 7fe838dabb80 1 -- 172.21.15.2:0/4247645771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f511d00 msgr2=0x555e4f51b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.719+0000 7fe838dabb80 1 --2- 172.21.15.2:0/4247645771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f511d00 0x555e4f51b330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe8180998b0 tx=0x7fe818096020 comp rx=0 tx=0).stop 2023-06-23T18:32:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 -- 172.21.15.2:0/4247645771 shutdown_connections 2023-06-23T18:32:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 --2- 172.21.15.2:0/4247645771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 0x555e4f50b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 --2- 172.21.15.2:0/4247645771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f511d00 0x555e4f51b330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 --2- 172.21.15.2:0/4247645771 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f513080 0x555e4f522ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 -- 172.21.15.2:0/4247645771 >> 172.21.15.2:0/4247645771 conn(0x555e4f507470 msgr2=0x555e4f5233f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 -- 172.21.15.2:0/4247645771 shutdown_connections 2023-06-23T18:32:33.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 -- 172.21.15.2:0/4247645771 wait complete. 2023-06-23T18:32:33.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 -- start start 2023-06-23T18:32:33.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.720+0000 7fe838dabb80 1 Processor -- start 2023-06-23T18:32:33.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f511d00 0x555e4f51dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f51e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe838dabb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 0x555e4f521280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe838dabb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f5217c0 con 0x555e4f511d00 2023-06-23T18:32:33.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe838dabb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f521930 con 0x555e4f513080 2023-06-23T18:32:33.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe838dabb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f521af0 con 0x555e4f51b870 2023-06-23T18:32:33.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe838dabb80 10 client.0 _ll_register_callbacks cb 0x555e4f4082e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:33.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f511d00 0x555e4f51dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f511d00 0x555e4f51dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56242/0 (socket says 172.21.15.2:56242) 2023-06-23T18:32:33.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 -- 172.21.15.2:0/691452694 learned_addr learned my addr 172.21.15.2:0/691452694 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe825d9b700 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f51e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 -- 172.21.15.2:0/691452694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 msgr2=0x555e4f521280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:32:33.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 0x555e4f521280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 -- 172.21.15.2:0/691452694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 msgr2=0x555e4f51e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f51e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 -- 172.21.15.2:0/691452694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8281a0020 con 0x555e4f511d00 2023-06-23T18:32:33.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe82659c700 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f511d00 0x555e4f51dd20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe818099b40 tx=0x7fe818097cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a50da92bb6cb20 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.721+0000 7fe80dffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:33.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe80dffb700 1 -- 172.21.15.2:0/691452694 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8180a2000 con 0x555e4f511d00 2023-06-23T18:32:33.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe80dffb700 1 -- 172.21.15.2:0/691452694 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe818099bb0 con 0x555e4f511d00 2023-06-23T18:32:33.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 1 -- 172.21.15.2:0/691452694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe82000f050 con 0x555e4f511d00 2023-06-23T18:32:33.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe80dffb700 1 -- 172.21.15.2:0/691452694 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8180990a0 con 0x555e4f511d00 2023-06-23T18:32:33.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 10 client.16302 fetch_fsmap learned FSMap version 34 2023-06-23T18:32:33.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 1 -- 172.21.15.2:0/691452694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x555e4f290e60 con 0x555e4f511d00 2023-06-23T18:32:33.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe80dffb700 1 -- 172.21.15.2:0/691452694 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe82000f050 con 0x555e4f511d00 2023-06-23T18:32:33.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe80dffb700 1 -- 172.21.15.2:0/691452694 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe818078ad0 con 0x555e4f511d00 2023-06-23T18:32:33.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 10 client.16302 fetch_fsmap finished waiting for FSMap version 34 2023-06-23T18:32:33.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:33.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 20 client.16302 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:33.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555e4f58e3d0 0x555e4f5907e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 4 client.16302 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:32:33.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.722+0000 7fe838dabb80 1 -- 172.21.15.2:0/691452694 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555e4f41e0d0 con 0x555e4f58e3d0 2023-06-23T18:32:33.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.723+0000 7fe825d9b700 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555e4f58e3d0 0x555e4f5907e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.723+0000 7fe80dffb700 10 client.16302 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:33.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.723+0000 7fe825d9b700 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555e4f58e3d0 0x555e4f5907e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17e943df47295749 server_cookie=55af910f9bfc3b78 in_seq=0 out_seq=0 2023-06-23T18:32:33.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe80dffb700 1 -- 172.21.15.2:0/691452694 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555e4f41e0d0 con 0x555e4f58e3d0 2023-06-23T18:32:33.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe80dffb700 10 client.16302 handle_command_reply: tid=0 2023-06-23T18:32:33.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 client.16302 shutdown 2023-06-23T18:32:33.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 -- 172.21.15.2:0/691452694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f511d00 msgr2=0x555e4f51dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f511d00 0x555e4f51dd20 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fe818099b40 tx=0x7fe818097cf0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 -- 172.21.15.2:0/691452694 shutdown_connections 2023-06-23T18:32:33.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x555e4f58e3d0 0x555e4f5907e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e4f51b870 0x555e4f521280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555e4f511d00 0x555e4f51dd20 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 --2- 172.21.15.2:0/691452694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555e4f513080 0x555e4f51e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 -- 172.21.15.2:0/691452694 >> 172.21.15.2:0/691452694 conn(0x555e4f507470 msgr2=0x555e4f510a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe80dffb700 0 client.16302 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:33.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 -- 172.21.15.2:0/691452694 shutdown_connections 2023-06-23T18:32:33.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.726+0000 7fe838dabb80 1 -- 172.21.15.2:0/691452694 wait complete. 2023-06-23T18:32:33.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe838dabb80 20 client.16302 trim_cache size 0 max 16384 2023-06-23T18:32:33.738 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:32:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:32:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "a5fecbaa-5816-42c7-afa5-5a75241424c5", 2023-06-23T18:32:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:32:33.739 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:32:33.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 -- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe8100a1150 msgr2=0x7fe8100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe8100a1150 0x7fe8100a3600 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe81c00b5c0 tx=0x7fe81c005fb0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 -- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828100600 msgr2=0x7fe828197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828100600 0x7fe828197990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe81800d980 tx=0x7fe81800dc90 comp rx=0 tx=0).stop 2023-06-23T18:32:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 -- 172.21.15.2:0/2079293335 shutdown_connections 2023-06-23T18:32:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe828105840 0x7fe82819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe828100600 0x7fe828197990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe8100a1150 0x7fe8100a3600 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 --2- 172.21.15.2:0/2079293335 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe828102f20 0x7fe828197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.730+0000 7fe82cc26700 1 -- 172.21.15.2:0/2079293335 >> 172.21.15.2:0/2079293335 conn(0x7fe8280fa0e0 msgr2=0x7fe828101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.731+0000 7fe82cc26700 1 -- 172.21.15.2:0/2079293335 shutdown_connections 2023-06-23T18:32:33.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.731+0000 7fe82cc26700 1 -- 172.21.15.2:0/2079293335 wait complete. 2023-06-23T18:32:33.751 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:32:33.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 -- 172.21.15.2:0/321060736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0105840 msgr2=0x7f99c0107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 --2- 172.21.15.2:0/321060736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0105840 0x7f99c0107ca0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b0a0 tx=0x7f99b800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 -- 172.21.15.2:0/321060736 shutdown_connections 2023-06-23T18:32:33.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 --2- 172.21.15.2:0/321060736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0105840 0x7f99c0107ca0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 --2- 172.21.15.2:0/321060736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c0100600 0x7f99c01029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 --2- 172.21.15.2:0/321060736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99c0102f20 0x7f99c0105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 -- 172.21.15.2:0/321060736 >> 172.21.15.2:0/321060736 conn(0x7f99c00fa0e0 msgr2=0x7f99c00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 -- 172.21.15.2:0/321060736 shutdown_connections 2023-06-23T18:32:33.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.896+0000 7f99c77c8700 1 -- 172.21.15.2:0/321060736 wait complete. 2023-06-23T18:32:33.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c77c8700 1 Processor -- start 2023-06-23T18:32:33.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c77c8700 1 -- start start 2023-06-23T18:32:33.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c77c8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0100600 0x7f99c0195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c77c8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99c0102f20 0x7f99c0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c77c8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c0105840 0x7f99c019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c77c8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019d400 con 0x7f99c0100600 2023-06-23T18:32:33.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c77c8700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019d540 con 0x7f99c0102f20 2023-06-23T18:32:33.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c77c8700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c019d6d0 con 0x7f99c0105840 2023-06-23T18:32:33.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0100600 0x7f99c0195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0100600 0x7f99c0195880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56256/0 (socket says 172.21.15.2:56256) 2023-06-23T18:32:33.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5564700 1 -- 172.21.15.2:0/148190554 learned_addr learned my addr 172.21.15.2:0/148190554 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5d65700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c0105840 0x7f99c019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c4d63700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99c0102f20 0x7f99c0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5564700 1 -- 172.21.15.2:0/148190554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c0105840 msgr2=0x7f99c019ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5564700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c0105840 0x7f99c019ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5564700 1 -- 172.21.15.2:0/148190554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99c0102f20 msgr2=0x7f99c0195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5564700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99c0102f20 0x7f99c0195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.897+0000 7f99c5564700 1 -- 172.21.15.2:0/148190554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8009d00 con 0x7f99c0100600 2023-06-23T18:32:33.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.898+0000 7f99c5564700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0100600 0x7f99c0195880 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e9a0 tx=0x7f99bc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.898+0000 7f99b67fc700 1 -- 172.21.15.2:0/148190554 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc009790 con 0x7f99c0100600 2023-06-23T18:32:33.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.898+0000 7f99c77c8700 1 -- 172.21.15.2:0/148190554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c019d9b0 con 0x7f99c0100600 2023-06-23T18:32:33.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.898+0000 7f99b67fc700 1 -- 172.21.15.2:0/148190554 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99bc010430 con 0x7f99c0100600 2023-06-23T18:32:33.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.898+0000 7f99b67fc700 1 -- 172.21.15.2:0/148190554 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc00cea0 con 0x7f99c0100600 2023-06-23T18:32:33.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.898+0000 7f99c77c8700 1 -- 172.21.15.2:0/148190554 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c019df10 con 0x7f99c0100600 2023-06-23T18:32:33.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.900+0000 7f99b67fc700 1 -- 172.21.15.2:0/148190554 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f99bc0105a0 con 0x7f99c0100600 2023-06-23T18:32:33.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.901+0000 7f99b67fc700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f99ac0a1150 0x7f99ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.901+0000 7f99b67fc700 1 -- 172.21.15.2:0/148190554 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f99bc014070 con 0x7f99c0100600 2023-06-23T18:32:33.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.901+0000 7f99c4d63700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f99ac0a1150 0x7f99ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.901+0000 7f99c4d63700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f99ac0a1150 0x7f99ac0a3600 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009c60 tx=0x7f99b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99d394db80 1 Processor -- start 2023-06-23T18:32:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99d394db80 1 -- start start 2023-06-23T18:32:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99d394db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99d394db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419d1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99d394db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 0x5566419d2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99d394db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566418d1cb0 con 0x5566419d1090 2023-06-23T18:32:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99d394db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556641924a40 con 0x5566419d2410 2023-06-23T18:32:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99d394db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55664191ded0 con 0x5566419d19e0 2023-06-23T18:32:33.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99c5564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99c5564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56264/0 (socket says 172.21.15.2:56264) 2023-06-23T18:32:33.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99c4d63700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419d1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.903+0000 7f99c5564700 1 -- 172.21.15.2:0/3583689303 learned_addr learned my addr 172.21.15.2:0/3583689303 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99c5d65700 1 --2- 172.21.15.2:0/3583689303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 0x5566419d2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99c4d63700 1 -- 172.21.15.2:0/3583689303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 msgr2=0x5566419d2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99c4d63700 1 --2- 172.21.15.2:0/3583689303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 0x5566419d2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99c4d63700 1 -- 172.21.15.2:0/3583689303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 msgr2=0x5566419d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99c4d63700 1 --2- 172.21.15.2:0/3583689303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99c4d63700 1 -- 172.21.15.2:0/3583689303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55664179f630 con 0x5566419d19e0 2023-06-23T18:32:33.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99c4d63700 1 --2- 172.21.15.2:0/3583689303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419d1e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f99b000e3b0 tx=0x7f99b0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a145314a063bd001 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99ab7fe700 1 -- 172.21.15.2:0/3583689303 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b000ed50 con 0x5566419d19e0 2023-06-23T18:32:33.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99ab7fe700 1 -- 172.21.15.2:0/3583689303 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99b0021070 con 0x5566419d19e0 2023-06-23T18:32:33.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.904+0000 7f99ab7fe700 1 -- 172.21.15.2:0/3583689303 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0003e10 con 0x5566419d19e0 2023-06-23T18:32:33.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 -- 172.21.15.2:0/3583689303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 msgr2=0x5566419d1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 --2- 172.21.15.2:0/3583689303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419d1e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f99b000e3b0 tx=0x7f99b0014000 comp rx=0 tx=0).stop 2023-06-23T18:32:33.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 -- 172.21.15.2:0/3583689303 shutdown_connections 2023-06-23T18:32:33.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 --2- 172.21.15.2:0/3583689303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 0x5566419d2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 --2- 172.21.15.2:0/3583689303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 --2- 172.21.15.2:0/3583689303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419d1e80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 -- 172.21.15.2:0/3583689303 >> 172.21.15.2:0/3583689303 conn(0x5566419c23f0 msgr2=0x5566419c7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 -- 172.21.15.2:0/3583689303 shutdown_connections 2023-06-23T18:32:33.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 -- 172.21.15.2:0/3583689303 wait complete. 2023-06-23T18:32:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 -- start start 2023-06-23T18:32:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.905+0000 7f99d394db80 1 Processor -- start 2023-06-23T18:32:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99d394db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419da920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99d394db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419dae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99d394db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 0x5566419e1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99d394db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566418bdac0 con 0x5566419d1090 2023-06-23T18:32:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99d394db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566418bd7c0 con 0x5566419d2410 2023-06-23T18:32:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99d394db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566417a1920 con 0x5566419d19e0 2023-06-23T18:32:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99d394db80 10 client.0 _ll_register_callbacks cb 0x5566418c3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c5564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419da920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c4d63700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419dae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c5564700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419da920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56270/0 (socket says 172.21.15.2:56270) 2023-06-23T18:32:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c5564700 1 -- 172.21.15.2:0/752744352 learned_addr learned my addr 172.21.15.2:0/752744352 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c5d65700 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 0x5566419e1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c4d63700 1 -- 172.21.15.2:0/752744352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 msgr2=0x5566419e1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c4d63700 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 0x5566419e1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c4d63700 1 -- 172.21.15.2:0/752744352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 msgr2=0x5566419da920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c4d63700 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419da920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c4d63700 1 -- 172.21.15.2:0/752744352 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99b0017070 con 0x5566419d19e0 2023-06-23T18:32:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.906+0000 7f99c5564700 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419da920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99c4d63700 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419dae60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f99b0011ab0 tx=0x7f99b0003860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d2eb69996f35f2b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:32:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99a8ff9700 1 -- 172.21.15.2:0/752744352 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b002a040 con 0x5566419d19e0 2023-06-23T18:32:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99a8ff9700 1 -- 172.21.15.2:0/752744352 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f99b00052c0 con 0x5566419d19e0 2023-06-23T18:32:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 1 -- 172.21.15.2:0/752744352 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566419bfbb0 con 0x5566419d19e0 2023-06-23T18:32:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99a8ff9700 1 -- 172.21.15.2:0/752744352 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b0003ea0 con 0x5566419d19e0 2023-06-23T18:32:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 10 client.26047 fetch_fsmap learned FSMap version 34 2023-06-23T18:32:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 1 -- 172.21.15.2:0/752744352 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55664174bde0 con 0x5566419d19e0 2023-06-23T18:32:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99a8ff9700 1 -- 172.21.15.2:0/752744352 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5566419bfbb0 con 0x5566419d19e0 2023-06-23T18:32:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99a8ff9700 1 -- 172.21.15.2:0/752744352 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f99b001d410 con 0x5566419d19e0 2023-06-23T18:32:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 10 client.26047 fetch_fsmap finished waiting for FSMap version 34 2023-06-23T18:32:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 20 client.26047 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566419e6510 0x5566419e8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 4 client.26047 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:32:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.907+0000 7f99d394db80 1 -- 172.21.15.2:0/752744352 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556641916eb0 con 0x5566419e6510 2023-06-23T18:32:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.908+0000 7f99c5564700 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566419e6510 0x5566419e8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.908+0000 7f99a8ff9700 10 client.26047 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.908+0000 7f99c5564700 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566419e6510 0x5566419e8920 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360d4a946d02472b server_cookie=5b8adb57046250a5 in_seq=0 out_seq=0 2023-06-23T18:32:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99a8ff9700 1 -- 172.21.15.2:0/752744352 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556641916eb0 con 0x5566419e6510 2023-06-23T18:32:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99a8ff9700 10 client.26047 handle_command_reply: tid=0 2023-06-23T18:32:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 client.26047 shutdown 2023-06-23T18:32:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 -- 172.21.15.2:0/752744352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 msgr2=0x5566419dae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419dae60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f99b0011ab0 tx=0x7f99b0003860 comp rx=0 tx=0).stop 2023-06-23T18:32:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 -- 172.21.15.2:0/752744352 shutdown_connections 2023-06-23T18:32:33.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566419e6510 0x5566419e8920 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d2410 0x5566419e1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99a8ff9700 0 client.26047 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:33.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d1090 0x5566419da920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 --2- 172.21.15.2:0/752744352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d19e0 0x5566419dae60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 -- 172.21.15.2:0/752744352 >> 172.21.15.2:0/752744352 conn(0x5566419c23f0 msgr2=0x5566419c6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 -- 172.21.15.2:0/752744352 shutdown_connections 2023-06-23T18:32:33.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 1 -- 172.21.15.2:0/752744352 wait complete. 2023-06-23T18:32:33.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.909+0000 7f99d394db80 20 client.26047 trim_cache size 0 max 16384 2023-06-23T18:32:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.934+0000 7f99d394db80 1 Processor -- start 2023-06-23T18:32:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.934+0000 7f99d394db80 1 -- start start 2023-06-23T18:32:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99d394db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419e2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99d394db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99d394db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419d5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99d394db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566419e60c0 con 0x5566419d19e0 2023-06-23T18:32:33.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99d394db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566419c1fa0 con 0x5566419d1090 2023-06-23T18:32:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99d394db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566419c8d80 con 0x5566419d2410 2023-06-23T18:32:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419e1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56278/0 (socket says 172.21.15.2:56278) 2023-06-23T18:32:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 -- 172.21.15.2:0/717719641 learned_addr learned my addr 172.21.15.2:0/717719641 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c5564700 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419e2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c5d65700 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419d5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 -- 172.21.15.2:0/717719641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 msgr2=0x5566419d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419d5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 -- 172.21.15.2:0/717719641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 msgr2=0x5566419e2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419e2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 -- 172.21.15.2:0/717719641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556641734c60 con 0x5566419d19e0 2023-06-23T18:32:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c5564700 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99c4d63700 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419e1e60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f99b000a010 tx=0x7f99b001d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abe91d03fb434db server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.935+0000 7f99abfff700 1 -- 172.21.15.2:0/717719641 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b003e070 con 0x5566419d19e0 2023-06-23T18:32:33.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99abfff700 1 -- 172.21.15.2:0/717719641 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99b00039e0 con 0x5566419d19e0 2023-06-23T18:32:33.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99abfff700 1 -- 172.21.15.2:0/717719641 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0003ea0 con 0x5566419d19e0 2023-06-23T18:32:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 -- 172.21.15.2:0/717719641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 msgr2=0x5566419e1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419e1e60 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f99b000a010 tx=0x7f99b001d9c0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 -- 172.21.15.2:0/717719641 shutdown_connections 2023-06-23T18:32:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419d5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419e2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 --2- 172.21.15.2:0/717719641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419e1e60 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 -- 172.21.15.2:0/717719641 >> 172.21.15.2:0/717719641 conn(0x5566419c23f0 msgr2=0x5566419ed240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 -- 172.21.15.2:0/717719641 shutdown_connections 2023-06-23T18:32:33.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.936+0000 7f99d394db80 1 -- 172.21.15.2:0/717719641 wait complete. 2023-06-23T18:32:33.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 1 -- start start 2023-06-23T18:32:33.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 1 Processor -- start 2023-06-23T18:32:33.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419ddef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419de430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419e1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566419e1990 con 0x5566419d19e0 2023-06-23T18:32:33.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566419e1b00 con 0x5566419d1090 2023-06-23T18:32:33.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566419e1cc0 con 0x5566419d2410 2023-06-23T18:32:33.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99d394db80 10 client.0 _ll_register_callbacks cb 0x5566418c3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:32:33.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99c4d63700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419de430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99c4d63700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419de430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56286/0 (socket says 172.21.15.2:56286) 2023-06-23T18:32:33.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99c4d63700 1 -- 172.21.15.2:0/2779426514 learned_addr learned my addr 172.21.15.2:0/2779426514 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:32:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99c5d65700 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419e1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.937+0000 7f99c5564700 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419ddef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99c4d63700 1 -- 172.21.15.2:0/2779426514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 msgr2=0x5566419e1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99c4d63700 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419e1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99c4d63700 1 -- 172.21.15.2:0/2779426514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 msgr2=0x5566419ddef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:32:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99c4d63700 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419ddef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99c4d63700 1 -- 172.21.15.2:0/2779426514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c019df10 con 0x5566419d19e0 2023-06-23T18:32:33.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99c5d65700 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:32:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99c4d63700 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419de430 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f99b001dd40 tx=0x7f99b001d6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cf03d124cc0da8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:32:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:32:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99a8ff9700 1 -- 172.21.15.2:0/2779426514 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b003e040 con 0x5566419d19e0 2023-06-23T18:32:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99a8ff9700 1 -- 172.21.15.2:0/2779426514 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f99b0035030 con 0x5566419d19e0 2023-06-23T18:32:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99d394db80 1 -- 172.21.15.2:0/2779426514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99b0021070 con 0x5566419d19e0 2023-06-23T18:32:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99a8ff9700 1 -- 172.21.15.2:0/2779426514 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b0003ea0 con 0x5566419d19e0 2023-06-23T18:32:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99d394db80 10 client.16320 fetch_fsmap learned FSMap version 34 2023-06-23T18:32:33.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99d394db80 1 -- 172.21.15.2:0/2779426514 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55664174bde0 con 0x5566419d19e0 2023-06-23T18:32:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.938+0000 7f99a8ff9700 1 -- 172.21.15.2:0/2779426514 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99b0021070 con 0x5566419d19e0 2023-06-23T18:32:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99a8ff9700 1 -- 172.21.15.2:0/2779426514 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 34) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f99b0026900 con 0x5566419d19e0 2023-06-23T18:32:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99d394db80 10 client.16320 fetch_fsmap finished waiting for FSMap version 34 2023-06-23T18:32:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99d394db80 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:32:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99d394db80 20 client.16320 populate_metadata read hostname 'smithi002' 2023-06-23T18:32:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99d394db80 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556641a3b1f0 0x556641a3d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:32:33.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99d394db80 4 client.16320 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:32:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99d394db80 1 -- 172.21.15.2:0/2779426514 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5566418d9050 con 0x556641a3b1f0 2023-06-23T18:32:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99c5564700 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556641a3b1f0 0x556641a3d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:32:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99a8ff9700 10 client.16320 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99c5564700 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556641a3b1f0 0x556641a3d5d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffcc16e660ba6aa6 server_cookie=95de3b130a7a31c3 in_seq=0 out_seq=0 2023-06-23T18:32:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99a8ff9700 1 -- 172.21.15.2:0/2779426514 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5566418d9050 con 0x556641a3b1f0 2023-06-23T18:32:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99a8ff9700 10 client.16320 handle_command_reply: tid=0 2023-06-23T18:32:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.939+0000 7f99d394db80 1 client.16320 shutdown 2023-06-23T18:32:33.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 -- 172.21.15.2:0/2779426514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 msgr2=0x5566419de430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419de430 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f99b001dd40 tx=0x7f99b001d6a0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 -- 172.21.15.2:0/2779426514 shutdown_connections 2023-06-23T18:32:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556641a3b1f0 0x556641a3d5d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566419d2410 0x5566419e1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566419d1090 0x5566419ddef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99a8ff9700 0 client.16320 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:32:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 --2- 172.21.15.2:0/2779426514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566419d19e0 0x5566419de430 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 -- 172.21.15.2:0/2779426514 >> 172.21.15.2:0/2779426514 conn(0x5566419c23f0 msgr2=0x5566419d8780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 -- 172.21.15.2:0/2779426514 shutdown_connections 2023-06-23T18:32:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 1 -- 172.21.15.2:0/2779426514 wait complete. 2023-06-23T18:32:33.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99d394db80 20 client.16320 trim_cache size 0 max 16384 2023-06-23T18:32:33.955 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:32:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:32:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:32:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "a5fecbaa-5816-42c7-afa5-5a75241424c5": { 2023-06-23T18:32:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:32:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "a5fecbaa-5816-42c7-afa5-5a75241424c5", 2023-06-23T18:32:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:32:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:32:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:32:33.956 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:32:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99c77c8700 1 -- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f99ac0a1150 msgr2=0x7f99ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99c77c8700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f99ac0a1150 0x7f99ac0a3600 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009c60 tx=0x7f99b0009260 comp rx=0 tx=0).stop 2023-06-23T18:32:33.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99c77c8700 1 -- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0100600 msgr2=0x7f99c0195880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:32:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99c77c8700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0100600 0x7f99c0195880 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00e9a0 tx=0x7f99bc00ecb0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99c77c8700 1 -- 172.21.15.2:0/148190554 shutdown_connections 2023-06-23T18:32:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.940+0000 7f99c77c8700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99c0105840 0x7f99c019ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.941+0000 7f99c77c8700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f99c0100600 0x7f99c0195880 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.941+0000 7f99c77c8700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f99ac0a1150 0x7f99ac0a3600 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.941+0000 7f99c77c8700 1 --2- 172.21.15.2:0/148190554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99c0102f20 0x7f99c0195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:32:33.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.941+0000 7f99c77c8700 1 -- 172.21.15.2:0/148190554 >> 172.21.15.2:0/148190554 conn(0x7f99c00fa0e0 msgr2=0x7f99c0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:32:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.941+0000 7f99c77c8700 1 -- 172.21.15.2:0/148190554 shutdown_connections 2023-06-23T18:32:33.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:32:33.941+0000 7f99c77c8700 1 -- 172.21.15.2:0/148190554 wait complete. 2023-06-23T18:32:33.962 INFO:tasks.cephfs.filesystem:scrub status for tag:a5fecbaa-5816-42c7-afa5-5a75241424c5 - {'path': '/', 'tag': 'a5fecbaa-5816-42c7-afa5-5a75241424c5', 'options': 'recursive,force'} 2023-06-23T18:32:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: pgmap v612: 97 pgs: 97 active+clean; 7.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 48 MiB/s wr, 87 op/s 2023-06-23T18:32:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:32:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:35.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:32:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:34 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: pgmap v612: 97 pgs: 97 active+clean; 7.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 48 MiB/s wr, 87 op/s 2023-06-23T18:32:35.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:35.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:32:35.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:35.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:35.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:35.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:35.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:35.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:32:35.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:34 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: pgmap v612: 97 pgs: 97 active+clean; 7.3 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 505 KiB/s rd, 48 MiB/s wr, 87 op/s 2023-06-23T18:32:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:32:35.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:35.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:35.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:35.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:35.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:35.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:32:35.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:35.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:34 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:32:37.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:36 smithi002 ceph-mon[95947]: pgmap v613: 97 pgs: 97 active+clean; 7.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 41 MiB/s wr, 124 op/s 2023-06-23T18:32:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:36 smithi082 ceph-mon[97889]: pgmap v613: 97 pgs: 97 active+clean; 7.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 41 MiB/s wr, 124 op/s 2023-06-23T18:32:37.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:36 smithi017 ceph-mon[97472]: pgmap v613: 97 pgs: 97 active+clean; 7.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 41 MiB/s wr, 124 op/s 2023-06-23T18:32:39.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:38 smithi002 ceph-mon[95947]: pgmap v614: 97 pgs: 97 active+clean; 7.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 39 MiB/s wr, 124 op/s 2023-06-23T18:32:39.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:38 smithi082 ceph-mon[97889]: pgmap v614: 97 pgs: 97 active+clean; 7.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 39 MiB/s wr, 124 op/s 2023-06-23T18:32:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:38 smithi017 ceph-mon[97472]: pgmap v614: 97 pgs: 97 active+clean; 7.4 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 39 MiB/s wr, 124 op/s 2023-06-23T18:32:40.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:39 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:40.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:39 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:40.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:39 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:32:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:40 smithi002 ceph-mon[95947]: pgmap v615: 97 pgs: 97 active+clean; 7.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 34 MiB/s wr, 124 op/s 2023-06-23T18:32:41.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:40 smithi082 ceph-mon[97889]: pgmap v615: 97 pgs: 97 active+clean; 7.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 34 MiB/s wr, 124 op/s 2023-06-23T18:32:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:40 smithi017 ceph-mon[97472]: pgmap v615: 97 pgs: 97 active+clean; 7.4 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 34 MiB/s wr, 124 op/s 2023-06-23T18:32:43.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:42 smithi002 ceph-mon[95947]: pgmap v616: 97 pgs: 97 active+clean; 7.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 28 MiB/s wr, 121 op/s 2023-06-23T18:32:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:42 smithi082 ceph-mon[97889]: pgmap v616: 97 pgs: 97 active+clean; 7.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 28 MiB/s wr, 121 op/s 2023-06-23T18:32:43.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:42 smithi017 ceph-mon[97472]: pgmap v616: 97 pgs: 97 active+clean; 7.5 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 28 MiB/s wr, 121 op/s 2023-06-23T18:32:45.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:44 smithi002 ceph-mon[95947]: pgmap v617: 97 pgs: 97 active+clean; 7.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 29 MiB/s wr, 111 op/s 2023-06-23T18:32:45.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:44 smithi082 ceph-mon[97889]: pgmap v617: 97 pgs: 97 active+clean; 7.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 29 MiB/s wr, 111 op/s 2023-06-23T18:32:45.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:44 smithi017 ceph-mon[97472]: pgmap v617: 97 pgs: 97 active+clean; 7.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 29 MiB/s wr, 111 op/s 2023-06-23T18:32:46.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:45 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:32:46.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:45 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:32:46.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:45 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:32:46.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:45 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:32:46.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:45 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:32:46.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:45 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:32:46.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:45 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:32:46.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:45 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:32:46.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:45 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:32:47.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:46 smithi002 ceph-mon[95947]: pgmap v618: 97 pgs: 97 active+clean; 7.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 28 MiB/s wr, 62 op/s 2023-06-23T18:32:47.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:46 smithi082 ceph-mon[97889]: pgmap v618: 97 pgs: 97 active+clean; 7.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 28 MiB/s wr, 62 op/s 2023-06-23T18:32:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:46 smithi017 ceph-mon[97472]: pgmap v618: 97 pgs: 97 active+clean; 7.6 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 28 MiB/s wr, 62 op/s 2023-06-23T18:32:49.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:48 smithi002 ceph-mon[95947]: pgmap v619: 97 pgs: 97 active+clean; 7.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 18 op/s 2023-06-23T18:32:49.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:48 smithi082 ceph-mon[97889]: pgmap v619: 97 pgs: 97 active+clean; 7.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 18 op/s 2023-06-23T18:32:49.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:48 smithi017 ceph-mon[97472]: pgmap v619: 97 pgs: 97 active+clean; 7.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 28 MiB/s wr, 18 op/s 2023-06-23T18:32:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:50 smithi002 ceph-mon[95947]: pgmap v620: 97 pgs: 97 active+clean; 7.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 20 op/s 2023-06-23T18:32:51.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:50 smithi082 ceph-mon[97889]: pgmap v620: 97 pgs: 97 active+clean; 7.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 20 op/s 2023-06-23T18:32:51.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:50 smithi017 ceph-mon[97472]: pgmap v620: 97 pgs: 97 active+clean; 7.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 29 MiB/s wr, 20 op/s 2023-06-23T18:32:53.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:52 smithi002 ceph-mon[95947]: pgmap v621: 97 pgs: 97 active+clean; 7.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 19 op/s 2023-06-23T18:32:53.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:52 smithi082 ceph-mon[97889]: pgmap v621: 97 pgs: 97 active+clean; 7.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 19 op/s 2023-06-23T18:32:53.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:52 smithi017 ceph-mon[97472]: pgmap v621: 97 pgs: 97 active+clean; 7.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 30 MiB/s wr, 19 op/s 2023-06-23T18:32:55.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:54 smithi002 ceph-mon[95947]: pgmap v622: 97 pgs: 97 active+clean; 7.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 19 op/s 2023-06-23T18:32:55.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:54 smithi082 ceph-mon[97889]: pgmap v622: 97 pgs: 97 active+clean; 7.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 19 op/s 2023-06-23T18:32:55.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:54 smithi017 ceph-mon[97472]: pgmap v622: 97 pgs: 97 active+clean; 7.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 19 op/s 2023-06-23T18:32:57.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:56 smithi002 ceph-mon[95947]: pgmap v623: 97 pgs: 97 active+clean; 7.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 16 op/s 2023-06-23T18:32:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:56 smithi082 ceph-mon[97889]: pgmap v623: 97 pgs: 97 active+clean; 7.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 16 op/s 2023-06-23T18:32:57.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:56 smithi017 ceph-mon[97472]: pgmap v623: 97 pgs: 97 active+clean; 7.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 30 MiB/s wr, 16 op/s 2023-06-23T18:32:59.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:32:58 smithi002 ceph-mon[95947]: pgmap v624: 97 pgs: 97 active+clean; 8.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 31 MiB/s wr, 15 op/s 2023-06-23T18:32:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:32:58 smithi082 ceph-mon[97889]: pgmap v624: 97 pgs: 97 active+clean; 8.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 31 MiB/s wr, 15 op/s 2023-06-23T18:32:59.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:32:58 smithi017 ceph-mon[97472]: pgmap v624: 97 pgs: 97 active+clean; 8.0 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 31 MiB/s wr, 15 op/s 2023-06-23T18:33:01.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:00 smithi002 ceph-mon[95947]: pgmap v625: 97 pgs: 97 active+clean; 8.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 31 MiB/s wr, 15 op/s 2023-06-23T18:33:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:00 smithi082 ceph-mon[97889]: pgmap v625: 97 pgs: 97 active+clean; 8.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 31 MiB/s wr, 15 op/s 2023-06-23T18:33:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:00 smithi017 ceph-mon[97472]: pgmap v625: 97 pgs: 97 active+clean; 8.0 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 31 MiB/s wr, 15 op/s 2023-06-23T18:33:02.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:01 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:02.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:01 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:02.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:01 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:02 smithi002 ceph-mon[95947]: pgmap v626: 97 pgs: 97 active+clean; 8.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-06-23T18:33:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:02 smithi082 ceph-mon[97889]: pgmap v626: 97 pgs: 97 active+clean; 8.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-06-23T18:33:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:02 smithi017 ceph-mon[97472]: pgmap v626: 97 pgs: 97 active+clean; 8.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-06-23T18:33:03.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:33:04.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 -- 172.21.15.2:0/3516188414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc0ffcf0 msgr2=0x7fc8bc100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/3516188414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc0ffcf0 0x7fc8bc100160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009990 tx=0x7fc8a8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 -- 172.21.15.2:0/3516188414 shutdown_connections 2023-06-23T18:33:04.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/3516188414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8bc1006a0 0x7fc8bc106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/3516188414 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8bc0fea60 0x7fc8bc0fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/3516188414 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc0ffcf0 0x7fc8bc100160 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 -- 172.21.15.2:0/3516188414 >> 172.21.15.2:0/3516188414 conn(0x7fc8bc0fa080 msgr2=0x7fc8bc0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 -- 172.21.15.2:0/3516188414 shutdown_connections 2023-06-23T18:33:04.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.114+0000 7fc8c2df1700 1 -- 172.21.15.2:0/3516188414 wait complete. 2023-06-23T18:33:04.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c2df1700 1 Processor -- start 2023-06-23T18:33:04.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c2df1700 1 -- start start 2023-06-23T18:33:04.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c2df1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8bc0fea60 0x7fc8bc195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c2df1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8bc0ffcf0 0x7fc8bc195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c2df1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc1006a0 0x7fc8bc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c2df1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc19d3b0 con 0x7fc8bc1006a0 2023-06-23T18:33:04.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c2df1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc19d4f0 con 0x7fc8bc0ffcf0 2023-06-23T18:33:04.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c2df1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc19d680 con 0x7fc8bc0fea60 2023-06-23T18:33:04.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c138e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc1006a0 0x7fc8bc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.115+0000 7fc8c0b8d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8bc0fea60 0x7fc8bc195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c138e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc1006a0 0x7fc8bc19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56302/0 (socket says 172.21.15.2:56302) 2023-06-23T18:33:04.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c138e700 1 -- 172.21.15.2:0/2511859294 learned_addr learned my addr 172.21.15.2:0/2511859294 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8bc0ffcf0 0x7fc8bc195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c0b8d700 1 -- 172.21.15.2:0/2511859294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8bc0ffcf0 msgr2=0x7fc8bc195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8bc0ffcf0 0x7fc8bc195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c0b8d700 1 -- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc1006a0 msgr2=0x7fc8bc19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc1006a0 0x7fc8bc19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c0b8d700 1 -- 172.21.15.2:0/2511859294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a80095f0 con 0x7fc8bc0fea60 2023-06-23T18:33:04.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8bc0ffcf0 0x7fc8bc195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:04.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c138e700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc1006a0 0x7fc8bc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:04.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8bc0fea60 0x7fc8bc195830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e9f0 tx=0x7fc8b000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8b9ffb700 1 -- 172.21.15.2:0/2511859294 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000c9c0 con 0x7fc8bc0fea60 2023-06-23T18:33:04.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8b9ffb700 1 -- 172.21.15.2:0/2511859294 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8b0010430 con 0x7fc8bc0fea60 2023-06-23T18:33:04.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8c2df1700 1 -- 172.21.15.2:0/2511859294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8bc107810 con 0x7fc8bc0fea60 2023-06-23T18:33:04.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.116+0000 7fc8b9ffb700 1 -- 172.21.15.2:0/2511859294 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0018680 con 0x7fc8bc0fea60 2023-06-23T18:33:04.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.117+0000 7fc8c2df1700 1 -- 172.21.15.2:0/2511859294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc107d20 con 0x7fc8bc0fea60 2023-06-23T18:33:04.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.119+0000 7fc8b9ffb700 1 -- 172.21.15.2:0/2511859294 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc8b00105a0 con 0x7fc8bc0fea60 2023-06-23T18:33:04.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.120+0000 7fc8b9ffb700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.120+0000 7fc8b9ffb700 1 -- 172.21.15.2:0/2511859294 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc8b0014070 con 0x7fc8bc0fea60 2023-06-23T18:33:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.120+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.120+0000 7fc8cef76b80 1 Processor -- start 2023-06-23T18:33:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.120+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8006010 tx=0x7fc8a8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8cef76b80 1 -- start start 2023-06-23T18:33:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe088c80 0x559dbe089090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe089a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe08a000 0x559dbe08a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbdf8dcb0 con 0x559dbe08a000 2023-06-23T18:33:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbdfe0a40 con 0x559dbe0895d0 2023-06-23T18:33:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbdfd9ed0 con 0x559dbe088c80 2023-06-23T18:33:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe08a000 0x559dbe08a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe08a000 0x559dbe08a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56310/0 (socket says 172.21.15.2:56310) 2023-06-23T18:33:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 -- 172.21.15.2:0/3334644561 learned_addr learned my addr 172.21.15.2:0/3334644561 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe089a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe088c80 0x559dbe089090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 -- 172.21.15.2:0/3334644561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe088c80 msgr2=0x559dbe089090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe088c80 0x559dbe089090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 -- 172.21.15.2:0/3334644561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 msgr2=0x559dbe089a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe089a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 -- 172.21.15.2:0/3334644561 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dbde5b630 con 0x559dbe08a000 2023-06-23T18:33:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.121+0000 7fc8c138e700 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe08a000 0x559dbe08a470 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4010d50 tx=0x7fc8b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f270f3d5e173179 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.122+0000 7fc8a6ffd700 1 -- 172.21.15.2:0/3334644561 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4015890 con 0x559dbe08a000 2023-06-23T18:33:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.122+0000 7fc8a6ffd700 1 -- 172.21.15.2:0/3334644561 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8b4016040 con 0x559dbe08a000 2023-06-23T18:33:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.122+0000 7fc8a6ffd700 1 -- 172.21.15.2:0/3334644561 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4007c60 con 0x559dbe08a000 2023-06-23T18:33:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.122+0000 7fc8cef76b80 1 -- 172.21.15.2:0/3334644561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe08a000 msgr2=0x559dbe08a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.122+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe08a000 0x559dbe08a470 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4010d50 tx=0x7fc8b400b040 comp rx=0 tx=0).stop 2023-06-23T18:33:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 -- 172.21.15.2:0/3334644561 shutdown_connections 2023-06-23T18:33:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe08a000 0x559dbe08a470 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe088c80 0x559dbe089090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/3334644561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe089a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 -- 172.21.15.2:0/3334644561 >> 172.21.15.2:0/3334644561 conn(0x559dbe07e3f0 msgr2=0x559dbe083520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 -- 172.21.15.2:0/3334644561 shutdown_connections 2023-06-23T18:33:04.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 -- 172.21.15.2:0/3334644561 wait complete. 2023-06-23T18:33:04.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 -- start start 2023-06-23T18:33:04.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.123+0000 7fc8cef76b80 1 Processor -- start 2023-06-23T18:33:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe088c80 0x559dbe092480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbdf79ac0 con 0x559dbe0895d0 2023-06-23T18:33:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbdf797c0 con 0x559dbe088c80 2023-06-23T18:33:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbde5d920 con 0x559dbe08a000 2023-06-23T18:33:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8cef76b80 10 client.0 _ll_register_callbacks cb 0x559dbdf7f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56314/0 (socket says 172.21.15.2:56314) 2023-06-23T18:33:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 -- 172.21.15.2:0/1355240172 learned_addr learned my addr 172.21.15.2:0/1355240172 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe088c80 0x559dbe092480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8c138e700 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 -- 172.21.15.2:0/1355240172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 msgr2=0x559dbe099a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 -- 172.21.15.2:0/1355240172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe088c80 msgr2=0x559dbe092480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe088c80 0x559dbe092480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 -- 172.21.15.2:0/1355240172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b4010a00 con 0x559dbe0895d0 2023-06-23T18:33:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8c138e700 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:33:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005cd0 tx=0x7fc8a801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd35b8c32f1748e1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.124+0000 7fc89bfff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc89bfff700 1 -- 172.21.15.2:0/1355240172 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a802b050 con 0x559dbe0895d0 2023-06-23T18:33:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc89bfff700 1 -- 172.21.15.2:0/1355240172 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc8a800fc50 con 0x559dbe0895d0 2023-06-23T18:33:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 1 -- 172.21.15.2:0/1355240172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dbe07bbb0 con 0x559dbe0895d0 2023-06-23T18:33:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc89bfff700 1 -- 172.21.15.2:0/1355240172 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a800fe10 con 0x559dbe0895d0 2023-06-23T18:33:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 10 client.16332 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 1 -- 172.21.15.2:0/1355240172 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559dbde07de0 con 0x559dbe0895d0 2023-06-23T18:33:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc89bfff700 1 -- 172.21.15.2:0/1355240172 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559dbe07bbb0 con 0x559dbe0895d0 2023-06-23T18:33:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc89bfff700 1 -- 172.21.15.2:0/1355240172 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8a80053d0 con 0x559dbe0895d0 2023-06-23T18:33:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:04.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 20 client.16332 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559dbe09e280 0x559dbe0a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 4 client.16332 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.125+0000 7fc8cef76b80 1 -- 172.21.15.2:0/1355240172 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dbdfd2eb0 con 0x559dbe09e280 2023-06-23T18:33:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.126+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559dbe09e280 0x559dbe0a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.126+0000 7fc89bfff700 10 client.16332 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.126+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559dbe09e280 0x559dbe0a0690 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e198e5147383f2e server_cookie=77666e79ee261ffc in_seq=0 out_seq=0 2023-06-23T18:33:04.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc89bfff700 1 -- 172.21.15.2:0/1355240172 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559dbdfd2eb0 con 0x559dbe09e280 2023-06-23T18:33:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc89bfff700 10 client.16332 handle_command_reply: tid=0 2023-06-23T18:33:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 client.16332 shutdown 2023-06-23T18:33:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 -- 172.21.15.2:0/1355240172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe0895d0 msgr2=0x559dbe0929c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8005cd0 tx=0x7fc8a801a050 comp rx=0 tx=0).stop 2023-06-23T18:33:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 -- 172.21.15.2:0/1355240172 shutdown_connections 2023-06-23T18:33:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559dbe09e280 0x559dbe0a0690 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe088c80 0x559dbe092480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/1355240172 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 -- 172.21.15.2:0/1355240172 >> 172.21.15.2:0/1355240172 conn(0x559dbe07e3f0 msgr2=0x559dbe082c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc89bfff700 0 client.16332 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 -- 172.21.15.2:0/1355240172 shutdown_connections 2023-06-23T18:33:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 1 -- 172.21.15.2:0/1355240172 wait complete. 2023-06-23T18:33:04.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.127+0000 7fc8cef76b80 20 client.16332 trim_cache size 0 max 16384 2023-06-23T18:33:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8cef76b80 1 Processor -- start 2023-06-23T18:33:04.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8cef76b80 1 -- start start 2023-06-23T18:33:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe092480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbe091930 con 0x559dbe088c80 2023-06-23T18:33:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbe091ff0 con 0x559dbe0895d0 2023-06-23T18:33:04.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbe084fb0 con 0x559dbe08a000 2023-06-23T18:33:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8c0b8d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe092480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8c0b8d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe092480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56320/0 (socket says 172.21.15.2:56320) 2023-06-23T18:33:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8c0b8d700 1 -- 172.21.15.2:0/2656947816 learned_addr learned my addr 172.21.15.2:0/2656947816 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.153+0000 7fc8c138e700 1 --2- 172.21.15.2:0/2656947816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8c0b8d700 1 -- 172.21.15.2:0/2656947816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 msgr2=0x559dbe099990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2656947816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8c0b8d700 1 -- 172.21.15.2:0/2656947816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 msgr2=0x559dbe0929c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2656947816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8c0b8d700 1 -- 172.21.15.2:0/2656947816 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dbddf0c60 con 0x559dbe088c80 2023-06-23T18:33:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2656947816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe092480 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0099af0 tx=0x7fc8b00ca3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b9ed60d83aaab5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8a77fe700 1 -- 172.21.15.2:0/2656947816 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0099df0 con 0x559dbe088c80 2023-06-23T18:33:04.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8a77fe700 1 -- 172.21.15.2:0/2656947816 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8b009de20 con 0x559dbe088c80 2023-06-23T18:33:04.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.154+0000 7fc8a77fe700 1 -- 172.21.15.2:0/2656947816 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b00992e0 con 0x559dbe088c80 2023-06-23T18:33:04.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2656947816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 msgr2=0x559dbe092480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2656947816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe092480 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0099af0 tx=0x7fc8b00ca3b0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2656947816 shutdown_connections 2023-06-23T18:33:04.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2656947816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe099990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2656947816 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe092480 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2656947816 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe0929c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2656947816 >> 172.21.15.2:0/2656947816 conn(0x559dbe07e3f0 msgr2=0x559dbe090330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2656947816 shutdown_connections 2023-06-23T18:33:04.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2656947816 wait complete. 2023-06-23T18:33:04.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.155+0000 7fc8cef76b80 1 -- start start 2023-06-23T18:33:04.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8cef76b80 1 Processor -- start 2023-06-23T18:33:04.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe099550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe095ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8cef76b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe096530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbe096c10 con 0x559dbe088c80 2023-06-23T18:33:04.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbe099bd0 con 0x559dbe0895d0 2023-06-23T18:33:04.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8cef76b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dbe099d60 con 0x559dbe08a000 2023-06-23T18:33:04.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8cef76b80 10 client.0 _ll_register_callbacks cb 0x559dbdf7f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:04.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8c0b8d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe099550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8c0b8d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe099550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56326/0 (socket says 172.21.15.2:56326) 2023-06-23T18:33:04.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8c138e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe096530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.156+0000 7fc8c0b8d700 1 -- 172.21.15.2:0/2467986656 learned_addr learned my addr 172.21.15.2:0/2467986656 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8bbfff700 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe095ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8c138e700 1 -- 172.21.15.2:0/2467986656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 msgr2=0x559dbe095ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8c138e700 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe095ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8c138e700 1 -- 172.21.15.2:0/2467986656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 msgr2=0x559dbe099550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8c138e700 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe099550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8c138e700 1 -- 172.21.15.2:0/2467986656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8bc107d20 con 0x559dbe08a000 2023-06-23T18:33:04.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe099550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:04.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8c138e700 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe096530 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4010d20 tx=0x7fc8b4003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d061d2d24f0b32 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc89bfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:33:04.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc89bfff700 1 -- 172.21.15.2:0/2467986656 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4007860 con 0x559dbe08a000 2023-06-23T18:33:04.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2467986656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8b4016040 con 0x559dbe08a000 2023-06-23T18:33:04.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc89bfff700 1 -- 172.21.15.2:0/2467986656 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc8b40079c0 con 0x559dbe08a000 2023-06-23T18:33:04.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc89bfff700 1 -- 172.21.15.2:0/2467986656 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8b4007bc0 con 0x559dbe08a000 2023-06-23T18:33:04.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8cef76b80 10 client.26071 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:04.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.157+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2467986656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559dbde2e250 con 0x559dbe08a000 2023-06-23T18:33:04.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.158+0000 7fc89bfff700 1 -- 172.21.15.2:0/2467986656 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8b4016040 con 0x559dbe08a000 2023-06-23T18:33:04.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.158+0000 7fc89bfff700 1 -- 172.21.15.2:0/2467986656 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc8b401b610 con 0x559dbe08a000 2023-06-23T18:33:04.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.158+0000 7fc8cef76b80 10 client.26071 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:04.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.158+0000 7fc8cef76b80 10 client.26071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:04.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.158+0000 7fc8cef76b80 20 client.26071 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:04.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.158+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559dbe0f38c0 0x559dbe0f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.158+0000 7fc8cef76b80 4 client.26071 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:33:04.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.158+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2467986656 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559dbdf95050 con 0x559dbe0f38c0 2023-06-23T18:33:04.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.159+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559dbe0f38c0 0x559dbe0f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc89bfff700 10 client.26071 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:04.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8c0b8d700 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559dbe0f38c0 0x559dbe0f5cd0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42a56321e5c3ca1 server_cookie=4d46f612266fad11 in_seq=0 out_seq=0 2023-06-23T18:33:04.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc89bfff700 1 -- 172.21.15.2:0/2467986656 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559dbdf95050 con 0x559dbe0f38c0 2023-06-23T18:33:04.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc89bfff700 10 client.26071 handle_command_reply: tid=0 2023-06-23T18:33:04.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 client.26071 shutdown 2023-06-23T18:33:04.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2467986656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 msgr2=0x559dbe096530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe096530 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4010d20 tx=0x7fc8b4003a00 comp rx=0 tx=0).stop 2023-06-23T18:33:04.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2467986656 shutdown_connections 2023-06-23T18:33:04.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559dbe0f38c0 0x559dbe0f5cd0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dbe08a000 0x559dbe096530 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dbe088c80 0x559dbe099550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 --2- 172.21.15.2:0/2467986656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dbe0895d0 0x559dbe095ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.160+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2467986656 >> 172.21.15.2:0/2467986656 conn(0x559dbe07e3f0 msgr2=0x559dbe08ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.161+0000 7fc89bfff700 0 client.26071 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:04.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.161+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2467986656 shutdown_connections 2023-06-23T18:33:04.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.161+0000 7fc8cef76b80 1 -- 172.21.15.2:0/2467986656 wait complete. 2023-06-23T18:33:04.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.161+0000 7fc8cef76b80 20 client.26071 trim_cache size 0 max 16384 2023-06-23T18:33:04.173 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:33:04.174 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:33:04.174 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:33:04.174 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:33:04.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 -- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8ac0a1150 msgr2=0x7fc8ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8006010 tx=0x7fc8a8005960 comp rx=0 tx=0).stop 2023-06-23T18:33:04.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 -- 172.21.15.2:0/2511859294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8bc0fea60 msgr2=0x7fc8bc195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8bc0fea60 0x7fc8bc195830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000e9f0 tx=0x7fc8b000ed00 comp rx=0 tx=0).stop 2023-06-23T18:33:04.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 -- 172.21.15.2:0/2511859294 shutdown_connections 2023-06-23T18:33:04.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc8bc1006a0 0x7fc8bc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8bc0fea60 0x7fc8bc195830 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc8ac0a1150 0x7fc8ac0a3600 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 --2- 172.21.15.2:0/2511859294 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8bc0ffcf0 0x7fc8bc195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 -- 172.21.15.2:0/2511859294 >> 172.21.15.2:0/2511859294 conn(0x7fc8bc0fa080 msgr2=0x7fc8bc103280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 -- 172.21.15.2:0/2511859294 shutdown_connections 2023-06-23T18:33:04.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.162+0000 7fc8c2df1700 1 -- 172.21.15.2:0/2511859294 wait complete. 2023-06-23T18:33:04.183 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:33:04.184 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:33:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.332+0000 7f53f9abc700 1 -- 172.21.15.2:0/522474210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f40fea30 msgr2=0x7f53f40fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.332+0000 7f53f9abc700 1 --2- 172.21.15.2:0/522474210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f40fea30 0x7f53f40fee40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009990 tx=0x7f53e4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.332+0000 7f53f9abc700 1 -- 172.21.15.2:0/522474210 shutdown_connections 2023-06-23T18:33:04.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.333+0000 7f53f9abc700 1 --2- 172.21.15.2:0/522474210 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53f4100670 0x7f53f4068930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.333+0000 7f53f9abc700 1 --2- 172.21.15.2:0/522474210 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f40fea30 0x7f53f40fee40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.333+0000 7f53f9abc700 1 --2- 172.21.15.2:0/522474210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53f40ffcc0 0x7f53f4100130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.333+0000 7f53f9abc700 1 -- 172.21.15.2:0/522474210 >> 172.21.15.2:0/522474210 conn(0x7f53f40fa050 msgr2=0x7f53f40fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.333+0000 7f53f9abc700 1 -- 172.21.15.2:0/522474210 shutdown_connections 2023-06-23T18:33:04.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.333+0000 7f53f9abc700 1 -- 172.21.15.2:0/522474210 wait complete. 2023-06-23T18:33:04.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.333+0000 7f53f9abc700 1 Processor -- start 2023-06-23T18:33:04.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f9abc700 1 -- start start 2023-06-23T18:33:04.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f9abc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53f40fea30 0x7f53f41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f9abc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53f40ffcc0 0x7f53f4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f9abc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f4100670 0x7f53f419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f9abc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419f560 con 0x7f53f4100670 2023-06-23T18:33:04.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f9abc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419f6a0 con 0x7f53f40fea30 2023-06-23T18:33:04.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f9abc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f419f830 con 0x7f53f40ffcc0 2023-06-23T18:33:04.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53f40fea30 0x7f53f41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53f40fea30 0x7f53f41979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46932/0 (socket says 172.21.15.2:46932) 2023-06-23T18:33:04.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f37fe700 1 -- 172.21.15.2:0/2640529216 learned_addr learned my addr 172.21.15.2:0/2640529216 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f2ffd700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53f40ffcc0 0x7f53f4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f3fff700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f4100670 0x7f53f419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f37fe700 1 -- 172.21.15.2:0/2640529216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53f40ffcc0 msgr2=0x7f53f4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f37fe700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53f40ffcc0 0x7f53f4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f37fe700 1 -- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f4100670 msgr2=0x7f53f419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f37fe700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f4100670 0x7f53f419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f37fe700 1 -- 172.21.15.2:0/2640529216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec009c40 con 0x7f53f40fea30 2023-06-23T18:33:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.334+0000 7f53f2ffd700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53f40ffcc0 0x7f53f4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:33:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.335+0000 7f53f37fe700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53f40fea30 0x7f53f41979e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f53e40098e0 tx=0x7f53e4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.335+0000 7f53f0ff9700 1 -- 172.21.15.2:0/2640529216 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e401d070 con 0x7f53f40fea30 2023-06-23T18:33:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.335+0000 7f53f0ff9700 1 -- 172.21.15.2:0/2640529216 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f53e400f460 con 0x7f53f40fea30 2023-06-23T18:33:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.335+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e40095f0 con 0x7f53f40fea30 2023-06-23T18:33:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.335+0000 7f53f0ff9700 1 -- 172.21.15.2:0/2640529216 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4021670 con 0x7f53f40fea30 2023-06-23T18:33:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.335+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f419fda0 con 0x7f53f40fea30 2023-06-23T18:33:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.336+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f41078b0 con 0x7f53f40fea30 2023-06-23T18:33:04.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.338+0000 7f53f0ff9700 1 -- 172.21.15.2:0/2640529216 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f53e400f5d0 con 0x7f53f40fea30 2023-06-23T18:33:04.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.338+0000 7f53f0ff9700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53dc0a11d0 0x7f53dc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.338+0000 7f53f0ff9700 1 -- 172.21.15.2:0/2640529216 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f53e40cdda0 con 0x7f53f40fea30 2023-06-23T18:33:04.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.338+0000 7f53f2ffd700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53dc0a11d0 0x7f53dc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.339+0000 7f53f2ffd700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53dc0a11d0 0x7f53dc0a3680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53e8005950 tx=0x7f53e80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.343+0000 7f53f0ff9700 1 -- 172.21.15.2:0/2640529216 <== mon.2 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f53e4097580 con 0x7f53f40fea30 2023-06-23T18:33:04.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.645+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53f402c810 con 0x7f53f40fea30 2023-06-23T18:33:04.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.646+0000 7f53f0ff9700 1 -- 172.21.15.2:0/2640529216 <== mon.2 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10780 (secure 0 0 0) 0x7f53e409aba0 con 0x7f53f40fea30 2023-06-23T18:33:04.650 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:33:04.651 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:33:00.759725+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:33:04.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53dc0a11d0 msgr2=0x7f53dc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53dc0a11d0 0x7f53dc0a3680 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53e8005950 tx=0x7f53e80058e0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53f40fea30 msgr2=0x7f53f41979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53f40fea30 0x7f53f41979e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f53e40098e0 tx=0x7f53e4005ef0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 shutdown_connections 2023-06-23T18:33:04.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f4100670 0x7f53f419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53dc0a11d0 0x7f53dc0a3680 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f53f40fea30 0x7f53f41979e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 --2- 172.21.15.2:0/2640529216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f53f40ffcc0 0x7f53f4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 >> 172.21.15.2:0/2640529216 conn(0x7f53f40fa050 msgr2=0x7f53f40fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 shutdown_connections 2023-06-23T18:33:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.649+0000 7f53f9abc700 1 -- 172.21.15.2:0/2640529216 wait complete. 2023-06-23T18:33:04.655 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 36 2023-06-23T18:33:04.667 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:33:04.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/4249701629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0ffc70 msgr2=0x7f8adc1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/4249701629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0ffc70 0x7f8adc1000c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009990 tx=0x7f8ad0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/4249701629 shutdown_connections 2023-06-23T18:33:04.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/4249701629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc100600 0x7f8adc106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/4249701629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8adc0fea80 0x7f8adc0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/4249701629 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0ffc70 0x7f8adc1000c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/4249701629 >> 172.21.15.2:0/4249701629 conn(0x7f8adc0fa0e0 msgr2=0x7f8adc0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/4249701629 shutdown_connections 2023-06-23T18:33:04.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.814+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/4249701629 wait complete. 2023-06-23T18:33:04.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.815+0000 7f8ae1b3b700 1 Processor -- start 2023-06-23T18:33:04.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.815+0000 7f8ae1b3b700 1 -- start start 2023-06-23T18:33:04.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.815+0000 7f8ae1b3b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0fea80 0x7f8adc109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8ae1b3b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0ffc70 0x7f8adc1079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8ae1b3b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8adc100600 0x7f8adc107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8ae1b3b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc108690 con 0x7f8adc0fea80 2023-06-23T18:33:04.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8ae1b3b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc108800 con 0x7f8adc0ffc70 2023-06-23T18:33:04.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8ae1b3b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc19f810 con 0x7f8adc100600 2023-06-23T18:33:04.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0fea80 0x7f8adc109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0fea80 0x7f8adc109370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56354/0 (socket says 172.21.15.2:56354) 2023-06-23T18:33:04.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adb7fe700 1 -- 172.21.15.2:0/2795259248 learned_addr learned my addr 172.21.15.2:0/2795259248 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0ffc70 0x7f8adc1079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adbfff700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8adc100600 0x7f8adc107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adaffd700 1 -- 172.21.15.2:0/2795259248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8adc100600 msgr2=0x7f8adc107ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8adc100600 0x7f8adc107ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adaffd700 1 -- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0fea80 msgr2=0x7f8adc109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0fea80 0x7f8adc109370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adaffd700 1 -- 172.21.15.2:0/2795259248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00095f0 con 0x7f8adc0ffc70 2023-06-23T18:33:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0fea80 0x7f8adc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0ffc70 0x7f8adc1079b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0006010 tx=0x7f8ad0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.816+0000 7f8ad8ff9700 1 -- 172.21.15.2:0/2795259248 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad001d070 con 0x7f8adc0ffc70 2023-06-23T18:33:04.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.817+0000 7f8ad8ff9700 1 -- 172.21.15.2:0/2795259248 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ad0003f40 con 0x7f8adc0ffc70 2023-06-23T18:33:04.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.817+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/2795259248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc19fa90 con 0x7f8adc0ffc70 2023-06-23T18:33:04.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.817+0000 7f8ad8ff9700 1 -- 172.21.15.2:0/2795259248 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0022b80 con 0x7f8adc0ffc70 2023-06-23T18:33:04.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.817+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/2795259248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc19ff10 con 0x7f8adc0ffc70 2023-06-23T18:33:04.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.819+0000 7f8ad8ff9700 1 -- 172.21.15.2:0/2795259248 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8ad0003ab0 con 0x7f8adc0ffc70 2023-06-23T18:33:04.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.819+0000 7f8ad8ff9700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8ac40a11d0 0x7f8ac40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.819+0000 7f8ad8ff9700 1 -- 172.21.15.2:0/2795259248 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8ad00cdf00 con 0x7f8adc0ffc70 2023-06-23T18:33:04.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.819+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8ac40a11d0 0x7f8ac40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.820+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8ac40a11d0 0x7f8ac40a3680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8acc007fd0 tx=0x7f8acc007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8aedcc0b80 1 Processor -- start 2023-06-23T18:33:04.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8aedcc0b80 1 -- start start 2023-06-23T18:33:04.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 0x559dedbaa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbaa5d0 0x559dedbaaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbab000 0x559dedbab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedaaecb0 con 0x559dedbaa5d0 2023-06-23T18:33:04.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedb01a40 con 0x559dedbab000 2023-06-23T18:33:04.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedafaed0 con 0x559dedba9c80 2023-06-23T18:33:04.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8adaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbaa5d0 0x559dedbaaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8adaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbaa5d0 0x559dedbaaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56362/0 (socket says 172.21.15.2:56362) 2023-06-23T18:33:04.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8adaffd700 1 -- 172.21.15.2:0/1872002528 learned_addr learned my addr 172.21.15.2:0/1872002528 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.821+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 0x559dedbaa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8adbfff700 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbab000 0x559dedbab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8adaffd700 1 -- 172.21.15.2:0/1872002528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 msgr2=0x559dedbaa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8adaffd700 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 0x559dedbaa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8adaffd700 1 -- 172.21.15.2:0/1872002528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbab000 msgr2=0x559dedbab470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8adaffd700 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbab000 0x559dedbab470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8adaffd700 1 -- 172.21.15.2:0/1872002528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ded97c630 con 0x559dedbaa5d0 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8adaffd700 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbaa5d0 0x559dedbaaa70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0000c00 tx=0x7f8ad00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c00f94d97db58ed server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8ac1ffb700 1 -- 172.21.15.2:0/1872002528 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00860f0 con 0x559dedbaa5d0 2023-06-23T18:33:04.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8ac1ffb700 1 -- 172.21.15.2:0/1872002528 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ad009b3a0 con 0x559dedbaa5d0 2023-06-23T18:33:04.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.822+0000 7f8ac1ffb700 1 -- 172.21.15.2:0/1872002528 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0099740 con 0x559dedbaa5d0 2023-06-23T18:33:04.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/1872002528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbaa5d0 msgr2=0x559dedbaaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbaa5d0 0x559dedbaaa70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0000c00 tx=0x7f8ad00047a0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/1872002528 shutdown_connections 2023-06-23T18:33:04.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbab000 0x559dedbab470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 0x559dedbaa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/1872002528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbaa5d0 0x559dedbaaa70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/1872002528 >> 172.21.15.2:0/1872002528 conn(0x559dedb9f3f0 msgr2=0x559dedba4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/1872002528 shutdown_connections 2023-06-23T18:33:04.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/1872002528 wait complete. 2023-06-23T18:33:04.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.823+0000 7f8aedcc0b80 1 -- start start 2023-06-23T18:33:04.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8aedcc0b80 1 Processor -- start 2023-06-23T18:33:04.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbb3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbb3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559deda9aac0 con 0x559dedbab000 2023-06-23T18:33:04.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559deda9a7c0 con 0x559dedba9c80 2023-06-23T18:33:04.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ded97e920 con 0x559dedbaa5d0 2023-06-23T18:33:04.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8aedcc0b80 10 client.0 _ll_register_callbacks cb 0x559dedaa0260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:04.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbb3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbb3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46960/0 (socket says 172.21.15.2:46960) 2023-06-23T18:33:04.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adb7fe700 1 -- 172.21.15.2:0/2307586149 learned_addr learned my addr 172.21.15.2:0/2307586149 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbb3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adbfff700 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adb7fe700 1 -- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 msgr2=0x559dedbb3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbb3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adb7fe700 1 -- 172.21.15.2:0/2307586149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 msgr2=0x559dedbba790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbba790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adb7fe700 1 -- 172.21.15.2:0/2307586149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc19ff10 con 0x559dedba9c80 2023-06-23T18:33:04.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.824+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbb3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:04.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbb3210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8acc011ae0 tx=0x7f8acc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77600fa7a9c94142 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8abaffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:33:04.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8abaffd700 1 -- 172.21.15.2:0/2307586149 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc01f840 con 0x559dedba9c80 2023-06-23T18:33:04.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8abaffd700 1 -- 172.21.15.2:0/2307586149 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8acc017070 con 0x559dedba9c80 2023-06-23T18:33:04.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307586149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dedb9cbb0 con 0x559dedba9c80 2023-06-23T18:33:04.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8abaffd700 1 -- 172.21.15.2:0/2307586149 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8acc01f9a0 con 0x559dedba9c80 2023-06-23T18:33:04.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8aedcc0b80 10 client.25889 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:04.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307586149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559ded928de0 con 0x559dedba9c80 2023-06-23T18:33:04.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.825+0000 7f8abaffd700 1 -- 172.21.15.2:0/2307586149 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559dedb9cbb0 con 0x559dedba9c80 2023-06-23T18:33:04.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8abaffd700 1 -- 172.21.15.2:0/2307586149 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8acc01ecb0 con 0x559dedba9c80 2023-06-23T18:33:04.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8aedcc0b80 10 client.25889 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:04.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8aedcc0b80 10 client.25889 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:33:04.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8aedcc0b80 20 client.25889 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:04.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x559dedbbf050 0x559dedbc1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8aedcc0b80 4 client.25889 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:04.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307586149 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dedaf3eb0 con 0x559dedbbf050 2023-06-23T18:33:04.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x559dedbbf050 0x559dedbc1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8abaffd700 10 client.25889 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:33:04.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.826+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x559dedbbf050 0x559dedbc1460 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=290854c51fc9fd55 server_cookie=ef81be1f1fce99c5 in_seq=0 out_seq=0 2023-06-23T18:33:04.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8abaffd700 1 -- 172.21.15.2:0/2307586149 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559dedaf3eb0 con 0x559dedbbf050 2023-06-23T18:33:04.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8abaffd700 10 client.25889 handle_command_reply: tid=0 2023-06-23T18:33:04.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 client.25889 shutdown 2023-06-23T18:33:04.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307586149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 msgr2=0x559dedbb3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbb3210 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8acc011ae0 tx=0x7f8acc0047c0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307586149 shutdown_connections 2023-06-23T18:33:04.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x559dedbbf050 0x559dedbc1460 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8abaffd700 0 client.25889 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:33:04.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbb3210 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307586149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbb3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307586149 >> 172.21.15.2:0/2307586149 conn(0x559dedb9f3f0 msgr2=0x559dedba3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307586149 shutdown_connections 2023-06-23T18:33:04.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307586149 wait complete. 2023-06-23T18:33:04.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.827+0000 7f8aedcc0b80 20 client.25889 trim_cache size 0 max 16384 2023-06-23T18:33:04.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8aedcc0b80 1 Processor -- start 2023-06-23T18:33:04.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8aedcc0b80 1 -- start start 2023-06-23T18:33:04.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbc5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedbb1ba0 con 0x559dedbab000 2023-06-23T18:33:04.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedaf3ee0 con 0x559dedba9c80 2023-06-23T18:33:04.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedbb26c0 con 0x559dedbaa5d0 2023-06-23T18:33:04.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8adaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8adaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbba720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56716/0 (socket says 172.21.15.2:56716) 2023-06-23T18:33:04.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.842+0000 7f8adaffd700 1 -- 172.21.15.2:0/2307061064 learned_addr learned my addr 172.21.15.2:0/2307061064 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbc5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adbfff700 1 -- 172.21.15.2:0/2307061064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 msgr2=0x559dedbba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adbfff700 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbba720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adbfff700 1 -- 172.21.15.2:0/2307061064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 msgr2=0x559dedbc5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adbfff700 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbc5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adbfff700 1 -- 172.21.15.2:0/2307061064 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ded911c60 con 0x559dedbab000 2023-06-23T18:33:04.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:04.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbc5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:04.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8adbfff700 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbade20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009650 tx=0x7f8ad40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7969d208bf131663 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8ac27fc700 1 -- 172.21.15.2:0/2307061064 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4009e60 con 0x559dedbab000 2023-06-23T18:33:04.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8ac27fc700 1 -- 172.21.15.2:0/2307061064 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ad400f070 con 0x559dedbab000 2023-06-23T18:33:04.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.843+0000 7f8ac27fc700 1 -- 172.21.15.2:0/2307061064 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4017e60 con 0x559dedbab000 2023-06-23T18:33:04.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307061064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 msgr2=0x559dedbade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbade20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009650 tx=0x7f8ad40077e0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307061064 shutdown_connections 2023-06-23T18:33:04.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbade20 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedba9c80 0x559dedbc5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2307061064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedbaa5d0 0x559dedbba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307061064 >> 172.21.15.2:0/2307061064 conn(0x559dedb9f3f0 msgr2=0x559dedba08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307061064 shutdown_connections 2023-06-23T18:33:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2307061064 wait complete. 2023-06-23T18:33:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.844+0000 7f8aedcc0b80 1 -- start start 2023-06-23T18:33:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8aedcc0b80 1 Processor -- start 2023-06-23T18:33:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 0x559dedbb6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbaa5d0 0x559dedbb6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8aedcc0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbb9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedbba0f0 con 0x559dedbab000 2023-06-23T18:33:04.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedbba260 con 0x559dedbaa5d0 2023-06-23T18:33:04.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8aedcc0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dedbba420 con 0x559dedba9c80 2023-06-23T18:33:04.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8aedcc0b80 10 client.0 _ll_register_callbacks cb 0x559dedaa0260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:04.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbb9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbb9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56386/0 (socket says 172.21.15.2:56386) 2023-06-23T18:33:04.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adbfff700 1 -- 172.21.15.2:0/2691347503 learned_addr learned my addr 172.21.15.2:0/2691347503 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:04.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbaa5d0 0x559dedbb6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 0x559dedbb6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adbfff700 1 -- 172.21.15.2:0/2691347503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 msgr2=0x559dedbb6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adbfff700 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 0x559dedbb6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adbfff700 1 -- 172.21.15.2:0/2691347503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbaa5d0 msgr2=0x559dedbb6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:04.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adbfff700 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbaa5d0 0x559dedbb6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adbfff700 1 -- 172.21.15.2:0/2691347503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ded928de0 con 0x559dedbab000 2023-06-23T18:33:04.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.845+0000 7f8adaffd700 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbaa5d0 0x559dedbb6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:04.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8adbfff700 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbb9bb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4004f20 tx=0x7f8ad4018de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40648a680fe482b6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:04.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8abaffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:04.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8abaffd700 1 -- 172.21.15.2:0/2691347503 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4024070 con 0x559dedbab000 2023-06-23T18:33:04.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8abaffd700 1 -- 172.21.15.2:0/2691347503 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8ad4017650 con 0x559dedbab000 2023-06-23T18:33:04.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2691347503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad009b3a0 con 0x559dedbab000 2023-06-23T18:33:04.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8abaffd700 1 -- 172.21.15.2:0/2691347503 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad4017810 con 0x559dedbab000 2023-06-23T18:33:04.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8aedcc0b80 10 client.16362 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:04.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2691347503 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x559ded97c630 con 0x559dedbab000 2023-06-23T18:33:04.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8abaffd700 1 -- 172.21.15.2:0/2691347503 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8ad009b3a0 con 0x559dedbab000 2023-06-23T18:33:04.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8abaffd700 1 -- 172.21.15.2:0/2691347503 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8ad4014050 con 0x559dedbab000 2023-06-23T18:33:04.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8aedcc0b80 10 client.16362 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:04.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8aedcc0b80 10 client.16362 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:33:04.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8aedcc0b80 20 client.16362 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:04.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.846+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x559dedc140b0 0x559dedc164c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:04.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.847+0000 7f8aedcc0b80 4 client.16362 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:33:04.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.847+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2691347503 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559dedab6050 con 0x559dedc140b0 2023-06-23T18:33:04.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.847+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x559dedc140b0 0x559dedc164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:04.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.847+0000 7f8abaffd700 10 client.16362 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:33:04.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.847+0000 7f8adb7fe700 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x559dedc140b0 0x559dedc164c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93456209933b95da server_cookie=35651a152b9344c9 in_seq=0 out_seq=0 2023-06-23T18:33:04.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.848+0000 7f8abaffd700 1 -- 172.21.15.2:0/2691347503 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559dedab6050 con 0x559dedc140b0 2023-06-23T18:33:04.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.848+0000 7f8abaffd700 10 client.16362 handle_command_reply: tid=0 2023-06-23T18:33:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 client.16362 shutdown 2023-06-23T18:33:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2691347503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 msgr2=0x559dedbb9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbb9bb0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4004f20 tx=0x7f8ad4018de0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2691347503 shutdown_connections 2023-06-23T18:33:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x559dedc140b0 0x559dedc164c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559dedbab000 0x559dedbb9bb0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8abaffd700 0 client.16362 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:33:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559dedba9c80 0x559dedbb6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 --2- 172.21.15.2:0/2691347503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559dedbaa5d0 0x559dedbb6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2691347503 >> 172.21.15.2:0/2691347503 conn(0x559dedb9f3f0 msgr2=0x559dedbc5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2691347503 shutdown_connections 2023-06-23T18:33:04.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.850+0000 7f8aedcc0b80 1 -- 172.21.15.2:0/2691347503 wait complete. 2023-06-23T18:33:04.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8aedcc0b80 20 client.16362 trim_cache size 0 max 16384 2023-06-23T18:33:04.862 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:33:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8ac40a11d0 msgr2=0x7f8ac40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8ac40a11d0 0x7f8ac40a3680 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8acc007fd0 tx=0x7f8acc007480 comp rx=0 tx=0).stop 2023-06-23T18:33:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/2795259248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0ffc70 msgr2=0x7f8adc1079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0ffc70 0x7f8adc1079b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0006010 tx=0x7f8ad0004730 comp rx=0 tx=0).stop 2023-06-23T18:33:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/2795259248 shutdown_connections 2023-06-23T18:33:04.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8adc100600 0x7f8adc107ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8adc0fea80 0x7f8adc109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8ac40a11d0 0x7f8ac40a3680 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 --2- 172.21.15.2:0/2795259248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8adc0ffc70 0x7f8adc1079b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/2795259248 >> 172.21.15.2:0/2795259248 conn(0x7f8adc0fa0e0 msgr2=0x7f8adc0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/2795259248 shutdown_connections 2023-06-23T18:33:04.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:04.851+0000 7f8ae1b3b700 1 -- 172.21.15.2:0/2795259248 wait complete. 2023-06-23T18:33:04.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:33:05.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 -- 172.21.15.2:0/3138762867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 msgr2=0x7fefa81000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 --2- 172.21.15.2:0/3138762867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 0x7fefa81000c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fef98009990 tx=0x7fef98009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 -- 172.21.15.2:0/3138762867 shutdown_connections 2023-06-23T18:33:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 --2- 172.21.15.2:0/3138762867 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa8100600 0x7fefa8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 --2- 172.21.15.2:0/3138762867 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa80fea80 0x7fefa80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 --2- 172.21.15.2:0/3138762867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 0x7fefa81000c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 -- 172.21.15.2:0/3138762867 >> 172.21.15.2:0/3138762867 conn(0x7fefa80fa0e0 msgr2=0x7fefa80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 -- 172.21.15.2:0/3138762867 shutdown_connections 2023-06-23T18:33:05.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 -- 172.21.15.2:0/3138762867 wait complete. 2023-06-23T18:33:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 Processor -- start 2023-06-23T18:33:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.035+0000 7fefafc31700 1 -- start start 2023-06-23T18:33:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefafc31700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa80fea80 0x7fefa81957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefafc31700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 0x7fefa8195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefafc31700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa8100600 0x7fefa819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefafc31700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa819d360 con 0x7fefa80ffc70 2023-06-23T18:33:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefafc31700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa819d4a0 con 0x7fefa8100600 2023-06-23T18:33:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefafc31700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa819d630 con 0x7fefa80fea80 2023-06-23T18:33:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefad1cc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 0x7fefa8195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefae1ce700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa8100600 0x7fefa819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefad1cc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 0x7fefa8195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56400/0 (socket says 172.21.15.2:56400) 2023-06-23T18:33:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefae1ce700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa8100600 0x7fefa819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:46990/0 (socket says 172.21.15.2:46990) 2023-06-23T18:33:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefad1cc700 1 -- 172.21.15.2:0/1937310676 learned_addr learned my addr 172.21.15.2:0/1937310676 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefad9cd700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa80fea80 0x7fefa81957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefae1ce700 1 -- 172.21.15.2:0/1937310676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa80fea80 msgr2=0x7fefa81957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefae1ce700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa80fea80 0x7fefa81957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefae1ce700 1 -- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 msgr2=0x7fefa8195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefae1ce700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 0x7fefa8195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefae1ce700 1 -- 172.21.15.2:0/1937310676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef980095f0 con 0x7fefa8100600 2023-06-23T18:33:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.036+0000 7fefae1ce700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa8100600 0x7fefa819cd90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefa000efc0 tx=0x7fefa000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.037+0000 7fef9effd700 1 -- 172.21.15.2:0/1937310676 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0018070 con 0x7fefa8100600 2023-06-23T18:33:05.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.037+0000 7fefafc31700 1 -- 172.21.15.2:0/1937310676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa819d8b0 con 0x7fefa8100600 2023-06-23T18:33:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.037+0000 7fef9effd700 1 -- 172.21.15.2:0/1937310676 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefa000f040 con 0x7fefa8100600 2023-06-23T18:33:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.037+0000 7fef9effd700 1 -- 172.21.15.2:0/1937310676 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa000cba0 con 0x7fefa8100600 2023-06-23T18:33:05.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.037+0000 7fefafc31700 1 -- 172.21.15.2:0/1937310676 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa819dd70 con 0x7fefa8100600 2023-06-23T18:33:05.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.040+0000 7fef9effd700 1 -- 172.21.15.2:0/1937310676 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fefa0003a00 con 0x7fefa8100600 2023-06-23T18:33:05.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.040+0000 7fef9effd700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fef940a1150 0x7fef940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.040+0000 7fefad9cd700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fef940a1150 0x7fef940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.040+0000 7fef9effd700 1 -- 172.21.15.2:0/1937310676 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fefa00cc3f0 con 0x7fefa8100600 2023-06-23T18:33:05.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.040+0000 7fefbbdb6b80 1 Processor -- start 2023-06-23T18:33:05.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefbbdb6b80 1 -- start start 2023-06-23T18:33:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d090 0x55675dd4d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd4de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4e410 0x55675dd4e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dc4dcb0 con 0x55675dd4e410 2023-06-23T18:33:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dca0a40 con 0x55675dd4d9e0 2023-06-23T18:33:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dc99ed0 con 0x55675dd4d090 2023-06-23T18:33:05.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefad9cd700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fef940a1150 0x7fef940a3600 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fefa80ffad0 tx=0x7fefa4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefae1ce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4e410 0x55675dd4e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefae1ce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4e410 0x55675dd4e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56406/0 (socket says 172.21.15.2:56406) 2023-06-23T18:33:05.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefae1ce700 1 -- 172.21.15.2:0/2632258439 learned_addr learned my addr 172.21.15.2:0/2632258439 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefad1cc700 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd4de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefad9cd700 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d090 0x55675dd4d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefad9cd700 1 -- 172.21.15.2:0/2632258439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 msgr2=0x55675dd4de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.041+0000 7fefad9cd700 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd4de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.042+0000 7fefad9cd700 1 -- 172.21.15.2:0/2632258439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4e410 msgr2=0x55675dd4e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.042+0000 7fefad9cd700 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4e410 0x55675dd4e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.042+0000 7fefad9cd700 1 -- 172.21.15.2:0/2632258439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55675db1b630 con 0x55675dd4d090 2023-06-23T18:33:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.042+0000 7fefae1ce700 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4e410 0x55675dd4e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.042+0000 7fefad9cd700 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d090 0x55675dd4d4a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefa4011c80 tx=0x7fefa4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65a077dd7dbaaf04 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.042+0000 7fef93fff700 1 -- 172.21.15.2:0/2632258439 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004220 con 0x55675dd4d090 2023-06-23T18:33:05.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.042+0000 7fef93fff700 1 -- 172.21.15.2:0/2632258439 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefa4021070 con 0x55675dd4d090 2023-06-23T18:33:05.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.042+0000 7fef93fff700 1 -- 172.21.15.2:0/2632258439 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4003a10 con 0x55675dd4d090 2023-06-23T18:33:05.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/2632258439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d090 msgr2=0x55675dd4d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d090 0x55675dd4d4a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fefa4011c80 tx=0x7fefa4014000 comp rx=0 tx=0).stop 2023-06-23T18:33:05.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/2632258439 shutdown_connections 2023-06-23T18:33:05.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4e410 0x55675dd4e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d090 0x55675dd4d4a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/2632258439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd4de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/2632258439 >> 172.21.15.2:0/2632258439 conn(0x55675dd3e3f0 msgr2=0x55675dd43520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/2632258439 shutdown_connections 2023-06-23T18:33:05.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.043+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/2632258439 wait complete. 2023-06-23T18:33:05.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 1 -- start start 2023-06-23T18:33:05.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 1 Processor -- start 2023-06-23T18:33:05.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d090 0x55675dd56980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d9e0 0x55675dd56ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 0x55675dd5df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dc39ac0 con 0x55675dd4d9e0 2023-06-23T18:33:05.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dc397c0 con 0x55675dd4d090 2023-06-23T18:33:05.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675db1d920 con 0x55675dd4e410 2023-06-23T18:33:05.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefbbdb6b80 10 client.0 _ll_register_callbacks cb 0x55675dc3f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:05.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefad9cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d090 0x55675dd56980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefad9cd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d090 0x55675dd56980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47002/0 (socket says 172.21.15.2:47002) 2023-06-23T18:33:05.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefad9cd700 1 -- 172.21.15.2:0/1266009628 learned_addr learned my addr 172.21.15.2:0/1266009628 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefae1ce700 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 0x55675dd5df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.044+0000 7fefad1cc700 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d9e0 0x55675dd56ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefad9cd700 1 -- 172.21.15.2:0/1266009628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 msgr2=0x55675dd5df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefad9cd700 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 0x55675dd5df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefad9cd700 1 -- 172.21.15.2:0/1266009628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d9e0 msgr2=0x55675dd56ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefad9cd700 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d9e0 0x55675dd56ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefad9cd700 1 -- 172.21.15.2:0/1266009628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa819dd70 con 0x55675dd4d090 2023-06-23T18:33:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefad1cc700 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d9e0 0x55675dd56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:33:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefad9cd700 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d090 0x55675dd56980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefa40045b0 tx=0x7fefa40039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9cb79cad426e173 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fef917fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:33:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fef917fa700 1 -- 172.21.15.2:0/1266009628 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa400ed90 con 0x55675dd4d090 2023-06-23T18:33:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1266009628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55675dd3bbb0 con 0x55675dd4d090 2023-06-23T18:33:05.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fef917fa700 1 -- 172.21.15.2:0/1266009628 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fefa4005050 con 0x55675dd4d090 2023-06-23T18:33:05.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fef917fa700 1 -- 172.21.15.2:0/1266009628 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa4005210 con 0x55675dd4d090 2023-06-23T18:33:05.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefbbdb6b80 10 client.25907 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:05.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1266009628 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fefa4017070 con 0x55675dd4d090 2023-06-23T18:33:05.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.045+0000 7fef917fa700 1 -- 172.21.15.2:0/1266009628 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55675dd3bbb0 con 0x55675dd4d090 2023-06-23T18:33:05.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fef917fa700 1 -- 172.21.15.2:0/1266009628 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fefa4004750 con 0x55675dd4d090 2023-06-23T18:33:05.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fefbbdb6b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:05.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fefbbdb6b80 10 client.25907 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:33:05.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fefbbdb6b80 20 client.25907 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:05.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55675dd625b0 0x55675dd649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fefbbdb6b80 4 client.25907 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:05.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1266009628 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55675dc92eb0 con 0x55675dd625b0 2023-06-23T18:33:05.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fefad1cc700 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55675dd625b0 0x55675dd649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fefad1cc700 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55675dd625b0 0x55675dd649c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aad40d3be41443c3 server_cookie=1d7939290d4b2262 in_seq=0 out_seq=0 2023-06-23T18:33:05.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.046+0000 7fef917fa700 10 client.25907 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:33:05.062 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:04 smithi002 ceph-mon[95947]: pgmap v627: 97 pgs: 97 active+clean; 8.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 31 MiB/s wr, 120 op/s 2023-06-23T18:33:05.062 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:04 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2640529216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:33:05.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fef917fa700 1 -- 172.21.15.2:0/1266009628 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55675dc92eb0 con 0x55675dd625b0 2023-06-23T18:33:05.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fef917fa700 10 client.25907 handle_command_reply: tid=0 2023-06-23T18:33:05.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 client.25907 shutdown 2023-06-23T18:33:05.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1266009628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d090 msgr2=0x55675dd56980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d090 0x55675dd56980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fefa40045b0 tx=0x7fefa40039b0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1266009628 shutdown_connections 2023-06-23T18:33:05.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55675dd625b0 0x55675dd649c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 0x55675dd5df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d090 0x55675dd56980 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fef917fa700 0 client.25907 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:33:05.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1266009628 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d9e0 0x55675dd56ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1266009628 >> 172.21.15.2:0/1266009628 conn(0x55675dd3e3f0 msgr2=0x55675dd42c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1266009628 shutdown_connections 2023-06-23T18:33:05.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.047+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1266009628 wait complete. 2023-06-23T18:33:05.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.048+0000 7fefbbdb6b80 20 client.25907 trim_cache size 0 max 16384 2023-06-23T18:33:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefbbdb6b80 1 Processor -- start 2023-06-23T18:33:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefbbdb6b80 1 -- start start 2023-06-23T18:33:05.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 0x55675dd5e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd5dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 0x55675dd56730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dd56550 con 0x55675dd4d090 2023-06-23T18:33:05.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dc92eb0 con 0x55675dd4d9e0 2023-06-23T18:33:05.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dd40470 con 0x55675dd4e410 2023-06-23T18:33:05.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefad1cc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd5dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefad1cc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd5dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47012/0 (socket says 172.21.15.2:47012) 2023-06-23T18:33:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefad1cc700 1 -- 172.21.15.2:0/1288786966 learned_addr learned my addr 172.21.15.2:0/1288786966 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefae1ce700 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 0x55675dd56730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.067+0000 7fefad9cd700 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 0x55675dd5e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fefad1cc700 1 -- 172.21.15.2:0/1288786966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 msgr2=0x55675dd56730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fefad1cc700 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 0x55675dd56730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fefad1cc700 1 -- 172.21.15.2:0/1288786966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 msgr2=0x55675dd5e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fefad1cc700 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 0x55675dd5e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fefad1cc700 1 -- 172.21.15.2:0/1288786966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55675dd4eab0 con 0x55675dd4d9e0 2023-06-23T18:33:05.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fefad1cc700 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd5dec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef980098e0 tx=0x7fef98009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1148b1d53205024 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fef9cdac700 1 -- 172.21.15.2:0/1288786966 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9801d070 con 0x55675dd4d9e0 2023-06-23T18:33:05.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fef9cdac700 1 -- 172.21.15.2:0/1288786966 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef9801e040 con 0x55675dd4d9e0 2023-06-23T18:33:05.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.068+0000 7fef9cdac700 1 -- 172.21.15.2:0/1288786966 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9800ba20 con 0x55675dd4d9e0 2023-06-23T18:33:05.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1288786966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 msgr2=0x55675dd5dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd5dec0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fef980098e0 tx=0x7fef98009310 comp rx=0 tx=0).stop 2023-06-23T18:33:05.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1288786966 shutdown_connections 2023-06-23T18:33:05.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4e410 0x55675dd56730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 0x55675dd5e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/1288786966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4d9e0 0x55675dd5dec0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1288786966 >> 172.21.15.2:0/1288786966 conn(0x55675dd3e3f0 msgr2=0x55675dd55670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1288786966 shutdown_connections 2023-06-23T18:33:05.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.069+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/1288786966 wait complete. 2023-06-23T18:33:05.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 1 -- start start 2023-06-23T18:33:05.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 1 Processor -- start 2023-06-23T18:33:05.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 0x55675dd4c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d9e0 0x55675dd4aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4e410 0x55675dd5de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dd4cca0 con 0x55675dd4d090 2023-06-23T18:33:05.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dd4afd0 con 0x55675dd4e410 2023-06-23T18:33:05.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55675dd4b110 con 0x55675dd4d9e0 2023-06-23T18:33:05.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefbbdb6b80 10 client.0 _ll_register_callbacks cb 0x55675dc3f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:05.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.070+0000 7fefad1cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d9e0 0x55675dd4aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefae1ce700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4e410 0x55675dd5de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefad1cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d9e0 0x55675dd4aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56764/0 (socket says 172.21.15.2:56764) 2023-06-23T18:33:05.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefad1cc700 1 -- 172.21.15.2:0/802151266 learned_addr learned my addr 172.21.15.2:0/802151266 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefad9cd700 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 0x55675dd4c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefae1ce700 1 -- 172.21.15.2:0/802151266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d9e0 msgr2=0x55675dd4aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefae1ce700 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d9e0 0x55675dd4aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefae1ce700 1 -- 172.21.15.2:0/802151266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 msgr2=0x55675dd4c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefae1ce700 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 0x55675dd4c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefae1ce700 1 -- 172.21.15.2:0/802151266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa80677a0 con 0x55675dd4e410 2023-06-23T18:33:05.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefad1cc700 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d9e0 0x55675dd4aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:05.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefae1ce700 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4e410 0x55675dd5de90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fefa00a1720 tx=0x7fefa00954b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c21f20420a9b851 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fef917fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:33:05.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fef917fa700 1 -- 172.21.15.2:0/802151266 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0099890 con 0x55675dd4e410 2023-06-23T18:33:05.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fef917fa700 1 -- 172.21.15.2:0/802151266 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fefa009d8c0 con 0x55675dd4e410 2023-06-23T18:33:05.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/802151266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefa4021070 con 0x55675dd4e410 2023-06-23T18:33:05.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fef917fa700 1 -- 172.21.15.2:0/802151266 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa00a1c50 con 0x55675dd4e410 2023-06-23T18:33:05.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.071+0000 7fefbbdb6b80 10 client.25913 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:05.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.072+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/802151266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55675dac7de0 con 0x55675dd4e410 2023-06-23T18:33:05.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.072+0000 7fef917fa700 1 -- 172.21.15.2:0/802151266 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefa4021070 con 0x55675dd4e410 2023-06-23T18:33:05.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fef917fa700 1 -- 172.21.15.2:0/802151266 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fefa00a17c0 con 0x55675dd4e410 2023-06-23T18:33:05.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefbbdb6b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:05.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefbbdb6b80 10 client.25913 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:33:05.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefbbdb6b80 20 client.25913 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:05.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55675ddc44c0 0x55675ddc68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefbbdb6b80 4 client.25913 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:33:05.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/802151266 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55675dc55050 con 0x55675ddc44c0 2023-06-23T18:33:05.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefad9cd700 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55675ddc44c0 0x55675ddc68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fef917fa700 10 client.25913 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:33:05.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefad9cd700 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55675ddc44c0 0x55675ddc68d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba6dc4d18f74e57a server_cookie=a174e290fbbc63b8 in_seq=0 out_seq=0 2023-06-23T18:33:05.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fef917fa700 1 -- 172.21.15.2:0/802151266 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55675dc55050 con 0x55675ddc44c0 2023-06-23T18:33:05.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fef917fa700 10 client.25913 handle_command_reply: tid=0 2023-06-23T18:33:05.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.073+0000 7fefbbdb6b80 1 client.25913 shutdown 2023-06-23T18:33:05.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/802151266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4e410 msgr2=0x55675dd5de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4e410 0x55675dd5de90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fefa00a1720 tx=0x7fefa00954b0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/802151266 shutdown_connections 2023-06-23T18:33:05.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55675ddc44c0 0x55675ddc68d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55675dd4e410 0x55675dd5de90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fef917fa700 0 client.25913 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:33:05.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55675dd4d090 0x55675dd4c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 --2- 172.21.15.2:0/802151266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55675dd4d9e0 0x55675dd4aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/802151266 >> 172.21.15.2:0/802151266 conn(0x55675dd3e3f0 msgr2=0x55675dd51370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/802151266 shutdown_connections 2023-06-23T18:33:05.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 1 -- 172.21.15.2:0/802151266 wait complete. 2023-06-23T18:33:05.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.074+0000 7fefbbdb6b80 20 client.25913 trim_cache size 0 max 16384 2023-06-23T18:33:05.089 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:33:05.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 -- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fef940a1150 msgr2=0x7fef940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fef940a1150 0x7fef940a3600 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fefa80ffad0 tx=0x7fefa4009260 comp rx=0 tx=0).stop 2023-06-23T18:33:05.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 -- 172.21.15.2:0/1937310676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa8100600 msgr2=0x7fefa819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa8100600 0x7fefa819cd90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fefa000efc0 tx=0x7fefa000c3a0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 -- 172.21.15.2:0/1937310676 shutdown_connections 2023-06-23T18:33:05.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fef940a1150 0x7fef940a3600 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fefa8100600 0x7fefa819cd90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fefa80fea80 0x7fefa81957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 --2- 172.21.15.2:0/1937310676 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefa80ffc70 0x7fefa8195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 -- 172.21.15.2:0/1937310676 >> 172.21.15.2:0/1937310676 conn(0x7fefa80fa0e0 msgr2=0x7fefa80fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 -- 172.21.15.2:0/1937310676 shutdown_connections 2023-06-23T18:33:05.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.075+0000 7fefafc31700 1 -- 172.21.15.2:0/1937310676 wait complete. 2023-06-23T18:33:05.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:33:05.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:04 smithi082 ceph-mon[97889]: pgmap v627: 97 pgs: 97 active+clean; 8.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 31 MiB/s wr, 120 op/s 2023-06-23T18:33:05.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:04 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2640529216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:33:05.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:04 smithi017 ceph-mon[97472]: pgmap v627: 97 pgs: 97 active+clean; 8.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 31 MiB/s wr, 120 op/s 2023-06-23T18:33:05.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:04 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2640529216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:33:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.261+0000 7f35147ce700 1 -- 172.21.15.2:0/3694807880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0fea80 msgr2=0x7f350c0fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.261+0000 7f35147ce700 1 --2- 172.21.15.2:0/3694807880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0fea80 0x7f350c0fee90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f350400b0a0 tx=0x7f350400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.261+0000 7f35147ce700 1 -- 172.21.15.2:0/3694807880 shutdown_connections 2023-06-23T18:33:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.261+0000 7f35147ce700 1 --2- 172.21.15.2:0/3694807880 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c100600 0x7f350c0677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.261+0000 7f35147ce700 1 --2- 172.21.15.2:0/3694807880 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0fea80 0x7f350c0fee90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.261+0000 7f35147ce700 1 --2- 172.21.15.2:0/3694807880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f350c0ffc70 0x7f350c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.261+0000 7f35147ce700 1 -- 172.21.15.2:0/3694807880 >> 172.21.15.2:0/3694807880 conn(0x7f350c0fa0e0 msgr2=0x7f350c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.262+0000 7f35147ce700 1 -- 172.21.15.2:0/3694807880 shutdown_connections 2023-06-23T18:33:05.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.262+0000 7f35147ce700 1 -- 172.21.15.2:0/3694807880 wait complete. 2023-06-23T18:33:05.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.262+0000 7f35147ce700 1 Processor -- start 2023-06-23T18:33:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.262+0000 7f35147ce700 1 -- start start 2023-06-23T18:33:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f35147ce700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f350c0fea80 0x7f350c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f35147ce700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0ffc70 0x7f350c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f35147ce700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c100600 0x7f350c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f35147ce700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19f6a0 con 0x7f350c0ffc70 2023-06-23T18:33:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f35147ce700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19f7e0 con 0x7f350c100600 2023-06-23T18:33:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f35147ce700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c19f970 con 0x7f350c0fea80 2023-06-23T18:33:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3511d69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0ffc70 0x7f350c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3512d6b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c100600 0x7f350c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3511d69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0ffc70 0x7f350c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56438/0 (socket says 172.21.15.2:56438) 2023-06-23T18:33:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3511d69700 1 -- 172.21.15.2:0/2897667767 learned_addr learned my addr 172.21.15.2:0/2897667767 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f351256a700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f350c0fea80 0x7f350c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3511d69700 1 -- 172.21.15.2:0/2897667767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f350c0fea80 msgr2=0x7f350c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3511d69700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f350c0fea80 0x7f350c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3511d69700 1 -- 172.21.15.2:0/2897667767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c100600 msgr2=0x7f350c19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3511d69700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c100600 0x7f350c19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3511d69700 1 -- 172.21.15.2:0/2897667767 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3504009d00 con 0x7f350c0ffc70 2023-06-23T18:33:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f351256a700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f350c0fea80 0x7f350c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.263+0000 7f3512d6b700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c100600 0x7f350c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.264+0000 7f3511d69700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0ffc70 0x7f350c198060 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f350000d950 tx=0x7f350000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.264+0000 7f34ff7fe700 1 -- 172.21.15.2:0/2897667767 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500009940 con 0x7f350c0ffc70 2023-06-23T18:33:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.264+0000 7f34ff7fe700 1 -- 172.21.15.2:0/2897667767 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3500010460 con 0x7f350c0ffc70 2023-06-23T18:33:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.264+0000 7f35147ce700 1 -- 172.21.15.2:0/2897667767 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c19fc50 con 0x7f350c0ffc70 2023-06-23T18:33:05.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.264+0000 7f34ff7fe700 1 -- 172.21.15.2:0/2897667767 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350000f620 con 0x7f350c0ffc70 2023-06-23T18:33:05.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.264+0000 7f35147ce700 1 -- 172.21.15.2:0/2897667767 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c1a01b0 con 0x7f350c0ffc70 2023-06-23T18:33:05.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.267+0000 7f34ff7fe700 1 -- 172.21.15.2:0/2897667767 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f35000105d0 con 0x7f350c0ffc70 2023-06-23T18:33:05.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.268+0000 7f34ff7fe700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34f80a11d0 0x7f34f80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.268+0000 7f351256a700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34f80a11d0 0x7f34f80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.268+0000 7f3520953b80 1 Processor -- start 2023-06-23T18:33:05.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.268+0000 7f34ff7fe700 1 -- 172.21.15.2:0/2897667767 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35000cc6c0 con 0x7f350c0ffc70 2023-06-23T18:33:05.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3520953b80 1 -- start start 2023-06-23T18:33:05.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3520953b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b114c80 0x56460b115090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3520953b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b115a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3520953b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b116470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3520953b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b019cb0 con 0x56460b114c80 2023-06-23T18:33:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3520953b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b06ca40 con 0x56460b116000 2023-06-23T18:33:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3520953b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b065ed0 con 0x56460b1155d0 2023-06-23T18:33:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3511d69700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3512d6b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b116470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3511d69700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b115a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56784/0 (socket says 172.21.15.2:56784) 2023-06-23T18:33:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3511d69700 1 -- 172.21.15.2:0/3537473250 learned_addr learned my addr 172.21.15.2:0/3537473250 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3512d6b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b116470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47040/0 (socket says 172.21.15.2:47040) 2023-06-23T18:33:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3511d69700 1 -- 172.21.15.2:0/3537473250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 msgr2=0x56460b116470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3511d69700 1 --2- 172.21.15.2:0/3537473250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b116470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3511d69700 1 -- 172.21.15.2:0/3537473250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b114c80 msgr2=0x56460b115090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3511d69700 1 --2- 172.21.15.2:0/3537473250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b114c80 0x56460b115090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.269+0000 7f3511d69700 1 -- 172.21.15.2:0/3537473250 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56460aee7630 con 0x56460b1155d0 2023-06-23T18:33:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.270+0000 7f3511d69700 1 --2- 172.21.15.2:0/3537473250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b115a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3500095fd0 tx=0x7f3500099430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f53a4b78b77c89a8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.270+0000 7f351256a700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34f80a11d0 0x7f34f80a3680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f350400b070 tx=0x7f3504009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.270+0000 7f34fcff9700 1 -- 172.21.15.2:0/3537473250 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500099ad0 con 0x56460b1155d0 2023-06-23T18:33:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.270+0000 7f34fcff9700 1 -- 172.21.15.2:0/3537473250 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f350009db00 con 0x56460b1155d0 2023-06-23T18:33:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.270+0000 7f34fcff9700 1 -- 172.21.15.2:0/3537473250 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000a1a90 con 0x56460b1155d0 2023-06-23T18:33:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 -- 172.21.15.2:0/3537473250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 msgr2=0x56460b115a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 --2- 172.21.15.2:0/3537473250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b115a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3500095fd0 tx=0x7f3500099430 comp rx=0 tx=0).stop 2023-06-23T18:33:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 -- 172.21.15.2:0/3537473250 shutdown_connections 2023-06-23T18:33:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 --2- 172.21.15.2:0/3537473250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b116470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 --2- 172.21.15.2:0/3537473250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b114c80 0x56460b115090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 --2- 172.21.15.2:0/3537473250 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b115a70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 -- 172.21.15.2:0/3537473250 >> 172.21.15.2:0/3537473250 conn(0x56460b10a3f0 msgr2=0x56460b10f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 -- 172.21.15.2:0/3537473250 shutdown_connections 2023-06-23T18:33:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 -- 172.21.15.2:0/3537473250 wait complete. 2023-06-23T18:33:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.271+0000 7f3520953b80 1 -- start start 2023-06-23T18:33:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3520953b80 1 Processor -- start 2023-06-23T18:33:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3520953b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 0x56460b11e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3520953b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b1155d0 0x56460b11e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3520953b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b125910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3520953b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b005ac0 con 0x56460b1155d0 2023-06-23T18:33:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3520953b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b0057c0 con 0x56460b116000 2023-06-23T18:33:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3520953b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460aee9920 con 0x56460b114c80 2023-06-23T18:33:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3520953b80 10 client.0 _ll_register_callbacks cb 0x56460b00b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3511d69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b1155d0 0x56460b11e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3511d69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b1155d0 0x56460b11e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56452/0 (socket says 172.21.15.2:56452) 2023-06-23T18:33:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3511d69700 1 -- 172.21.15.2:0/2671656097 learned_addr learned my addr 172.21.15.2:0/2671656097 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3512d6b700 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b125910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f351256a700 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 0x56460b11e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3512d6b700 1 -- 172.21.15.2:0/2671656097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 msgr2=0x56460b11e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3512d6b700 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 0x56460b11e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.272+0000 7f3512d6b700 1 -- 172.21.15.2:0/2671656097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b1155d0 msgr2=0x56460b11e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f3512d6b700 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b1155d0 0x56460b11e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f3512d6b700 1 -- 172.21.15.2:0/2671656097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c1a01b0 con 0x56460b116000 2023-06-23T18:33:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f3511d69700 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b1155d0 0x56460b11e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f3512d6b700 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b125910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3508003930 tx=0x7f3508003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ec8fb7bfec9e98b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f34f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:33:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2671656097 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350801e070 con 0x56460b116000 2023-06-23T18:33:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f3520953b80 1 -- 172.21.15.2:0/2671656097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56460b107bb0 con 0x56460b116000 2023-06-23T18:33:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2671656097 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3508004830 con 0x56460b116000 2023-06-23T18:33:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2671656097 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35080049f0 con 0x56460b116000 2023-06-23T18:33:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f3520953b80 10 client.25925 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.273+0000 7f3520953b80 1 -- 172.21.15.2:0/2671656097 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56460ae93de0 con 0x56460b116000 2023-06-23T18:33:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2671656097 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56460b107bb0 con 0x56460b116000 2023-06-23T18:33:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2671656097 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f350800f690 con 0x56460b116000 2023-06-23T18:33:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f3520953b80 10 client.25925 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f3520953b80 10 client.25925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:05.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f3520953b80 20 client.25925 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:05.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f3520953b80 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56460b12a2c0 0x56460b12c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f3520953b80 4 client.25925 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:05.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f3520953b80 1 -- 172.21.15.2:0/2671656097 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56460b05eeb0 con 0x56460b12a2c0 2023-06-23T18:33:05.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f351256a700 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56460b12a2c0 0x56460b12c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.275+0000 7f34f5ffb700 10 client.25925 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:05.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.274+0000 7f351256a700 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56460b12a2c0 0x56460b12c6d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dad44e956598a8ab server_cookie=512c234f0cd4fbb2 in_seq=0 out_seq=0 2023-06-23T18:33:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.275+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2671656097 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56460b05eeb0 con 0x56460b12a2c0 2023-06-23T18:33:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.275+0000 7f34f5ffb700 10 client.25925 handle_command_reply: tid=0 2023-06-23T18:33:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 client.25925 shutdown 2023-06-23T18:33:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 -- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 msgr2=0x56460b125910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b125910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3508003930 tx=0x7f3508003c40 comp rx=0 tx=0).stop 2023-06-23T18:33:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 -- 172.21.15.2:0/2671656097 shutdown_connections 2023-06-23T18:33:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56460b12a2c0 0x56460b12c6d0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b116000 0x56460b125910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 0x56460b11e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 --2- 172.21.15.2:0/2671656097 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b1155d0 0x56460b11e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 -- 172.21.15.2:0/2671656097 >> 172.21.15.2:0/2671656097 conn(0x56460b10a3f0 msgr2=0x56460b10ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f34f5ffb700 0 client.25925 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 -- 172.21.15.2:0/2671656097 shutdown_connections 2023-06-23T18:33:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 1 -- 172.21.15.2:0/2671656097 wait complete. 2023-06-23T18:33:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.276+0000 7f3520953b80 20 client.25925 trim_cache size 0 max 16384 2023-06-23T18:33:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3520953b80 1 Processor -- start 2023-06-23T18:33:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3520953b80 1 -- start start 2023-06-23T18:33:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3520953b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 0x56460b11d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3520953b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b1155d0 0x56460b1258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3520953b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b12c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3520953b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b11d840 con 0x56460b116000 2023-06-23T18:33:05.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3520953b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b11df00 con 0x56460b1155d0 2023-06-23T18:33:05.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3520953b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b10ffa0 con 0x56460b114c80 2023-06-23T18:33:05.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3512d6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b12c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f351256a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 0x56460b11d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3511d69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b1155d0 0x56460b1258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3512d6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b12c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56462/0 (socket says 172.21.15.2:56462) 2023-06-23T18:33:05.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.293+0000 7f3512d6b700 1 -- 172.21.15.2:0/1688772620 learned_addr learned my addr 172.21.15.2:0/1688772620 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f3511d69700 1 -- 172.21.15.2:0/1688772620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 msgr2=0x56460b11d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f3511d69700 1 --2- 172.21.15.2:0/1688772620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 0x56460b11d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f3511d69700 1 -- 172.21.15.2:0/1688772620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 msgr2=0x56460b12c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f3511d69700 1 --2- 172.21.15.2:0/1688772620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b12c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f3511d69700 1 -- 172.21.15.2:0/1688772620 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56460ae7cc60 con 0x56460b1155d0 2023-06-23T18:33:05.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f3511d69700 1 --2- 172.21.15.2:0/1688772620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b1155d0 0x56460b1258d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3500009520 tx=0x7f3500005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeb4497341b699a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f34fd7fa700 1 -- 172.21.15.2:0/1688772620 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500097c10 con 0x56460b1155d0 2023-06-23T18:33:05.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f34fd7fa700 1 -- 172.21.15.2:0/1688772620 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3500097e70 con 0x56460b1155d0 2023-06-23T18:33:05.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.294+0000 7f34fd7fa700 1 -- 172.21.15.2:0/1688772620 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000a1cf0 con 0x56460b1155d0 2023-06-23T18:33:05.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 -- 172.21.15.2:0/1688772620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b1155d0 msgr2=0x56460b1258d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 --2- 172.21.15.2:0/1688772620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b1155d0 0x56460b1258d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3500009520 tx=0x7f3500005dc0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 -- 172.21.15.2:0/1688772620 shutdown_connections 2023-06-23T18:33:05.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 --2- 172.21.15.2:0/1688772620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b12c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 --2- 172.21.15.2:0/1688772620 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b114c80 0x56460b11d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 --2- 172.21.15.2:0/1688772620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b1155d0 0x56460b1258d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 -- 172.21.15.2:0/1688772620 >> 172.21.15.2:0/1688772620 conn(0x56460b10a3f0 msgr2=0x56460b131030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 -- 172.21.15.2:0/1688772620 shutdown_connections 2023-06-23T18:33:05.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.295+0000 7f3520953b80 1 -- 172.21.15.2:0/1688772620 wait complete. 2023-06-23T18:33:05.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 1 -- start start 2023-06-23T18:33:05.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 1 Processor -- start 2023-06-23T18:33:05.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b114c80 0x56460b11b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b1258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b17c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b17c8b0 con 0x56460b116000 2023-06-23T18:33:05.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b17ca20 con 0x56460b114c80 2023-06-23T18:33:05.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56460b17cbe0 con 0x56460b1155d0 2023-06-23T18:33:05.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3520953b80 10 client.0 _ll_register_callbacks cb 0x56460b00b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:05.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f351256a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b114c80 0x56460b11b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f351256a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b114c80 0x56460b11b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47056/0 (socket says 172.21.15.2:47056) 2023-06-23T18:33:05.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f351256a700 1 -- 172.21.15.2:0/2299473745 learned_addr learned my addr 172.21.15.2:0/2299473745 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:05.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3512d6b700 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b17c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.296+0000 7f3511d69700 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b1258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3511d69700 1 -- 172.21.15.2:0/2299473745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b114c80 msgr2=0x56460b11b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3511d69700 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b114c80 0x56460b11b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3511d69700 1 -- 172.21.15.2:0/2299473745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 msgr2=0x56460b17c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:05.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3511d69700 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b17c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3511d69700 1 -- 172.21.15.2:0/2299473745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56460aee7630 con 0x56460b1155d0 2023-06-23T18:33:05.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3512d6b700 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b17c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:05.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3511d69700 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b1258d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3500009520 tx=0x7f3500099430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f107dcdd324eb841 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:05.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f34f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:33:05.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2299473745 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3500099260 con 0x56460b1155d0 2023-06-23T18:33:05.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2299473745 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3500095ad0 con 0x56460b1155d0 2023-06-23T18:33:05.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3520953b80 1 -- 172.21.15.2:0/2299473745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f350009db00 con 0x56460b1155d0 2023-06-23T18:33:05.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2299473745 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3500098a30 con 0x56460b1155d0 2023-06-23T18:33:05.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3520953b80 10 client.26128 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:05.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f3520953b80 1 -- 172.21.15.2:0/2299473745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56460aeba250 con 0x56460b1155d0 2023-06-23T18:33:05.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.297+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2299473745 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f350009db00 con 0x56460b1155d0 2023-06-23T18:33:05.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2299473745 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f35000b9180 con 0x56460b1155d0 2023-06-23T18:33:05.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f3520953b80 10 client.26128 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:05.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f3520953b80 10 client.26128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:05.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f3520953b80 20 client.26128 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:05.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f3520953b80 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56460b17e040 0x56460b180420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f3520953b80 4 client.26128 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:33:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f3520953b80 1 -- 172.21.15.2:0/2299473745 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56460b021050 con 0x56460b17e040 2023-06-23T18:33:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f351256a700 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56460b17e040 0x56460b180420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f34f5ffb700 10 client.26128 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.298+0000 7f351256a700 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56460b17e040 0x56460b180420 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=984cd49557005b7b server_cookie=1260f0dceb470b63 in_seq=0 out_seq=0 2023-06-23T18:33:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f34f5ffb700 1 -- 172.21.15.2:0/2299473745 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56460b021050 con 0x56460b17e040 2023-06-23T18:33:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f34f5ffb700 10 client.26128 handle_command_reply: tid=0 2023-06-23T18:33:05.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 client.26128 shutdown 2023-06-23T18:33:05.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 -- 172.21.15.2:0/2299473745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 msgr2=0x56460b1258d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b1258d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3500009520 tx=0x7f3500099430 comp rx=0 tx=0).stop 2023-06-23T18:33:05.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 -- 172.21.15.2:0/2299473745 shutdown_connections 2023-06-23T18:33:05.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56460b17e040 0x56460b180420 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56460b116000 0x56460b17c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f34f5ffb700 0 client.26128 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:05.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56460b114c80 0x56460b11b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 --2- 172.21.15.2:0/2299473745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56460b1155d0 0x56460b1258d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 -- 172.21.15.2:0/2299473745 >> 172.21.15.2:0/2299473745 conn(0x56460b10a3f0 msgr2=0x56460b11d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 -- 172.21.15.2:0/2299473745 shutdown_connections 2023-06-23T18:33:05.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 1 -- 172.21.15.2:0/2299473745 wait complete. 2023-06-23T18:33:05.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.299+0000 7f3520953b80 20 client.26128 trim_cache size 0 max 16384 2023-06-23T18:33:05.312 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:33:05.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 -- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34f80a11d0 msgr2=0x7f34f80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34f80a11d0 0x7f34f80a3680 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f350400b070 tx=0x7f3504009550 comp rx=0 tx=0).stop 2023-06-23T18:33:05.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 -- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0ffc70 msgr2=0x7f350c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:05.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0ffc70 0x7f350c198060 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f350000d950 tx=0x7f350000dc60 comp rx=0 tx=0).stop 2023-06-23T18:33:05.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 -- 172.21.15.2:0/2897667767 shutdown_connections 2023-06-23T18:33:05.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f34f80a11d0 0x7f34f80a3680 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f350c100600 0x7f350c19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f350c0fea80 0x7f350c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 --2- 172.21.15.2:0/2897667767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350c0ffc70 0x7f350c198060 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:05.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.300+0000 7f35147ce700 1 -- 172.21.15.2:0/2897667767 >> 172.21.15.2:0/2897667767 conn(0x7f350c0fa0e0 msgr2=0x7f350c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:05.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.301+0000 7f35147ce700 1 -- 172.21.15.2:0/2897667767 shutdown_connections 2023-06-23T18:33:05.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:05.301+0000 7f35147ce700 1 -- 172.21.15.2:0/2897667767 wait complete. 2023-06-23T18:33:06.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:05 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:33:06.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:05 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:33:06.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:05 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:33:06.321 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:33:06.322 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:33:06.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.491+0000 7f02c9f0b700 1 -- 172.21.15.2:0/3109697983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 msgr2=0x7f02c4100460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.491+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/3109697983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 0x7f02c4100460 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009990 tx=0x7f02b4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.492+0000 7f02c9f0b700 1 -- 172.21.15.2:0/3109697983 shutdown_connections 2023-06-23T18:33:06.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.492+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/3109697983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02c40ff650 0x7f02c4106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.492+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/3109697983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 0x7f02c4100460 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.492+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/3109697983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c40feca0 0x7f02c40ff110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.492+0000 7f02c9f0b700 1 -- 172.21.15.2:0/3109697983 >> 172.21.15.2:0/3109697983 conn(0x7f02c40f9820 msgr2=0x7f02c40fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.492+0000 7f02c9f0b700 1 -- 172.21.15.2:0/3109697983 shutdown_connections 2023-06-23T18:33:06.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.492+0000 7f02c9f0b700 1 -- 172.21.15.2:0/3109697983 wait complete. 2023-06-23T18:33:06.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.492+0000 7f02c9f0b700 1 Processor -- start 2023-06-23T18:33:06.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c9f0b700 1 -- start start 2023-06-23T18:33:06.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c9f0b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c40feca0 0x7f02c41956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c9f0b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02c40ff650 0x7f02c4195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c9f0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 0x7f02c419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c9f0b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c419d250 con 0x7f02c4100050 2023-06-23T18:33:06.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c9f0b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c419d390 con 0x7f02c40feca0 2023-06-23T18:33:06.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c9f0b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c419d520 con 0x7f02c40ff650 2023-06-23T18:33:06.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 0x7f02c419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 0x7f02c419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56482/0 (socket says 172.21.15.2:56482) 2023-06-23T18:33:06.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 -- 172.21.15.2:0/2281274924 learned_addr learned my addr 172.21.15.2:0/2281274924 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c37fe700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c40feca0 0x7f02c41956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 -- 172.21.15.2:0/2281274924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02c40ff650 msgr2=0x7f02c4195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02c40ff650 0x7f02c4195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 -- 172.21.15.2:0/2281274924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c40feca0 msgr2=0x7f02c41956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c40feca0 0x7f02c41956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 -- 172.21.15.2:0/2281274924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b40095f0 con 0x7f02c4100050 2023-06-23T18:33:06.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c37fe700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c40feca0 0x7f02c41956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:06.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.493+0000 7f02c3fff700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 0x7f02c419cc80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f02bc007d70 tx=0x7f02bc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.494+0000 7f02c0ff9700 1 -- 172.21.15.2:0/2281274924 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00ad40 con 0x7f02c4100050 2023-06-23T18:33:06.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.494+0000 7f02c0ff9700 1 -- 172.21.15.2:0/2281274924 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02bc0128b0 con 0x7f02c4100050 2023-06-23T18:33:06.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.494+0000 7f02c0ff9700 1 -- 172.21.15.2:0/2281274924 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0116b0 con 0x7f02c4100050 2023-06-23T18:33:06.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.494+0000 7f02c9f0b700 1 -- 172.21.15.2:0/2281274924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c419d800 con 0x7f02c4100050 2023-06-23T18:33:06.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.494+0000 7f02c9f0b700 1 -- 172.21.15.2:0/2281274924 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c419dcc0 con 0x7f02c4100050 2023-06-23T18:33:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.497+0000 7f02c0ff9700 1 -- 172.21.15.2:0/2281274924 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f02bc012a20 con 0x7f02c4100050 2023-06-23T18:33:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.497+0000 7f02c0ff9700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f02ac0a11d0 0x7f02ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.497+0000 7f02c0ff9700 1 -- 172.21.15.2:0/2281274924 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f02bc0ccc00 con 0x7f02c4100050 2023-06-23T18:33:06.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.498+0000 7f02c37fe700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f02ac0a11d0 0x7f02ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.498+0000 7f02c37fe700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f02ac0a11d0 0x7f02ac0a3680 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f02b40095c0 tx=0x7f02b4009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.501+0000 7f02d6090b80 1 Processor -- start 2023-06-23T18:33:06.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.501+0000 7f02d6090b80 1 -- start start 2023-06-23T18:33:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.501+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036bd00 0x56502036c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.501+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x56502036caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.501+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036d080 0x56502036d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02d6090b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x565020270d30 con 0x56502036c650 2023-06-23T18:33:06.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02d6090b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650202c3ac0 con 0x56502036d080 2023-06-23T18:33:06.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02d6090b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650202bcf50 con 0x56502036bd00 2023-06-23T18:33:06.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x56502036caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x56502036caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56486/0 (socket says 172.21.15.2:56486) 2023-06-23T18:33:06.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 -- 172.21.15.2:0/274045062 learned_addr learned my addr 172.21.15.2:0/274045062 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c37fe700 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036bd00 0x56502036c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c3fff700 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036d080 0x56502036d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 -- 172.21.15.2:0/274045062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036bd00 msgr2=0x56502036c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036bd00 0x56502036c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 -- 172.21.15.2:0/274045062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036d080 msgr2=0x56502036d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036d080 0x56502036d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 -- 172.21.15.2:0/274045062 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56502013e6b0 con 0x56502036c650 2023-06-23T18:33:06.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c3fff700 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036d080 0x56502036d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:06.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.502+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x56502036caf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f02b800ab20 tx=0x7f02b800ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f523a17017ee104 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.503+0000 7f02a9ffb700 1 -- 172.21.15.2:0/274045062 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b800cc10 con 0x56502036c650 2023-06-23T18:33:06.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.503+0000 7f02a9ffb700 1 -- 172.21.15.2:0/274045062 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b800cd70 con 0x56502036c650 2023-06-23T18:33:06.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.503+0000 7f02a9ffb700 1 -- 172.21.15.2:0/274045062 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b8018660 con 0x56502036c650 2023-06-23T18:33:06.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 -- 172.21.15.2:0/274045062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 msgr2=0x56502036caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x56502036caf0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f02b800ab20 tx=0x7f02b800ae30 comp rx=0 tx=0).stop 2023-06-23T18:33:06.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 -- 172.21.15.2:0/274045062 shutdown_connections 2023-06-23T18:33:06.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036d080 0x56502036d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036bd00 0x56502036c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 --2- 172.21.15.2:0/274045062 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x56502036caf0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 -- 172.21.15.2:0/274045062 >> 172.21.15.2:0/274045062 conn(0x565020361470 msgr2=0x5650203665a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 -- 172.21.15.2:0/274045062 shutdown_connections 2023-06-23T18:33:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 -- 172.21.15.2:0/274045062 wait complete. 2023-06-23T18:33:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 -- start start 2023-06-23T18:33:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.504+0000 7f02d6090b80 1 Processor -- start 2023-06-23T18:33:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036c650 0x565020375af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036d080 0x56502037cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02d6090b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56502025cb40 con 0x56502036d080 2023-06-23T18:33:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02d6090b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56502025c840 con 0x56502036bd00 2023-06-23T18:33:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02d6090b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650201409a0 con 0x56502036c650 2023-06-23T18:33:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02d6090b80 10 client.0 _ll_register_callbacks cb 0x5650202622e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c2ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036c650 0x565020375af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036d080 0x56502037cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47084/0 (socket says 172.21.15.2:47084) 2023-06-23T18:33:06.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c37fe700 1 -- 172.21.15.2:0/1165336648 learned_addr learned my addr 172.21.15.2:0/1165336648 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c2ffd700 1 -- 172.21.15.2:0/1165336648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 msgr2=0x5650203755b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c2ffd700 1 -- 172.21.15.2:0/1165336648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036d080 msgr2=0x56502037cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036d080 0x56502037cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c2ffd700 1 -- 172.21.15.2:0/1165336648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b800a750 con 0x56502036c650 2023-06-23T18:33:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036c650 0x565020375af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x56502036ca60 tx=0x7f02b800f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d683eaeb035b9575 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.505+0000 7f02a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:33:06.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02a2ffd700 1 -- 172.21.15.2:0/1165336648 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b800cee0 con 0x56502036c650 2023-06-23T18:33:06.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02a2ffd700 1 -- 172.21.15.2:0/1165336648 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f02b800f8b0 con 0x56502036c650 2023-06-23T18:33:06.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 1 -- 172.21.15.2:0/1165336648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56502035ec30 con 0x56502036c650 2023-06-23T18:33:06.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02a2ffd700 1 -- 172.21.15.2:0/1165336648 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b8018810 con 0x56502036c650 2023-06-23T18:33:06.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 10 client.26134 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:06.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 1 -- 172.21.15.2:0/1165336648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5650200eae60 con 0x56502036c650 2023-06-23T18:33:06.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02a2ffd700 1 -- 172.21.15.2:0/1165336648 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56502035ec30 con 0x56502036c650 2023-06-23T18:33:06.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02a2ffd700 1 -- 172.21.15.2:0/1165336648 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02b8016020 con 0x56502036c650 2023-06-23T18:33:06.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 10 client.26134 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:06.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:06.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 20 client.26134 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:06.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x565020381050 0x565020383460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 4 client.26134 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:06.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02d6090b80 1 -- 172.21.15.2:0/1165336648 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650202b5f30 con 0x565020381050 2023-06-23T18:33:06.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.506+0000 7f02c37fe700 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x565020381050 0x565020383460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.507+0000 7f02a2ffd700 10 client.26134 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:06.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.507+0000 7f02c37fe700 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x565020381050 0x565020383460 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb3fa147b0228dd server_cookie=5b98ca6329bccfd1 in_seq=0 out_seq=0 2023-06-23T18:33:06.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.507+0000 7f02a2ffd700 1 -- 172.21.15.2:0/1165336648 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5650202b5f30 con 0x565020381050 2023-06-23T18:33:06.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02a2ffd700 10 client.26134 handle_command_reply: tid=0 2023-06-23T18:33:06.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 client.26134 shutdown 2023-06-23T18:33:06.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 -- 172.21.15.2:0/1165336648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036c650 msgr2=0x565020375af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036c650 0x565020375af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x56502036ca60 tx=0x7f02b800f680 comp rx=0 tx=0).stop 2023-06-23T18:33:06.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 -- 172.21.15.2:0/1165336648 shutdown_connections 2023-06-23T18:33:06.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x565020381050 0x565020383460 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036d080 0x56502037cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02a2ffd700 0 client.26134 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:06.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1165336648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036c650 0x565020375af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 -- 172.21.15.2:0/1165336648 >> 172.21.15.2:0/1165336648 conn(0x565020361470 msgr2=0x565020365cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 -- 172.21.15.2:0/1165336648 shutdown_connections 2023-06-23T18:33:06.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 1 -- 172.21.15.2:0/1165336648 wait complete. 2023-06-23T18:33:06.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.508+0000 7f02d6090b80 20 client.26134 trim_cache size 0 max 16384 2023-06-23T18:33:06.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02d6090b80 1 Processor -- start 2023-06-23T18:33:06.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02d6090b80 1 -- start start 2023-06-23T18:33:06.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036c650 0x565020375af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 0x56502037caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02d6090b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650203680d0 con 0x56502036bd00 2023-06-23T18:33:06.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02d6090b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x565020375120 con 0x56502036c650 2023-06-23T18:33:06.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02d6090b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x565020360ff0 con 0x56502036d080 2023-06-23T18:33:06.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02c37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02c37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56498/0 (socket says 172.21.15.2:56498) 2023-06-23T18:33:06.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02c37fe700 1 -- 172.21.15.2:0/1609920965 learned_addr learned my addr 172.21.15.2:0/1609920965 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036c650 0x565020375af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02c3fff700 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 0x56502037caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.527+0000 7f02c37fe700 1 -- 172.21.15.2:0/1609920965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 msgr2=0x56502037caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.528+0000 7f02c37fe700 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 0x56502037caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.528+0000 7f02c37fe700 1 -- 172.21.15.2:0/1609920965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036c650 msgr2=0x565020375af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.528+0000 7f02c37fe700 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036c650 0x565020375af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.528+0000 7f02c37fe700 1 -- 172.21.15.2:0/1609920965 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56502036d720 con 0x56502036bd00 2023-06-23T18:33:06.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.528+0000 7f02c37fe700 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036bd00 0x5650203755b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f02b400bf30 tx=0x7f02b4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac8623708325c46 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.528+0000 7f02aa7fc700 1 -- 172.21.15.2:0/1609920965 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b402c040 con 0x56502036bd00 2023-06-23T18:33:06.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.528+0000 7f02aa7fc700 1 -- 172.21.15.2:0/1609920965 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b4033480 con 0x56502036bd00 2023-06-23T18:33:06.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.528+0000 7f02aa7fc700 1 -- 172.21.15.2:0/1609920965 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b401e070 con 0x56502036bd00 2023-06-23T18:33:06.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 -- 172.21.15.2:0/1609920965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036bd00 msgr2=0x5650203755b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036bd00 0x5650203755b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f02b400bf30 tx=0x7f02b4021000 comp rx=0 tx=0).stop 2023-06-23T18:33:06.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 -- 172.21.15.2:0/1609920965 shutdown_connections 2023-06-23T18:33:06.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 0x56502037caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036bd00 0x5650203755b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 --2- 172.21.15.2:0/1609920965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036c650 0x565020375af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 -- 172.21.15.2:0/1609920965 >> 172.21.15.2:0/1609920965 conn(0x565020361470 msgr2=0x5650203742e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 -- 172.21.15.2:0/1609920965 shutdown_connections 2023-06-23T18:33:06.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 -- 172.21.15.2:0/1609920965 wait complete. 2023-06-23T18:33:06.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 -- start start 2023-06-23T18:33:06.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.529+0000 7f02d6090b80 1 Processor -- start 2023-06-23T18:33:06.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x56502037ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x565020379520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02d6090b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 0x565020379a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02d6090b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56502037a170 con 0x56502036c650 2023-06-23T18:33:06.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02d6090b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650203d4f00 con 0x56502036bd00 2023-06-23T18:33:06.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02d6090b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650203d50c0 con 0x56502036d080 2023-06-23T18:33:06.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02d6090b80 10 client.0 _ll_register_callbacks cb 0x5650202622e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:06.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x565020379520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x56502037ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x56502037ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47096/0 (socket says 172.21.15.2:47096) 2023-06-23T18:33:06.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c37fe700 1 -- 172.21.15.2:0/4259480163 learned_addr learned my addr 172.21.15.2:0/4259480163 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x565020379520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56506/0 (socket says 172.21.15.2:56506) 2023-06-23T18:33:06.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c3fff700 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 0x565020379a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c37fe700 1 -- 172.21.15.2:0/4259480163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 msgr2=0x565020379a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c37fe700 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 0x565020379a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c37fe700 1 -- 172.21.15.2:0/4259480163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 msgr2=0x565020379520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c37fe700 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x565020379520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c37fe700 1 -- 172.21.15.2:0/4259480163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c41077f0 con 0x56502036bd00 2023-06-23T18:33:06.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.530+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x565020379520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:06.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.531+0000 7f02c37fe700 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x56502037ca80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009350 tx=0x7f02b4004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba37896e3f8ee4f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.531+0000 7f02a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:33:06.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.531+0000 7f02a2ffd700 1 -- 172.21.15.2:0/4259480163 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b401e050 con 0x56502036bd00 2023-06-23T18:33:06.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.531+0000 7f02a2ffd700 1 -- 172.21.15.2:0/4259480163 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f02b4033cb0 con 0x56502036bd00 2023-06-23T18:33:06.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.531+0000 7f02d6090b80 1 -- 172.21.15.2:0/4259480163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02b800cd70 con 0x56502036bd00 2023-06-23T18:33:06.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.531+0000 7f02a2ffd700 1 -- 172.21.15.2:0/4259480163 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b402c040 con 0x56502036bd00 2023-06-23T18:33:06.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.531+0000 7f02d6090b80 10 client.25940 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:06.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.531+0000 7f02d6090b80 1 -- 172.21.15.2:0/4259480163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x5650200d3ce0 con 0x56502036bd00 2023-06-23T18:33:06.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02a2ffd700 1 -- 172.21.15.2:0/4259480163 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b404e020 con 0x56502036bd00 2023-06-23T18:33:06.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02a2ffd700 1 -- 172.21.15.2:0/4259480163 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f02b4003b20 con 0x56502036bd00 2023-06-23T18:33:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02d6090b80 10 client.25940 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02d6090b80 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02d6090b80 20 client.25940 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02d6090b80 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5650203d6690 0x5650203d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02d6090b80 4 client.25940 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:33:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02d6090b80 1 -- 172.21.15.2:0/4259480163 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5650202780d0 con 0x5650203d6690 2023-06-23T18:33:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5650203d6690 0x5650203d8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02a2ffd700 10 client.25940 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:06.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.532+0000 7f02c2ffd700 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5650203d6690 0x5650203d8a70 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9db843b6e1eca539 server_cookie=d04449f1a5975a6d in_seq=0 out_seq=0 2023-06-23T18:33:06.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02a2ffd700 1 -- 172.21.15.2:0/4259480163 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5650202780d0 con 0x5650203d6690 2023-06-23T18:33:06.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02a2ffd700 10 client.25940 handle_command_reply: tid=0 2023-06-23T18:33:06.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 client.25940 shutdown 2023-06-23T18:33:06.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 -- 172.21.15.2:0/4259480163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 msgr2=0x56502037ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x56502037ca80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009350 tx=0x7f02b4004500 comp rx=0 tx=0).stop 2023-06-23T18:33:06.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 -- 172.21.15.2:0/4259480163 shutdown_connections 2023-06-23T18:33:06.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5650203d6690 0x5650203d8a70 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56502036d080 0x565020379a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56502036bd00 0x56502037ca80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 --2- 172.21.15.2:0/4259480163 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56502036c650 0x565020379520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 -- 172.21.15.2:0/4259480163 >> 172.21.15.2:0/4259480163 conn(0x565020361470 msgr2=0x565020375550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02a2ffd700 0 client.25940 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:06.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 -- 172.21.15.2:0/4259480163 shutdown_connections 2023-06-23T18:33:06.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 1 -- 172.21.15.2:0/4259480163 wait complete. 2023-06-23T18:33:06.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.534+0000 7f02d6090b80 20 client.25940 trim_cache size 0 max 16384 2023-06-23T18:33:06.547 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:33:06.547 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:33:06.547 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "7ac6f3d0-c050-4430-9d58-b4e14ba6adb5", 2023-06-23T18:33:06.547 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:33:06.548 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:33:06.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 -- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f02ac0a11d0 msgr2=0x7f02ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f02ac0a11d0 0x7f02ac0a3680 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f02b40095c0 tx=0x7f02b4009550 comp rx=0 tx=0).stop 2023-06-23T18:33:06.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 -- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 msgr2=0x7f02c419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 0x7f02c419cc80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f02bc007d70 tx=0x7f02bc00a340 comp rx=0 tx=0).stop 2023-06-23T18:33:06.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 -- 172.21.15.2:0/2281274924 shutdown_connections 2023-06-23T18:33:06.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f02c4100050 0x7f02c419cc80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f02ac0a11d0 0x7f02ac0a3680 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c40feca0 0x7f02c41956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 --2- 172.21.15.2:0/2281274924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02c40ff650 0x7f02c4195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 -- 172.21.15.2:0/2281274924 >> 172.21.15.2:0/2281274924 conn(0x7f02c40f9820 msgr2=0x7f02c40fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 -- 172.21.15.2:0/2281274924 shutdown_connections 2023-06-23T18:33:06.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.535+0000 7f02c9f0b700 1 -- 172.21.15.2:0/2281274924 wait complete. 2023-06-23T18:33:06.556 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:33:06.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 -- 172.21.15.2:0/3364936463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc105840 msgr2=0x7fdbfc107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 --2- 172.21.15.2:0/3364936463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc105840 0x7fdbfc107ca0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009f90 tx=0x7fdbf400b330 comp rx=0 tx=0).stop 2023-06-23T18:33:06.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 -- 172.21.15.2:0/3364936463 shutdown_connections 2023-06-23T18:33:06.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 --2- 172.21.15.2:0/3364936463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc105840 0x7fdbfc107ca0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 --2- 172.21.15.2:0/3364936463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbfc100600 0x7fdbfc1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 --2- 172.21.15.2:0/3364936463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbfc102f20 0x7fdbfc105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 -- 172.21.15.2:0/3364936463 >> 172.21.15.2:0/3364936463 conn(0x7fdbfc0fa0e0 msgr2=0x7fdbfc0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 -- 172.21.15.2:0/3364936463 shutdown_connections 2023-06-23T18:33:06.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.733+0000 7fdc00d77700 1 -- 172.21.15.2:0/3364936463 wait complete. 2023-06-23T18:33:06.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdc00d77700 1 Processor -- start 2023-06-23T18:33:06.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdc00d77700 1 -- start start 2023-06-23T18:33:06.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdc00d77700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc100600 0x7fdbfc19be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdc00d77700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbfc102f20 0x7fdbfc19c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdc00d77700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbfc105840 0x7fdbfc1a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdc00d77700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1a39d0 con 0x7fdbfc105840 2023-06-23T18:33:06.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdc00d77700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1a3b10 con 0x7fdbfc102f20 2023-06-23T18:33:06.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdc00d77700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1a3ca0 con 0x7fdbfc100600 2023-06-23T18:33:06.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdbfa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc100600 0x7fdbfc19be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdbf9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbfc102f20 0x7fdbfc19c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdbfa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc100600 0x7fdbfc19be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56856/0 (socket says 172.21.15.2:56856) 2023-06-23T18:33:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdbfa59c700 1 -- 172.21.15.2:0/1006047585 learned_addr learned my addr 172.21.15.2:0/1006047585 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.734+0000 7fdbfad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbfc105840 0x7fdbfc1a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbfa59c700 1 -- 172.21.15.2:0/1006047585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbfc102f20 msgr2=0x7fdbfc19c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbfc102f20 0x7fdbfc19c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbfa59c700 1 -- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbfc105840 msgr2=0x7fdbfc1a3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbfc105840 0x7fdbfc1a3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbfa59c700 1 -- 172.21.15.2:0/1006047585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf4009c40 con 0x7fdbfc100600 2023-06-23T18:33:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbfc102f20 0x7fdbfc19c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbfad9d700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbfc105840 0x7fdbfc1a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:33:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc100600 0x7fdbfc19be50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e9c0 tx=0x7fdbec00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbeb7fe700 1 -- 172.21.15.2:0/1006047585 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00cbd0 con 0x7fdbfc100600 2023-06-23T18:33:06.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbeb7fe700 1 -- 172.21.15.2:0/1006047585 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbec00eeb0 con 0x7fdbfc100600 2023-06-23T18:33:06.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdc00d77700 1 -- 172.21.15.2:0/1006047585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc1a3f80 con 0x7fdbfc100600 2023-06-23T18:33:06.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdbeb7fe700 1 -- 172.21.15.2:0/1006047585 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec005410 con 0x7fdbfc100600 2023-06-23T18:33:06.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.735+0000 7fdc00d77700 1 -- 172.21.15.2:0/1006047585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc1a43b0 con 0x7fdbfc100600 2023-06-23T18:33:06.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.738+0000 7fdbeb7fe700 1 -- 172.21.15.2:0/1006047585 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdbec0055b0 con 0x7fdbfc100600 2023-06-23T18:33:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.739+0000 7fdbeb7fe700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdbe40a1250 0x7fdbe40a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.739+0000 7fdbeb7fe700 1 -- 172.21.15.2:0/1006047585 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdbec014070 con 0x7fdbfc100600 2023-06-23T18:33:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.739+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdbe40a1250 0x7fdbe40a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.739+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdbe40a1250 0x7fdbe40a3700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0011490 tx=0x7fdbf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdc0cefcb80 1 Processor -- start 2023-06-23T18:33:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdc0cefcb80 1 -- start start 2023-06-23T18:33:06.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130e9b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 0x558130e9ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130e9c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130d9fcb0 con 0x558130e9ac80 2023-06-23T18:33:06.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130df2a40 con 0x558130e9c000 2023-06-23T18:33:06.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130debed0 con 0x558130e9b5d0 2023-06-23T18:33:06.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdbfa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130e9b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdbfa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130e9b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56526/0 (socket says 172.21.15.2:56526) 2023-06-23T18:33:06.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdbfa59c700 1 -- 172.21.15.2:0/1635596651 learned_addr learned my addr 172.21.15.2:0/1635596651 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.740+0000 7fdbfad9d700 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130e9c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbfa59c700 1 -- 172.21.15.2:0/1635596651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 msgr2=0x558130e9ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 0x558130e9ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbfa59c700 1 -- 172.21.15.2:0/1635596651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 msgr2=0x558130e9c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130e9c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbfa59c700 1 -- 172.21.15.2:0/1635596651 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558130c6d630 con 0x558130e9ac80 2023-06-23T18:33:06.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbfad9d700 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130e9c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:06.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130e9b090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdbec099c10 tx=0x7fdbec009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1de21b275778d45 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbe8ff9700 1 -- 172.21.15.2:0/1635596651 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec0a2360 con 0x558130e9ac80 2023-06-23T18:33:06.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbe8ff9700 1 -- 172.21.15.2:0/1635596651 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbec0ca710 con 0x558130e9ac80 2023-06-23T18:33:06.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.741+0000 7fdbe8ff9700 1 -- 172.21.15.2:0/1635596651 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec099660 con 0x558130e9ac80 2023-06-23T18:33:06.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.742+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1635596651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 msgr2=0x558130e9b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.742+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130e9b090 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdbec099c10 tx=0x7fdbec009550 comp rx=0 tx=0).stop 2023-06-23T18:33:06.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.742+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1635596651 shutdown_connections 2023-06-23T18:33:06.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.742+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130e9c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.742+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130e9b090 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.742+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/1635596651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 0x558130e9ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.742+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1635596651 >> 172.21.15.2:0/1635596651 conn(0x558130e903f0 msgr2=0x558130e95520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1635596651 shutdown_connections 2023-06-23T18:33:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1635596651 wait complete. 2023-06-23T18:33:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 -- start start 2023-06-23T18:33:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 Processor -- start 2023-06-23T18:33:06.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9ac80 0x558130ea4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9b5d0 0x558130ea4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9c000 0x558130eabb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130d8bac0 con 0x558130e9b5d0 2023-06-23T18:33:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130d8b7c0 con 0x558130e9ac80 2023-06-23T18:33:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130c6f920 con 0x558130e9c000 2023-06-23T18:33:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdc0cefcb80 10 client.0 _ll_register_callbacks cb 0x558130d91260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdbf9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9b5d0 0x558130ea4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.743+0000 7fdbf9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9b5d0 0x558130ea4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56532/0 (socket says 172.21.15.2:56532) 2023-06-23T18:33:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbf9d9b700 1 -- 172.21.15.2:0/957189533 learned_addr learned my addr 172.21.15.2:0/957189533 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbfad9d700 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9c000 0x558130eabb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbf9d9b700 1 -- 172.21.15.2:0/957189533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9c000 msgr2=0x558130eabb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9c000 0x558130eabb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbf9d9b700 1 -- 172.21.15.2:0/957189533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9ac80 msgr2=0x558130ea4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9ac80 0x558130ea4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9ac80 0x558130ea4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbf9d9b700 1 -- 172.21.15.2:0/957189533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc108890 con 0x558130e9b5d0 2023-06-23T18:33:06.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9b5d0 0x558130ea4ad0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0003a00 tx=0x7fdbf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3fb3cecd70bcbe server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.744+0000 7fdbe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:06.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/957189533 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf000ebd0 con 0x558130e9b5d0 2023-06-23T18:33:06.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/957189533 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdbf000ed30 con 0x558130e9b5d0 2023-06-23T18:33:06.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/957189533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558130e8dbb0 con 0x558130e9b5d0 2023-06-23T18:33:06.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/957189533 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbf0003e70 con 0x558130e9b5d0 2023-06-23T18:33:06.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdc0cefcb80 10 client.16425 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/957189533 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x7fdbfc1a43b0 con 0x558130e9b5d0 2023-06-23T18:33:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/957189533 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558130e8dbb0 con 0x558130e9b5d0 2023-06-23T18:33:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/957189533 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdbf0024020 con 0x558130e9b5d0 2023-06-23T18:33:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdc0cefcb80 10 client.16425 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.745+0000 7fdc0cefcb80 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.746+0000 7fdc0cefcb80 20 client.16425 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:06.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.746+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558130eb0370 0x558130eb2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.746+0000 7fdc0cefcb80 4 client.16425 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.746+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/957189533 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558130de4eb0 con 0x558130eb0370 2023-06-23T18:33:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.746+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558130eb0370 0x558130eb2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.746+0000 7fdbe1ffb700 10 client.16425 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.746+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558130eb0370 0x558130eb2780 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275c88723a9272a0 server_cookie=9827e2466453e37f in_seq=0 out_seq=0 2023-06-23T18:33:06.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/957189533 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558130de4eb0 con 0x558130eb0370 2023-06-23T18:33:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdbe1ffb700 10 client.16425 handle_command_reply: tid=0 2023-06-23T18:33:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 client.16425 shutdown 2023-06-23T18:33:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/957189533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9b5d0 msgr2=0x558130ea4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9b5d0 0x558130ea4ad0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0003a00 tx=0x7fdbf0004750 comp rx=0 tx=0).stop 2023-06-23T18:33:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/957189533 shutdown_connections 2023-06-23T18:33:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558130eb0370 0x558130eb2780 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9c000 0x558130eabb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9ac80 0x558130ea4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/957189533 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9b5d0 0x558130ea4ad0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdbe1ffb700 0 client.16425 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/957189533 >> 172.21.15.2:0/957189533 conn(0x558130e903f0 msgr2=0x558130e94c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/957189533 shutdown_connections 2023-06-23T18:33:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.747+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/957189533 wait complete. 2023-06-23T18:33:06.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.748+0000 7fdc0cefcb80 20 client.16425 trim_cache size 0 max 16384 2023-06-23T18:33:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdc0cefcb80 1 Processor -- start 2023-06-23T18:33:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdc0cefcb80 1 -- start start 2023-06-23T18:33:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130ea23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 0x558130eabad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130ea4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130ead800 con 0x558130e9ac80 2023-06-23T18:33:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130ea3a40 con 0x558130e9c000 2023-06-23T18:33:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130ea4100 con 0x558130e9b5d0 2023-06-23T18:33:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdbfa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130ea23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.772+0000 7fdbfad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130ea4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbfad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130ea4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47136/0 (socket says 172.21.15.2:47136) 2023-06-23T18:33:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbfad9d700 1 -- 172.21.15.2:0/1516449400 learned_addr learned my addr 172.21.15.2:0/1516449400 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 0x558130eabad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbfad9d700 1 -- 172.21.15.2:0/1516449400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 msgr2=0x558130eabad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbfad9d700 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 0x558130eabad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbfad9d700 1 -- 172.21.15.2:0/1516449400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 msgr2=0x558130ea23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbfad9d700 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130ea23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbfad9d700 1 -- 172.21.15.2:0/1516449400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558130c19de0 con 0x558130e9c000 2023-06-23T18:33:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 0x558130eabad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbfad9d700 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130ea4590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007420 tx=0x7fdbf4007450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aa9b267ab59377a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.773+0000 7fdbe97fa700 1 -- 172.21.15.2:0/1516449400 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400e040 con 0x558130e9c000 2023-06-23T18:33:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdbe97fa700 1 -- 172.21.15.2:0/1516449400 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbf4004a50 con 0x558130e9c000 2023-06-23T18:33:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdbe97fa700 1 -- 172.21.15.2:0/1516449400 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4025470 con 0x558130e9c000 2023-06-23T18:33:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1516449400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 msgr2=0x558130ea4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130ea4590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4007420 tx=0x7fdbf4007450 comp rx=0 tx=0).stop 2023-06-23T18:33:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1516449400 shutdown_connections 2023-06-23T18:33:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9c000 0x558130ea4590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9ac80 0x558130ea23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/1516449400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9b5d0 0x558130eabad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1516449400 >> 172.21.15.2:0/1516449400 conn(0x558130e903f0 msgr2=0x558130ea3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.774+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1516449400 shutdown_connections 2023-06-23T18:33:06.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/1516449400 wait complete. 2023-06-23T18:33:06.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 -- start start 2023-06-23T18:33:06.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 Processor -- start 2023-06-23T18:33:06.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9ac80 0x558130ea5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9b5d0 0x558130ea62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9c000 0x558130eab300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130eab840 con 0x558130e9c000 2023-06-23T18:33:06.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130eab9b0 con 0x558130e9b5d0 2023-06-23T18:33:06.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558130eabb70 con 0x558130e9ac80 2023-06-23T18:33:06.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.775+0000 7fdc0cefcb80 10 client.0 _ll_register_callbacks cb 0x558130d91260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:06.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9b5d0 0x558130ea62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9b5d0 0x558130ea62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47140/0 (socket says 172.21.15.2:47140) 2023-06-23T18:33:06.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 -- 172.21.15.2:0/2604143598 learned_addr learned my addr 172.21.15.2:0/2604143598 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:06.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbfad9d700 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9c000 0x558130eab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9ac80 0x558130ea5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 -- 172.21.15.2:0/2604143598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9ac80 msgr2=0x558130ea5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9ac80 0x558130ea5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 -- 172.21.15.2:0/2604143598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9c000 msgr2=0x558130eab300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:06.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9c000 0x558130eab300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 -- 172.21.15.2:0/2604143598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc1a43b0 con 0x558130e9b5d0 2023-06-23T18:33:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbf9d9b700 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9b5d0 0x558130ea62d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0026400 tx=0x7fdbf0027c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6d855b38ea32fd server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbe1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:33:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.776+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/2604143598 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf001c380 con 0x558130e9b5d0 2023-06-23T18:33:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/2604143598 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdbf00266f0 con 0x558130e9b5d0 2023-06-23T18:33:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/2604143598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbec0ca710 con 0x558130e9b5d0 2023-06-23T18:33:06.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/2604143598 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbf0003e70 con 0x558130e9b5d0 2023-06-23T18:33:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 10 client.25955 fetch_fsmap learned FSMap version 36 2023-06-23T18:33:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/2604143598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x558130c02c60 con 0x558130e9b5d0 2023-06-23T18:33:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/2604143598 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbec0ca710 con 0x558130e9b5d0 2023-06-23T18:33:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/2604143598 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 36) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdbf0003ae0 con 0x558130e9b5d0 2023-06-23T18:33:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 10 client.25955 fetch_fsmap finished waiting for FSMap version 36 2023-06-23T18:33:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 10 client.25955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 20 client.25955 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:06.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558130f16550 0x558130f18960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 4 client.25955 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:33:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/2604143598 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558130da7050 con 0x558130f16550 2023-06-23T18:33:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.777+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558130f16550 0x558130f18960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.778+0000 7fdbe1ffb700 10 client.25955 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.778+0000 7fdbfa59c700 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558130f16550 0x558130f18960 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bcce149e08468bf server_cookie=bbb6c8e9cba89f61 in_seq=0 out_seq=0 2023-06-23T18:33:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.778+0000 7fdbe1ffb700 1 -- 172.21.15.2:0/2604143598 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558130da7050 con 0x558130f16550 2023-06-23T18:33:06.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.778+0000 7fdbe1ffb700 10 client.25955 handle_command_reply: tid=0 2023-06-23T18:33:06.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.778+0000 7fdc0cefcb80 1 client.25955 shutdown 2023-06-23T18:33:06.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.778+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/2604143598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9b5d0 msgr2=0x558130ea62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.778+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9b5d0 0x558130ea62d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0026400 tx=0x7fdbf0027c10 comp rx=0 tx=0).stop 2023-06-23T18:33:06.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/2604143598 shutdown_connections 2023-06-23T18:33:06.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558130f16550 0x558130f18960 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558130e9c000 0x558130eab300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558130e9ac80 0x558130ea5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdbe1ffb700 0 client.25955 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:06.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 1 --2- 172.21.15.2:0/2604143598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558130e9b5d0 0x558130ea62d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/2604143598 >> 172.21.15.2:0/2604143598 conn(0x558130e903f0 msgr2=0x558130ea23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/2604143598 shutdown_connections 2023-06-23T18:33:06.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 1 -- 172.21.15.2:0/2604143598 wait complete. 2023-06-23T18:33:06.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.779+0000 7fdc0cefcb80 20 client.25955 trim_cache size 0 max 16384 2023-06-23T18:33:06.796 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: "7ac6f3d0-c050-4430-9d58-b4e14ba6adb5": { 2023-06-23T18:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7ac6f3d0-c050-4430-9d58-b4e14ba6adb5", 2023-06-23T18:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:33:06.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:33:06.798 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:33:06.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.780+0000 7fdc00d77700 1 -- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdbe40a1250 msgr2=0x7fdbe40a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.780+0000 7fdc00d77700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdbe40a1250 0x7fdbe40a3700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdbf0011490 tx=0x7fdbf0009260 comp rx=0 tx=0).stop 2023-06-23T18:33:06.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.780+0000 7fdc00d77700 1 -- 172.21.15.2:0/1006047585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc100600 msgr2=0x7fdbfc19be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:06.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.780+0000 7fdc00d77700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc100600 0x7fdbfc19be50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00e9c0 tx=0x7fdbec00ecd0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.782+0000 7fdc00d77700 1 -- 172.21.15.2:0/1006047585 shutdown_connections 2023-06-23T18:33:06.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.782+0000 7fdc00d77700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdbfc105840 0x7fdbfc1a3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.782+0000 7fdc00d77700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbfc100600 0x7fdbfc19be50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.782+0000 7fdc00d77700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fdbe40a1250 0x7fdbe40a3700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.782+0000 7fdc00d77700 1 --2- 172.21.15.2:0/1006047585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdbfc102f20 0x7fdbfc19c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.782+0000 7fdc00d77700 1 -- 172.21.15.2:0/1006047585 >> 172.21.15.2:0/1006047585 conn(0x7fdbfc0fa0e0 msgr2=0x7fdbfc101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.783+0000 7fdc00d77700 1 -- 172.21.15.2:0/1006047585 shutdown_connections 2023-06-23T18:33:06.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:06.783+0000 7fdc00d77700 1 -- 172.21.15.2:0/1006047585 wait complete. 2023-06-23T18:33:06.804 INFO:tasks.cephfs.filesystem:scrub status for tag:7ac6f3d0-c050-4430-9d58-b4e14ba6adb5 - {'path': '/', 'tag': '7ac6f3d0-c050-4430-9d58-b4e14ba6adb5', 'options': 'recursive,force'} 2023-06-23T18:33:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:06 smithi002 ceph-mon[95947]: pgmap v628: 97 pgs: 97 active+clean; 8.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 30 MiB/s wr, 233 op/s 2023-06-23T18:33:07.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:06 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:33:07.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:06 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:07.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:06 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:33:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:06 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:33:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:06 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:33:07.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:06 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:06 smithi082 ceph-mon[97889]: pgmap v628: 97 pgs: 97 active+clean; 8.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 30 MiB/s wr, 233 op/s 2023-06-23T18:33:07.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:06 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:33:07.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:06 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:07.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:06 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:33:07.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:06 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:33:07.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:06 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:33:07.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:06 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:06 smithi017 ceph-mon[97472]: pgmap v628: 97 pgs: 97 active+clean; 8.2 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 30 MiB/s wr, 233 op/s 2023-06-23T18:33:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:06 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:33:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:06 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:06 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:33:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:06 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:33:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:06 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:33:07.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:06 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:08.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:07 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:08.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:07 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:08.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:07 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:07 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:08.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:07 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:08.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:07 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:07 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:07 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:08.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:07 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:08 smithi002 ceph-mon[95947]: pgmap v629: 97 pgs: 97 active+clean; 8.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 31 MiB/s wr, 273 op/s 2023-06-23T18:33:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:08 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:08 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:08 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:33:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:08 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:33:09.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:08 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:08 smithi082 ceph-mon[97889]: pgmap v629: 97 pgs: 97 active+clean; 8.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 31 MiB/s wr, 273 op/s 2023-06-23T18:33:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:08 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:08 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:08 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:33:09.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:08 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:33:09.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:08 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:08 smithi017 ceph-mon[97472]: pgmap v629: 97 pgs: 97 active+clean; 8.3 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 31 MiB/s wr, 273 op/s 2023-06-23T18:33:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:08 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:08 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:08 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:33:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:08 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:33:09.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:08 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:33:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:10 smithi002 ceph-mon[95947]: pgmap v630: 97 pgs: 97 active+clean; 8.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 32 MiB/s wr, 334 op/s 2023-06-23T18:33:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:10 smithi082 ceph-mon[97889]: pgmap v630: 97 pgs: 97 active+clean; 8.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 32 MiB/s wr, 334 op/s 2023-06-23T18:33:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:10 smithi017 ceph-mon[97472]: pgmap v630: 97 pgs: 97 active+clean; 8.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 32 MiB/s wr, 334 op/s 2023-06-23T18:33:13.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:12 smithi002 ceph-mon[95947]: pgmap v631: 97 pgs: 97 active+clean; 8.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 36 MiB/s wr, 369 op/s 2023-06-23T18:33:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:12 smithi082 ceph-mon[97889]: pgmap v631: 97 pgs: 97 active+clean; 8.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 36 MiB/s wr, 369 op/s 2023-06-23T18:33:13.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:12 smithi017 ceph-mon[97472]: pgmap v631: 97 pgs: 97 active+clean; 8.3 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 36 MiB/s wr, 369 op/s 2023-06-23T18:33:15.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:14 smithi002 ceph-mon[95947]: pgmap v632: 97 pgs: 97 active+clean; 8.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 38 MiB/s wr, 391 op/s 2023-06-23T18:33:15.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:14 smithi082 ceph-mon[97889]: pgmap v632: 97 pgs: 97 active+clean; 8.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 38 MiB/s wr, 391 op/s 2023-06-23T18:33:15.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:14 smithi017 ceph-mon[97472]: pgmap v632: 97 pgs: 97 active+clean; 8.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 38 MiB/s wr, 391 op/s 2023-06-23T18:33:17.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:16 smithi002 ceph-mon[95947]: pgmap v633: 97 pgs: 97 active+clean; 8.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 36 MiB/s wr, 295 op/s 2023-06-23T18:33:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:16 smithi082 ceph-mon[97889]: pgmap v633: 97 pgs: 97 active+clean; 8.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 36 MiB/s wr, 295 op/s 2023-06-23T18:33:17.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:16 smithi017 ceph-mon[97472]: pgmap v633: 97 pgs: 97 active+clean; 8.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 36 MiB/s wr, 295 op/s 2023-06-23T18:33:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:18 smithi002 ceph-mon[95947]: pgmap v634: 97 pgs: 97 active+clean; 8.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 39 MiB/s wr, 231 op/s 2023-06-23T18:33:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:18 smithi082 ceph-mon[97889]: pgmap v634: 97 pgs: 97 active+clean; 8.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 39 MiB/s wr, 231 op/s 2023-06-23T18:33:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:18 smithi017 ceph-mon[97472]: pgmap v634: 97 pgs: 97 active+clean; 8.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 39 MiB/s wr, 231 op/s 2023-06-23T18:33:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:20 smithi082 ceph-mon[97889]: pgmap v635: 97 pgs: 97 active+clean; 8.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 194 op/s 2023-06-23T18:33:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:20 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:33:21.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:20 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:33:21.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:20 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:33:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:20 smithi017 ceph-mon[97472]: pgmap v635: 97 pgs: 97 active+clean; 8.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 194 op/s 2023-06-23T18:33:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:20 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:33:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:20 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:33:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:20 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:33:21.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:20 smithi002 ceph-mon[95947]: pgmap v635: 97 pgs: 97 active+clean; 8.5 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 41 MiB/s wr, 194 op/s 2023-06-23T18:33:21.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:20 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:33:21.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:20 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:33:21.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:20 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:33:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:22 smithi082 ceph-mon[97889]: pgmap v636: 97 pgs: 97 active+clean; 8.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 41 MiB/s wr, 154 op/s 2023-06-23T18:33:23.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:22 smithi017 ceph-mon[97472]: pgmap v636: 97 pgs: 97 active+clean; 8.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 41 MiB/s wr, 154 op/s 2023-06-23T18:33:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:22 smithi002 ceph-mon[95947]: pgmap v636: 97 pgs: 97 active+clean; 8.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 41 MiB/s wr, 154 op/s 2023-06-23T18:33:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:24 smithi082 ceph-mon[97889]: pgmap v637: 97 pgs: 97 active+clean; 8.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 40 MiB/s wr, 121 op/s 2023-06-23T18:33:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:24 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:25.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:24 smithi017 ceph-mon[97472]: pgmap v637: 97 pgs: 97 active+clean; 8.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 40 MiB/s wr, 121 op/s 2023-06-23T18:33:25.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:24 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:25.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:24 smithi002 ceph-mon[95947]: pgmap v637: 97 pgs: 97 active+clean; 8.6 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 40 MiB/s wr, 121 op/s 2023-06-23T18:33:25.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:24 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:26 smithi082 ceph-mon[97889]: pgmap v638: 97 pgs: 97 active+clean; 8.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 MiB/s wr, 93 op/s 2023-06-23T18:33:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:26 smithi017 ceph-mon[97472]: pgmap v638: 97 pgs: 97 active+clean; 8.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 MiB/s wr, 93 op/s 2023-06-23T18:33:27.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:26 smithi002 ceph-mon[95947]: pgmap v638: 97 pgs: 97 active+clean; 8.6 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 41 MiB/s wr, 93 op/s 2023-06-23T18:33:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:28 smithi082 ceph-mon[97889]: pgmap v639: 97 pgs: 97 active+clean; 8.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 44 MiB/s wr, 92 op/s 2023-06-23T18:33:29.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:28 smithi017 ceph-mon[97472]: pgmap v639: 97 pgs: 97 active+clean; 8.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 44 MiB/s wr, 92 op/s 2023-06-23T18:33:29.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:28 smithi002 ceph-mon[95947]: pgmap v639: 97 pgs: 97 active+clean; 8.7 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 44 MiB/s wr, 92 op/s 2023-06-23T18:33:30.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:29 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:30.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:29 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:30.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:29 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:33:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:30 smithi082 ceph-mon[97889]: pgmap v640: 97 pgs: 97 active+clean; 8.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 MiB/s wr, 44 op/s 2023-06-23T18:33:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:30 smithi017 ceph-mon[97472]: pgmap v640: 97 pgs: 97 active+clean; 8.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 MiB/s wr, 44 op/s 2023-06-23T18:33:31.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:30 smithi002 ceph-mon[95947]: pgmap v640: 97 pgs: 97 active+clean; 8.7 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 44 MiB/s wr, 44 op/s 2023-06-23T18:33:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:32 smithi082 ceph-mon[97889]: pgmap v641: 97 pgs: 97 active+clean; 8.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 42 MiB/s wr, 62 op/s 2023-06-23T18:33:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:32 smithi017 ceph-mon[97472]: pgmap v641: 97 pgs: 97 active+clean; 8.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 42 MiB/s wr, 62 op/s 2023-06-23T18:33:33.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:32 smithi002 ceph-mon[95947]: pgmap v641: 97 pgs: 97 active+clean; 8.7 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 42 MiB/s wr, 62 op/s 2023-06-23T18:33:35.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:34 smithi082 ceph-mon[97889]: pgmap v642: 97 pgs: 97 active+clean; 8.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 44 MiB/s wr, 45 op/s 2023-06-23T18:33:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:34 smithi017 ceph-mon[97472]: pgmap v642: 97 pgs: 97 active+clean; 8.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 44 MiB/s wr, 45 op/s 2023-06-23T18:33:35.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:34 smithi002 ceph-mon[95947]: pgmap v642: 97 pgs: 97 active+clean; 8.8 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 44 MiB/s wr, 45 op/s 2023-06-23T18:33:36.805 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:33:36.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 -- 172.21.15.2:0/1965472762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80fe7f0 msgr2=0x7f8dc80fec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:36.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/1965472762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80fe7f0 0x7f8dc80fec00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009990 tx=0x7f8db8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 -- 172.21.15.2:0/1965472762 shutdown_connections 2023-06-23T18:33:36.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/1965472762 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dc8100370 0x7f8dc8106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/1965472762 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80fe7f0 0x7f8dc80fec00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/1965472762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dc80ff9e0 0x7f8dc80ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 -- 172.21.15.2:0/1965472762 >> 172.21.15.2:0/1965472762 conn(0x7f8dc80f9df0 msgr2=0x7f8dc80fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:36.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 -- 172.21.15.2:0/1965472762 shutdown_connections 2023-06-23T18:33:36.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.970+0000 7f8dc7fff700 1 -- 172.21.15.2:0/1965472762 wait complete. 2023-06-23T18:33:36.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.971+0000 7f8dc7fff700 1 Processor -- start 2023-06-23T18:33:36.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.971+0000 7f8dc7fff700 1 -- start start 2023-06-23T18:33:36.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.971+0000 7f8dc7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dc80fe7f0 0x7f8dc8194490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80ff9e0 0x7f8dc81949d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dc8100370 0x7f8dc819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc7fff700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc819c010 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc7fff700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc819c150 con 0x7f8dc80fe7f0 2023-06-23T18:33:36.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc7fff700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc819c2e0 con 0x7f8dc8100370 2023-06-23T18:33:36.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80ff9e0 0x7f8dc81949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80ff9e0 0x7f8dc81949d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56564/0 (socket says 172.21.15.2:56564) 2023-06-23T18:33:36.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 -- 172.21.15.2:0/2759330336 learned_addr learned my addr 172.21.15.2:0/2759330336 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:36.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dc80fe7f0 0x7f8dc8194490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc77fe700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dc8100370 0x7f8dc819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 -- 172.21.15.2:0/2759330336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dc8100370 msgr2=0x7f8dc819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:36.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dc8100370 0x7f8dc819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 -- 172.21.15.2:0/2759330336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dc80fe7f0 msgr2=0x7f8dc8194490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:36.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dc80fe7f0 0x7f8dc8194490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 -- 172.21.15.2:0/2759330336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db80095f0 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8dc67fc700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80ff9e0 0x7f8dc81949d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00b890 tx=0x7f8dbc00bba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:36.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.972+0000 7f8daffff700 1 -- 172.21.15.2:0/2759330336 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc009790 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.973+0000 7f8daffff700 1 -- 172.21.15.2:0/2759330336 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8dbc00ed40 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.973+0000 7f8dc7fff700 1 -- 172.21.15.2:0/2759330336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc8068730 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.973+0000 7f8daffff700 1 -- 172.21.15.2:0/2759330336 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc017e60 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.973+0000 7f8dc7fff700 1 -- 172.21.15.2:0/2759330336 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc8068c40 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.975+0000 7f8daffff700 1 -- 172.21.15.2:0/2759330336 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8dbc00e8b0 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.976+0000 7f8daffff700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8db00a1150 0x7f8db00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.976+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8db00a1150 0x7f8db00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.976+0000 7f8daffff700 1 -- 172.21.15.2:0/2759330336 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8dbc0cc580 con 0x7f8dc80ff9e0 2023-06-23T18:33:36.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.976+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8db00a1150 0x7f8db00a3600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8db80098e0 tx=0x7f8db800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:36.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.977+0000 7f8dd9986b80 1 Processor -- start 2023-06-23T18:33:36.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.977+0000 7f8dd9986b80 1 -- start start 2023-06-23T18:33:36.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.977+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1998130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.977+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d1998fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.977+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.977+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d189ccb0 con 0x55d0d1999520 2023-06-23T18:33:36.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.977+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d18efa40 con 0x55d0d1998b70 2023-06-23T18:33:36.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.977+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d18e8ed0 con 0x55d0d1997d20 2023-06-23T18:33:36.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56574/0 (socket says 172.21.15.2:56574) 2023-06-23T18:33:36.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 -- 172.21.15.2:0/947366450 learned_addr learned my addr 172.21.15.2:0/947366450 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:36.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1998130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 -- 172.21.15.2:0/947366450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 msgr2=0x55d0d1998130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1998130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 -- 172.21.15.2:0/947366450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 msgr2=0x55d0d1998fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d1998fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 -- 172.21.15.2:0/947366450 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d176a630 con 0x55d0d1999520 2023-06-23T18:33:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1998130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dc77fe700 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199fdc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0010920 tx=0x7f8dc000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cc43fc0bd6e4ef server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dad7fa700 1 -- 172.21.15.2:0/947366450 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc001f070 con 0x55d0d1999520 2023-06-23T18:33:36.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dad7fa700 1 -- 172.21.15.2:0/947366450 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8dc0016040 con 0x55d0d1999520 2023-06-23T18:33:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.978+0000 7f8dad7fa700 1 -- 172.21.15.2:0/947366450 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc0004780 con 0x55d0d1999520 2023-06-23T18:33:36.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 -- 172.21.15.2:0/947366450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 msgr2=0x55d0d199fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:36.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199fdc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8dc0010920 tx=0x7f8dc000b040 comp rx=0 tx=0).stop 2023-06-23T18:33:36.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 -- 172.21.15.2:0/947366450 shutdown_connections 2023-06-23T18:33:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199fdc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1998130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/947366450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d1998fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 -- 172.21.15.2:0/947366450 >> 172.21.15.2:0/947366450 conn(0x55d0d198d3f0 msgr2=0x55d0d1992780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 -- 172.21.15.2:0/947366450 shutdown_connections 2023-06-23T18:33:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.979+0000 7f8dd9986b80 1 -- 172.21.15.2:0/947366450 wait complete. 2023-06-23T18:33:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 1 -- start start 2023-06-23T18:33:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 1 Processor -- start 2023-06-23T18:33:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d19a1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d19a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d1888ac0 con 0x55d0d1999520 2023-06-23T18:33:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d18887c0 con 0x55d0d1998b70 2023-06-23T18:33:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d176c920 con 0x55d0d1997d20 2023-06-23T18:33:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dd9986b80 10 client.0 _ll_register_callbacks cb 0x55d0d188e260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d19a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d19a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:56914/0 (socket says 172.21.15.2:56914) 2023-06-23T18:33:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dc6ffd700 1 -- 172.21.15.2:0/285532902 learned_addr learned my addr 172.21.15.2:0/285532902 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56580/0 (socket says 172.21.15.2:56580) 2023-06-23T18:33:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.980+0000 7f8dc67fc700 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d19a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dc6ffd700 1 -- 172.21.15.2:0/285532902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 msgr2=0x55d0d19a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d19a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dc6ffd700 1 -- 172.21.15.2:0/285532902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 msgr2=0x55d0d19a8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dc6ffd700 1 -- 172.21.15.2:0/285532902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00105d0 con 0x55d0d1997d20 2023-06-23T18:33:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dc77fe700 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d19a1780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8db8015000 tx=0x7f8db801a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb2ee9ed1d724af server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8daaff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:33:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8daaff5700 1 -- 172.21.15.2:0/285532902 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800bb40 con 0x55d0d1997d20 2023-06-23T18:33:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dd9986b80 1 -- 172.21.15.2:0/285532902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0d198abb0 con 0x55d0d1997d20 2023-06-23T18:33:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8daaff5700 1 -- 172.21.15.2:0/285532902 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8db800bca0 con 0x55d0d1997d20 2023-06-23T18:33:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8daaff5700 1 -- 172.21.15.2:0/285532902 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8db800be60 con 0x55d0d1997d20 2023-06-23T18:33:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dd9986b80 10 client.26164 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.981+0000 7f8dd9986b80 1 -- 172.21.15.2:0/285532902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d0d1716de0 con 0x55d0d1997d20 2023-06-23T18:33:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.982+0000 7f8daaff5700 1 -- 172.21.15.2:0/285532902 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d0d198abb0 con 0x55d0d1997d20 2023-06-23T18:33:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.982+0000 7f8daaff5700 1 -- 172.21.15.2:0/285532902 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8db800f5a0 con 0x55d0d1997d20 2023-06-23T18:33:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.982+0000 7f8dd9986b80 10 client.26164 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:36.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.982+0000 7f8dd9986b80 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:36.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.982+0000 7f8dd9986b80 20 client.26164 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:36.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.982+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d0d19ad2e0 0x55d0d19af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:36.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.982+0000 7f8dd9986b80 4 client.26164 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:36.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.982+0000 7f8dd9986b80 1 -- 172.21.15.2:0/285532902 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0d18e1eb0 con 0x55d0d19ad2e0 2023-06-23T18:33:36.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.983+0000 7f8dc67fc700 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d0d19ad2e0 0x55d0d19af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:36.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.984+0000 7f8dc67fc700 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d0d19ad2e0 0x55d0d19af6f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea670df87e2b5a0 server_cookie=5ce8ad3debea33d2 in_seq=0 out_seq=0 2023-06-23T18:33:36.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.984+0000 7f8daaff5700 10 client.26164 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:36.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8daaff5700 1 -- 172.21.15.2:0/285532902 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d0d18e1eb0 con 0x55d0d19ad2e0 2023-06-23T18:33:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8daaff5700 10 client.26164 handle_command_reply: tid=0 2023-06-23T18:33:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 client.26164 shutdown 2023-06-23T18:33:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 -- 172.21.15.2:0/285532902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 msgr2=0x55d0d19a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d19a1780 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8db8015000 tx=0x7f8db801a050 comp rx=0 tx=0).stop 2023-06-23T18:33:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 -- 172.21.15.2:0/285532902 shutdown_connections 2023-06-23T18:33:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d0d19ad2e0 0x55d0d19af6f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8daaff5700 0 client.26164 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d19a1780 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/285532902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d19a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 -- 172.21.15.2:0/285532902 >> 172.21.15.2:0/285532902 conn(0x55d0d198d3f0 msgr2=0x55d0d1991e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 -- 172.21.15.2:0/285532902 shutdown_connections 2023-06-23T18:33:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 1 -- 172.21.15.2:0/285532902 wait complete. 2023-06-23T18:33:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:36.985+0000 7f8dd9986b80 20 client.26164 trim_cache size 0 max 16384 2023-06-23T18:33:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.005+0000 7f8dd9986b80 1 Processor -- start 2023-06-23T18:33:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.005+0000 7f8dd9986b80 1 -- start start 2023-06-23T18:33:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1993460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d19a8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d19a2970 con 0x55d0d1999520 2023-06-23T18:33:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d19a0a80 con 0x55d0d1998b70 2023-06-23T18:33:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d19a0c30 con 0x55d0d1997d20 2023-06-23T18:33:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56588/0 (socket says 172.21.15.2:56588) 2023-06-23T18:33:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc77fe700 1 -- 172.21.15.2:0/3987747489 learned_addr learned my addr 172.21.15.2:0/3987747489 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1993460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc67fc700 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d19a8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc6ffd700 1 -- 172.21.15.2:0/3987747489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 msgr2=0x55d0d19a8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d19a8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc6ffd700 1 -- 172.21.15.2:0/3987747489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 msgr2=0x55d0d199c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc6ffd700 1 -- 172.21.15.2:0/3987747489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0d16ffc60 con 0x55d0d1997d20 2023-06-23T18:33:37.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.006+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1993460 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8db8003f60 tx=0x7f8db8009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57ec6ebe6704df5a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dadffb700 1 -- 172.21.15.2:0/3987747489 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db800f4e0 con 0x55d0d1997d20 2023-06-23T18:33:37.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dadffb700 1 -- 172.21.15.2:0/3987747489 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8db802a050 con 0x55d0d1997d20 2023-06-23T18:33:37.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dadffb700 1 -- 172.21.15.2:0/3987747489 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8db80049a0 con 0x55d0d1997d20 2023-06-23T18:33:37.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3987747489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 msgr2=0x55d0d1993460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1993460 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8db8003f60 tx=0x7f8db8009910 comp rx=0 tx=0).stop 2023-06-23T18:33:37.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3987747489 shutdown_connections 2023-06-23T18:33:37.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d199c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1997d20 0x55d0d1993460 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3987747489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1998b70 0x55d0d19a8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3987747489 >> 172.21.15.2:0/3987747489 conn(0x55d0d198d3f0 msgr2=0x55d0d19a95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3987747489 shutdown_connections 2023-06-23T18:33:37.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.007+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3987747489 wait complete. 2023-06-23T18:33:37.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 1 -- start start 2023-06-23T18:33:37.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 1 Processor -- start 2023-06-23T18:33:37.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1997d20 0x55d0d19a6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1998b70 0x55d0d19a6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d19a83b0 con 0x55d0d1999520 2023-06-23T18:33:37.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d19a8520 con 0x55d0d1997d20 2023-06-23T18:33:37.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0d19a86e0 con 0x55d0d1998b70 2023-06-23T18:33:37.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dd9986b80 10 client.0 _ll_register_callbacks cb 0x55d0d188e260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:37.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1997d20 0x55d0d19a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dc67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1998b70 0x55d0d19a6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dc77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56594/0 (socket says 172.21.15.2:56594) 2023-06-23T18:33:37.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.008+0000 7f8dc77fe700 1 -- 172.21.15.2:0/3655503579 learned_addr learned my addr 172.21.15.2:0/3655503579 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dc77fe700 1 -- 172.21.15.2:0/3655503579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1998b70 msgr2=0x55d0d19a6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dc77fe700 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1998b70 0x55d0d19a6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dc77fe700 1 -- 172.21.15.2:0/3655503579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1997d20 msgr2=0x55d0d19a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dc77fe700 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1997d20 0x55d0d19a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dc77fe700 1 -- 172.21.15.2:0/3655503579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0d1716de0 con 0x55d0d1999520 2023-06-23T18:33:37.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dc67fc700 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1998b70 0x55d0d19a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:37.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1997d20 0x55d0d19a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:37.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dc77fe700 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a7e70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00108f0 tx=0x7f8dc0009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdeca7b5803e3190 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8daaff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8daaff5700 1 -- 172.21.15.2:0/3655503579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc001f040 con 0x55d0d1999520 2023-06-23T18:33:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8daaff5700 1 -- 172.21.15.2:0/3655503579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8dc0004be0 con 0x55d0d1999520 2023-06-23T18:33:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3655503579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dc0016040 con 0x55d0d1999520 2023-06-23T18:33:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8daaff5700 1 -- 172.21.15.2:0/3655503579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dc0004da0 con 0x55d0d1999520 2023-06-23T18:33:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dd9986b80 10 client.16449 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.009+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3655503579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d0d173d250 con 0x55d0d1999520 2023-06-23T18:33:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8daaff5700 1 -- 172.21.15.2:0/3655503579 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8dc0016040 con 0x55d0d1999520 2023-06-23T18:33:37.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8daaff5700 1 -- 172.21.15.2:0/3655503579 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8dc001b510 con 0x55d0d1999520 2023-06-23T18:33:37.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8dd9986b80 10 client.16449 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:37.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8dd9986b80 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:37.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8dd9986b80 20 client.16449 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:37.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d0d1a02030 0x55d0d1a04440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8dd9986b80 4 client.16449 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:33:37.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3655503579 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d0d18a4050 con 0x55d0d1a02030 2023-06-23T18:33:37.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d0d1a02030 0x55d0d1a04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8daaff5700 10 client.16449 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:37.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.010+0000 7f8dc6ffd700 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d0d1a02030 0x55d0d1a04440 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d55c3cfde1c2a server_cookie=7495af4acbf9cc83 in_seq=0 out_seq=0 2023-06-23T18:33:37.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8daaff5700 1 -- 172.21.15.2:0/3655503579 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d0d18a4050 con 0x55d0d1a02030 2023-06-23T18:33:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8daaff5700 10 client.16449 handle_command_reply: tid=0 2023-06-23T18:33:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 client.16449 shutdown 2023-06-23T18:33:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3655503579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 msgr2=0x55d0d19a7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a7e70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8dc00108f0 tx=0x7f8dc0009510 comp rx=0 tx=0).stop 2023-06-23T18:33:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3655503579 shutdown_connections 2023-06-23T18:33:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d0d1a02030 0x55d0d1a04440 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d0d1999520 0x55d0d19a7e70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d0d1997d20 0x55d0d19a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8daaff5700 0 client.16449 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 --2- 172.21.15.2:0/3655503579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0d1998b70 0x55d0d19a6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3655503579 >> 172.21.15.2:0/3655503579 conn(0x55d0d198d3f0 msgr2=0x55d0d1993270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3655503579 shutdown_connections 2023-06-23T18:33:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 1 -- 172.21.15.2:0/3655503579 wait complete. 2023-06-23T18:33:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.011+0000 7f8dd9986b80 20 client.16449 trim_cache size 0 max 16384 2023-06-23T18:33:37.026 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:33:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:33:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:33:37.026 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:33:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 -- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8db00a1150 msgr2=0x7f8db00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8db00a1150 0x7f8db00a3600 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8db80098e0 tx=0x7f8db800b260 comp rx=0 tx=0).stop 2023-06-23T18:33:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 -- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80ff9e0 msgr2=0x7f8dc81949d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80ff9e0 0x7f8dc81949d0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00b890 tx=0x7f8dbc00bba0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 -- 172.21.15.2:0/2759330336 shutdown_connections 2023-06-23T18:33:37.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8dc8100370 0x7f8dc819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8db00a1150 0x7f8db00a3600 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8dc80fe7f0 0x7f8dc8194490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 --2- 172.21.15.2:0/2759330336 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8dc80ff9e0 0x7f8dc81949d0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 -- 172.21.15.2:0/2759330336 >> 172.21.15.2:0/2759330336 conn(0x7f8dc80f9df0 msgr2=0x7f8dc8102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 -- 172.21.15.2:0/2759330336 shutdown_connections 2023-06-23T18:33:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.012+0000 7f8dc7fff700 1 -- 172.21.15.2:0/2759330336 wait complete. 2023-06-23T18:33:37.030 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:33:37.030 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:33:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:36 smithi082 ceph-mon[97889]: pgmap v643: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 43 MiB/s wr, 45 op/s 2023-06-23T18:33:37.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 -- 172.21.15.2:0/3573773618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62c80ff9b0 msgr2=0x7f62c80ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 --2- 172.21.15.2:0/3573773618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62c80ff9b0 0x7f62c80ffe00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009990 tx=0x7f62bc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 -- 172.21.15.2:0/3573773618 shutdown_connections 2023-06-23T18:33:37.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 --2- 172.21.15.2:0/3573773618 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c8100340 0x7f62c8106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 --2- 172.21.15.2:0/3573773618 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62c80fe7c0 0x7f62c80febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 --2- 172.21.15.2:0/3573773618 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62c80ff9b0 0x7f62c80ffe00 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 -- 172.21.15.2:0/3573773618 >> 172.21.15.2:0/3573773618 conn(0x7f62c80f9e00 msgr2=0x7f62c80fc210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 -- 172.21.15.2:0/3573773618 shutdown_connections 2023-06-23T18:33:37.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.177+0000 7f62cd39d700 1 -- 172.21.15.2:0/3573773618 wait complete. 2023-06-23T18:33:37.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62cd39d700 1 Processor -- start 2023-06-23T18:33:37.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62cd39d700 1 -- start start 2023-06-23T18:33:37.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62cd39d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62c80fe7c0 0x7f62c8194440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62cd39d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c80ff9b0 0x7f62c8194980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62cd39d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62c8100340 0x7f62c819b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62cd39d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c819bfc0 con 0x7f62c8100340 2023-06-23T18:33:37.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62cd39d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c819c100 con 0x7f62c80ff9b0 2023-06-23T18:33:37.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62cd39d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c819c290 con 0x7f62c80fe7c0 2023-06-23T18:33:37.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62c77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c80ff9b0 0x7f62c8194980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62c77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c80ff9b0 0x7f62c8194980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47198/0 (socket says 172.21.15.2:47198) 2023-06-23T18:33:37.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62c77fe700 1 -- 172.21.15.2:0/2176980916 learned_addr learned my addr 172.21.15.2:0/2176980916 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62ccb9c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62c8100340 0x7f62c819b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.178+0000 7f62c7fff700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62c80fe7c0 0x7f62c8194440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c77fe700 1 -- 172.21.15.2:0/2176980916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62c80fe7c0 msgr2=0x7f62c8194440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c77fe700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62c80fe7c0 0x7f62c8194440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c77fe700 1 -- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62c8100340 msgr2=0x7f62c819b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c77fe700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62c8100340 0x7f62c819b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c77fe700 1 -- 172.21.15.2:0/2176980916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc0095f0 con 0x7f62c80ff9b0 2023-06-23T18:33:37.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c77fe700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c80ff9b0 0x7f62c8194980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62bc005f50 tx=0x7f62bc0047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c57fa700 1 -- 172.21.15.2:0/2176980916 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc01d070 con 0x7f62c80ff9b0 2023-06-23T18:33:37.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c57fa700 1 -- 172.21.15.2:0/2176980916 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62bc00be50 con 0x7f62c80ff9b0 2023-06-23T18:33:37.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c819c510 con 0x7f62c80ff9b0 2023-06-23T18:33:37.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62c57fa700 1 -- 172.21.15.2:0/2176980916 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc005320 con 0x7f62c80ff9b0 2023-06-23T18:33:37.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.179+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c819c930 con 0x7f62c80ff9b0 2023-06-23T18:33:37.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.180+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c81057a0 con 0x7f62c80ff9b0 2023-06-23T18:33:37.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.181+0000 7f62c57fa700 1 -- 172.21.15.2:0/2176980916 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f62bc004a80 con 0x7f62c80ff9b0 2023-06-23T18:33:37.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.182+0000 7f62c57fa700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f62b00a1150 0x7f62b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.182+0000 7f62c7fff700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f62b00a1150 0x7f62b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.182+0000 7f62c57fa700 1 -- 172.21.15.2:0/2176980916 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f62bc0cde60 con 0x7f62c80ff9b0 2023-06-23T18:33:37.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.183+0000 7f62c7fff700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f62b00a1150 0x7f62b00a3600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f62c80ff810 tx=0x7f62b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:36 smithi017 ceph-mon[97472]: pgmap v643: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 43 MiB/s wr, 45 op/s 2023-06-23T18:33:37.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.185+0000 7f62c57fa700 1 -- 172.21.15.2:0/2176980916 <== mon.2 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f62bc097640 con 0x7f62c80ff9b0 2023-06-23T18:33:37.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:36 smithi002 ceph-mon[95947]: pgmap v643: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 43 MiB/s wr, 45 op/s 2023-06-23T18:33:37.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.460+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62c802c810 con 0x7f62c80ff9b0 2023-06-23T18:33:37.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.461+0000 7f62c57fa700 1 -- 172.21.15.2:0/2176980916 <== mon.2 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10783 (secure 0 0 0) 0x7f62bc09ac60 con 0x7f62c80ff9b0 2023-06-23T18:33:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:33:37.466 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:33:28.815886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:33:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f62b00a1150 msgr2=0x7f62b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f62b00a1150 0x7f62b00a3600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f62c80ff810 tx=0x7f62b8009260 comp rx=0 tx=0).stop 2023-06-23T18:33:37.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c80ff9b0 msgr2=0x7f62c8194980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c80ff9b0 0x7f62c8194980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f62bc005f50 tx=0x7f62bc0047a0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 shutdown_connections 2023-06-23T18:33:37.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62c8100340 0x7f62c819b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62c80fe7c0 0x7f62c8194440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f62b00a1150 0x7f62b00a3600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 --2- 172.21.15.2:0/2176980916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62c80ff9b0 0x7f62c8194980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 >> 172.21.15.2:0/2176980916 conn(0x7f62c80f9e00 msgr2=0x7f62c80fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 shutdown_connections 2023-06-23T18:33:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.463+0000 7f62cd39d700 1 -- 172.21.15.2:0/2176980916 wait complete. 2023-06-23T18:33:37.470 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 38 2023-06-23T18:33:37.481 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:33:37.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/2959931043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a40fea80 msgr2=0x7fb2a40fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/2959931043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a40fea80 0x7fb2a40fee90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb294009990 tx=0x7fb294009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/2959931043 shutdown_connections 2023-06-23T18:33:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/2959931043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a4100600 0x7fb2a4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/2959931043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a40fea80 0x7fb2a40fee90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/2959931043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a40ffc70 0x7fb2a41000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/2959931043 >> 172.21.15.2:0/2959931043 conn(0x7fb2a40fa0e0 msgr2=0x7fb2a40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/2959931043 shutdown_connections 2023-06-23T18:33:37.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.618+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/2959931043 wait complete. 2023-06-23T18:33:37.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a8c1e700 1 Processor -- start 2023-06-23T18:33:37.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a8c1e700 1 -- start start 2023-06-23T18:33:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a8c1e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a40fea80 0x7fb2a41957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a8c1e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a40ffc70 0x7fb2a4195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a8c1e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a4100600 0x7fb2a419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a8c1e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419d320 con 0x7fb2a4100600 2023-06-23T18:33:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a8c1e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419d460 con 0x7fb2a40ffc70 2023-06-23T18:33:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a8c1e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419d5f0 con 0x7fb2a40fea80 2023-06-23T18:33:37.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.619+0000 7fb2a2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a4100600 0x7fb2a419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a4100600 0x7fb2a419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56622/0 (socket says 172.21.15.2:56622) 2023-06-23T18:33:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a2d9d700 1 -- 172.21.15.2:0/997925719 learned_addr learned my addr 172.21.15.2:0/997925719 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a259c700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a40fea80 0x7fb2a41957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a40ffc70 0x7fb2a4195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a2d9d700 1 -- 172.21.15.2:0/997925719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a40fea80 msgr2=0x7fb2a41957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a2d9d700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a40fea80 0x7fb2a41957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a2d9d700 1 -- 172.21.15.2:0/997925719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a40ffc70 msgr2=0x7fb2a4195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a2d9d700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a40ffc70 0x7fb2a4195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a2d9d700 1 -- 172.21.15.2:0/997925719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2940095f0 con 0x7fb2a4100600 2023-06-23T18:33:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a2d9d700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a4100600 0x7fb2a419cd50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb29c007d70 tx=0x7fb29c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2937fe700 1 -- 172.21.15.2:0/997925719 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00ac90 con 0x7fb2a4100600 2023-06-23T18:33:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/997925719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a419d8d0 con 0x7fb2a4100600 2023-06-23T18:33:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2937fe700 1 -- 172.21.15.2:0/997925719 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb29c00adf0 con 0x7fb2a4100600 2023-06-23T18:33:37.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2937fe700 1 -- 172.21.15.2:0/997925719 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c0115e0 con 0x7fb2a4100600 2023-06-23T18:33:37.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.620+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/997925719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a419dde0 con 0x7fb2a4100600 2023-06-23T18:33:37.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.623+0000 7fb2937fe700 1 -- 172.21.15.2:0/997925719 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb29c012420 con 0x7fb2a4100600 2023-06-23T18:33:37.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.624+0000 7fb2937fe700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb28c0a1150 0x7fb28c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.624+0000 7fb2a259c700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb28c0a1150 0x7fb28c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.624+0000 7fb2937fe700 1 -- 172.21.15.2:0/997925719 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb29c0ccae0 con 0x7fb2a4100600 2023-06-23T18:33:37.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.624+0000 7fb2b4da3b80 1 Processor -- start 2023-06-23T18:33:37.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2b4da3b80 1 -- start start 2023-06-23T18:33:37.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cdc80 0x55b2187ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cf000 0x55b2187cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2186d2cb0 con 0x55b2187ce5d0 2023-06-23T18:33:37.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b218725a40 con 0x55b2187cf000 2023-06-23T18:33:37.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b21871eed0 con 0x55b2187cdc80 2023-06-23T18:33:37.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2a259c700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb28c0a1150 0x7fb28c0a3600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb2940098e0 tx=0x7fb29400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56626/0 (socket says 172.21.15.2:56626) 2023-06-23T18:33:37.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2a1d9b700 1 -- 172.21.15.2:0/456111359 learned_addr learned my addr 172.21.15.2:0/456111359 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.625+0000 7fb2a2d9d700 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cf000 0x55b2187cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb2a259c700 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cdc80 0x55b2187ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb2a1d9b700 1 -- 172.21.15.2:0/456111359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cdc80 msgr2=0x55b2187ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cdc80 0x55b2187ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb2a1d9b700 1 -- 172.21.15.2:0/456111359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cf000 msgr2=0x55b2187cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cf000 0x55b2187cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb2a1d9b700 1 -- 172.21.15.2:0/456111359 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2185a0630 con 0x55b2187ce5d0 2023-06-23T18:33:37.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb2a259c700 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cdc80 0x55b2187ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:37.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187cea70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb298009d80 tx=0x7fb298011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee9c4350efb7fec server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb290ff9700 1 -- 172.21.15.2:0/456111359 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298004230 con 0x55b2187ce5d0 2023-06-23T18:33:37.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb290ff9700 1 -- 172.21.15.2:0/456111359 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb298004390 con 0x55b2187ce5d0 2023-06-23T18:33:37.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.626+0000 7fb290ff9700 1 -- 172.21.15.2:0/456111359 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2980197e0 con 0x55b2187ce5d0 2023-06-23T18:33:37.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/456111359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 msgr2=0x55b2187cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187cea70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb298009d80 tx=0x7fb298011720 comp rx=0 tx=0).stop 2023-06-23T18:33:37.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/456111359 shutdown_connections 2023-06-23T18:33:37.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cf000 0x55b2187cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cdc80 0x55b2187ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/456111359 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187cea70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/456111359 >> 172.21.15.2:0/456111359 conn(0x55b2187c33f0 msgr2=0x55b2187c8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/456111359 shutdown_connections 2023-06-23T18:33:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.627+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/456111359 wait complete. 2023-06-23T18:33:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 1 -- start start 2023-06-23T18:33:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 1 Processor -- start 2023-06-23T18:33:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cdc80 0x55b2187d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187ce5d0 0x55b2187d7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2186beac0 con 0x55b2187cdc80 2023-06-23T18:33:37.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2186be7c0 con 0x55b2187ce5d0 2023-06-23T18:33:37.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2185a2920 con 0x55b2187cf000 2023-06-23T18:33:37.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2b4da3b80 10 client.0 _ll_register_callbacks cb 0x55b2186c4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:37.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.628+0000 7fb2a259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cdc80 0x55b2187d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cdc80 0x55b2187d74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56632/0 (socket says 172.21.15.2:56632) 2023-06-23T18:33:37.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a259c700 1 -- 172.21.15.2:0/1852846367 learned_addr learned my addr 172.21.15.2:0/1852846367 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a2d9d700 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a259c700 1 -- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 msgr2=0x55b2187dea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187dea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a259c700 1 -- 172.21.15.2:0/1852846367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187ce5d0 msgr2=0x55b2187d7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187ce5d0 0x55b2187d7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a259c700 1 -- 172.21.15.2:0/1852846367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2980099e0 con 0x55b2187cdc80 2023-06-23T18:33:37.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a2d9d700 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:37.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cdc80 0x55b2187d74d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40ffad0 tx=0x7fb29401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffacd7215a263a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb289ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb289ffb700 1 -- 172.21.15.2:0/1852846367 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294004500 con 0x55b2187cdc80 2023-06-23T18:33:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb289ffb700 1 -- 172.21.15.2:0/1852846367 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb294004660 con 0x55b2187cdc80 2023-06-23T18:33:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1852846367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2187c0bb0 con 0x55b2187cdc80 2023-06-23T18:33:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.629+0000 7fb289ffb700 1 -- 172.21.15.2:0/1852846367 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb294004820 con 0x55b2187cdc80 2023-06-23T18:33:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2b4da3b80 10 client.16467 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:37.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1852846367 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b21854cde0 con 0x55b2187cdc80 2023-06-23T18:33:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb289ffb700 1 -- 172.21.15.2:0/1852846367 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b2187c0bb0 con 0x55b2187cdc80 2023-06-23T18:33:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb289ffb700 1 -- 172.21.15.2:0/1852846367 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb29400f8b0 con 0x55b2187cdc80 2023-06-23T18:33:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2b4da3b80 10 client.16467 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2b4da3b80 10 client.16467 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:33:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2b4da3b80 20 client.16467 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55b2187e7560 0x55b2187e9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2b4da3b80 4 client.16467 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:37.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1852846367 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b218717eb0 con 0x55b2187e7560 2023-06-23T18:33:37.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.630+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55b2187e7560 0x55b2187e9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.631+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55b2187e7560 0x55b2187e9970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f60a292052264e0 server_cookie=eea82d420bc9a204 in_seq=0 out_seq=0 2023-06-23T18:33:37.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.631+0000 7fb289ffb700 10 client.16467 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:33:37.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.632+0000 7fb289ffb700 1 -- 172.21.15.2:0/1852846367 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b218717eb0 con 0x55b2187e7560 2023-06-23T18:33:37.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.632+0000 7fb289ffb700 10 client.16467 handle_command_reply: tid=0 2023-06-23T18:33:37.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.632+0000 7fb2b4da3b80 1 client.16467 shutdown 2023-06-23T18:33:37.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1852846367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cdc80 msgr2=0x55b2187d74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cdc80 0x55b2187d74d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40ffad0 tx=0x7fb29401a050 comp rx=0 tx=0).stop 2023-06-23T18:33:37.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1852846367 shutdown_connections 2023-06-23T18:33:37.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55b2187e7560 0x55b2187e9970 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb289ffb700 0 client.16467 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:33:37.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cdc80 0x55b2187d74d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1852846367 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187ce5d0 0x55b2187d7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1852846367 >> 172.21.15.2:0/1852846367 conn(0x55b2187c33f0 msgr2=0x55b2187c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1852846367 shutdown_connections 2023-06-23T18:33:37.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1852846367 wait complete. 2023-06-23T18:33:37.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.633+0000 7fb2b4da3b80 20 client.16467 trim_cache size 0 max 16384 2023-06-23T18:33:37.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.655+0000 7fb2b4da3b80 1 Processor -- start 2023-06-23T18:33:37.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.655+0000 7fb2b4da3b80 1 -- start start 2023-06-23T18:33:37.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.655+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.655+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187ce5d0 0x55b2187dea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.655+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cf000 0x55b2187d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.655+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2187c9070 con 0x55b2187cf000 2023-06-23T18:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2187d5110 con 0x55b2187cdc80 2023-06-23T18:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2187d2a00 con 0x55b2187ce5d0 2023-06-23T18:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cf000 0x55b2187d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cf000 0x55b2187d5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56644/0 (socket says 172.21.15.2:56644) 2023-06-23T18:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a2d9d700 1 -- 172.21.15.2:0/1296481270 learned_addr learned my addr 172.21.15.2:0/1296481270 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187e36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187ce5d0 0x55b2187dea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a259c700 1 -- 172.21.15.2:0/1296481270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187ce5d0 msgr2=0x55b2187dea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187ce5d0 0x55b2187dea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a259c700 1 -- 172.21.15.2:0/1296481270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cf000 msgr2=0x55b2187d5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cf000 0x55b2187d5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a259c700 1 -- 172.21.15.2:0/1296481270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b218535c60 con 0x55b2187cdc80 2023-06-23T18:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a2d9d700 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cf000 0x55b2187d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187e36f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb29400fc90 tx=0x7fb2940092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b96fb24dc28a3ff3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.656+0000 7fb2917fa700 1 -- 172.21.15.2:0/1296481270 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400b990 con 0x55b2187cdc80 2023-06-23T18:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2917fa700 1 -- 172.21.15.2:0/1296481270 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb29402a050 con 0x55b2187cdc80 2023-06-23T18:33:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2917fa700 1 -- 172.21.15.2:0/1296481270 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294024b90 con 0x55b2187cdc80 2023-06-23T18:33:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1296481270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 msgr2=0x55b2187e36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187e36f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb29400fc90 tx=0x7fb2940092d0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1296481270 shutdown_connections 2023-06-23T18:33:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187cf000 0x55b2187d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187e36f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1296481270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187ce5d0 0x55b2187dea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1296481270 >> 172.21.15.2:0/1296481270 conn(0x55b2187c33f0 msgr2=0x55b2187ee2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1296481270 shutdown_connections 2023-06-23T18:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.657+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1296481270 wait complete. 2023-06-23T18:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 1 -- start start 2023-06-23T18:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 1 Processor -- start 2023-06-23T18:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2187de240 con 0x55b2187ce5d0 2023-06-23T18:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2187de3b0 con 0x55b2187cdc80 2023-06-23T18:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2187de570 con 0x55b2187cf000 2023-06-23T18:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2b4da3b80 10 client.0 _ll_register_callbacks cb 0x55b2186c4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187dace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56648/0 (socket says 172.21.15.2:56648) 2023-06-23T18:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a1d9b700 1 -- 172.21.15.2:0/1012237690 learned_addr learned my addr 172.21.15.2:0/1012237690 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a2d9d700 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a1d9b700 1 -- 172.21.15.2:0/1012237690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 msgr2=0x55b2187ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a1d9b700 1 -- 172.21.15.2:0/1012237690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 msgr2=0x55b2187da7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.658+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187da7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb2a1d9b700 1 -- 172.21.15.2:0/1012237690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a419dde0 con 0x55b2187ce5d0 2023-06-23T18:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb2a1d9b700 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187dace0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb29801b040 tx=0x7fb298004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c9d0cc702f5bb1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb289ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb289ffb700 1 -- 172.21.15.2:0/1012237690 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298029330 con 0x55b2187ce5d0 2023-06-23T18:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1012237690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb298004390 con 0x55b2187ce5d0 2023-06-23T18:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb289ffb700 1 -- 172.21.15.2:0/1012237690 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb298029e20 con 0x55b2187ce5d0 2023-06-23T18:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb289ffb700 1 -- 172.21.15.2:0/1012237690 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb298029490 con 0x55b2187ce5d0 2023-06-23T18:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb2b4da3b80 10 client.16473 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1012237690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55b21854cde0 con 0x55b2187ce5d0 2023-06-23T18:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.659+0000 7fb289ffb700 1 -- 172.21.15.2:0/1012237690 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb298004390 con 0x55b2187ce5d0 2023-06-23T18:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb289ffb700 1 -- 172.21.15.2:0/1012237690 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb298011440 con 0x55b2187ce5d0 2023-06-23T18:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2b4da3b80 10 client.16473 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2b4da3b80 10 client.16473 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2b4da3b80 20 client.16473 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55b21883c330 0x55b21883e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2b4da3b80 4 client.16473 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:33:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1012237690 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b2186da050 con 0x55b21883c330 2023-06-23T18:33:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55b21883c330 0x55b21883e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2a259c700 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55b21883c330 0x55b21883e740 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c0391d53ca878a4c server_cookie=d19aa55fb9571398 in_seq=0 out_seq=0 2023-06-23T18:33:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb289ffb700 10 client.16473 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:33:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb289ffb700 1 -- 172.21.15.2:0/1012237690 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b2186da050 con 0x55b21883c330 2023-06-23T18:33:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb289ffb700 10 client.16473 handle_command_reply: tid=0 2023-06-23T18:33:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.660+0000 7fb2b4da3b80 1 client.16473 shutdown 2023-06-23T18:33:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1012237690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 msgr2=0x55b2187dace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187dace0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb29801b040 tx=0x7fb298004130 comp rx=0 tx=0).stop 2023-06-23T18:33:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1012237690 shutdown_connections 2023-06-23T18:33:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55b21883c330 0x55b21883e740 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b2187cf000 0x55b2187ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b2187cdc80 0x55b2187da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 --2- 172.21.15.2:0/1012237690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55b2187ce5d0 0x55b2187dace0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb289ffb700 0 client.16473 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:33:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1012237690 >> 172.21.15.2:0/1012237690 conn(0x55b2187c33f0 msgr2=0x55b2187d5360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1012237690 shutdown_connections 2023-06-23T18:33:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 1 -- 172.21.15.2:0/1012237690 wait complete. 2023-06-23T18:33:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2b4da3b80 20 client.16473 trim_cache size 0 max 16384 2023-06-23T18:33:37.676 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:33:37.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb28c0a1150 msgr2=0x7fb28c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb28c0a1150 0x7fb28c0a3600 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb2940098e0 tx=0x7fb29400b260 comp rx=0 tx=0).stop 2023-06-23T18:33:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a4100600 msgr2=0x7fb2a419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a4100600 0x7fb2a419cd50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb29c007d70 tx=0x7fb29c00a340 comp rx=0 tx=0).stop 2023-06-23T18:33:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/997925719 shutdown_connections 2023-06-23T18:33:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb2a4100600 0x7fb2a419cd50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2a40fea80 0x7fb2a41957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb28c0a1150 0x7fb28c0a3600 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 --2- 172.21.15.2:0/997925719 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2a40ffc70 0x7fb2a4195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.661+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/997925719 >> 172.21.15.2:0/997925719 conn(0x7fb2a40fa0e0 msgr2=0x7fb2a40fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.662+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/997925719 shutdown_connections 2023-06-23T18:33:37.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.662+0000 7fb2a8c1e700 1 -- 172.21.15.2:0/997925719 wait complete. 2023-06-23T18:33:37.680 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:33:37.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 -- 172.21.15.2:0/315884903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80ffc70 msgr2=0x7f7ca81000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 --2- 172.21.15.2:0/315884903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80ffc70 0x7f7ca81000c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009990 tx=0x7f7c98009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 -- 172.21.15.2:0/315884903 shutdown_connections 2023-06-23T18:33:37.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 --2- 172.21.15.2:0/315884903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8100600 0x7f7ca8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 --2- 172.21.15.2:0/315884903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ca80fea80 0x7f7ca80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 --2- 172.21.15.2:0/315884903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80ffc70 0x7f7ca81000c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 -- 172.21.15.2:0/315884903 >> 172.21.15.2:0/315884903 conn(0x7f7ca80fa0e0 msgr2=0x7f7ca80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 -- 172.21.15.2:0/315884903 shutdown_connections 2023-06-23T18:33:37.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.822+0000 7f7cafa20700 1 -- 172.21.15.2:0/315884903 wait complete. 2023-06-23T18:33:37.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cafa20700 1 Processor -- start 2023-06-23T18:33:37.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cafa20700 1 -- start start 2023-06-23T18:33:37.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cafa20700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80fea80 0x7f7ca8068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cafa20700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ca80ffc70 0x7f7ca819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cafa20700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8100600 0x7f7ca819cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cafa20700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819d2c0 con 0x7f7ca80fea80 2023-06-23T18:33:37.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cafa20700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819d430 con 0x7f7ca80ffc70 2023-06-23T18:33:37.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cafa20700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca819d5f0 con 0x7f7ca8100600 2023-06-23T18:33:37.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80fea80 0x7f7ca8068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80fea80 0x7f7ca8068310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56660/0 (socket says 172.21.15.2:56660) 2023-06-23T18:33:37.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 -- 172.21.15.2:0/2378153295 learned_addr learned my addr 172.21.15.2:0/2378153295 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ca80ffc70 0x7f7ca819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8100600 0x7f7ca819cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 -- 172.21.15.2:0/2378153295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8100600 msgr2=0x7f7ca819cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8100600 0x7f7ca819cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 -- 172.21.15.2:0/2378153295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ca80ffc70 msgr2=0x7f7ca819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ca80ffc70 0x7f7ca819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 -- 172.21.15.2:0/2378153295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c980095f0 con 0x7f7ca80fea80 2023-06-23T18:33:37.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8100600 0x7f7ca819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:37.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.823+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80fea80 0x7f7ca8068310 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400d980 tx=0x7f7ca400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.824+0000 7f7c9e7fc700 1 -- 172.21.15.2:0/2378153295 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4004020 con 0x7f7ca80fea80 2023-06-23T18:33:37.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.824+0000 7f7c9e7fc700 1 -- 172.21.15.2:0/2378153295 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca4009730 con 0x7f7ca80fea80 2023-06-23T18:33:37.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.824+0000 7f7cafa20700 1 -- 172.21.15.2:0/2378153295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca819d8d0 con 0x7f7ca80fea80 2023-06-23T18:33:37.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.824+0000 7f7c9e7fc700 1 -- 172.21.15.2:0/2378153295 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4003a00 con 0x7f7ca80fea80 2023-06-23T18:33:37.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.824+0000 7f7cafa20700 1 -- 172.21.15.2:0/2378153295 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca819de30 con 0x7f7ca80fea80 2023-06-23T18:33:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.826+0000 7f7c9e7fc700 1 -- 172.21.15.2:0/2378153295 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7ca40098a0 con 0x7f7ca80fea80 2023-06-23T18:33:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.827+0000 7f7c9e7fc700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c940a11d0 0x7f7c940a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.827+0000 7f7c9e7fc700 1 -- 172.21.15.2:0/2378153295 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ca40cc580 con 0x7f7ca80fea80 2023-06-23T18:33:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.827+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c940a11d0 0x7f7c940a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c940a11d0 0x7f7c940a3680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c010 tx=0x7f7c98005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cbbba5b80 1 Processor -- start 2023-06-23T18:33:37.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cbbba5b80 1 -- start start 2023-06-23T18:33:37.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0e91a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0e92470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0d95cb0 con 0x55c3c0e90c80 2023-06-23T18:33:37.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0de8a40 con 0x55c3c0e92000 2023-06-23T18:33:37.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0de1ed0 con 0x55c3c0e915d0 2023-06-23T18:33:37.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cad7bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cad7bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e91090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56668/0 (socket says 172.21.15.2:56668) 2023-06-23T18:33:37.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cad7bc700 1 -- 172.21.15.2:0/1406758861 learned_addr learned my addr 172.21.15.2:0/1406758861 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.828+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0e91a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0e92470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7cad7bc700 1 -- 172.21.15.2:0/1406758861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 msgr2=0x55c3c0e91a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0e91a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7cad7bc700 1 -- 172.21.15.2:0/1406758861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 msgr2=0x55c3c0e92470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0e92470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7cad7bc700 1 -- 172.21.15.2:0/1406758861 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c0c63630 con 0x55c3c0e90c80 2023-06-23T18:33:37.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e91090 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4096400 tx=0x7f7ca40992f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3ae7b84d9cd0a1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7c937fe700 1 -- 172.21.15.2:0/1406758861 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4009b80 con 0x55c3c0e90c80 2023-06-23T18:33:37.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7c937fe700 1 -- 172.21.15.2:0/1406758861 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca4099990 con 0x55c3c0e90c80 2023-06-23T18:33:37.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.829+0000 7f7c937fe700 1 -- 172.21.15.2:0/1406758861 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca40990e0 con 0x55c3c0e90c80 2023-06-23T18:33:37.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/1406758861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 msgr2=0x55c3c0e91090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e91090 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4096400 tx=0x7f7ca40992f0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/1406758861 shutdown_connections 2023-06-23T18:33:37.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0e92470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e91090 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/1406758861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0e91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/1406758861 >> 172.21.15.2:0/1406758861 conn(0x55c3c0e863f0 msgr2=0x55c3c0e8b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/1406758861 shutdown_connections 2023-06-23T18:33:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/1406758861 wait complete. 2023-06-23T18:33:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 -- start start 2023-06-23T18:33:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.830+0000 7f7cbbba5b80 1 Processor -- start 2023-06-23T18:33:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e9a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0e9aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0ea1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0d81ac0 con 0x55c3c0e90c80 2023-06-23T18:33:37.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0d817c0 con 0x55c3c0e92000 2023-06-23T18:33:37.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0c65920 con 0x55c3c0e915d0 2023-06-23T18:33:37.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cad7bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e9a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cbbba5b80 10 client.0 _ll_register_callbacks cb 0x55c3c0d87260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:37.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cad7bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e9a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56674/0 (socket says 172.21.15.2:56674) 2023-06-23T18:33:37.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cad7bc700 1 -- 172.21.15.2:0/4043692459 learned_addr learned my addr 172.21.15.2:0/4043692459 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0e9aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0ea1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cad7bc700 1 -- 172.21.15.2:0/4043692459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 msgr2=0x55c3c0e9aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0e9aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cad7bc700 1 -- 172.21.15.2:0/4043692459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 msgr2=0x55c3c0ea1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0ea1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cad7bc700 1 -- 172.21.15.2:0/4043692459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca819de30 con 0x55c3c0e90c80 2023-06-23T18:33:37.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.831+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0ea1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:37.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e9a550 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40046a0 tx=0x7f7ca40993e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6150397e2ec99e6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7c90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:37.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7c90ff9700 1 -- 172.21.15.2:0/4043692459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4099080 con 0x55c3c0e90c80 2023-06-23T18:33:37.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7c90ff9700 1 -- 172.21.15.2:0/4043692459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ca409d9c0 con 0x55c3c0e90c80 2023-06-23T18:33:37.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/4043692459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3c0e83bb0 con 0x55c3c0e90c80 2023-06-23T18:33:37.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7c90ff9700 1 -- 172.21.15.2:0/4043692459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca40a1950 con 0x55c3c0e90c80 2023-06-23T18:33:37.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 10 client.16485 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:37.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/4043692459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c3c0c0fde0 con 0x55c3c0e90c80 2023-06-23T18:33:37.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7c90ff9700 1 -- 172.21.15.2:0/4043692459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c3c0e83bb0 con 0x55c3c0e90c80 2023-06-23T18:33:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7c90ff9700 1 -- 172.21.15.2:0/4043692459 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7ca4003b60 con 0x55c3c0e90c80 2023-06-23T18:33:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 10 client.16485 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 10 client.16485 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:33:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 20 client.16485 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55c3c0ea6040 0x55c3c0ea8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 4 client.16485 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:37.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.832+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/4043692459 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3c0ddaeb0 con 0x55c3c0ea6040 2023-06-23T18:33:37.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.833+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55c3c0ea6040 0x55c3c0ea8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.833+0000 7f7c90ff9700 10 client.16485 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:33:37.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.833+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55c3c0ea6040 0x55c3c0ea8450 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dccd4893df7e29c2 server_cookie=36d2855f131ae5fe in_seq=0 out_seq=0 2023-06-23T18:33:37.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7c90ff9700 1 -- 172.21.15.2:0/4043692459 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c3c0ddaeb0 con 0x55c3c0ea6040 2023-06-23T18:33:37.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7c90ff9700 10 client.16485 handle_command_reply: tid=0 2023-06-23T18:33:37.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 client.16485 shutdown 2023-06-23T18:33:37.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/4043692459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 msgr2=0x55c3c0e9a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e9a550 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7ca40046a0 tx=0x7f7ca40993e0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/4043692459 shutdown_connections 2023-06-23T18:33:37.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55c3c0ea6040 0x55c3c0ea8450 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e92000 0x55c3c0ea1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7c90ff9700 0 client.16485 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:33:37.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e90c80 0x55c3c0e9a550 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/4043692459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0e9aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/4043692459 >> 172.21.15.2:0/4043692459 conn(0x55c3c0e863f0 msgr2=0x55c3c0e8ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/4043692459 shutdown_connections 2023-06-23T18:33:37.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/4043692459 wait complete. 2023-06-23T18:33:37.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.834+0000 7f7cbbba5b80 20 client.16485 trim_cache size 0 max 16384 2023-06-23T18:33:37.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.853+0000 7f7cbbba5b80 1 Processor -- start 2023-06-23T18:33:37.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.853+0000 7f7cbbba5b80 1 -- start start 2023-06-23T18:33:37.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.853+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e90c80 0x55c3c0e94f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.853+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.853+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e95440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.853+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0ddaeb0 con 0x55c3c0e92000 2023-06-23T18:33:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.853+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0e88470 con 0x55c3c0e90c80 2023-06-23T18:33:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.853+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0e8c0f0 con 0x55c3c0e915d0 2023-06-23T18:33:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e95440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.860 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e95440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56684/0 (socket says 172.21.15.2:56684) 2023-06-23T18:33:37.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 -- 172.21.15.2:0/2685720436 learned_addr learned my addr 172.21.15.2:0/2685720436 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e90c80 0x55c3c0e94f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 -- 172.21.15.2:0/2685720436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 msgr2=0x55c3c0ea1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 -- 172.21.15.2:0/2685720436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e90c80 msgr2=0x55c3c0e94f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e90c80 0x55c3c0e94f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 -- 172.21.15.2:0/2685720436 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c0bf8c60 con 0x55c3c0e92000 2023-06-23T18:33:37.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:37.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e95440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009c10 tx=0x7f7ca0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4255670ae0a5a96 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.854+0000 7f7c93fff700 1 -- 172.21.15.2:0/2685720436 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000f070 con 0x55c3c0e92000 2023-06-23T18:33:37.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7c93fff700 1 -- 172.21.15.2:0/2685720436 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca0007cb0 con 0x55c3c0e92000 2023-06-23T18:33:37.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7c93fff700 1 -- 172.21.15.2:0/2685720436 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00155f0 con 0x55c3c0e92000 2023-06-23T18:33:37.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/2685720436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 msgr2=0x55c3c0e95440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e95440 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0009c10 tx=0x7f7ca0009040 comp rx=0 tx=0).stop 2023-06-23T18:33:37.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/2685720436 shutdown_connections 2023-06-23T18:33:37.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e95440 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e90c80 0x55c3c0e94f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/2685720436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/2685720436 >> 172.21.15.2:0/2685720436 conn(0x55c3c0e863f0 msgr2=0x55c3c0e99200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/2685720436 shutdown_connections 2023-06-23T18:33:37.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.855+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/2685720436 wait complete. 2023-06-23T18:33:37.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 1 -- start start 2023-06-23T18:33:37.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 1 Processor -- start 2023-06-23T18:33:37.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e90c80 0x55c3c0ea15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e9e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0e9e850 con 0x55c3c0e92000 2023-06-23T18:33:37.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0e9e990 con 0x55c3c0e915d0 2023-06-23T18:33:37.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c0e9eb50 con 0x55c3c0e90c80 2023-06-23T18:33:37.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cbbba5b80 10 client.0 _ll_register_callbacks cb 0x55c3c0d87260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:37.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cadfbd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e9e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cad7bc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e90c80 0x55c3c0ea15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cadfbd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e9e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56690/0 (socket says 172.21.15.2:56690) 2023-06-23T18:33:37.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cadfbd700 1 -- 172.21.15.2:0/172502770 learned_addr learned my addr 172.21.15.2:0/172502770 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:37.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.856+0000 7f7cacfbb700 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cadfbd700 1 -- 172.21.15.2:0/172502770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e90c80 msgr2=0x55c3c0ea15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e90c80 0x55c3c0ea15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cadfbd700 1 -- 172.21.15.2:0/172502770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e915d0 msgr2=0x55c3c0ea1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:37.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cadfbd700 1 -- 172.21.15.2:0/172502770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca0009840 con 0x55c3c0e92000 2023-06-23T18:33:37.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e90c80 0x55c3c0ea15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:37.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cadfbd700 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e9e170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0016660 tx=0x7f7ca0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d054624366c6a7a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:37.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7c90ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:37.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7c90ff9700 1 -- 172.21.15.2:0/172502770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000f030 con 0x55c3c0e92000 2023-06-23T18:33:37.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7c90ff9700 1 -- 172.21.15.2:0/172502770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7ca0007e20 con 0x55c3c0e92000 2023-06-23T18:33:37.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/172502770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ca4099990 con 0x55c3c0e92000 2023-06-23T18:33:37.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7c90ff9700 1 -- 172.21.15.2:0/172502770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca00168f0 con 0x55c3c0e92000 2023-06-23T18:33:37.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cbbba5b80 10 client.16491 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:37.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.857+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/172502770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55c3c0c36250 con 0x55c3c0e92000 2023-06-23T18:33:37.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7c90ff9700 1 -- 172.21.15.2:0/172502770 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ca4099990 con 0x55c3c0e92000 2023-06-23T18:33:37.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7c90ff9700 1 -- 172.21.15.2:0/172502770 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7ca0026020 con 0x55c3c0e92000 2023-06-23T18:33:37.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7cbbba5b80 10 client.16491 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:37.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7cbbba5b80 10 client.16491 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:33:37.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7cbbba5b80 20 client.16491 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:37.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55c3c0efb600 0x55c3c0efda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:37.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7cbbba5b80 4 client.16491 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:33:37.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/172502770 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3c0d9d050 con 0x55c3c0efb600 2023-06-23T18:33:37.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.858+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55c3c0efb600 0x55c3c0efda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:37.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7c90ff9700 10 client.16491 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:33:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cad7bc700 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55c3c0efb600 0x55c3c0efda10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=47daf4add080d2f4 server_cookie=6b1c1d67592e2c88 in_seq=0 out_seq=0 2023-06-23T18:33:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7c90ff9700 1 -- 172.21.15.2:0/172502770 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3c0d9d050 con 0x55c3c0efb600 2023-06-23T18:33:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7c90ff9700 10 client.16491 handle_command_reply: tid=0 2023-06-23T18:33:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 client.16491 shutdown 2023-06-23T18:33:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/172502770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 msgr2=0x55c3c0e9e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e9e170 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7ca0016660 tx=0x7f7ca0004f90 comp rx=0 tx=0).stop 2023-06-23T18:33:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/172502770 shutdown_connections 2023-06-23T18:33:37.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55c3c0efb600 0x55c3c0efda10 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c3c0e92000 0x55c3c0e9e170 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c3c0e90c80 0x55c3c0ea15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 --2- 172.21.15.2:0/172502770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c3c0e915d0 0x55c3c0ea1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/172502770 >> 172.21.15.2:0/172502770 conn(0x55c3c0e863f0 msgr2=0x55c3c0e94f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7c90ff9700 0 client.16491 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:33:37.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/172502770 shutdown_connections 2023-06-23T18:33:37.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 1 -- 172.21.15.2:0/172502770 wait complete. 2023-06-23T18:33:37.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.859+0000 7f7cbbba5b80 20 client.16491 trim_cache size 0 max 16384 2023-06-23T18:33:37.874 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:33:37.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 -- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c940a11d0 msgr2=0x7f7c940a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c940a11d0 0x7f7c940a3680 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800c010 tx=0x7f7c98005960 comp rx=0 tx=0).stop 2023-06-23T18:33:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 -- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80fea80 msgr2=0x7f7ca8068310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80fea80 0x7f7ca8068310 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7ca400d980 tx=0x7f7ca400dc90 comp rx=0 tx=0).stop 2023-06-23T18:33:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 -- 172.21.15.2:0/2378153295 shutdown_connections 2023-06-23T18:33:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ca8100600 0x7f7ca819cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7ca80fea80 0x7f7ca8068310 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f7c940a11d0 0x7f7c940a3680 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 --2- 172.21.15.2:0/2378153295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7ca80ffc70 0x7f7ca819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:37.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 -- 172.21.15.2:0/2378153295 >> 172.21.15.2:0/2378153295 conn(0x7f7ca80fa0e0 msgr2=0x7f7ca81031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:37.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 -- 172.21.15.2:0/2378153295 shutdown_connections 2023-06-23T18:33:37.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:37.860+0000 7f7cafa20700 1 -- 172.21.15.2:0/2378153295 wait complete. 2023-06-23T18:33:37.877 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:33:38.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.011+0000 7fbfb6536700 1 -- 172.21.15.2:0/3555021380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 msgr2=0x7fbfb0106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.011+0000 7fbfb6536700 1 --2- 172.21.15.2:0/3555021380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 0x7fbfb0106f40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800b0a0 tx=0x7fbfa800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.011+0000 7fbfb6536700 1 -- 172.21.15.2:0/3555021380 shutdown_connections 2023-06-23T18:33:38.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.011+0000 7fbfb6536700 1 --2- 172.21.15.2:0/3555021380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 0x7fbfb0106f40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.011+0000 7fbfb6536700 1 --2- 172.21.15.2:0/3555021380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb00fea60 0x7fbfb00fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.011+0000 7fbfb6536700 1 --2- 172.21.15.2:0/3555021380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb0100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.011+0000 7fbfb6536700 1 -- 172.21.15.2:0/3555021380 >> 172.21.15.2:0/3555021380 conn(0x7fbfb00fa080 msgr2=0x7fbfb00fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:38.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.012+0000 7fbfb6536700 1 -- 172.21.15.2:0/3555021380 shutdown_connections 2023-06-23T18:33:38.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.012+0000 7fbfb6536700 1 -- 172.21.15.2:0/3555021380 wait complete. 2023-06-23T18:33:38.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.012+0000 7fbfb6536700 1 Processor -- start 2023-06-23T18:33:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.012+0000 7fbfb6536700 1 -- start start 2023-06-23T18:33:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb6536700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb0194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb6536700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb0194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb6536700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 0x7fbfb019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb6536700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb019c410 con 0x7fbfb01006a0 2023-06-23T18:33:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb6536700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb019c550 con 0x7fbfb00fea60 2023-06-23T18:33:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb6536700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb019c6e0 con 0x7fbfb00ffcf0 2023-06-23T18:33:38.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb4ad3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 0x7fbfb019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb4ad3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 0x7fbfb019be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56704/0 (socket says 172.21.15.2:56704) 2023-06-23T18:33:38.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb4ad3700 1 -- 172.21.15.2:0/2052767593 learned_addr learned my addr 172.21.15.2:0/2052767593 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:38.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb0194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb0194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb4ad3700 1 -- 172.21.15.2:0/2052767593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb00ffcf0 msgr2=0x7fbfb0194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb0194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb4ad3700 1 -- 172.21.15.2:0/2052767593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 msgr2=0x7fbfb0194890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb0194890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.013+0000 7fbfb4ad3700 1 -- 172.21.15.2:0/2052767593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8009d00 con 0x7fbfb01006a0 2023-06-23T18:33:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.014+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 0x7fbfb019be40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800b780 tx=0x7fbfa800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.014+0000 7fbfad7fa700 1 -- 172.21.15.2:0/2052767593 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800e040 con 0x7fbfb01006a0 2023-06-23T18:33:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.014+0000 7fbfad7fa700 1 -- 172.21.15.2:0/2052767593 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfa8004e60 con 0x7fbfb01006a0 2023-06-23T18:33:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.014+0000 7fbfb6536700 1 -- 172.21.15.2:0/2052767593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb019c960 con 0x7fbfb01006a0 2023-06-23T18:33:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.014+0000 7fbfad7fa700 1 -- 172.21.15.2:0/2052767593 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80052a0 con 0x7fbfb01006a0 2023-06-23T18:33:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.014+0000 7fbfb6536700 1 -- 172.21.15.2:0/2052767593 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb019cd90 con 0x7fbfb01006a0 2023-06-23T18:33:38.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.017+0000 7fbfad7fa700 1 -- 172.21.15.2:0/2052767593 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbfa801d470 con 0x7fbfb01006a0 2023-06-23T18:33:38.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.017+0000 7fbfad7fa700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf980a1150 0x7fbf980a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.017+0000 7fbfad7fa700 1 -- 172.21.15.2:0/2052767593 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfa80cdfc0 con 0x7fbfb01006a0 2023-06-23T18:33:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.017+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf980a1150 0x7fbf980a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfc26bbb80 1 Processor -- start 2023-06-23T18:33:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfaffff700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf980a1150 0x7fbf980a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009c60 tx=0x7fbfa0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfc26bbb80 1 -- start start 2023-06-23T18:33:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea7c30 0x562277ea8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ea8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea8fb0 0x562277ea9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277daccb0 con 0x562277ea8580 2023-06-23T18:33:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277dffa40 con 0x562277ea7c30 2023-06-23T18:33:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277df8ed0 con 0x562277ea8fb0 2023-06-23T18:33:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ea8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ea8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56710/0 (socket says 172.21.15.2:56710) 2023-06-23T18:33:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/524289610 learned_addr learned my addr 172.21.15.2:0/524289610 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.018+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea8fb0 0x562277ea9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbfaffff700 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea7c30 0x562277ea8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/524289610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea8fb0 msgr2=0x562277ea9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea8fb0 0x562277ea9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/524289610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea7c30 msgr2=0x562277ea8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea7c30 0x562277ea8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/524289610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562277c7a630 con 0x562277ea8580 2023-06-23T18:33:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea8fb0 0x562277ea9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ea8a70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400bcf0 tx=0x7fbfa40103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1058d94830def93b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbf967fc700 1 -- 172.21.15.2:0/524289610 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4010c50 con 0x562277ea8580 2023-06-23T18:33:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbf967fc700 1 -- 172.21.15.2:0/524289610 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfa4015070 con 0x562277ea8580 2023-06-23T18:33:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.019+0000 7fbf967fc700 1 -- 172.21.15.2:0/524289610 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4004e80 con 0x562277ea8580 2023-06-23T18:33:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/524289610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 msgr2=0x562277ea8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ea8a70 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400bcf0 tx=0x7fbfa40103d0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/524289610 shutdown_connections 2023-06-23T18:33:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea8fb0 0x562277ea9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea7c30 0x562277ea8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/524289610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ea8a70 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/524289610 >> 172.21.15.2:0/524289610 conn(0x562277e9d3f0 msgr2=0x562277ea2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/524289610 shutdown_connections 2023-06-23T18:33:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.020+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/524289610 wait complete. 2023-06-23T18:33:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 1 -- start start 2023-06-23T18:33:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 1 Processor -- start 2023-06-23T18:33:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277eb9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277eba240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 0x562277ec1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277d98ac0 con 0x562277ea8580 2023-06-23T18:33:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277d987c0 con 0x562277ea8fb0 2023-06-23T18:33:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277c7c920 con 0x562277ea7c30 2023-06-23T18:33:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfc26bbb80 10 client.0 _ll_register_callbacks cb 0x562277d9e260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277eba240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277eba240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56716/0 (socket says 172.21.15.2:56716) 2023-06-23T18:33:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/1905055116 learned_addr learned my addr 172.21.15.2:0/1905055116 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 0x562277ec1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.021+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277eb9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/1905055116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 msgr2=0x562277eb9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277eb9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/1905055116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 msgr2=0x562277ec1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 0x562277ec1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/1905055116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb019cd90 con 0x562277ea8580 2023-06-23T18:33:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277eb9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:33:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277eba240 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4010cb0 tx=0x7fbfa4010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b8cd3f2c76a318 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbf8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1905055116 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4020070 con 0x562277ea8580 2023-06-23T18:33:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1905055116 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfa4017070 con 0x562277ea8580 2023-06-23T18:33:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1905055116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562277e9abb0 con 0x562277ea8580 2023-06-23T18:33:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1905055116 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfa4004250 con 0x562277ea8580 2023-06-23T18:33:38.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfc26bbb80 10 client.16503 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1905055116 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fbfa400b920 con 0x562277ea8580 2023-06-23T18:33:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.022+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1905055116 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562277e9abb0 con 0x562277ea8580 2023-06-23T18:33:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1905055116 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbfa4011460 con 0x562277ea8580 2023-06-23T18:33:38.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbfc26bbb80 10 client.16503 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbfc26bbb80 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbfc26bbb80 20 client.16503 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562277ec5a00 0x562277ec7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbfc26bbb80 4 client.16503 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1905055116 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562277df1eb0 con 0x562277ec5a00 2023-06-23T18:33:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562277ec5a00 0x562277ec7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562277ec5a00 0x562277ec7e10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f0eaa961bb042d server_cookie=9d27b996c1765765 in_seq=0 out_seq=0 2023-06-23T18:33:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.023+0000 7fbf8f7fe700 10 client.16503 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:38.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.024+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1905055116 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562277df1eb0 con 0x562277ec5a00 2023-06-23T18:33:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.024+0000 7fbf8f7fe700 10 client.16503 handle_command_reply: tid=0 2023-06-23T18:33:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.024+0000 7fbfc26bbb80 1 client.16503 shutdown 2023-06-23T18:33:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.024+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1905055116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 msgr2=0x562277eba240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.024+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277eba240 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4010cb0 tx=0x7fbfa4010ce0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1905055116 shutdown_connections 2023-06-23T18:33:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562277ec5a00 0x562277ec7e10 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 0x562277ec1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277eb9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1905055116 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277eba240 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1905055116 >> 172.21.15.2:0/1905055116 conn(0x562277e9d3f0 msgr2=0x562277ea1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbf8f7fe700 0 client.16503 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1905055116 shutdown_connections 2023-06-23T18:33:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1905055116 wait complete. 2023-06-23T18:33:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.025+0000 7fbfc26bbb80 20 client.16503 trim_cache size 0 max 16384 2023-06-23T18:33:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfc26bbb80 1 Processor -- start 2023-06-23T18:33:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfc26bbb80 1 -- start start 2023-06-23T18:33:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ea9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8580 0x562277ec1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8fb0 0x562277eb9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277eb91c0 con 0x562277ea8fb0 2023-06-23T18:33:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277e9cf70 con 0x562277ea8580 2023-06-23T18:33:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277eba980 con 0x562277ea7c30 2023-06-23T18:33:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfb4ad3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8fb0 0x562277eb9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfb4ad3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8fb0 0x562277eb9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56726/0 (socket says 172.21.15.2:56726) 2023-06-23T18:33:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfb4ad3700 1 -- 172.21.15.2:0/934668946 learned_addr learned my addr 172.21.15.2:0/934668946 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8580 0x562277ec1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.044+0000 7fbfaffff700 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ea9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbfb4ad3700 1 -- 172.21.15.2:0/934668946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 msgr2=0x562277ea9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:38.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ea9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbfb4ad3700 1 -- 172.21.15.2:0/934668946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8580 msgr2=0x562277ec1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:38.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8580 0x562277ec1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbfb4ad3700 1 -- 172.21.15.2:0/934668946 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562277eb8d50 con 0x562277ea8fb0 2023-06-23T18:33:38.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8fb0 0x562277eb9cc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80971e0 tx=0x7fbfa8086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188a05a2b8c864f1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbf96ffd700 1 -- 172.21.15.2:0/934668946 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80b1070 con 0x562277ea8fb0 2023-06-23T18:33:38.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbf96ffd700 1 -- 172.21.15.2:0/934668946 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfa809afa0 con 0x562277ea8fb0 2023-06-23T18:33:38.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.045+0000 7fbf96ffd700 1 -- 172.21.15.2:0/934668946 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80860b0 con 0x562277ea8fb0 2023-06-23T18:33:38.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/934668946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8fb0 msgr2=0x562277eb9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8fb0 0x562277eb9cc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fbfa80971e0 tx=0x7fbfa8086040 comp rx=0 tx=0).stop 2023-06-23T18:33:38.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/934668946 shutdown_connections 2023-06-23T18:33:38.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8fb0 0x562277eb9cc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ea9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/934668946 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8580 0x562277ec1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/934668946 >> 172.21.15.2:0/934668946 conn(0x562277e9d3f0 msgr2=0x562277ec7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:38.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/934668946 shutdown_connections 2023-06-23T18:33:38.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/934668946 wait complete. 2023-06-23T18:33:38.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.046+0000 7fbfc26bbb80 1 -- start start 2023-06-23T18:33:38.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfc26bbb80 1 Processor -- start 2023-06-23T18:33:38.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ebeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ebeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfc26bbb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 0x562277ec0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277ec0540 con 0x562277ea8580 2023-06-23T18:33:38.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277ec06b0 con 0x562277ea8fb0 2023-06-23T18:33:38.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfc26bbb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562277ec0870 con 0x562277ea7c30 2023-06-23T18:33:38.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfc26bbb80 10 client.0 _ll_register_callbacks cb 0x562277d9e260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:38.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfaffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ebeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ebeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfaffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ebeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57066/0 (socket says 172.21.15.2:57066) 2023-06-23T18:33:38.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfaffff700 1 -- 172.21.15.2:0/1238693206 learned_addr learned my addr 172.21.15.2:0/1238693206 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:38.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfb4ad3700 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 0x562277ec0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/1238693206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 msgr2=0x562277ebeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:38.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ebeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/1238693206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 msgr2=0x562277ec0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:38.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.047+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 0x562277ec0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbfaf7fe700 1 -- 172.21.15.2:0/1238693206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb019c960 con 0x562277ea8580 2023-06-23T18:33:38.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ebeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:38.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbfaf7fe700 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ebeff0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4010410 tx=0x7fbfa4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8789529d72fad099 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:38.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbf8f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:38.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1238693206 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4020040 con 0x562277ea8580 2023-06-23T18:33:38.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1238693206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfa4015070 con 0x562277ea8580 2023-06-23T18:33:38.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1238693206 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbfa4000e40 con 0x562277ea8580 2023-06-23T18:33:38.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1238693206 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfa4004660 con 0x562277ea8580 2023-06-23T18:33:38.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbfc26bbb80 10 client.16509 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:38.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1238693206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x562277c26de0 con 0x562277ea8580 2023-06-23T18:33:38.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1238693206 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbfa4015070 con 0x562277ea8580 2023-06-23T18:33:38.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.048+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1238693206 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbfa4030460 con 0x562277ea8580 2023-06-23T18:33:38.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbfc26bbb80 10 client.16509 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:38.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbfc26bbb80 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:38.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbfc26bbb80 20 client.16509 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:38.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562277f19e60 0x562277f1c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:38.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbfc26bbb80 4 client.16509 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:33:38.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1238693206 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562277db4050 con 0x562277f19e60 2023-06-23T18:33:38.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562277f19e60 0x562277f1c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:38.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbfaffff700 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562277f19e60 0x562277f1c270 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af099d76045da6d8 server_cookie=7b90f02ae2cd346c in_seq=0 out_seq=0 2023-06-23T18:33:38.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.049+0000 7fbf8f7fe700 10 client.16509 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:38.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbf8f7fe700 1 -- 172.21.15.2:0/1238693206 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562277db4050 con 0x562277f19e60 2023-06-23T18:33:38.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbf8f7fe700 10 client.16509 handle_command_reply: tid=0 2023-06-23T18:33:38.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 client.16509 shutdown 2023-06-23T18:33:38.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1238693206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 msgr2=0x562277ebeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ebeff0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4010410 tx=0x7fbfa4011ba0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1238693206 shutdown_connections 2023-06-23T18:33:38.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562277f19e60 0x562277f1c270 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562277ea8fb0 0x562277ec0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562277ea7c30 0x562277ebeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbf8f7fe700 0 client.16509 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:38.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 --2- 172.21.15.2:0/1238693206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562277ea8580 0x562277ebeff0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1238693206 >> 172.21.15.2:0/1238693206 conn(0x562277e9d3f0 msgr2=0x562277ec1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:38.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1238693206 shutdown_connections 2023-06-23T18:33:38.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 1 -- 172.21.15.2:0/1238693206 wait complete. 2023-06-23T18:33:38.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.050+0000 7fbfc26bbb80 20 client.16509 trim_cache size 0 max 16384 2023-06-23T18:33:38.065 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:33:38.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 -- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf980a1150 msgr2=0x7fbf980a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf980a1150 0x7fbf980a3600 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009c60 tx=0x7fbfa0009260 comp rx=0 tx=0).stop 2023-06-23T18:33:38.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 -- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 msgr2=0x7fbfb019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:38.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 0x7fbfb019be40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800b780 tx=0x7fbfa800bd40 comp rx=0 tx=0).stop 2023-06-23T18:33:38.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 -- 172.21.15.2:0/2052767593 shutdown_connections 2023-06-23T18:33:38.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbfb01006a0 0x7fbfb019be40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbf980a1150 0x7fbf980a3600 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfb00fea60 0x7fbfb0194890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 --2- 172.21.15.2:0/2052767593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbfb00ffcf0 0x7fbfb0194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:38.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 -- 172.21.15.2:0/2052767593 >> 172.21.15.2:0/2052767593 conn(0x7fbfb00fa080 msgr2=0x7fbfb0103300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:38.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 -- 172.21.15.2:0/2052767593 shutdown_connections 2023-06-23T18:33:38.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:38.051+0000 7fbfb6536700 1 -- 172.21.15.2:0/2052767593 wait complete. 2023-06-23T18:33:38.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:37 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2176980916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:33:38.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:37 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2176980916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:33:38.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:37 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2176980916' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:33:39.071 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:33:39.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:33:39.098 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:38 smithi002 ceph-mon[95947]: pgmap v644: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 42 op/s 2023-06-23T18:33:39.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:38 smithi082 ceph-mon[97889]: pgmap v644: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 42 op/s 2023-06-23T18:33:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:38 smithi017 ceph-mon[97472]: pgmap v644: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 42 MiB/s wr, 42 op/s 2023-06-23T18:33:39.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.213+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1162675900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 msgr2=0x7fd2641029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.213+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1162675900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 0x7fd2641029e0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd254009990 tx=0x7fd254009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.213+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1162675900 shutdown_connections 2023-06-23T18:33:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.213+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1162675900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd264105840 0x7fd264107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.213+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1162675900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 0x7fd2641029e0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.213+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1162675900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264102f20 0x7fd264105300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.213+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1162675900 >> 172.21.15.2:0/1162675900 conn(0x7fd2640fa0e0 msgr2=0x7fd2640fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.214+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1162675900 shutdown_connections 2023-06-23T18:33:39.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.214+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1162675900 wait complete. 2023-06-23T18:33:39.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.214+0000 7fd26a8d9700 1 Processor -- start 2023-06-23T18:33:39.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.214+0000 7fd26a8d9700 1 -- start start 2023-06-23T18:33:39.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd26a8d9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 0x7fd2641957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd26a8d9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd264102f20 0x7fd264195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd26a8d9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264105840 0x7fd26419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd26a8d9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26419d330 con 0x7fd264100600 2023-06-23T18:33:39.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd26a8d9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26419d470 con 0x7fd264102f20 2023-06-23T18:33:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd26a8d9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd26419d600 con 0x7fd264105840 2023-06-23T18:33:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd263fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 0x7fd2641957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd263fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 0x7fd2641957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56742/0 (socket says 172.21.15.2:56742) 2023-06-23T18:33:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd263fff700 1 -- 172.21.15.2:0/1801110913 learned_addr learned my addr 172.21.15.2:0/1801110913 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd268e76700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264105840 0x7fd26419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd2637fe700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd264102f20 0x7fd264195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd263fff700 1 -- 172.21.15.2:0/1801110913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264105840 msgr2=0x7fd26419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd263fff700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264105840 0x7fd26419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd263fff700 1 -- 172.21.15.2:0/1801110913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd264102f20 msgr2=0x7fd264195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.215+0000 7fd263fff700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd264102f20 0x7fd264195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.216+0000 7fd263fff700 1 -- 172.21.15.2:0/1801110913 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2540095f0 con 0x7fd264100600 2023-06-23T18:33:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.216+0000 7fd263fff700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 0x7fd2641957b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd254006010 tx=0x7fd254003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.216+0000 7fd2617fa700 1 -- 172.21.15.2:0/1801110913 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25401d070 con 0x7fd264100600 2023-06-23T18:33:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.216+0000 7fd2617fa700 1 -- 172.21.15.2:0/1801110913 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd25400bd70 con 0x7fd264100600 2023-06-23T18:33:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.216+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1801110913 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd26419d880 con 0x7fd264100600 2023-06-23T18:33:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.216+0000 7fd2617fa700 1 -- 172.21.15.2:0/1801110913 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd254022c40 con 0x7fd264100600 2023-06-23T18:33:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.216+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1801110913 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26419dca0 con 0x7fd264100600 2023-06-23T18:33:39.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.218+0000 7fd2617fa700 1 -- 172.21.15.2:0/1801110913 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd254022470 con 0x7fd264100600 2023-06-23T18:33:39.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.219+0000 7fd2617fa700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd24c0a1150 0x7fd24c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.220+0000 7fd2617fa700 1 -- 172.21.15.2:0/1801110913 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2540cdf20 con 0x7fd264100600 2023-06-23T18:33:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.220+0000 7fd2637fe700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd24c0a1150 0x7fd24c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.220+0000 7fd2637fe700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd24c0a1150 0x7fd24c0a3600 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd258005950 tx=0x7fd25800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd276a5eb80 1 Processor -- start 2023-06-23T18:33:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd276a5eb80 1 -- start start 2023-06-23T18:33:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b5f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f5080 0x56248b5f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b4f8d30 con 0x56248b5f5080 2023-06-23T18:33:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b54bac0 con 0x56248b5f4650 2023-06-23T18:33:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b544f50 con 0x56248b5f3d00 2023-06-23T18:33:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd268e76700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f5080 0x56248b5f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd263fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd268e76700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f5080 0x56248b5f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56754/0 (socket says 172.21.15.2:56754) 2023-06-23T18:33:39.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd268e76700 1 -- 172.21.15.2:0/2663246898 learned_addr learned my addr 172.21.15.2:0/2663246898 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd2637fe700 1 --2- 172.21.15.2:0/2663246898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b5f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd268e76700 1 -- 172.21.15.2:0/2663246898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 msgr2=0x56248b5f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd268e76700 1 --2- 172.21.15.2:0/2663246898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd268e76700 1 -- 172.21.15.2:0/2663246898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 msgr2=0x56248b5f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd268e76700 1 --2- 172.21.15.2:0/2663246898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b5f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.221+0000 7fd268e76700 1 -- 172.21.15.2:0/2663246898 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56248b3c66b0 con 0x56248b5f5080 2023-06-23T18:33:39.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.222+0000 7fd268e76700 1 --2- 172.21.15.2:0/2663246898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f5080 0x56248b5f54f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0098f0 tx=0x7fd25c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c449f87297986da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.222+0000 7fd24a7fc700 1 -- 172.21.15.2:0/2663246898 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c019070 con 0x56248b5f5080 2023-06-23T18:33:39.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.222+0000 7fd24a7fc700 1 -- 172.21.15.2:0/2663246898 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd25c00f040 con 0x56248b5f5080 2023-06-23T18:33:39.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.222+0000 7fd24a7fc700 1 -- 172.21.15.2:0/2663246898 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c007a50 con 0x56248b5f5080 2023-06-23T18:33:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2663246898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f5080 msgr2=0x56248b5f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/2663246898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f5080 0x56248b5f54f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0098f0 tx=0x7fd25c009c00 comp rx=0 tx=0).stop 2023-06-23T18:33:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2663246898 shutdown_connections 2023-06-23T18:33:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/2663246898 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f5080 0x56248b5f54f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/2663246898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/2663246898 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b5f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2663246898 >> 172.21.15.2:0/2663246898 conn(0x56248b5e9470 msgr2=0x56248b5ee5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2663246898 shutdown_connections 2023-06-23T18:33:39.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2663246898 wait complete. 2023-06-23T18:33:39.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.223+0000 7fd276a5eb80 1 -- start start 2023-06-23T18:33:39.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd276a5eb80 1 Processor -- start 2023-06-23T18:33:39.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f3d00 0x56248b5fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b5fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b604b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b4e4b40 con 0x56248b5f4650 2023-06-23T18:33:39.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b4e4840 con 0x56248b5f3d00 2023-06-23T18:33:39.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b3c89a0 con 0x56248b5f5080 2023-06-23T18:33:39.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd276a5eb80 10 client.0 _ll_register_callbacks cb 0x56248b4ea2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:39.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b5fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd2637fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b5fdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56758/0 (socket says 172.21.15.2:56758) 2023-06-23T18:33:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd2637fe700 1 -- 172.21.15.2:0/341397374 learned_addr learned my addr 172.21.15.2:0/341397374 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd263fff700 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f3d00 0x56248b5fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.224+0000 7fd268e76700 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b604b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2637fe700 1 -- 172.21.15.2:0/341397374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 msgr2=0x56248b604b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2637fe700 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b604b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2637fe700 1 -- 172.21.15.2:0/341397374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f3d00 msgr2=0x56248b5fd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2637fe700 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f3d00 0x56248b5fd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2637fe700 1 -- 172.21.15.2:0/341397374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c003c50 con 0x56248b5f4650 2023-06-23T18:33:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2637fe700 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b5fdb20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd258011770 tx=0x7fd2580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666cea28d7501a83 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2437fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2437fe700 1 -- 172.21.15.2:0/341397374 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd258004a30 con 0x56248b5f4650 2023-06-23T18:33:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2437fe700 1 -- 172.21.15.2:0/341397374 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd258017070 con 0x56248b5f4650 2023-06-23T18:33:39.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd276a5eb80 1 -- 172.21.15.2:0/341397374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56248b5e6c30 con 0x56248b5f4650 2023-06-23T18:33:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd2437fe700 1 -- 172.21.15.2:0/341397374 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd258010ed0 con 0x56248b5f4650 2023-06-23T18:33:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd276a5eb80 10 client.16521 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.225+0000 7fd276a5eb80 1 -- 172.21.15.2:0/341397374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56248b372e60 con 0x56248b5f4650 2023-06-23T18:33:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd2437fe700 1 -- 172.21.15.2:0/341397374 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56248b5e6c30 con 0x56248b5f4650 2023-06-23T18:33:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd2437fe700 1 -- 172.21.15.2:0/341397374 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd258027420 con 0x56248b5f4650 2023-06-23T18:33:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd276a5eb80 10 client.16521 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:39.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd276a5eb80 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd276a5eb80 20 client.16521 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56248b6093f0 0x56248b60b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd276a5eb80 4 client.16521 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd276a5eb80 1 -- 172.21.15.2:0/341397374 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56248b53df30 con 0x56248b6093f0 2023-06-23T18:33:39.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.226+0000 7fd263fff700 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56248b6093f0 0x56248b60b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.227+0000 7fd263fff700 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56248b6093f0 0x56248b60b800 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a1bfff6601483c8 server_cookie=7d1eacc45d4846e3 in_seq=0 out_seq=0 2023-06-23T18:33:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.227+0000 7fd2437fe700 10 client.16521 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd2437fe700 1 -- 172.21.15.2:0/341397374 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56248b53df30 con 0x56248b6093f0 2023-06-23T18:33:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd2437fe700 10 client.16521 handle_command_reply: tid=0 2023-06-23T18:33:39.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 client.16521 shutdown 2023-06-23T18:33:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 -- 172.21.15.2:0/341397374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 msgr2=0x56248b5fdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b5fdb20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fd258011770 tx=0x7fd2580047c0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 -- 172.21.15.2:0/341397374 shutdown_connections 2023-06-23T18:33:39.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56248b6093f0 0x56248b60b800 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b604b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f3d00 0x56248b5fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd2437fe700 0 client.16521 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:39.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/341397374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b5fdb20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 -- 172.21.15.2:0/341397374 >> 172.21.15.2:0/341397374 conn(0x56248b5e9470 msgr2=0x56248b5edcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 -- 172.21.15.2:0/341397374 shutdown_connections 2023-06-23T18:33:39.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 1 -- 172.21.15.2:0/341397374 wait complete. 2023-06-23T18:33:39.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.228+0000 7fd276a5eb80 20 client.16521 trim_cache size 0 max 16384 2023-06-23T18:33:39.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.247+0000 7fd276a5eb80 1 Processor -- start 2023-06-23T18:33:39.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.247+0000 7fd276a5eb80 1 -- start start 2023-06-23T18:33:39.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.247+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.247+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b604b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.247+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f5080 0x56248b5f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.247+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b5fd190 con 0x56248b5f4650 2023-06-23T18:33:39.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.247+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b5ef480 con 0x56248b5f5080 2023-06-23T18:33:39.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.247+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b5e9020 con 0x56248b5f3d00 2023-06-23T18:33:39.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57102/0 (socket says 172.21.15.2:57102) 2023-06-23T18:33:39.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 -- 172.21.15.2:0/2671848319 learned_addr learned my addr 172.21.15.2:0/2671848319 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd268e76700 1 --2- 172.21.15.2:0/2671848319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f5080 0x56248b5f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 -- 172.21.15.2:0/2671848319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f5080 msgr2=0x56248b5f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 --2- 172.21.15.2:0/2671848319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f5080 0x56248b5f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 -- 172.21.15.2:0/2671848319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 msgr2=0x56248b604b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 --2- 172.21.15.2:0/2671848319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b604b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 -- 172.21.15.2:0/2671848319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56248b35bce0 con 0x56248b5f3d00 2023-06-23T18:33:39.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd263fff700 1 --2- 172.21.15.2:0/2671848319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f7f80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd2540cba20 tx=0x7fd254097040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d53e42247fdb3746 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.248+0000 7fd24affd700 1 -- 172.21.15.2:0/2671848319 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2540a3780 con 0x56248b5f3d00 2023-06-23T18:33:39.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.249+0000 7fd24affd700 1 -- 172.21.15.2:0/2671848319 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2540cbb30 con 0x56248b5f3d00 2023-06-23T18:33:39.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.249+0000 7fd24affd700 1 -- 172.21.15.2:0/2671848319 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25409b620 con 0x56248b5f3d00 2023-06-23T18:33:39.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.250+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2671848319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 msgr2=0x56248b5f7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.250+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/2671848319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f7f80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd2540cba20 tx=0x7fd254097040 comp rx=0 tx=0).stop 2023-06-23T18:33:39.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.251+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2671848319 shutdown_connections 2023-06-23T18:33:39.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.251+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/2671848319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f5080 0x56248b5f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.251+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/2671848319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f3d00 0x56248b5f7f80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.251+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/2671848319 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f4650 0x56248b604b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.251+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2671848319 >> 172.21.15.2:0/2671848319 conn(0x56248b5e9470 msgr2=0x56248b5fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.251+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2671848319 shutdown_connections 2023-06-23T18:33:39.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.251+0000 7fd276a5eb80 1 -- 172.21.15.2:0/2671848319 wait complete. 2023-06-23T18:33:39.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.252+0000 7fd276a5eb80 1 -- start start 2023-06-23T18:33:39.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.252+0000 7fd276a5eb80 1 Processor -- start 2023-06-23T18:33:39.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f3d00 0x56248b601b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b602050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd276a5eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b603060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b6035a0 con 0x56248b5f3d00 2023-06-23T18:33:39.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b603710 con 0x56248b5f4650 2023-06-23T18:33:39.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd276a5eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56248b6038d0 con 0x56248b5f5080 2023-06-23T18:33:39.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd276a5eb80 10 client.0 _ll_register_callbacks cb 0x56248b4ea2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:39.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd263fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f3d00 0x56248b601b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd263fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f3d00 0x56248b601b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56770/0 (socket says 172.21.15.2:56770) 2023-06-23T18:33:39.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd263fff700 1 -- 172.21.15.2:0/4105627167 learned_addr learned my addr 172.21.15.2:0/4105627167 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd263fff700 1 -- 172.21.15.2:0/4105627167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 msgr2=0x56248b603060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd268e76700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b603060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd2637fe700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b602050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd263fff700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b603060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd263fff700 1 -- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 msgr2=0x56248b602050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd263fff700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b602050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.253+0000 7fd263fff700 1 -- 172.21.15.2:0/4105627167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd26419dca0 con 0x56248b5f3d00 2023-06-23T18:33:39.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.254+0000 7fd263fff700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f3d00 0x56248b601b10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd2540a3250 tx=0x7fd25409b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a511447b90c5eb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.254+0000 7fd2437fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:39.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.254+0000 7fd2437fe700 1 -- 172.21.15.2:0/4105627167 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25409aac0 con 0x56248b5f3d00 2023-06-23T18:33:39.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.254+0000 7fd2437fe700 1 -- 172.21.15.2:0/4105627167 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd25409b620 con 0x56248b5f3d00 2023-06-23T18:33:39.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.254+0000 7fd276a5eb80 1 -- 172.21.15.2:0/4105627167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd25c00f040 con 0x56248b5f3d00 2023-06-23T18:33:39.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.254+0000 7fd2637fe700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b602050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:39.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.254+0000 7fd2437fe700 1 -- 172.21.15.2:0/4105627167 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd25409b7e0 con 0x56248b5f3d00 2023-06-23T18:33:39.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd276a5eb80 10 client.16524 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:39.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd276a5eb80 1 -- 172.21.15.2:0/4105627167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56248b3992d0 con 0x56248b5f3d00 2023-06-23T18:33:39.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd2437fe700 1 -- 172.21.15.2:0/4105627167 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2540c9020 con 0x56248b5f3d00 2023-06-23T18:33:39.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd2437fe700 1 -- 172.21.15.2:0/4105627167 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd254097340 con 0x56248b5f3d00 2023-06-23T18:33:39.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd276a5eb80 10 client.16524 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:39.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd276a5eb80 10 client.16524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:39.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd276a5eb80 20 client.16524 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:39.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56248b65f2b0 0x56248b661690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd276a5eb80 4 client.16524 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:33:39.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.255+0000 7fd276a5eb80 1 -- 172.21.15.2:0/4105627167 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56248b5000d0 con 0x56248b65f2b0 2023-06-23T18:33:39.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.256+0000 7fd268e76700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b603060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:39.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.256+0000 7fd2637fe700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56248b65f2b0 0x56248b661690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.258+0000 7fd2437fe700 10 client.16524 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:39.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.258+0000 7fd2637fe700 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56248b65f2b0 0x56248b661690 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c90c18054f216d server_cookie=ebe1b50e2ee84423 in_seq=0 out_seq=0 2023-06-23T18:33:39.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.259+0000 7fd2437fe700 1 -- 172.21.15.2:0/4105627167 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56248b5000d0 con 0x56248b65f2b0 2023-06-23T18:33:39.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.259+0000 7fd2437fe700 10 client.16524 handle_command_reply: tid=0 2023-06-23T18:33:39.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.259+0000 7fd276a5eb80 1 client.16524 shutdown 2023-06-23T18:33:39.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 -- 172.21.15.2:0/4105627167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f3d00 msgr2=0x56248b601b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f3d00 0x56248b601b10 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd2540a3250 tx=0x7fd25409b030 comp rx=0 tx=0).stop 2023-06-23T18:33:39.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 -- 172.21.15.2:0/4105627167 shutdown_connections 2023-06-23T18:33:39.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x56248b65f2b0 0x56248b661690 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56248b5f5080 0x56248b603060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56248b5f3d00 0x56248b601b10 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 --2- 172.21.15.2:0/4105627167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56248b5f4650 0x56248b602050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 -- 172.21.15.2:0/4105627167 >> 172.21.15.2:0/4105627167 conn(0x56248b5e9470 msgr2=0x56248b5f7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd2437fe700 0 client.16524 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:39.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 -- 172.21.15.2:0/4105627167 shutdown_connections 2023-06-23T18:33:39.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 1 -- 172.21.15.2:0/4105627167 wait complete. 2023-06-23T18:33:39.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.260+0000 7fd276a5eb80 20 client.16524 trim_cache size 0 max 16384 2023-06-23T18:33:39.277 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:33:39.277 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:33:39.278 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "7488922a-c311-4e79-af15-3c147b2f8a22", 2023-06-23T18:33:39.278 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:33:39.278 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:33:39.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd24c0a1150 msgr2=0x7fd24c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd24c0a1150 0x7fd24c0a3600 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd258005950 tx=0x7fd25800b410 comp rx=0 tx=0).stop 2023-06-23T18:33:39.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 msgr2=0x7fd2641957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 0x7fd2641957b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd254006010 tx=0x7fd254003a50 comp rx=0 tx=0).stop 2023-06-23T18:33:39.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1801110913 shutdown_connections 2023-06-23T18:33:39.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd264105840 0x7fd26419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd264100600 0x7fd2641957b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd24c0a1150 0x7fd24c0a3600 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 --2- 172.21.15.2:0/1801110913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd264102f20 0x7fd264195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1801110913 >> 172.21.15.2:0/1801110913 conn(0x7fd2640fa0e0 msgr2=0x7fd264101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1801110913 shutdown_connections 2023-06-23T18:33:39.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.261+0000 7fd26a8d9700 1 -- 172.21.15.2:0/1801110913 wait complete. 2023-06-23T18:33:39.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:33:39.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.455+0000 7f465e420700 1 -- 172.21.15.2:0/665865065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 msgr2=0x7f46581000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.455+0000 7f465e420700 1 --2- 172.21.15.2:0/665865065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 0x7f46581000c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f464c009990 tx=0x7f464c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.455+0000 7f465e420700 1 -- 172.21.15.2:0/665865065 shutdown_connections 2023-06-23T18:33:39.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.455+0000 7f465e420700 1 --2- 172.21.15.2:0/665865065 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4658100600 0x7f4658106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.455+0000 7f465e420700 1 --2- 172.21.15.2:0/665865065 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46580fea80 0x7f46580fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.455+0000 7f465e420700 1 --2- 172.21.15.2:0/665865065 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 0x7f46581000c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.455+0000 7f465e420700 1 -- 172.21.15.2:0/665865065 >> 172.21.15.2:0/665865065 conn(0x7f46580fa0e0 msgr2=0x7f46580fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.456+0000 7f465e420700 1 -- 172.21.15.2:0/665865065 shutdown_connections 2023-06-23T18:33:39.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.456+0000 7f465e420700 1 -- 172.21.15.2:0/665865065 wait complete. 2023-06-23T18:33:39.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.456+0000 7f465e420700 1 Processor -- start 2023-06-23T18:33:39.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.456+0000 7f465e420700 1 -- start start 2023-06-23T18:33:39.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f465e420700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46580fea80 0x7f46581957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f465e420700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 0x7f4658195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f465e420700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4658100600 0x7f465819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f465e420700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465819d320 con 0x7f46580ffc70 2023-06-23T18:33:39.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f465e420700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465819d460 con 0x7f4658100600 2023-06-23T18:33:39.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f465e420700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465819d5f0 con 0x7f46580fea80 2023-06-23T18:33:39.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 0x7f4658195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f4657fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46580fea80 0x7f46581957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 0x7f4658195ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56786/0 (socket says 172.21.15.2:56786) 2023-06-23T18:33:39.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 -- 172.21.15.2:0/3674185131 learned_addr learned my addr 172.21.15.2:0/3674185131 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f4657fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46580fea80 0x7f46581957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57122/0 (socket says 172.21.15.2:57122) 2023-06-23T18:33:39.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f465c9bd700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4658100600 0x7f465819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 -- 172.21.15.2:0/3674185131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46580fea80 msgr2=0x7f46581957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46580fea80 0x7f46581957a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 -- 172.21.15.2:0/3674185131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4658100600 msgr2=0x7f465819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4658100600 0x7f465819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 -- 172.21.15.2:0/3674185131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4650009c40 con 0x7f46580ffc70 2023-06-23T18:33:39.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f465c9bd700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4658100600 0x7f465819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:33:39.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f4657fff700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46580fea80 0x7f46581957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:39.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.457+0000 7f46577fe700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 0x7f4658195ce0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f464c005f50 tx=0x7f464c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.458+0000 7f46557fa700 1 -- 172.21.15.2:0/3674185131 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c01d070 con 0x7f46580ffc70 2023-06-23T18:33:39.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.458+0000 7f46557fa700 1 -- 172.21.15.2:0/3674185131 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f464c004810 con 0x7f46580ffc70 2023-06-23T18:33:39.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.458+0000 7f465e420700 1 -- 172.21.15.2:0/3674185131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c0095f0 con 0x7f46580ffc70 2023-06-23T18:33:39.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.458+0000 7f46557fa700 1 -- 172.21.15.2:0/3674185131 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c005340 con 0x7f46580ffc70 2023-06-23T18:33:39.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.458+0000 7f465e420700 1 -- 172.21.15.2:0/3674185131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465819db00 con 0x7f46580ffc70 2023-06-23T18:33:39.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.460+0000 7f46557fa700 1 -- 172.21.15.2:0/3674185131 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f464c004980 con 0x7f46580ffc70 2023-06-23T18:33:39.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.461+0000 7f46557fa700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46400a1150 0x7f46400a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.461+0000 7f4657fff700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46400a1150 0x7f46400a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.461+0000 7f46557fa700 1 -- 172.21.15.2:0/3674185131 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f464c0cdb50 con 0x7f46580ffc70 2023-06-23T18:33:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.461+0000 7f4657fff700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46400a1150 0x7f46400a3600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f46580ffad0 tx=0x7f464800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f466a5a5b80 1 Processor -- start 2023-06-23T18:33:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f466a5a5b80 1 -- start start 2023-06-23T18:33:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369cc80 0x56331369d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x56331369da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369e000 0x56331369e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633135a1cb0 con 0x56331369e000 2023-06-23T18:33:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633135f4a40 con 0x56331369cc80 2023-06-23T18:33:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633135eded0 con 0x56331369d5d0 2023-06-23T18:33:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f465c9bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369e000 0x56331369e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f46577fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x56331369da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f465c9bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369e000 0x56331369e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56796/0 (socket says 172.21.15.2:56796) 2023-06-23T18:33:39.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f465c9bd700 1 -- 172.21.15.2:0/3612666812 learned_addr learned my addr 172.21.15.2:0/3612666812 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f4657fff700 1 --2- 172.21.15.2:0/3612666812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369cc80 0x56331369d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f46577fe700 1 -- 172.21.15.2:0/3612666812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369cc80 msgr2=0x56331369d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f46577fe700 1 --2- 172.21.15.2:0/3612666812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369cc80 0x56331369d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f46577fe700 1 -- 172.21.15.2:0/3612666812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369e000 msgr2=0x56331369e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f46577fe700 1 --2- 172.21.15.2:0/3612666812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369e000 0x56331369e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.462+0000 7f46577fe700 1 -- 172.21.15.2:0/3612666812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56331346f630 con 0x56331369d5d0 2023-06-23T18:33:39.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.463+0000 7f46577fe700 1 --2- 172.21.15.2:0/3612666812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x56331369da70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f464c005910 tx=0x7f464c09b070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7747751a0b95ea1c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.463+0000 7f463e7fc700 1 -- 172.21.15.2:0/3612666812 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c099390 con 0x56331369d5d0 2023-06-23T18:33:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.463+0000 7f463e7fc700 1 -- 172.21.15.2:0/3612666812 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f464c09b250 con 0x56331369d5d0 2023-06-23T18:33:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.463+0000 7f463e7fc700 1 -- 172.21.15.2:0/3612666812 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c0a3120 con 0x56331369d5d0 2023-06-23T18:33:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3612666812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 msgr2=0x56331369da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3612666812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x56331369da70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f464c005910 tx=0x7f464c09b070 comp rx=0 tx=0).stop 2023-06-23T18:33:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3612666812 shutdown_connections 2023-06-23T18:33:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3612666812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369e000 0x56331369e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3612666812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369cc80 0x56331369d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3612666812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x56331369da70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3612666812 >> 172.21.15.2:0/3612666812 conn(0x5633136923f0 msgr2=0x563313697520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3612666812 shutdown_connections 2023-06-23T18:33:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3612666812 wait complete. 2023-06-23T18:33:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 -- start start 2023-06-23T18:33:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 Processor -- start 2023-06-23T18:33:39.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.464+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369cc80 0x5633136a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369d5d0 0x5633136a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 0x5633136ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331358dac0 con 0x56331369d5d0 2023-06-23T18:33:39.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331358d7c0 con 0x56331369e000 2023-06-23T18:33:39.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563313471920 con 0x56331369cc80 2023-06-23T18:33:39.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f466a5a5b80 10 client.0 _ll_register_callbacks cb 0x563313593260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:39.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369d5d0 0x5633136a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f4657fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369cc80 0x5633136a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369d5d0 0x5633136a66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56800/0 (socket says 172.21.15.2:56800) 2023-06-23T18:33:39.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 -- 172.21.15.2:0/2264165102 learned_addr learned my addr 172.21.15.2:0/2264165102 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f465c9bd700 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 0x5633136ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 -- 172.21.15.2:0/2264165102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369cc80 msgr2=0x5633136a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369cc80 0x5633136a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 -- 172.21.15.2:0/2264165102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 msgr2=0x5633136ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 0x5633136ad720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 -- 172.21.15.2:0/2264165102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465819db00 con 0x56331369d5d0 2023-06-23T18:33:39.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f4657fff700 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369cc80 0x5633136a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:33:39.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46577fe700 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369d5d0 0x5633136a66e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f464c0974f0 tx=0x7f464c0a32e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27c249dbe9f723a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46377fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:39.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46377fe700 1 -- 172.21.15.2:0/2264165102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c09b3c0 con 0x56331369d5d0 2023-06-23T18:33:39.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.465+0000 7f46377fe700 1 -- 172.21.15.2:0/2264165102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f464c0b9180 con 0x56331369d5d0 2023-06-23T18:33:39.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2264165102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56331368fbb0 con 0x56331369d5d0 2023-06-23T18:33:39.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f46377fe700 1 -- 172.21.15.2:0/2264165102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f464c09b520 con 0x56331369d5d0 2023-06-23T18:33:39.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 10 client.16536 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:39.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2264165102 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56331341bde0 con 0x56331369d5d0 2023-06-23T18:33:39.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f46377fe700 1 -- 172.21.15.2:0/2264165102 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56331368fbb0 con 0x56331369d5d0 2023-06-23T18:33:39.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f46377fe700 1 -- 172.21.15.2:0/2264165102 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f464c0cbb10 con 0x56331369d5d0 2023-06-23T18:33:39.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 10 client.16536 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:39.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:39.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 20 client.16536 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:39.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5633136b1a50 0x5633136b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 4 client.16536 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:33:39.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2264165102 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633135e6eb0 con 0x5633136b1a50 2023-06-23T18:33:39.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.466+0000 7f4657fff700 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5633136b1a50 0x5633136b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.467+0000 7f46377fe700 10 client.16536 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:39.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.467+0000 7f4657fff700 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5633136b1a50 0x5633136b3e30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0080a8b5fbade85 server_cookie=cb27da891e5ca77a in_seq=0 out_seq=0 2023-06-23T18:33:39.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.467+0000 7f46377fe700 1 -- 172.21.15.2:0/2264165102 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5633135e6eb0 con 0x5633136b1a50 2023-06-23T18:33:39.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.467+0000 7f46377fe700 10 client.16536 handle_command_reply: tid=0 2023-06-23T18:33:39.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 client.16536 shutdown 2023-06-23T18:33:39.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2264165102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369d5d0 msgr2=0x5633136a66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369d5d0 0x5633136a66e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f464c0974f0 tx=0x7f464c0a32e0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2264165102 shutdown_connections 2023-06-23T18:33:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5633136b1a50 0x5633136b3e30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 0x5633136ad720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369cc80 0x5633136a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f46377fe700 0 client.16536 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2264165102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369d5d0 0x5633136a66e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2264165102 >> 172.21.15.2:0/2264165102 conn(0x5633136923f0 msgr2=0x563313696c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2264165102 shutdown_connections 2023-06-23T18:33:39.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2264165102 wait complete. 2023-06-23T18:33:39.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.468+0000 7f466a5a5b80 20 client.16536 trim_cache size 0 max 16384 2023-06-23T18:33:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f466a5a5b80 1 Processor -- start 2023-06-23T18:33:39.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f466a5a5b80 1 -- start start 2023-06-23T18:33:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369d5d0 0x5633136ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369e000 0x5633136a40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563313698d50 con 0x56331369cc80 2023-06-23T18:33:39.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563313691f70 con 0x56331369d5d0 2023-06-23T18:33:39.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633136a7320 con 0x56331369e000 2023-06-23T18:33:39.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f4657fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f4657fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56806/0 (socket says 172.21.15.2:56806) 2023-06-23T18:33:39.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.489+0000 7f4657fff700 1 -- 172.21.15.2:0/2019730975 learned_addr learned my addr 172.21.15.2:0/2019730975 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f465c9bd700 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369e000 0x5633136a40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f46577fe700 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369d5d0 0x5633136ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f4657fff700 1 -- 172.21.15.2:0/2019730975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369e000 msgr2=0x5633136a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f4657fff700 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369e000 0x5633136a40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f4657fff700 1 -- 172.21.15.2:0/2019730975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369d5d0 msgr2=0x5633136ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f4657fff700 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369d5d0 0x5633136ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f4657fff700 1 -- 172.21.15.2:0/2019730975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563313404c60 con 0x56331369cc80 2023-06-23T18:33:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f46577fe700 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369d5d0 0x5633136ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:33:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f4657fff700 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136adfb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f464800bda0 tx=0x7f4648013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aa59ea8f43b468 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f463effd700 1 -- 172.21.15.2:0/2019730975 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648003cd0 con 0x56331369cc80 2023-06-23T18:33:39.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f463effd700 1 -- 172.21.15.2:0/2019730975 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f46480275c0 con 0x56331369cc80 2023-06-23T18:33:39.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.490+0000 7f463effd700 1 -- 172.21.15.2:0/2019730975 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648011440 con 0x56331369cc80 2023-06-23T18:33:39.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2019730975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 msgr2=0x5633136adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136adfb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f464800bda0 tx=0x7f4648013000 comp rx=0 tx=0).stop 2023-06-23T18:33:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2019730975 shutdown_connections 2023-06-23T18:33:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369e000 0x5633136a40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136adfb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/2019730975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369d5d0 0x5633136ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2019730975 >> 172.21.15.2:0/2019730975 conn(0x5633136923f0 msgr2=0x5633136b8790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2019730975 shutdown_connections 2023-06-23T18:33:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 -- 172.21.15.2:0/2019730975 wait complete. 2023-06-23T18:33:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 -- start start 2023-06-23T18:33:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.491+0000 7f466a5a5b80 1 Processor -- start 2023-06-23T18:33:39.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x5633136a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f466a5a5b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 0x5633136acb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633136ad080 con 0x56331369cc80 2023-06-23T18:33:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633136ad1f0 con 0x56331369e000 2023-06-23T18:33:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f466a5a5b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633136ad3b0 con 0x56331369d5d0 2023-06-23T18:33:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f466a5a5b80 10 client.0 _ll_register_callbacks cb 0x563313593260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:33:39.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56812/0 (socket says 172.21.15.2:56812) 2023-06-23T18:33:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 -- 172.21.15.2:0/3293572254 learned_addr learned my addr 172.21.15.2:0/3293572254 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:33:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f46577fe700 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x5633136a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f465c9bd700 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 0x5633136acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 -- 172.21.15.2:0/3293572254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 msgr2=0x5633136a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x5633136a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 -- 172.21.15.2:0/3293572254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 msgr2=0x5633136acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:33:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 0x5633136acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 -- 172.21.15.2:0/3293572254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46480040c0 con 0x56331369cc80 2023-06-23T18:33:39.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f4657fff700 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136a95e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4648010fe0 tx=0x7f4648028d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfe013ea61206d1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:33:39.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f46377fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:33:39.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.492+0000 7f46377fe700 1 -- 172.21.15.2:0/3293572254 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4648027330 con 0x56331369cc80 2023-06-23T18:33:39.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f46377fe700 1 -- 172.21.15.2:0/3293572254 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4648017040 con 0x56331369cc80 2023-06-23T18:33:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3293572254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f464c09b250 con 0x56331369cc80 2023-06-23T18:33:39.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f46377fe700 1 -- 172.21.15.2:0/3293572254 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4648027940 con 0x56331369cc80 2023-06-23T18:33:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 10 client.16542 fetch_fsmap learned FSMap version 38 2023-06-23T18:33:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3293572254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x563313442250 con 0x56331369cc80 2023-06-23T18:33:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f46377fe700 1 -- 172.21.15.2:0/3293572254 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f464c09b250 con 0x56331369cc80 2023-06-23T18:33:39.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f46377fe700 1 -- 172.21.15.2:0/3293572254 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f464802e070 con 0x56331369cc80 2023-06-23T18:33:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 10 client.16542 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:33:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 10 client.16542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:33:39.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 20 client.16542 populate_metadata read hostname 'smithi002' 2023-06-23T18:33:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5633137068b0 0x563313708cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:33:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 4 client.16542 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:33:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3293572254 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633135a9050 con 0x5633137068b0 2023-06-23T18:33:39.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.493+0000 7f46577fe700 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5633137068b0 0x563313708cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:33:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f46377fe700 10 client.16542 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f46577fe700 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5633137068b0 0x563313708cc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab7bbda3e3d27a7f server_cookie=20e292ba72943ccb in_seq=0 out_seq=0 2023-06-23T18:33:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f46377fe700 1 -- 172.21.15.2:0/3293572254 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5633135a9050 con 0x5633137068b0 2023-06-23T18:33:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f46377fe700 10 client.16542 handle_command_reply: tid=0 2023-06-23T18:33:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 client.16542 shutdown 2023-06-23T18:33:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3293572254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 msgr2=0x5633136a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136a95e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4648010fe0 tx=0x7f4648028d90 comp rx=0 tx=0).stop 2023-06-23T18:33:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3293572254 shutdown_connections 2023-06-23T18:33:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5633137068b0 0x563313708cc0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x56331369e000 0x5633136acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x56331369cc80 0x5633136a95e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f46377fe700 0 client.16542 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:33:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 --2- 172.21.15.2:0/3293572254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56331369d5d0 0x5633136a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3293572254 >> 172.21.15.2:0/3293572254 conn(0x5633136923f0 msgr2=0x5633136a6140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3293572254 shutdown_connections 2023-06-23T18:33:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 1 -- 172.21.15.2:0/3293572254 wait complete. 2023-06-23T18:33:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.494+0000 7f466a5a5b80 20 client.16542 trim_cache size 0 max 16384 2023-06-23T18:33:39.521 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:33:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:33:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:33:39.522 INFO:teuthology.orchestra.run.smithi002.stdout: "7488922a-c311-4e79-af15-3c147b2f8a22": { 2023-06-23T18:33:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:33:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "7488922a-c311-4e79-af15-3c147b2f8a22", 2023-06-23T18:33:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:33:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:33:39.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:33:39.524 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:33:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 -- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46400a1150 msgr2=0x7f46400a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46400a1150 0x7f46400a3600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f46580ffad0 tx=0x7f464800b410 comp rx=0 tx=0).stop 2023-06-23T18:33:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 -- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 msgr2=0x7f4658195ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:33:39.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 0x7f4658195ce0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f464c005f50 tx=0x7f464c0047a0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 -- 172.21.15.2:0/3674185131 shutdown_connections 2023-06-23T18:33:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46400a1150 0x7f46400a3600 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4658100600 0x7f465819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46580fea80 0x7f46581957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 --2- 172.21.15.2:0/3674185131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46580ffc70 0x7f4658195ce0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:33:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 -- 172.21.15.2:0/3674185131 >> 172.21.15.2:0/3674185131 conn(0x7f46580fa0e0 msgr2=0x7f4658067800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:33:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 -- 172.21.15.2:0/3674185131 shutdown_connections 2023-06-23T18:33:39.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:33:39.495+0000 7f465e420700 1 -- 172.21.15.2:0/3674185131 wait complete. 2023-06-23T18:33:39.527 INFO:tasks.cephfs.filesystem:scrub status for tag:7488922a-c311-4e79-af15-3c147b2f8a22 - {'path': '/', 'tag': '7488922a-c311-4e79-af15-3c147b2f8a22', 'options': 'recursive,force'} 2023-06-23T18:33:40.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: pgmap v645: 97 pgs: 97 active+clean; 8.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 43 op/s 2023-06-23T18:33:40.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:33:40.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:33:40.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:33:40.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:33:40.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:40 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: pgmap v645: 97 pgs: 97 active+clean; 8.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 43 op/s 2023-06-23T18:33:40.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:33:40.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:33:40.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:33:40.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:33:40.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:40 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: pgmap v645: 97 pgs: 97 active+clean; 8.9 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 43 MiB/s wr, 43 op/s 2023-06-23T18:33:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:33:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:33:40.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:33:40.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:33:40.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:40.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:40 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:33:42.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:42 smithi002 ceph-mon[95947]: pgmap v646: 97 pgs: 97 active+clean; 9.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 45 MiB/s wr, 43 op/s 2023-06-23T18:33:42.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:42 smithi082 ceph-mon[97889]: pgmap v646: 97 pgs: 97 active+clean; 9.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 45 MiB/s wr, 43 op/s 2023-06-23T18:33:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:42 smithi017 ceph-mon[97472]: pgmap v646: 97 pgs: 97 active+clean; 9.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 45 MiB/s wr, 43 op/s 2023-06-23T18:33:44.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:44 smithi002 ceph-mon[95947]: pgmap v647: 97 pgs: 97 active+clean; 9.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:33:44.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:44 smithi082 ceph-mon[97889]: pgmap v647: 97 pgs: 97 active+clean; 9.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:33:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:44 smithi017 ceph-mon[97472]: pgmap v647: 97 pgs: 97 active+clean; 9.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 46 MiB/s wr, 25 op/s 2023-06-23T18:33:46.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:46 smithi002 ceph-mon[95947]: pgmap v648: 97 pgs: 97 active+clean; 9.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:33:46.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:46 smithi082 ceph-mon[97889]: pgmap v648: 97 pgs: 97 active+clean; 9.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:33:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:46 smithi017 ceph-mon[97472]: pgmap v648: 97 pgs: 97 active+clean; 9.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 41 MiB/s wr, 25 op/s 2023-06-23T18:33:48.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:48 smithi002 ceph-mon[95947]: pgmap v649: 97 pgs: 97 active+clean; 9.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 24 op/s 2023-06-23T18:33:48.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:48 smithi082 ceph-mon[97889]: pgmap v649: 97 pgs: 97 active+clean; 9.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 24 op/s 2023-06-23T18:33:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:48 smithi017 ceph-mon[97472]: pgmap v649: 97 pgs: 97 active+clean; 9.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 41 MiB/s wr, 24 op/s 2023-06-23T18:33:50.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:50 smithi002 ceph-mon[95947]: pgmap v650: 97 pgs: 97 active+clean; 9.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 40 op/s 2023-06-23T18:33:50.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:50 smithi082 ceph-mon[97889]: pgmap v650: 97 pgs: 97 active+clean; 9.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 40 op/s 2023-06-23T18:33:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:50 smithi017 ceph-mon[97472]: pgmap v650: 97 pgs: 97 active+clean; 9.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 43 MiB/s wr, 40 op/s 2023-06-23T18:33:51.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:51 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:33:51.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:51 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:33:51.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:51 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:33:51.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:51 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:33:51.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:51 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:33:51.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:51 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:33:51.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:51 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:33:51.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:51 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:33:51.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:51 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:33:51.861 INFO:teuthology.orchestra.run.smithi002.stderr:done. 2023-06-23T18:33:51.864 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-06-23T18:33:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:starting vacuum...end. 2023-06-23T18:33:52.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:52 smithi002 ceph-mon[95947]: pgmap v651: 97 pgs: 97 active+clean; 9.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 42 MiB/s wr, 73 op/s 2023-06-23T18:33:52.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:52 smithi082 ceph-mon[97889]: pgmap v651: 97 pgs: 97 active+clean; 9.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 42 MiB/s wr, 73 op/s 2023-06-23T18:33:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:52 smithi017 ceph-mon[97472]: pgmap v651: 97 pgs: 97 active+clean; 9.2 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 42 MiB/s wr, 73 op/s 2023-06-23T18:33:54.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:54 smithi002 ceph-mon[95947]: pgmap v652: 97 pgs: 97 active+clean; 9.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 40 MiB/s wr, 75 op/s 2023-06-23T18:33:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:54 smithi082 ceph-mon[97889]: pgmap v652: 97 pgs: 97 active+clean; 9.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 40 MiB/s wr, 75 op/s 2023-06-23T18:33:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:54 smithi017 ceph-mon[97472]: pgmap v652: 97 pgs: 97 active+clean; 9.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 40 MiB/s wr, 75 op/s 2023-06-23T18:33:56.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:56 smithi002 ceph-mon[95947]: pgmap v653: 97 pgs: 97 active+clean; 9.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 36 MiB/s wr, 82 op/s 2023-06-23T18:33:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:56 smithi082 ceph-mon[97889]: pgmap v653: 97 pgs: 97 active+clean; 9.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 36 MiB/s wr, 82 op/s 2023-06-23T18:33:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:56 smithi017 ceph-mon[97472]: pgmap v653: 97 pgs: 97 active+clean; 9.0 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 36 MiB/s wr, 82 op/s 2023-06-23T18:33:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 5.0 s, 382.6 tps, lat 165.412 ms stddev 193.237 2023-06-23T18:33:58.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:33:58 smithi002 ceph-mon[95947]: pgmap v654: 97 pgs: 97 active+clean; 8.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 33 MiB/s wr, 138 op/s 2023-06-23T18:33:58.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:33:58 smithi082 ceph-mon[97889]: pgmap v654: 97 pgs: 97 active+clean; 8.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 33 MiB/s wr, 138 op/s 2023-06-23T18:33:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:33:58 smithi017 ceph-mon[97472]: pgmap v654: 97 pgs: 97 active+clean; 8.7 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 33 MiB/s wr, 138 op/s 2023-06-23T18:34:00.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:00 smithi002 ceph-mon[95947]: pgmap v655: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 29 MiB/s wr, 247 op/s 2023-06-23T18:34:00.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:00 smithi082 ceph-mon[97889]: pgmap v655: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 29 MiB/s wr, 247 op/s 2023-06-23T18:34:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:00 smithi017 ceph-mon[97472]: pgmap v655: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 29 MiB/s wr, 247 op/s 2023-06-23T18:34:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 10.0 s, 366.0 tps, lat 21.864 ms stddev 6.725 2023-06-23T18:34:02.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:02 smithi002 ceph-mon[95947]: pgmap v656: 97 pgs: 97 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 24 MiB/s wr, 311 op/s 2023-06-23T18:34:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:02 smithi082 ceph-mon[97889]: pgmap v656: 97 pgs: 97 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 24 MiB/s wr, 311 op/s 2023-06-23T18:34:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:02 smithi017 ceph-mon[97472]: pgmap v656: 97 pgs: 97 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 24 MiB/s wr, 311 op/s 2023-06-23T18:34:04.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:04 smithi002 ceph-mon[95947]: pgmap v657: 97 pgs: 97 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 17 MiB/s wr, 374 op/s 2023-06-23T18:34:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:04 smithi082 ceph-mon[97889]: pgmap v657: 97 pgs: 97 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 17 MiB/s wr, 374 op/s 2023-06-23T18:34:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:04 smithi017 ceph-mon[97472]: pgmap v657: 97 pgs: 97 active+clean; 8.3 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 17 MiB/s wr, 374 op/s 2023-06-23T18:34:06.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:06 smithi002 ceph-mon[95947]: pgmap v658: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 11 MiB/s wr, 434 op/s 2023-06-23T18:34:06.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:06 smithi082 ceph-mon[97889]: pgmap v658: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 11 MiB/s wr, 434 op/s 2023-06-23T18:34:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:06 smithi017 ceph-mon[97472]: pgmap v658: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 11 MiB/s wr, 434 op/s 2023-06-23T18:34:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 15.0 s, 561.4 tps, lat 394.604 ms stddev 1305.318 2023-06-23T18:34:08.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:08 smithi002 ceph-mon[95947]: pgmap v659: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 10 MiB/s wr, 510 op/s 2023-06-23T18:34:08.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:08 smithi082 ceph-mon[97889]: pgmap v659: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 10 MiB/s wr, 510 op/s 2023-06-23T18:34:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:08 smithi017 ceph-mon[97472]: pgmap v659: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 10 MiB/s wr, 510 op/s 2023-06-23T18:34:09.528 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:34:09.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:09 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:34:09.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:09 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:34:09.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.681+0000 7f84e8459700 1 -- 172.21.15.2:0/1870309174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84e0100df0 msgr2=0x7f84e0101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.681+0000 7f84e8459700 1 --2- 172.21.15.2:0/1870309174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84e0100df0 0x7f84e0101260 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f84d00098f0 tx=0x7f84d0009c00 comp rx=0 tx=0).stop 2023-06-23T18:34:09.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.682+0000 7f84e8459700 1 -- 172.21.15.2:0/1870309174 shutdown_connections 2023-06-23T18:34:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.682+0000 7f84e8459700 1 --2- 172.21.15.2:0/1870309174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e01017a0 0x7f84e0108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.682+0000 7f84e8459700 1 --2- 172.21.15.2:0/1870309174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84e00ffb60 0x7f84e00fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.682+0000 7f84e8459700 1 --2- 172.21.15.2:0/1870309174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84e0100df0 0x7f84e0101260 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.682+0000 7f84e8459700 1 -- 172.21.15.2:0/1870309174 >> 172.21.15.2:0/1870309174 conn(0x7f84e00fb180 msgr2=0x7f84e00fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:09.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.682+0000 7f84e8459700 1 -- 172.21.15.2:0/1870309174 shutdown_connections 2023-06-23T18:34:09.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.682+0000 7f84e8459700 1 -- 172.21.15.2:0/1870309174 wait complete. 2023-06-23T18:34:09.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e8459700 1 Processor -- start 2023-06-23T18:34:09.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e8459700 1 -- start start 2023-06-23T18:34:09.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e8459700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84e00ffb60 0x7f84e0074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e8459700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84e0100df0 0x7f84e00728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e8459700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e01017a0 0x7f84e0072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e8459700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e00735a0 con 0x7f84e0100df0 2023-06-23T18:34:09.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e8459700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e0073710 con 0x7f84e00ffb60 2023-06-23T18:34:09.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e8459700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e019f890 con 0x7f84e01017a0 2023-06-23T18:34:09.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e59f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84e0100df0 0x7f84e00728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84e00ffb60 0x7f84e0074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84e00ffb60 0x7f84e0074280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47418/0 (socket says 172.21.15.2:47418) 2023-06-23T18:34:09.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e69f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e01017a0 0x7f84e0072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e61f5700 1 -- 172.21.15.2:0/3249971298 learned_addr learned my addr 172.21.15.2:0/3249971298 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:09.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e61f5700 1 -- 172.21.15.2:0/3249971298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e01017a0 msgr2=0x7f84e0072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e01017a0 0x7f84e0072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e61f5700 1 -- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84e0100df0 msgr2=0x7f84e00728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84e0100df0 0x7f84e00728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.683+0000 7f84e61f5700 1 -- 172.21.15.2:0/3249971298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc0095f0 con 0x7f84e00ffb60 2023-06-23T18:34:09.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.684+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84e00ffb60 0x7f84e0074280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00ec10 tx=0x7f84dc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:09.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.684+0000 7f84d77fe700 1 -- 172.21.15.2:0/3249971298 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc00cc90 con 0x7f84e00ffb60 2023-06-23T18:34:09.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.684+0000 7f84d77fe700 1 -- 172.21.15.2:0/3249971298 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84dc00cdf0 con 0x7f84e00ffb60 2023-06-23T18:34:09.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.684+0000 7f84e8459700 1 -- 172.21.15.2:0/3249971298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d0009520 con 0x7f84e00ffb60 2023-06-23T18:34:09.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.684+0000 7f84d77fe700 1 -- 172.21.15.2:0/3249971298 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc003e80 con 0x7f84e00ffb60 2023-06-23T18:34:09.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.684+0000 7f84e8459700 1 -- 172.21.15.2:0/3249971298 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e019ff40 con 0x7f84e00ffb60 2023-06-23T18:34:09.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.687+0000 7f84d77fe700 1 -- 172.21.15.2:0/3249971298 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f84dc004000 con 0x7f84e00ffb60 2023-06-23T18:34:09.699 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:09 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:34:09.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.688+0000 7f84d77fe700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f84cc0a11d0 0x7f84cc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.688+0000 7f84d77fe700 1 -- 172.21.15.2:0/3249971298 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84dc014070 con 0x7f84e00ffb60 2023-06-23T18:34:09.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.688+0000 7f84e59f4700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f84cc0a11d0 0x7f84cc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.688+0000 7f84f45deb80 1 Processor -- start 2023-06-23T18:34:09.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84f45deb80 1 -- start start 2023-06-23T18:34:09.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e59f4700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f84cc0a11d0 0x7f84cc0a3680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f84d0005e90 tx=0x7f84d0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:09.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53b8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84f45deb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c52bbcb0 con 0x5630c53b75d0 2023-06-23T18:34:09.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84f45deb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c530ea40 con 0x5630c53b6c80 2023-06-23T18:34:09.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84f45deb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c5307ed0 con 0x5630c53b8000 2023-06-23T18:34:09.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e59f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e69f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53b8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e59f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53b7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56834/0 (socket says 172.21.15.2:56834) 2023-06-23T18:34:09.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47428/0 (socket says 172.21.15.2:47428) 2023-06-23T18:34:09.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e59f4700 1 -- 172.21.15.2:0/1571610951 learned_addr learned my addr 172.21.15.2:0/1571610951 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:09.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e61f5700 1 -- 172.21.15.2:0/1571610951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 msgr2=0x5630c53b8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:09.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e61f5700 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53b8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e61f5700 1 -- 172.21.15.2:0/1571610951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 msgr2=0x5630c53b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:09.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e61f5700 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e61f5700 1 -- 172.21.15.2:0/1571610951 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630c5189630 con 0x5630c53b6c80 2023-06-23T18:34:09.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e69f6700 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:09.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.689+0000 7f84e59f4700 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:09.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.690+0000 7f84e61f5700 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53b7090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0965b0 tx=0x7f84dc004640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d036241664215811 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:09.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.690+0000 7f84d4dac700 1 -- 172.21.15.2:0/1571610951 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0a2500 con 0x5630c53b6c80 2023-06-23T18:34:09.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.690+0000 7f84d4dac700 1 -- 172.21.15.2:0/1571610951 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84dc0981f0 con 0x5630c53b6c80 2023-06-23T18:34:09.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.690+0000 7f84d4dac700 1 -- 172.21.15.2:0/1571610951 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc09a570 con 0x5630c53b6c80 2023-06-23T18:34:09.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.690+0000 7f84f45deb80 1 -- 172.21.15.2:0/1571610951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 msgr2=0x5630c53b7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.690+0000 7f84f45deb80 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53b7090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0965b0 tx=0x7f84dc004640 comp rx=0 tx=0).stop 2023-06-23T18:34:09.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 -- 172.21.15.2:0/1571610951 shutdown_connections 2023-06-23T18:34:09.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53b8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53b7090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 --2- 172.21.15.2:0/1571610951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 -- 172.21.15.2:0/1571610951 >> 172.21.15.2:0/1571610951 conn(0x5630c53ac3f0 msgr2=0x5630c53b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:09.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 -- 172.21.15.2:0/1571610951 shutdown_connections 2023-06-23T18:34:09.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 -- 172.21.15.2:0/1571610951 wait complete. 2023-06-23T18:34:09.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 -- start start 2023-06-23T18:34:09.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 Processor -- start 2023-06-23T18:34:09.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b6c80 0x5630c53c0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b75d0 0x5630c53c0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b8000 0x5630c53c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c52a7ac0 con 0x5630c53b8000 2023-06-23T18:34:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c52a77c0 con 0x5630c53b75d0 2023-06-23T18:34:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c518b920 con 0x5630c53b6c80 2023-06-23T18:34:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84f45deb80 10 client.0 _ll_register_callbacks cb 0x5630c52ad260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b6c80 0x5630c53c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b6c80 0x5630c53c0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57178/0 (socket says 172.21.15.2:57178) 2023-06-23T18:34:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.691+0000 7f84e61f5700 1 -- 172.21.15.2:0/3139547325 learned_addr learned my addr 172.21.15.2:0/3139547325 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:09.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84e59f4700 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b75d0 0x5630c53c0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84e69f6700 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b8000 0x5630c53c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84e61f5700 1 -- 172.21.15.2:0/3139547325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b75d0 msgr2=0x5630c53c0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:09.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b75d0 0x5630c53c0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84e61f5700 1 -- 172.21.15.2:0/3139547325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b8000 msgr2=0x5630c53c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:09.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b8000 0x5630c53c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84e61f5700 1 -- 172.21.15.2:0/3139547325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d8011230 con 0x5630c53b6c80 2023-06-23T18:34:09.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b6c80 0x5630c53c0560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f84dc09a270 tx=0x7f84dc09e1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d479ecafcf71d8c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:09.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:09.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84c9ffb700 1 -- 172.21.15.2:0/3139547325 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0a2500 con 0x5630c53b6c80 2023-06-23T18:34:09.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84c9ffb700 1 -- 172.21.15.2:0/3139547325 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84dc003680 con 0x5630c53b6c80 2023-06-23T18:34:09.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84f45deb80 1 -- 172.21.15.2:0/3139547325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630c53a9bb0 con 0x5630c53b6c80 2023-06-23T18:34:09.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84c9ffb700 1 -- 172.21.15.2:0/3139547325 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84dc09a520 con 0x5630c53b6c80 2023-06-23T18:34:09.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84f45deb80 10 client.26263 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:09.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.692+0000 7f84f45deb80 1 -- 172.21.15.2:0/3139547325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f84e019ff40 con 0x5630c53b6c80 2023-06-23T18:34:09.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84c9ffb700 1 -- 172.21.15.2:0/3139547325 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5630c53a9bb0 con 0x5630c53b6c80 2023-06-23T18:34:09.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84c9ffb700 1 -- 172.21.15.2:0/3139547325 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84dc0ca260 con 0x5630c53b6c80 2023-06-23T18:34:09.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84f45deb80 10 client.26263 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:09.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84f45deb80 10 client.26263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:09.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84f45deb80 20 client.26263 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:09.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630c53cbe90 0x5630c53ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84f45deb80 4 client.26263 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84f45deb80 1 -- 172.21.15.2:0/3139547325 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630c5300eb0 con 0x5630c53cbe90 2023-06-23T18:34:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84e59f4700 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630c53cbe90 0x5630c53ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84c9ffb700 10 client.26263 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.693+0000 7f84e59f4700 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630c53cbe90 0x5630c53ce2a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4629c1be2637897d server_cookie=f72a22de02097dd6 in_seq=0 out_seq=0 2023-06-23T18:34:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84c9ffb700 1 -- 172.21.15.2:0/3139547325 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5630c5300eb0 con 0x5630c53cbe90 2023-06-23T18:34:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84c9ffb700 10 client.26263 handle_command_reply: tid=0 2023-06-23T18:34:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 client.26263 shutdown 2023-06-23T18:34:09.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 -- 172.21.15.2:0/3139547325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b6c80 msgr2=0x5630c53c0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b6c80 0x5630c53c0560 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f84dc09a270 tx=0x7f84dc09e1a0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 -- 172.21.15.2:0/3139547325 shutdown_connections 2023-06-23T18:34:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630c53cbe90 0x5630c53ce2a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b8000 0x5630c53c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b6c80 0x5630c53c0560 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3139547325 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b75d0 0x5630c53c0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 -- 172.21.15.2:0/3139547325 >> 172.21.15.2:0/3139547325 conn(0x5630c53ac3f0 msgr2=0x5630c53b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:09.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84c9ffb700 0 client.26263 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:09.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 -- 172.21.15.2:0/3139547325 shutdown_connections 2023-06-23T18:34:09.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 1 -- 172.21.15.2:0/3139547325 wait complete. 2023-06-23T18:34:09.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.695+0000 7f84f45deb80 20 client.26263 trim_cache size 0 max 16384 2023-06-23T18:34:09.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.717+0000 7f84f45deb80 1 Processor -- start 2023-06-23T18:34:09.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84f45deb80 1 -- start start 2023-06-23T18:34:09.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b6c80 0x5630c53be390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b75d0 0x5630c53be8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b8000 0x5630c53c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84f45deb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c53c0130 con 0x5630c53b6c80 2023-06-23T18:34:09.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84f45deb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c53bf0b0 con 0x5630c53b8000 2023-06-23T18:34:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84f45deb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c5300ee0 con 0x5630c53b75d0 2023-06-23T18:34:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b6c80 0x5630c53be390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e69f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b8000 0x5630c53c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e69f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b8000 0x5630c53c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47440/0 (socket says 172.21.15.2:47440) 2023-06-23T18:34:09.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e69f6700 1 -- 172.21.15.2:0/3739888199 learned_addr learned my addr 172.21.15.2:0/3739888199 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:09.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e59f4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b75d0 0x5630c53be8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b6c80 0x5630c53be390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56848/0 (socket says 172.21.15.2:56848) 2023-06-23T18:34:09.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e61f5700 1 -- 172.21.15.2:0/3739888199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b75d0 msgr2=0x5630c53be8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:09.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3739888199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b75d0 0x5630c53be8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e61f5700 1 -- 172.21.15.2:0/3739888199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b8000 msgr2=0x5630c53c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:09.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3739888199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b8000 0x5630c53c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e61f5700 1 -- 172.21.15.2:0/3739888199 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630c53b86a0 con 0x5630c53b6c80 2023-06-23T18:34:09.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e69f6700 1 --2- 172.21.15.2:0/3739888199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b8000 0x5630c53c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:09.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.718+0000 7f84e61f5700 1 --2- 172.21.15.2:0/3739888199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b6c80 0x5630c53be390 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f84dc096b20 tx=0x7f84dc0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=917c2b3e18079ab2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:09.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84d55ad700 1 -- 172.21.15.2:0/3739888199 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004390 con 0x5630c53b6c80 2023-06-23T18:34:09.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84d55ad700 1 -- 172.21.15.2:0/3739888199 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84dc09a520 con 0x5630c53b6c80 2023-06-23T18:34:09.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84d55ad700 1 -- 172.21.15.2:0/3739888199 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0ca450 con 0x5630c53b6c80 2023-06-23T18:34:09.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84f45deb80 1 -- 172.21.15.2:0/3739888199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b6c80 msgr2=0x5630c53be390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3739888199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b6c80 0x5630c53be390 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f84dc096b20 tx=0x7f84dc0045b0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84f45deb80 1 -- 172.21.15.2:0/3739888199 shutdown_connections 2023-06-23T18:34:09.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3739888199 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b8000 0x5630c53c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3739888199 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b6c80 0x5630c53be390 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84f45deb80 1 --2- 172.21.15.2:0/3739888199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b75d0 0x5630c53be8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84f45deb80 1 -- 172.21.15.2:0/3739888199 >> 172.21.15.2:0/3739888199 conn(0x5630c53ac3f0 msgr2=0x5630c53d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:09.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.719+0000 7f84f45deb80 1 -- 172.21.15.2:0/3739888199 shutdown_connections 2023-06-23T18:34:09.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 -- 172.21.15.2:0/3739888199 wait complete. 2023-06-23T18:34:09.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 -- start start 2023-06-23T18:34:09.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 Processor -- start 2023-06-23T18:34:09.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c53c7190 con 0x5630c53b75d0 2023-06-23T18:34:09.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c53c7300 con 0x5630c53b6c80 2023-06-23T18:34:09.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630c53c74c0 con 0x5630c53b8000 2023-06-23T18:34:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84f45deb80 10 client.0 _ll_register_callbacks cb 0x5630c52ad260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84e59f4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84e69f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84e69f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57194/0 (socket says 172.21.15.2:57194) 2023-06-23T18:34:09.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84e61f5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53c36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47444/0 (socket says 172.21.15.2:47444) 2023-06-23T18:34:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.720+0000 7f84e69f6700 1 -- 172.21.15.2:0/669975340 learned_addr learned my addr 172.21.15.2:0/669975340 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84e69f6700 1 -- 172.21.15.2:0/669975340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 msgr2=0x5630c53c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84e69f6700 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84e69f6700 1 -- 172.21.15.2:0/669975340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 msgr2=0x5630c53c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84e69f6700 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84e69f6700 1 -- 172.21.15.2:0/669975340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d8009290 con 0x5630c53b8000 2023-06-23T18:34:09.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84e69f6700 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53c6c50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84d8004580 tx=0x7f84d8004890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=206889fddcd54051 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84c9ffb700 1 -- 172.21.15.2:0/669975340 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d801c070 con 0x5630c53b8000 2023-06-23T18:34:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84c9ffb700 1 -- 172.21.15.2:0/669975340 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f84d8008ad0 con 0x5630c53b8000 2023-06-23T18:34:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84f45deb80 1 -- 172.21.15.2:0/669975340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84dc0981f0 con 0x5630c53b8000 2023-06-23T18:34:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84c9ffb700 1 -- 172.21.15.2:0/669975340 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84d800e040 con 0x5630c53b8000 2023-06-23T18:34:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84f45deb80 10 client.26269 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:09.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84f45deb80 1 -- 172.21.15.2:0/669975340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f84e019ff40 con 0x5630c53b8000 2023-06-23T18:34:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.721+0000 7f84c9ffb700 1 -- 172.21.15.2:0/669975340 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84dc0981f0 con 0x5630c53b8000 2023-06-23T18:34:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84c9ffb700 1 -- 172.21.15.2:0/669975340 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84d8020cf0 con 0x5630c53b8000 2023-06-23T18:34:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84f45deb80 10 client.26269 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84f45deb80 10 client.26269 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84f45deb80 20 client.26269 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84f45deb80 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630c5420c10 0x5630c5423020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84f45deb80 4 client.26269 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:34:09.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84f45deb80 1 -- 172.21.15.2:0/669975340 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5630c52c3050 con 0x5630c5420c10 2023-06-23T18:34:09.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84e61f5700 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630c5420c10 0x5630c5423020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84e61f5700 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630c5420c10 0x5630c5423020 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33b8619c4d6d68e server_cookie=c1903b0541e7f57b in_seq=0 out_seq=0 2023-06-23T18:34:09.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.722+0000 7f84c9ffb700 10 client.26269 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:09.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84c9ffb700 1 -- 172.21.15.2:0/669975340 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5630c52c3050 con 0x5630c5420c10 2023-06-23T18:34:09.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84c9ffb700 10 client.26269 handle_command_reply: tid=0 2023-06-23T18:34:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 client.26269 shutdown 2023-06-23T18:34:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 -- 172.21.15.2:0/669975340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 msgr2=0x5630c53c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53c6c50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84d8004580 tx=0x7f84d8004890 comp rx=0 tx=0).stop 2023-06-23T18:34:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 -- 172.21.15.2:0/669975340 shutdown_connections 2023-06-23T18:34:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5630c5420c10 0x5630c5423020 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5630c53b8000 0x5630c53c6c50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5630c53b6c80 0x5630c53c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84c9ffb700 0 client.26269 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 --2- 172.21.15.2:0/669975340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5630c53b75d0 0x5630c53c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 -- 172.21.15.2:0/669975340 >> 172.21.15.2:0/669975340 conn(0x5630c53ac3f0 msgr2=0x5630c53be390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 -- 172.21.15.2:0/669975340 shutdown_connections 2023-06-23T18:34:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 1 -- 172.21.15.2:0/669975340 wait complete. 2023-06-23T18:34:09.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.723+0000 7f84f45deb80 20 client.26269 trim_cache size 0 max 16384 2023-06-23T18:34:09.747 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:34:09.747 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:34:09.748 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:34:09.748 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:34:09.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 -- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f84cc0a11d0 msgr2=0x7f84cc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f84cc0a11d0 0x7f84cc0a3680 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f84d0005e90 tx=0x7f84d0019040 comp rx=0 tx=0).stop 2023-06-23T18:34:09.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 -- 172.21.15.2:0/3249971298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84e00ffb60 msgr2=0x7f84e0074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84e00ffb60 0x7f84e0074280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00ec10 tx=0x7f84dc00ef20 comp rx=0 tx=0).stop 2023-06-23T18:34:09.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 -- 172.21.15.2:0/3249971298 shutdown_connections 2023-06-23T18:34:09.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84e01017a0 0x7f84e0072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f84cc0a11d0 0x7f84cc0a3680 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84e00ffb60 0x7f84e0074280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 --2- 172.21.15.2:0/3249971298 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f84e0100df0 0x7f84e00728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 -- 172.21.15.2:0/3249971298 >> 172.21.15.2:0/3249971298 conn(0x7f84e00fb180 msgr2=0x7f84e00fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:09.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 -- 172.21.15.2:0/3249971298 shutdown_connections 2023-06-23T18:34:09.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.724+0000 7f84e8459700 1 -- 172.21.15.2:0/3249971298 wait complete. 2023-06-23T18:34:09.753 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:34:09.753 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:34:09.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 -- 172.21.15.2:0/3502217012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c1066a0 msgr2=0x7f778c106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 --2- 172.21.15.2:0/3502217012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c1066a0 0x7f778c106ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f77800098f0 tx=0x7f7780009c00 comp rx=0 tx=0).stop 2023-06-23T18:34:09.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 -- 172.21.15.2:0/3502217012 shutdown_connections 2023-06-23T18:34:09.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 --2- 172.21.15.2:0/3502217012 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f778c107d60 0x7f778c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 --2- 172.21.15.2:0/3502217012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c1066a0 0x7f778c106ab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 --2- 172.21.15.2:0/3502217012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f778c1073b0 0x7f778c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 -- 172.21.15.2:0/3502217012 >> 172.21.15.2:0/3502217012 conn(0x7f778c0751c0 msgr2=0x7f778c077610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:09.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 -- 172.21.15.2:0/3502217012 shutdown_connections 2023-06-23T18:34:09.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.960+0000 7f7791a17700 1 -- 172.21.15.2:0/3502217012 wait complete. 2023-06-23T18:34:09.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f7791a17700 1 Processor -- start 2023-06-23T18:34:09.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f7791a17700 1 -- start start 2023-06-23T18:34:09.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f7791a17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f778c1066a0 0x7f778c19be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f7791a17700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f778c1073b0 0x7f778c19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f7791a17700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c107d60 0x7f778c1a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f7791a17700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c1a3980 con 0x7f778c1066a0 2023-06-23T18:34:09.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f7791a17700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c1a3ac0 con 0x7f778c1073b0 2023-06-23T18:34:09.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f7791a17700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c1a3c50 con 0x7f778c107d60 2023-06-23T18:34:09.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f778affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f778c1066a0 0x7f778c19be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f778affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f778c1066a0 0x7f778c19be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56866/0 (socket says 172.21.15.2:56866) 2023-06-23T18:34:09.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f778affd700 1 -- 172.21.15.2:0/3087067892 learned_addr learned my addr 172.21.15.2:0/3087067892 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:09.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f778a7fc700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f778c1073b0 0x7f778c19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.961+0000 7f778b7fe700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c107d60 0x7f778c1a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f778affd700 1 -- 172.21.15.2:0/3087067892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c107d60 msgr2=0x7f778c1a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f778affd700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c107d60 0x7f778c1a33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f778affd700 1 -- 172.21.15.2:0/3087067892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f778c1073b0 msgr2=0x7f778c19c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:09.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f778affd700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f778c1073b0 0x7f778c19c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:09.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f778affd700 1 -- 172.21.15.2:0/3087067892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7784009d00 con 0x7f778c1066a0 2023-06-23T18:34:09.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f778b7fe700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c107d60 0x7f778c1a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:09.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f778affd700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f778c1066a0 0x7f778c19be00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f778000b790 tx=0x7f778000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:09.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f7790a15700 1 -- 172.21.15.2:0/3087067892 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778001c070 con 0x7f778c1066a0 2023-06-23T18:34:09.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f7790a15700 1 -- 172.21.15.2:0/3087067892 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f778000f7b0 con 0x7f778c1066a0 2023-06-23T18:34:09.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7780009520 con 0x7f778c1066a0 2023-06-23T18:34:09.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f7790a15700 1 -- 172.21.15.2:0/3087067892 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780004930 con 0x7f778c1066a0 2023-06-23T18:34:09.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.962+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c1a4160 con 0x7f778c1066a0 2023-06-23T18:34:09.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.963+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778c10d850 con 0x7f778c1066a0 2023-06-23T18:34:09.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.965+0000 7f7790a15700 1 -- 172.21.15.2:0/3087067892 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7780020070 con 0x7f778c1066a0 2023-06-23T18:34:09.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.966+0000 7f7790a15700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77740a1150 0x7f77740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:09.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.966+0000 7f7790a15700 1 -- 172.21.15.2:0/3087067892 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f77800cdd20 con 0x7f778c1066a0 2023-06-23T18:34:09.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.966+0000 7f778a7fc700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77740a1150 0x7f77740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:09.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.967+0000 7f778a7fc700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77740a1150 0x7f77740a3600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f777c009c60 tx=0x7f777c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:09.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:09.971+0000 7f7790a15700 1 -- 172.21.15.2:0/3087067892 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7780097500 con 0x7f778c1066a0 2023-06-23T18:34:10.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.328+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f778c02c810 con 0x7f778c1066a0 2023-06-23T18:34:10.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.329+0000 7f7790a15700 1 -- 172.21.15.2:0/3087067892 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10783 (secure 0 0 0) 0x7f778009ab20 con 0x7f778c1066a0 2023-06-23T18:34:10.334 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:34:10.334 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:33:28.815886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:34:10.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77740a1150 msgr2=0x7f77740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77740a1150 0x7f77740a3600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f777c009c60 tx=0x7f777c009260 comp rx=0 tx=0).stop 2023-06-23T18:34:10.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f778c1066a0 msgr2=0x7f778c19be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f778c1066a0 0x7f778c19be00 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f778000b790 tx=0x7f778000b7c0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 shutdown_connections 2023-06-23T18:34:10.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f778c107d60 0x7f778c1a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f778c1066a0 0x7f778c19be00 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f77740a1150 0x7f77740a3600 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 --2- 172.21.15.2:0/3087067892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f778c1073b0 0x7f778c19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 >> 172.21.15.2:0/3087067892 conn(0x7f778c0751c0 msgr2=0x7f778c10a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 shutdown_connections 2023-06-23T18:34:10.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.332+0000 7f7791a17700 1 -- 172.21.15.2:0/3087067892 wait complete. 2023-06-23T18:34:10.340 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 38 2023-06-23T18:34:10.354 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:34:10.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.518+0000 7ffb61b1b700 1 -- 172.21.15.2:0/1217788795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c105900 msgr2=0x7ffb5c107ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.518+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/1217788795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c105900 0x7ffb5c107ce0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009f90 tx=0x7ffb5400b330 comp rx=0 tx=0).stop 2023-06-23T18:34:10.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.518+0000 7ffb61b1b700 1 -- 172.21.15.2:0/1217788795 shutdown_connections 2023-06-23T18:34:10.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.518+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/1217788795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c105900 0x7ffb5c107ce0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.518+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/1217788795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb5c1006c0 0x7ffb5c102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.518+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/1217788795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb5c102fe0 0x7ffb5c1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.518+0000 7ffb61b1b700 1 -- 172.21.15.2:0/1217788795 >> 172.21.15.2:0/1217788795 conn(0x7ffb5c0fa0c0 msgr2=0x7ffb5c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.518+0000 7ffb61b1b700 1 -- 172.21.15.2:0/1217788795 shutdown_connections 2023-06-23T18:34:10.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.519+0000 7ffb61b1b700 1 -- 172.21.15.2:0/1217788795 wait complete. 2023-06-23T18:34:10.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.519+0000 7ffb61b1b700 1 Processor -- start 2023-06-23T18:34:10.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb61b1b700 1 -- start start 2023-06-23T18:34:10.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb61b1b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c1006c0 0x7ffb5c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb61b1b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb5c102fe0 0x7ffb5c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb61b1b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb5c105900 0x7ffb5c19f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb61b1b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c19f610 con 0x7ffb5c102fe0 2023-06-23T18:34:10.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb61b1b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c19f750 con 0x7ffb5c1006c0 2023-06-23T18:34:10.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb61b1b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c19f8e0 con 0x7ffb5c105900 2023-06-23T18:34:10.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c1006c0 0x7ffb5c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c1006c0 0x7ffb5c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47472/0 (socket says 172.21.15.2:47472) 2023-06-23T18:34:10.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/436379123 learned_addr learned my addr 172.21.15.2:0/436379123 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5bfff700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb5c105900 0x7ffb5c19f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5affd700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb5c102fe0 0x7ffb5c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/436379123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb5c105900 msgr2=0x7ffb5c19f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb5c105900 0x7ffb5c19f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb5c102fe0 msgr2=0x7ffb5c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb5c102fe0 0x7ffb5c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.520+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/436379123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb54009c40 con 0x7ffb5c1006c0 2023-06-23T18:34:10.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.521+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c1006c0 0x7ffb5c197a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00e9c0 tx=0x7ffb4c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.521+0000 7ffb58ff9700 1 -- 172.21.15.2:0/436379123 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c004a30 con 0x7ffb5c1006c0 2023-06-23T18:34:10.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.521+0000 7ffb58ff9700 1 -- 172.21.15.2:0/436379123 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb4c00cc50 con 0x7ffb5c1006c0 2023-06-23T18:34:10.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.521+0000 7ffb58ff9700 1 -- 172.21.15.2:0/436379123 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c005410 con 0x7ffb5c1006c0 2023-06-23T18:34:10.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.521+0000 7ffb61b1b700 1 -- 172.21.15.2:0/436379123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c19fbc0 con 0x7ffb5c1006c0 2023-06-23T18:34:10.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.521+0000 7ffb61b1b700 1 -- 172.21.15.2:0/436379123 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c1a0080 con 0x7ffb5c1006c0 2023-06-23T18:34:10.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.523+0000 7ffb58ff9700 1 -- 172.21.15.2:0/436379123 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ffb4c005570 con 0x7ffb5c1006c0 2023-06-23T18:34:10.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.524+0000 7ffb58ff9700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffb440a11d0 0x7ffb440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.524+0000 7ffb58ff9700 1 -- 172.21.15.2:0/436379123 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffb4c014070 con 0x7ffb5c1006c0 2023-06-23T18:34:10.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.524+0000 7ffb5affd700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffb440a11d0 0x7ffb440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb5affd700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffb440a11d0 0x7ffb440a3680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009c60 tx=0x7ffb50009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb6dca0b80 1 Processor -- start 2023-06-23T18:34:10.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb6dca0b80 1 -- start start 2023-06-23T18:34:10.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90995c80 0x55ef90996090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef909965d0 0x55ef90996a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef90997470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9089acb0 con 0x55ef90997000 2023-06-23T18:34:10.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef908eda40 con 0x55ef90995c80 2023-06-23T18:34:10.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef908e6ed0 con 0x55ef909965d0 2023-06-23T18:34:10.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef90997470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef90997470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56892/0 (socket says 172.21.15.2:56892) 2023-06-23T18:34:10.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb5bfff700 1 -- 172.21.15.2:0/4066659989 learned_addr learned my addr 172.21.15.2:0/4066659989 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb5affd700 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef909965d0 0x55ef90996a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.525+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90995c80 0x55ef90996090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb5bfff700 1 -- 172.21.15.2:0/4066659989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef909965d0 msgr2=0x55ef90996a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb5bfff700 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef909965d0 0x55ef90996a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb5bfff700 1 -- 172.21.15.2:0/4066659989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90995c80 msgr2=0x55ef90996090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb5bfff700 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90995c80 0x55ef90996090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb5bfff700 1 -- 172.21.15.2:0/4066659989 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef90768630 con 0x55ef90997000 2023-06-23T18:34:10.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb5bfff700 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef90997470 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009b60 tx=0x7ffb54009b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a20fb238e19c70 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb41ffb700 1 -- 172.21.15.2:0/4066659989 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400e040 con 0x55ef90997000 2023-06-23T18:34:10.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb41ffb700 1 -- 172.21.15.2:0/4066659989 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb54003a00 con 0x55ef90997000 2023-06-23T18:34:10.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.526+0000 7ffb41ffb700 1 -- 172.21.15.2:0/4066659989 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5400bc50 con 0x55ef90997000 2023-06-23T18:34:10.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4066659989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 msgr2=0x55ef90997470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef90997470 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009b60 tx=0x7ffb54009b90 comp rx=0 tx=0).stop 2023-06-23T18:34:10.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4066659989 shutdown_connections 2023-06-23T18:34:10.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef90997470 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90995c80 0x55ef90996090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4066659989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef909965d0 0x55ef90996a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4066659989 >> 172.21.15.2:0/4066659989 conn(0x55ef9098b3f0 msgr2=0x55ef90990520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4066659989 shutdown_connections 2023-06-23T18:34:10.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4066659989 wait complete. 2023-06-23T18:34:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.527+0000 7ffb6dca0b80 1 -- start start 2023-06-23T18:34:10.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb6dca0b80 1 Processor -- start 2023-06-23T18:34:10.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909a8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef90886ac0 con 0x55ef90997000 2023-06-23T18:34:10.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef908867c0 con 0x55ef909965d0 2023-06-23T18:34:10.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9076a920 con 0x55ef90995c80 2023-06-23T18:34:10.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb6dca0b80 10 client.0 _ll_register_callbacks cb 0x55ef9088c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb5b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909af260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56898/0 (socket says 172.21.15.2:56898) 2023-06-23T18:34:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb5bfff700 1 -- 172.21.15.2:0/4153956514 learned_addr learned my addr 172.21.15.2:0/4153956514 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.528+0000 7ffb5affd700 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/4153956514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 msgr2=0x55ef909a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/4153956514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 msgr2=0x55ef909af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/4153956514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c0fe9a0 con 0x55ef90995c80 2023-06-23T18:34:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb5affd700 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909a7ce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00f010 tx=0x7ffb4c00b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2d4755e097c5f82 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb3affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:10.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb3affd700 1 -- 172.21.15.2:0/4153956514 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c09a200 con 0x55ef90995c80 2023-06-23T18:34:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb3affd700 1 -- 172.21.15.2:0/4153956514 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffb4c0b1070 con 0x55ef90995c80 2023-06-23T18:34:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4153956514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef90988bb0 con 0x55ef90995c80 2023-06-23T18:34:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb3affd700 1 -- 172.21.15.2:0/4153956514 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb4c09ddd0 con 0x55ef90995c80 2023-06-23T18:34:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb6dca0b80 10 client.26287 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:10.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.529+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4153956514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ef90714de0 con 0x55ef90995c80 2023-06-23T18:34:10.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb3affd700 1 -- 172.21.15.2:0/4153956514 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb4c07c080 con 0x55ef90995c80 2023-06-23T18:34:10.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb3affd700 1 -- 172.21.15.2:0/4153956514 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffb4c0a1d60 con 0x55ef90995c80 2023-06-23T18:34:10.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb6dca0b80 10 client.26287 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:10.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb6dca0b80 10 client.26287 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:34:10.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb6dca0b80 20 client.26287 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:10.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55ef909b3ae0 0x55ef909b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb6dca0b80 4 client.26287 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4153956514 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef908dfeb0 con 0x55ef909b3ae0 2023-06-23T18:34:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb5affd700 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55ef909b3ae0 0x55ef909b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb3affd700 10 client.26287 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:34:10.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.530+0000 7ffb5affd700 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55ef909b3ae0 0x55ef909b5ef0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c84e3087e972fd44 server_cookie=1c6dfa5ffee9de90 in_seq=0 out_seq=0 2023-06-23T18:34:10.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.531+0000 7ffb3affd700 1 -- 172.21.15.2:0/4153956514 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef908dfeb0 con 0x55ef909b3ae0 2023-06-23T18:34:10.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.531+0000 7ffb3affd700 10 client.26287 handle_command_reply: tid=0 2023-06-23T18:34:10.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.531+0000 7ffb6dca0b80 1 client.26287 shutdown 2023-06-23T18:34:10.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4153956514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 msgr2=0x55ef909a7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909a7ce0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00f010 tx=0x7ffb4c00b2c0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4153956514 shutdown_connections 2023-06-23T18:34:10.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55ef909b3ae0 0x55ef909b5ef0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909a7ce0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/4153956514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4153956514 >> 172.21.15.2:0/4153956514 conn(0x55ef9098b3f0 msgr2=0x55ef9098fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb3affd700 0 client.26287 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:34:10.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4153956514 shutdown_connections 2023-06-23T18:34:10.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/4153956514 wait complete. 2023-06-23T18:34:10.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.532+0000 7ffb6dca0b80 20 client.26287 trim_cache size 0 max 16384 2023-06-23T18:34:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:10 smithi002 ceph-mon[95947]: pgmap v660: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 10 MiB/s wr, 543 op/s 2023-06-23T18:34:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:34:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:34:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:34:10.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.551+0000 7ffb6dca0b80 1 Processor -- start 2023-06-23T18:34:10.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.551+0000 7ffb6dca0b80 1 -- start start 2023-06-23T18:34:10.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909ba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef909965d0 0x55ef909af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90997000 0x55ef9099d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef90991810 con 0x55ef909965d0 2023-06-23T18:34:10.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef9099e980 con 0x55ef90997000 2023-06-23T18:34:10.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef909913f0 con 0x55ef90995c80 2023-06-23T18:34:10.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef909965d0 0x55ef909af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90997000 0x55ef9099d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909ba850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef909965d0 0x55ef909af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56904/0 (socket says 172.21.15.2:56904) 2023-06-23T18:34:10.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909ba850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57240/0 (socket says 172.21.15.2:57240) 2023-06-23T18:34:10.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90997000 0x55ef9099d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47498/0 (socket says 172.21.15.2:47498) 2023-06-23T18:34:10.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 -- 172.21.15.2:0/3985493002 learned_addr learned my addr 172.21.15.2:0/3985493002 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 -- 172.21.15.2:0/3985493002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 msgr2=0x55ef909ba850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 --2- 172.21.15.2:0/3985493002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909ba850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 -- 172.21.15.2:0/3985493002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90997000 msgr2=0x55ef9099d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 --2- 172.21.15.2:0/3985493002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90997000 0x55ef9099d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 -- 172.21.15.2:0/3985493002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef909978f0 con 0x55ef909965d0 2023-06-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/3985493002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.552+0000 7ffb5affd700 1 --2- 172.21.15.2:0/3985493002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef909965d0 0x55ef909af220 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ffb50011ab0 tx=0x7ffb50014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97d4cf1fc9c06e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.553+0000 7ffb427fc700 1 -- 172.21.15.2:0/3985493002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50022070 con 0x55ef909965d0 2023-06-23T18:34:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.553+0000 7ffb427fc700 1 -- 172.21.15.2:0/3985493002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb50023040 con 0x55ef909965d0 2023-06-23T18:34:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.553+0000 7ffb427fc700 1 -- 172.21.15.2:0/3985493002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5001d670 con 0x55ef909965d0 2023-06-23T18:34:10.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.553+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/3985493002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef909965d0 msgr2=0x55ef909af220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.553+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/3985493002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef909965d0 0x55ef909af220 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ffb50011ab0 tx=0x7ffb50014000 comp rx=0 tx=0).stop 2023-06-23T18:34:10.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/3985493002 shutdown_connections 2023-06-23T18:34:10.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/3985493002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef90997000 0x55ef9099d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/3985493002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909ba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/3985493002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef909965d0 0x55ef909af220 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/3985493002 >> 172.21.15.2:0/3985493002 conn(0x55ef9098b3f0 msgr2=0x55ef9098c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/3985493002 shutdown_connections 2023-06-23T18:34:10.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/3985493002 wait complete. 2023-06-23T18:34:10.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 -- start start 2023-06-23T18:34:10.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.554+0000 7ffb6dca0b80 1 Processor -- start 2023-06-23T18:34:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909aac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb6dca0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef909ae700 con 0x55ef90997000 2023-06-23T18:34:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef909ae870 con 0x55ef909965d0 2023-06-23T18:34:10.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb6dca0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef909aea30 con 0x55ef90995c80 2023-06-23T18:34:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb6dca0b80 10 client.0 _ll_register_callbacks cb 0x55ef9088c260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57246/0 (socket says 172.21.15.2:57246) 2023-06-23T18:34:10.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/1750191317 learned_addr learned my addr 172.21.15.2:0/1750191317 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5affd700 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5bfff700 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/1750191317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 msgr2=0x55ef909ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909ab1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/1750191317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 msgr2=0x55ef909ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.555+0000 7ffb5b7fe700 1 -- 172.21.15.2:0/1750191317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef90714de0 con 0x55ef90995c80 2023-06-23T18:34:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.556+0000 7ffb5b7fe700 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909aac60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0967c0 tx=0x7ffb4c099b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a812347ae620d625 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.556+0000 7ffb3affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.556+0000 7ffb3affd700 1 -- 172.21.15.2:0/1750191317 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c099560 con 0x55ef90995c80 2023-06-23T18:34:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.556+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/1750191317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb54003a00 con 0x55ef90995c80 2023-06-23T18:34:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.556+0000 7ffb3affd700 1 -- 172.21.15.2:0/1750191317 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffb4c0b1040 con 0x55ef90995c80 2023-06-23T18:34:10.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.556+0000 7ffb3affd700 1 -- 172.21.15.2:0/1750191317 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb4c09ddd0 con 0x55ef90995c80 2023-06-23T18:34:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.556+0000 7ffb6dca0b80 10 client.26293 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.556+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/1750191317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7ffb50017070 con 0x55ef90995c80 2023-06-23T18:34:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.557+0000 7ffb3affd700 1 -- 172.21.15.2:0/1750191317 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb4c0bd080 con 0x55ef90995c80 2023-06-23T18:34:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.557+0000 7ffb3affd700 1 -- 172.21.15.2:0/1750191317 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffb4c0b9b50 con 0x55ef90995c80 2023-06-23T18:34:10.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.557+0000 7ffb6dca0b80 10 client.26293 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.557+0000 7ffb6dca0b80 10 client.26293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:34:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.557+0000 7ffb6dca0b80 20 client.26293 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.557+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55ef90a083d0 0x55ef90a0a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.557+0000 7ffb6dca0b80 4 client.26293 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:34:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.557+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/1750191317 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ef908a2050 con 0x55ef90a083d0 2023-06-23T18:34:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.558+0000 7ffb5affd700 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55ef90a083d0 0x55ef90a0a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.558+0000 7ffb5affd700 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55ef90a083d0 0x55ef90a0a7e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5bade6007cf82248 server_cookie=3de8d9330c74e631 in_seq=0 out_seq=0 2023-06-23T18:34:10.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.558+0000 7ffb3affd700 10 client.26293 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:34:10.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb3affd700 1 -- 172.21.15.2:0/1750191317 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ef908a2050 con 0x55ef90a083d0 2023-06-23T18:34:10.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb3affd700 10 client.26293 handle_command_reply: tid=0 2023-06-23T18:34:10.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 client.26293 shutdown 2023-06-23T18:34:10.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/1750191317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 msgr2=0x55ef909aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909aac60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0967c0 tx=0x7ffb4c099b70 comp rx=0 tx=0).stop 2023-06-23T18:34:10.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/1750191317 shutdown_connections 2023-06-23T18:34:10.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55ef90a083d0 0x55ef90a0a7e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ef90997000 0x55ef909ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ef90995c80 0x55ef909aac60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 --2- 172.21.15.2:0/1750191317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ef909965d0 0x55ef909ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/1750191317 >> 172.21.15.2:0/1750191317 conn(0x55ef9098b3f0 msgr2=0x55ef909ba850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb3affd700 0 client.26293 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:34:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/1750191317 shutdown_connections 2023-06-23T18:34:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 1 -- 172.21.15.2:0/1750191317 wait complete. 2023-06-23T18:34:10.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.560+0000 7ffb6dca0b80 20 client.26293 trim_cache size 0 max 16384 2023-06-23T18:34:10.583 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:34:10.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 -- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffb440a11d0 msgr2=0x7ffb440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffb440a11d0 0x7ffb440a3680 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ffb50009c60 tx=0x7ffb50009260 comp rx=0 tx=0).stop 2023-06-23T18:34:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 -- 172.21.15.2:0/436379123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c1006c0 msgr2=0x7ffb5c197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c1006c0 0x7ffb5c197a90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00e9c0 tx=0x7ffb4c00ecd0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 -- 172.21.15.2:0/436379123 shutdown_connections 2023-06-23T18:34:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb5c105900 0x7ffb5c19f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ffb440a11d0 0x7ffb440a3680 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb5c1006c0 0x7ffb5c197a90 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 --2- 172.21.15.2:0/436379123 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffb5c102fe0 0x7ffb5c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 -- 172.21.15.2:0/436379123 >> 172.21.15.2:0/436379123 conn(0x7ffb5c0fa0c0 msgr2=0x7ffb5c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 -- 172.21.15.2:0/436379123 shutdown_connections 2023-06-23T18:34:10.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.561+0000 7ffb61b1b700 1 -- 172.21.15.2:0/436379123 wait complete. 2023-06-23T18:34:10.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:34:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:10 smithi082 ceph-mon[97889]: pgmap v660: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 10 MiB/s wr, 543 op/s 2023-06-23T18:34:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:34:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:34:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:34:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:10 smithi017 ceph-mon[97472]: pgmap v660: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 10 MiB/s wr, 543 op/s 2023-06-23T18:34:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:34:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:34:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 -- 172.21.15.2:0/2964297942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80ffc70 msgr2=0x7efde81000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 --2- 172.21.15.2:0/2964297942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80ffc70 0x7efde81000c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efddc009990 tx=0x7efddc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 -- 172.21.15.2:0/2964297942 shutdown_connections 2023-06-23T18:34:10.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 --2- 172.21.15.2:0/2964297942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde8100600 0x7efde8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 --2- 172.21.15.2:0/2964297942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde80fea80 0x7efde80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 --2- 172.21.15.2:0/2964297942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80ffc70 0x7efde81000c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 -- 172.21.15.2:0/2964297942 >> 172.21.15.2:0/2964297942 conn(0x7efde80fa0e0 msgr2=0x7efde80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 -- 172.21.15.2:0/2964297942 shutdown_connections 2023-06-23T18:34:10.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.759+0000 7efdedc1d700 1 -- 172.21.15.2:0/2964297942 wait complete. 2023-06-23T18:34:10.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efdedc1d700 1 Processor -- start 2023-06-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efdedc1d700 1 -- start start 2023-06-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efdedc1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80fea80 0x7efde8109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efdedc1d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde80ffc70 0x7efde81079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efdedc1d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde8100600 0x7efde8107ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efdedc1d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde8199670 con 0x7efde80fea80 2023-06-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efdedc1d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde8108430 con 0x7efde80ffc70 2023-06-23T18:34:10.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efdedc1d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde81085f0 con 0x7efde8100600 2023-06-23T18:34:10.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efde77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80fea80 0x7efde8109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efde6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde80ffc70 0x7efde81079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efde77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80fea80 0x7efde8109370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56922/0 (socket says 172.21.15.2:56922) 2023-06-23T18:34:10.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efde7fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde8100600 0x7efde8107ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.760+0000 7efde77fe700 1 -- 172.21.15.2:0/601656104 learned_addr learned my addr 172.21.15.2:0/601656104 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde7fff700 1 -- 172.21.15.2:0/601656104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde80ffc70 msgr2=0x7efde81079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde7fff700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde80ffc70 0x7efde81079b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde7fff700 1 -- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80fea80 msgr2=0x7efde8109370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde7fff700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80fea80 0x7efde8109370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde7fff700 1 -- 172.21.15.2:0/601656104 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0095f0 con 0x7efde8100600 2023-06-23T18:34:10.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde7fff700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde8100600 0x7efde8107ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efde000efc0 tx=0x7efde000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde4ff9700 1 -- 172.21.15.2:0/601656104 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde0018070 con 0x7efde8100600 2023-06-23T18:34:10.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde4ff9700 1 -- 172.21.15.2:0/601656104 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efde000f040 con 0x7efde8100600 2023-06-23T18:34:10.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efdedc1d700 1 -- 172.21.15.2:0/601656104 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde81997e0 con 0x7efde8100600 2023-06-23T18:34:10.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efde4ff9700 1 -- 172.21.15.2:0/601656104 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde000cba0 con 0x7efde8100600 2023-06-23T18:34:10.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.761+0000 7efdedc1d700 1 -- 172.21.15.2:0/601656104 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde8199cf0 con 0x7efde8100600 2023-06-23T18:34:10.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.764+0000 7efde4ff9700 1 -- 172.21.15.2:0/601656104 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7efde0003a00 con 0x7efde8100600 2023-06-23T18:34:10.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.765+0000 7efde4ff9700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdd00a1220 0x7efdd00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.765+0000 7efde4ff9700 1 -- 172.21.15.2:0/601656104 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efde00cc390 con 0x7efde8100600 2023-06-23T18:34:10.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.765+0000 7efde77fe700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdd00a1220 0x7efdd00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efdf9da2b80 1 Processor -- start 2023-06-23T18:34:10.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efdf9da2b80 1 -- start start 2023-06-23T18:34:10.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efde77fe700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdd00a1220 0x7efdd00a36d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005950 tx=0x7efdd8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b93c80 0x555727b94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727b94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727b95470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727a98cb0 con 0x555727b95000 2023-06-23T18:34:10.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727aeba40 con 0x555727b93c80 2023-06-23T18:34:10.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727ae4ed0 con 0x555727b945d0 2023-06-23T18:34:10.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efde77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b93c80 0x555727b94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efde7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727b95470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efde77fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b93c80 0x555727b94090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47522/0 (socket says 172.21.15.2:47522) 2023-06-23T18:34:10.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efde77fe700 1 -- 172.21.15.2:0/1021680669 learned_addr learned my addr 172.21.15.2:0/1021680669 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efde6ffd700 1 --2- 172.21.15.2:0/1021680669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727b94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.766+0000 7efde77fe700 1 -- 172.21.15.2:0/1021680669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 msgr2=0x555727b94a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.767+0000 7efde77fe700 1 --2- 172.21.15.2:0/1021680669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727b94a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.767+0000 7efde77fe700 1 -- 172.21.15.2:0/1021680669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 msgr2=0x555727b95470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.767+0000 7efde77fe700 1 --2- 172.21.15.2:0/1021680669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727b95470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.767+0000 7efde77fe700 1 -- 172.21.15.2:0/1021680669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555727966630 con 0x555727b93c80 2023-06-23T18:34:10.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.767+0000 7efde77fe700 1 --2- 172.21.15.2:0/1021680669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b93c80 0x555727b94090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009800 tx=0x7efdd8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68a6d0569cd63018 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.767+0000 7efdcdffb700 1 -- 172.21.15.2:0/1021680669 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8026430 con 0x555727b93c80 2023-06-23T18:34:10.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.767+0000 7efdcdffb700 1 -- 172.21.15.2:0/1021680669 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdd8021070 con 0x555727b93c80 2023-06-23T18:34:10.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.767+0000 7efdcdffb700 1 -- 172.21.15.2:0/1021680669 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd801d670 con 0x555727b93c80 2023-06-23T18:34:10.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1021680669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b93c80 msgr2=0x555727b94090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1021680669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b93c80 0x555727b94090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009800 tx=0x7efdd8014000 comp rx=0 tx=0).stop 2023-06-23T18:34:10.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1021680669 shutdown_connections 2023-06-23T18:34:10.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1021680669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727b95470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1021680669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b93c80 0x555727b94090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1021680669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727b94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1021680669 >> 172.21.15.2:0/1021680669 conn(0x555727b893f0 msgr2=0x555727b8e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1021680669 shutdown_connections 2023-06-23T18:34:10.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.768+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1021680669 wait complete. 2023-06-23T18:34:10.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 1 -- start start 2023-06-23T18:34:10.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 1 Processor -- start 2023-06-23T18:34:10.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727baea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727a84ac0 con 0x555727b93c80 2023-06-23T18:34:10.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727a847c0 con 0x555727b95000 2023-06-23T18:34:10.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727968920 con 0x555727b945d0 2023-06-23T18:34:10.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.769+0000 7efdf9da2b80 10 client.0 _ll_register_callbacks cb 0x555727a8a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:10.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56936/0 (socket says 172.21.15.2:56936) 2023-06-23T18:34:10.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 -- 172.21.15.2:0/2844506964 learned_addr learned my addr 172.21.15.2:0/2844506964 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde6ffd700 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727baea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde7fff700 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 -- 172.21.15.2:0/2844506964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 msgr2=0x555727baea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727baea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 -- 172.21.15.2:0/2844506964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 msgr2=0x555727bb5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 -- 172.21.15.2:0/2844506964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd8017070 con 0x555727b93c80 2023-06-23T18:34:10.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efde77fe700 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bae510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efdd801cfa0 tx=0x7efdd801cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bea7c86efa6d2a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efdc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:34:10.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efdc6ffd700 1 -- 172.21.15.2:0/2844506964 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd801c620 con 0x555727b93c80 2023-06-23T18:34:10.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efdc6ffd700 1 -- 172.21.15.2:0/2844506964 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efdd801c780 con 0x555727b93c80 2023-06-23T18:34:10.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.770+0000 7efdf9da2b80 1 -- 172.21.15.2:0/2844506964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555727b86bb0 con 0x555727b93c80 2023-06-23T18:34:10.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdc6ffd700 1 -- 172.21.15.2:0/2844506964 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdd801c940 con 0x555727b93c80 2023-06-23T18:34:10.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdf9da2b80 10 client.16596 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:10.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdf9da2b80 1 -- 172.21.15.2:0/2844506964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x555727912de0 con 0x555727b93c80 2023-06-23T18:34:10.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdc6ffd700 1 -- 172.21.15.2:0/2844506964 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555727b86bb0 con 0x555727b93c80 2023-06-23T18:34:10.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdc6ffd700 1 -- 172.21.15.2:0/2844506964 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7efdd8005590 con 0x555727b93c80 2023-06-23T18:34:10.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdf9da2b80 10 client.16596 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:10.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdf9da2b80 10 client.16596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:34:10.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdf9da2b80 20 client.16596 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:10.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x555727bba180 0x555727bbc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdf9da2b80 4 client.16596 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:10.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.771+0000 7efdf9da2b80 1 -- 172.21.15.2:0/2844506964 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555727addeb0 con 0x555727bba180 2023-06-23T18:34:10.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.772+0000 7efde6ffd700 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x555727bba180 0x555727bbc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.772+0000 7efdc6ffd700 10 client.16596 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:34:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.772+0000 7efde6ffd700 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x555727bba180 0x555727bbc590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1d57314819a7b36 server_cookie=bfa83fc17f454437 in_seq=0 out_seq=0 2023-06-23T18:34:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.772+0000 7efdc6ffd700 1 -- 172.21.15.2:0/2844506964 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555727addeb0 con 0x555727bba180 2023-06-23T18:34:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.772+0000 7efdc6ffd700 10 client.16596 handle_command_reply: tid=0 2023-06-23T18:34:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 client.16596 shutdown 2023-06-23T18:34:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 -- 172.21.15.2:0/2844506964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 msgr2=0x555727bae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bae510 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efdd801cfa0 tx=0x7efdd801cfd0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 -- 172.21.15.2:0/2844506964 shutdown_connections 2023-06-23T18:34:10.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x555727bba180 0x555727bbc590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bae510 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdc6ffd700 0 client.16596 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:34:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/2844506964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727baea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 -- 172.21.15.2:0/2844506964 >> 172.21.15.2:0/2844506964 conn(0x555727b893f0 msgr2=0x555727b8dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 -- 172.21.15.2:0/2844506964 shutdown_connections 2023-06-23T18:34:10.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 1 -- 172.21.15.2:0/2844506964 wait complete. 2023-06-23T18:34:10.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.773+0000 7efdf9da2b80 20 client.16596 trim_cache size 0 max 16384 2023-06-23T18:34:10.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efdf9da2b80 1 Processor -- start 2023-06-23T18:34:10.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efdf9da2b80 1 -- start start 2023-06-23T18:34:10.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b93c80 0x555727b953d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b945d0 0x555727bb5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727bc0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727b8fd50 con 0x555727b95000 2023-06-23T18:34:10.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727b989b0 con 0x555727b945d0 2023-06-23T18:34:10.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727bc1430 con 0x555727b93c80 2023-06-23T18:34:10.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b93c80 0x555727b953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727bc0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b93c80 0x555727b953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57282/0 (socket says 172.21.15.2:57282) 2023-06-23T18:34:10.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde77fe700 1 -- 172.21.15.2:0/1615809261 learned_addr learned my addr 172.21.15.2:0/1615809261 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde6ffd700 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b945d0 0x555727bb5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde77fe700 1 -- 172.21.15.2:0/1615809261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b945d0 msgr2=0x555727bb5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde77fe700 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b945d0 0x555727bb5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde77fe700 1 -- 172.21.15.2:0/1615809261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 msgr2=0x555727bc0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde77fe700 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727bc0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde77fe700 1 -- 172.21.15.2:0/1615809261 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557278fbc60 con 0x555727b93c80 2023-06-23T18:34:10.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.792+0000 7efde6ffd700 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b945d0 0x555727bb5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:10.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.793+0000 7efde7fff700 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727bc0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:10.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.793+0000 7efde77fe700 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b93c80 0x555727b953d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efdd800e510 tx=0x7efdd8009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64aa4897bc6031cb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.793+0000 7efdce7fc700 1 -- 172.21.15.2:0/1615809261 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd803f070 con 0x555727b93c80 2023-06-23T18:34:10.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.793+0000 7efdce7fc700 1 -- 172.21.15.2:0/1615809261 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdd8004a80 con 0x555727b93c80 2023-06-23T18:34:10.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.793+0000 7efdce7fc700 1 -- 172.21.15.2:0/1615809261 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8004210 con 0x555727b93c80 2023-06-23T18:34:10.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1615809261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b93c80 msgr2=0x555727b953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b93c80 0x555727b953d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efdd800e510 tx=0x7efdd8009350 comp rx=0 tx=0).stop 2023-06-23T18:34:10.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1615809261 shutdown_connections 2023-06-23T18:34:10.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b95000 0x555727bc0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b93c80 0x555727b953d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1615809261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b945d0 0x555727bb5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1615809261 >> 172.21.15.2:0/1615809261 conn(0x555727b893f0 msgr2=0x555727bad270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1615809261 shutdown_connections 2023-06-23T18:34:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1615809261 wait complete. 2023-06-23T18:34:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.794+0000 7efdf9da2b80 1 -- start start 2023-06-23T18:34:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efdf9da2b80 1 Processor -- start 2023-06-23T18:34:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bb1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727bb1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efdf9da2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727bb53a0 con 0x555727b93c80 2023-06-23T18:34:10.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727bb5510 con 0x555727b95000 2023-06-23T18:34:10.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efdf9da2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555727bb56d0 con 0x555727b945d0 2023-06-23T18:34:10.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efdf9da2b80 10 client.0 _ll_register_callbacks cb 0x555727a8a260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:10.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bb1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727bb1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47546/0 (socket says 172.21.15.2:47546) 2023-06-23T18:34:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bb1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:56950/0 (socket says 172.21.15.2:56950) 2023-06-23T18:34:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde7fff700 1 -- 172.21.15.2:0/1471346339 learned_addr learned my addr 172.21.15.2:0/1471346339 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde6ffd700 1 -- 172.21.15.2:0/1471346339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 msgr2=0x555727bb4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde6ffd700 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.795+0000 7efde6ffd700 1 -- 172.21.15.2:0/1471346339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 msgr2=0x555727bb1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efde6ffd700 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bb1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efde6ffd700 1 -- 172.21.15.2:0/1471346339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde80677a0 con 0x555727b945d0 2023-06-23T18:34:10.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efde6ffd700 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727bb1e40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efddc000c00 tx=0x7efddc0051f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2155dbac9263adb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efdc6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:10.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efdc6ffd700 1 -- 172.21.15.2:0/1471346339 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc02c070 con 0x555727b945d0 2023-06-23T18:34:10.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efdc6ffd700 1 -- 172.21.15.2:0/1471346339 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efddc0054f0 con 0x555727b945d0 2023-06-23T18:34:10.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1471346339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdd8021070 con 0x555727b945d0 2023-06-23T18:34:10.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efdc6ffd700 1 -- 172.21.15.2:0/1471346339 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7efddc01d040 con 0x555727b945d0 2023-06-23T18:34:10.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efdf9da2b80 10 client.26311 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.796+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1471346339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7efde8199cf0 con 0x555727b945d0 2023-06-23T18:34:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdc6ffd700 1 -- 172.21.15.2:0/1471346339 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efdd8021070 con 0x555727b945d0 2023-06-23T18:34:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdc6ffd700 1 -- 172.21.15.2:0/1471346339 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7efddc00ba00 con 0x555727b945d0 2023-06-23T18:34:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdf9da2b80 10 client.26311 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdf9da2b80 10 client.26311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:34:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdf9da2b80 20 client.26311 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:10.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x555727c0f200 0x555727c11610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdf9da2b80 4 client.26311 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:34:10.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1471346339 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555727aa0050 con 0x555727c0f200 2023-06-23T18:34:10.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efde77fe700 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x555727c0f200 0x555727c11610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdc6ffd700 10 client.26311 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:34:10.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efde77fe700 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x555727c0f200 0x555727c11610 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf99fc0ece81eec6 server_cookie=37767597266b407c in_seq=0 out_seq=0 2023-06-23T18:34:10.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.797+0000 7efdc6ffd700 1 -- 172.21.15.2:0/1471346339 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555727aa0050 con 0x555727c0f200 2023-06-23T18:34:10.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdc6ffd700 10 client.26311 handle_command_reply: tid=0 2023-06-23T18:34:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 client.26311 shutdown 2023-06-23T18:34:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1471346339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 msgr2=0x555727bb1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727bb1e40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efddc000c00 tx=0x7efddc0051f0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1471346339 shutdown_connections 2023-06-23T18:34:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x555727c0f200 0x555727c11610 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x555727b95000 0x555727bb4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x555727b93c80 0x555727bb1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 --2- 172.21.15.2:0/1471346339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555727b945d0 0x555727bb1e40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1471346339 >> 172.21.15.2:0/1471346339 conn(0x555727b893f0 msgr2=0x555727bad270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdc6ffd700 0 client.26311 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:34:10.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1471346339 shutdown_connections 2023-06-23T18:34:10.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 1 -- 172.21.15.2:0/1471346339 wait complete. 2023-06-23T18:34:10.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.798+0000 7efdf9da2b80 20 client.26311 trim_cache size 0 max 16384 2023-06-23T18:34:10.814 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:34:10.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 -- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdd00a1220 msgr2=0x7efdd00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdd00a1220 0x7efdd00a36d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005950 tx=0x7efdd8009260 comp rx=0 tx=0).stop 2023-06-23T18:34:10.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 -- 172.21.15.2:0/601656104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde8100600 msgr2=0x7efde8107ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde8100600 0x7efde8107ef0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efde000efc0 tx=0x7efde000c3a0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 -- 172.21.15.2:0/601656104 shutdown_connections 2023-06-23T18:34:10.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efde8100600 0x7efde8107ef0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efde80fea80 0x7efde8109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efdd00a1220 0x7efdd00a36d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 --2- 172.21.15.2:0/601656104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde80ffc70 0x7efde81079b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 -- 172.21.15.2:0/601656104 >> 172.21.15.2:0/601656104 conn(0x7efde80fa0e0 msgr2=0x7efde80fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 -- 172.21.15.2:0/601656104 shutdown_connections 2023-06-23T18:34:10.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.799+0000 7efdedc1d700 1 -- 172.21.15.2:0/601656104 wait complete. 2023-06-23T18:34:10.821 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:34:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.984+0000 7f620fa91700 1 -- 172.21.15.2:0/2984154002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6208100040 msgr2=0x7f6208100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.984+0000 7f620fa91700 1 --2- 172.21.15.2:0/2984154002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6208100040 0x7f6208100450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f62040098f0 tx=0x7f6204009c00 comp rx=0 tx=0).stop 2023-06-23T18:34:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.984+0000 7f620fa91700 1 -- 172.21.15.2:0/2984154002 shutdown_connections 2023-06-23T18:34:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.984+0000 7f620fa91700 1 --2- 172.21.15.2:0/2984154002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62080ff640 0x7f6208106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.984+0000 7f620fa91700 1 --2- 172.21.15.2:0/2984154002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6208100040 0x7f6208100450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.984+0000 7f620fa91700 1 --2- 172.21.15.2:0/2984154002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f62080fec90 0x7f62080ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.984+0000 7f620fa91700 1 -- 172.21.15.2:0/2984154002 >> 172.21.15.2:0/2984154002 conn(0x7f62080f9850 msgr2=0x7f62080fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.985+0000 7f620fa91700 1 -- 172.21.15.2:0/2984154002 shutdown_connections 2023-06-23T18:34:10.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.985+0000 7f620fa91700 1 -- 172.21.15.2:0/2984154002 wait complete. 2023-06-23T18:34:10.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.985+0000 7f620fa91700 1 Processor -- start 2023-06-23T18:34:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.985+0000 7f620fa91700 1 -- start start 2023-06-23T18:34:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620fa91700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62080fec90 0x7f6208109360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620fa91700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62080ff640 0x7f62081079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620fa91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6208100040 0x7f6208107ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620fa91700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6208108680 con 0x7f6208100040 2023-06-23T18:34:10.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620fa91700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62081087f0 con 0x7f62080ff640 2023-06-23T18:34:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620fa91700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620819f870 con 0x7f62080fec90 2023-06-23T18:34:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62080ff640 0x7f62081079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d82d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62080fec90 0x7f6208109360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62080ff640 0x7f62081079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47558/0 (socket says 172.21.15.2:47558) 2023-06-23T18:34:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620e02e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6208100040 0x7f6208107ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 -- 172.21.15.2:0/1775384859 learned_addr learned my addr 172.21.15.2:0/1775384859 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:10.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 -- 172.21.15.2:0/1775384859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62080fec90 msgr2=0x7f6208109360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62080fec90 0x7f6208109360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 -- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6208100040 msgr2=0x7f6208107ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:10.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6208100040 0x7f6208107ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:10.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 -- 172.21.15.2:0/1775384859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204009520 con 0x7f62080ff640 2023-06-23T18:34:10.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.986+0000 7f620d02c700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62080ff640 0x7f62081079a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61f800e9c0 tx=0x7f61f800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.987+0000 7f61feffd700 1 -- 172.21.15.2:0/1775384859 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8004a30 con 0x7f62080ff640 2023-06-23T18:34:10.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.987+0000 7f61feffd700 1 -- 172.21.15.2:0/1775384859 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61f800cc50 con 0x7f62080ff640 2023-06-23T18:34:10.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.987+0000 7f61feffd700 1 -- 172.21.15.2:0/1775384859 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8005410 con 0x7f62080ff640 2023-06-23T18:34:10.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.987+0000 7f620fa91700 1 -- 172.21.15.2:0/1775384859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620819fb50 con 0x7f62080ff640 2023-06-23T18:34:10.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.987+0000 7f620fa91700 1 -- 172.21.15.2:0/1775384859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62081a0010 con 0x7f62080ff640 2023-06-23T18:34:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.989+0000 7f61feffd700 1 -- 172.21.15.2:0/1775384859 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f61f8005570 con 0x7f62080ff640 2023-06-23T18:34:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.990+0000 7f61feffd700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f61f40a1150 0x7f61f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.990+0000 7f61feffd700 1 -- 172.21.15.2:0/1775384859 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f61f8014070 con 0x7f62080ff640 2023-06-23T18:34:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.990+0000 7f620d82d700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f61f40a1150 0x7f61f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:10.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.991+0000 7f620d82d700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f61f40a1150 0x7f61f40a3600 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f620400f9d0 tx=0x7f6204019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.991+0000 7f621bc16b80 1 Processor -- start 2023-06-23T18:34:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f621bc16b80 1 -- start start 2023-06-23T18:34:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x5617840f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x5617840f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x5617840f5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f621bc16b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561783ff8cb0 con 0x5617840f3c80 2023-06-23T18:34:10.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f621bc16b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178404ba40 con 0x5617840f5000 2023-06-23T18:34:10.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f621bc16b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561784044ed0 con 0x5617840f45d0 2023-06-23T18:34:11.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f620e02e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x5617840f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f620d82d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x5617840f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f620d02c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x5617840f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f620e02e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x5617840f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47564/0 (socket says 172.21.15.2:47564) 2023-06-23T18:34:11.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.992+0000 7f620e02e700 1 -- 172.21.15.2:0/2045926694 learned_addr learned my addr 172.21.15.2:0/2045926694 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:11.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f620d02c700 1 -- 172.21.15.2:0/2045926694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 msgr2=0x5617840f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:11.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f620d02c700 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x5617840f5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f620d02c700 1 -- 172.21.15.2:0/2045926694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 msgr2=0x5617840f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:11.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f620d02c700 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x5617840f4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f620d02c700 1 -- 172.21.15.2:0/2045926694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561783ec6630 con 0x5617840f45d0 2023-06-23T18:34:11.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f620e02e700 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x5617840f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:11.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f620d82d700 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x5617840f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:11.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f620d02c700 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x5617840f4a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f61f8099b00 tx=0x7f61f8099a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f739499bc7e666 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:11.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f61f3fff700 1 -- 172.21.15.2:0/2045926694 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f809de30 con 0x5617840f45d0 2023-06-23T18:34:11.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f61f3fff700 1 -- 172.21.15.2:0/2045926694 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61f80a2250 con 0x5617840f45d0 2023-06-23T18:34:11.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.993+0000 7f61f3fff700 1 -- 172.21.15.2:0/2045926694 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f809a2c0 con 0x5617840f45d0 2023-06-23T18:34:11.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 -- 172.21.15.2:0/2045926694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 msgr2=0x5617840f4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:11.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x5617840f4a70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f61f8099b00 tx=0x7f61f8099a50 comp rx=0 tx=0).stop 2023-06-23T18:34:11.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 -- 172.21.15.2:0/2045926694 shutdown_connections 2023-06-23T18:34:11.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x5617840f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x5617840f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 --2- 172.21.15.2:0/2045926694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x5617840f4a70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 -- 172.21.15.2:0/2045926694 >> 172.21.15.2:0/2045926694 conn(0x5617840e93f0 msgr2=0x5617840ee520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:11.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 -- 172.21.15.2:0/2045926694 shutdown_connections 2023-06-23T18:34:11.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.994+0000 7f621bc16b80 1 -- 172.21.15.2:0/2045926694 wait complete. 2023-06-23T18:34:11.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 1 -- start start 2023-06-23T18:34:11.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 1 Processor -- start 2023-06-23T18:34:11.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x561784105da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f5000 0x56178410d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561783fe4ac0 con 0x5617840f3c80 2023-06-23T18:34:11.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561783fe47c0 con 0x5617840f45d0 2023-06-23T18:34:11.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561783ec8920 con 0x5617840f5000 2023-06-23T18:34:11.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.995+0000 7f621bc16b80 10 client.0 _ll_register_callbacks cb 0x561783fea260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:11.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f5000 0x56178410d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620d82d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x561784105da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f5000 0x56178410d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57320/0 (socket says 172.21.15.2:57320) 2023-06-23T18:34:11.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 -- 172.21.15.2:0/3308140989 learned_addr learned my addr 172.21.15.2:0/3308140989 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:11.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620d02c700 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 -- 172.21.15.2:0/3308140989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 msgr2=0x5617841062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:11.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 -- 172.21.15.2:0/3308140989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 msgr2=0x561784105da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:11.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x561784105da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 -- 172.21.15.2:0/3308140989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62000112b0 con 0x5617840f5000 2023-06-23T18:34:11.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f620e02e700 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f5000 0x56178410d320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62000145d0 tx=0x7f62000148e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8c4f5c6d2da1f3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:11.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f61f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:11.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f61f17fa700 1 -- 172.21.15.2:0/3308140989 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620000e040 con 0x5617840f5000 2023-06-23T18:34:11.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f61f17fa700 1 -- 172.21.15.2:0/3308140989 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f620001a070 con 0x5617840f5000 2023-06-23T18:34:11.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.996+0000 7f621bc16b80 1 -- 172.21.15.2:0/3308140989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617840e6bb0 con 0x5617840f5000 2023-06-23T18:34:11.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f61f17fa700 1 -- 172.21.15.2:0/3308140989 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6200011e50 con 0x5617840f5000 2023-06-23T18:34:11.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f621bc16b80 10 client.26323 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:11.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f621bc16b80 1 -- 172.21.15.2:0/3308140989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f62081a0010 con 0x5617840f5000 2023-06-23T18:34:11.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f61f17fa700 1 -- 172.21.15.2:0/3308140989 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5617840e6bb0 con 0x5617840f5000 2023-06-23T18:34:11.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f61f17fa700 1 -- 172.21.15.2:0/3308140989 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6200007d10 con 0x5617840f5000 2023-06-23T18:34:11.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f621bc16b80 10 client.26323 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:11.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f621bc16b80 10 client.26323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:11.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f621bc16b80 20 client.26323 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:11.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561784111c80 0x561784114090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f621bc16b80 4 client.26323 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:11.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.997+0000 7f621bc16b80 1 -- 172.21.15.2:0/3308140989 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56178403deb0 con 0x561784111c80 2023-06-23T18:34:11.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.998+0000 7f620d82d700 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561784111c80 0x561784114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.998+0000 7f61f17fa700 10 client.26323 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:11.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:10.998+0000 7f620d82d700 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561784111c80 0x561784114090 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae95a229093e9b1f server_cookie=817bef255935600e in_seq=0 out_seq=0 2023-06-23T18:34:11.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f61f17fa700 1 -- 172.21.15.2:0/3308140989 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56178403deb0 con 0x561784111c80 2023-06-23T18:34:11.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f61f17fa700 10 client.26323 handle_command_reply: tid=0 2023-06-23T18:34:11.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 client.26323 shutdown 2023-06-23T18:34:11.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 -- 172.21.15.2:0/3308140989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f5000 msgr2=0x56178410d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:11.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f5000 0x56178410d320 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62000145d0 tx=0x7f62000148e0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 -- 172.21.15.2:0/3308140989 shutdown_connections 2023-06-23T18:34:11.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561784111c80 0x561784114090 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f5000 0x56178410d320 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f61f17fa700 0 client.26323 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:11.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x561784105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3308140989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 -- 172.21.15.2:0/3308140989 >> 172.21.15.2:0/3308140989 conn(0x5617840e93f0 msgr2=0x5617840edc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:11.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 -- 172.21.15.2:0/3308140989 shutdown_connections 2023-06-23T18:34:11.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.000+0000 7f621bc16b80 1 -- 172.21.15.2:0/3308140989 wait complete. 2023-06-23T18:34:11.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.001+0000 7f621bc16b80 20 client.26323 trim_cache size 0 max 16384 2023-06-23T18:34:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.021+0000 7f621bc16b80 1 Processor -- start 2023-06-23T18:34:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.021+0000 7f621bc16b80 1 -- start start 2023-06-23T18:34:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x5617840fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x56178410d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x561784105da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f621bc16b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178403deb0 con 0x5617840f3c80 2023-06-23T18:34:11.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f621bc16b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617840eb470 con 0x5617840f5000 2023-06-23T18:34:11.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f621bc16b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178410f010 con 0x5617840f45d0 2023-06-23T18:34:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620d02c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x56178410d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620d02c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x56178410d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57324/0 (socket says 172.21.15.2:57324) 2023-06-23T18:34:11.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620d02c700 1 -- 172.21.15.2:0/1177427338 learned_addr learned my addr 172.21.15.2:0/1177427338 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620e02e700 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x561784105da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620d02c700 1 -- 172.21.15.2:0/1177427338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 msgr2=0x561784105da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620d02c700 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x561784105da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620d02c700 1 -- 172.21.15.2:0/1177427338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 msgr2=0x5617840fb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620d02c700 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x5617840fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620d02c700 1 -- 172.21.15.2:0/1177427338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561783e72de0 con 0x5617840f45d0 2023-06-23T18:34:11.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.022+0000 7f620e02e700 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x561784105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:34:11.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.023+0000 7f620d02c700 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x56178410d2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61f800f010 tx=0x7f61f8099810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2795a16b02db15d5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:11.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.023+0000 7f61fcdac700 1 -- 172.21.15.2:0/1177427338 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f8097f40 con 0x5617840f45d0 2023-06-23T18:34:11.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.023+0000 7f61fcdac700 1 -- 172.21.15.2:0/1177427338 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61f80981a0 con 0x5617840f45d0 2023-06-23T18:34:11.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.023+0000 7f61fcdac700 1 -- 172.21.15.2:0/1177427338 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f809a060 con 0x5617840f45d0 2023-06-23T18:34:11.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 -- 172.21.15.2:0/1177427338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 msgr2=0x56178410d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:11.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x56178410d2e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f61f800f010 tx=0x7f61f8099810 comp rx=0 tx=0).stop 2023-06-23T18:34:11.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 -- 172.21.15.2:0/1177427338 shutdown_connections 2023-06-23T18:34:11.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f5000 0x561784105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f3c80 0x5617840fb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 --2- 172.21.15.2:0/1177427338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f45d0 0x56178410d2e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 -- 172.21.15.2:0/1177427338 >> 172.21.15.2:0/1177427338 conn(0x5617840e93f0 msgr2=0x5617841189f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:11.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 -- 172.21.15.2:0/1177427338 shutdown_connections 2023-06-23T18:34:11.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 -- 172.21.15.2:0/1177427338 wait complete. 2023-06-23T18:34:11.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.024+0000 7f621bc16b80 1 -- start start 2023-06-23T18:34:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f621bc16b80 1 Processor -- start 2023-06-23T18:34:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f3c80 0x561784108eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f621bc16b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f5000 0x56178410c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f621bc16b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178410c950 con 0x5617840f5000 2023-06-23T18:34:11.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f621bc16b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178410cac0 con 0x5617840f45d0 2023-06-23T18:34:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f621bc16b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56178410cc80 con 0x5617840f3c80 2023-06-23T18:34:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f621bc16b80 10 client.0 _ll_register_callbacks cb 0x561783fea260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f620d02c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f620e02e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f5000 0x56178410c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f620d02c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47584/0 (socket says 172.21.15.2:47584) 2023-06-23T18:34:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f620d02c700 1 -- 172.21.15.2:0/3868676846 learned_addr learned my addr 172.21.15.2:0/3868676846 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.025+0000 7f620d82d700 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f3c80 0x561784108eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f620e02e700 1 -- 172.21.15.2:0/3868676846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f3c80 msgr2=0x561784108eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f620e02e700 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f3c80 0x561784108eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f620e02e700 1 -- 172.21.15.2:0/3868676846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 msgr2=0x5617841093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f620e02e700 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841093f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f620e02e700 1 -- 172.21.15.2:0/3868676846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561783ec6630 con 0x5617840f5000 2023-06-23T18:34:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f620e02e700 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f5000 0x56178410c410 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6200000f80 tx=0x7f6200011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43bfb256adde178 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f61f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:34:11.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f61f17fa700 1 -- 172.21.15.2:0/3868676846 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620000e070 con 0x5617840f5000 2023-06-23T18:34:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f61f17fa700 1 -- 172.21.15.2:0/3868676846 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f620001a040 con 0x5617840f5000 2023-06-23T18:34:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f621bc16b80 1 -- 172.21.15.2:0/3868676846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61f80a2250 con 0x5617840f5000 2023-06-23T18:34:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f61f17fa700 1 -- 172.21.15.2:0/3868676846 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f620001ec60 con 0x5617840f5000 2023-06-23T18:34:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f621bc16b80 10 client.16617 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.026+0000 7f621bc16b80 1 -- 172.21.15.2:0/3868676846 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x561783e5bc60 con 0x5617840f5000 2023-06-23T18:34:11.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f61f17fa700 1 -- 172.21.15.2:0/3868676846 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f61f80a2250 con 0x5617840f5000 2023-06-23T18:34:11.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f61f17fa700 1 -- 172.21.15.2:0/3868676846 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f620002f060 con 0x5617840f5000 2023-06-23T18:34:11.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f621bc16b80 10 client.16617 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:11.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f621bc16b80 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:11.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f621bc16b80 20 client.16617 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:11.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5617841668f0 0x561784168d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:11.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f621bc16b80 4 client.16617 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:34:11.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f621bc16b80 1 -- 172.21.15.2:0/3868676846 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561784000050 con 0x5617841668f0 2023-06-23T18:34:11.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.027+0000 7f620d82d700 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5617841668f0 0x561784168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:11.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.028+0000 7f620d82d700 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5617841668f0 0x561784168d00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f06bef56bc2cca server_cookie=66a2083015089f65 in_seq=0 out_seq=0 2023-06-23T18:34:11.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.028+0000 7f61f17fa700 10 client.16617 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:11.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.028+0000 7f61f17fa700 1 -- 172.21.15.2:0/3868676846 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561784000050 con 0x5617841668f0 2023-06-23T18:34:11.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.028+0000 7f61f17fa700 10 client.16617 handle_command_reply: tid=0 2023-06-23T18:34:11.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.028+0000 7f621bc16b80 1 client.16617 shutdown 2023-06-23T18:34:11.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.028+0000 7f621bc16b80 1 -- 172.21.15.2:0/3868676846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f5000 msgr2=0x56178410c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:11.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.028+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f5000 0x56178410c410 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6200000f80 tx=0x7f6200011de0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 1 -- 172.21.15.2:0/3868676846 shutdown_connections 2023-06-23T18:34:11.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5617841668f0 0x561784168d00 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5617840f5000 0x56178410c410 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5617840f3c80 0x561784108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f61f17fa700 0 client.16617 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:11.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 1 --2- 172.21.15.2:0/3868676846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5617840f45d0 0x5617841093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 1 -- 172.21.15.2:0/3868676846 >> 172.21.15.2:0/3868676846 conn(0x5617840e93f0 msgr2=0x5617840fb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:11.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 1 -- 172.21.15.2:0/3868676846 shutdown_connections 2023-06-23T18:34:11.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 1 -- 172.21.15.2:0/3868676846 wait complete. 2023-06-23T18:34:11.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f621bc16b80 20 client.16617 trim_cache size 0 max 16384 2023-06-23T18:34:11.042 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:34:11.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.029+0000 7f620fa91700 1 -- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f61f40a1150 msgr2=0x7f61f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:11.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f61f40a1150 0x7f61f40a3600 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f620400f9d0 tx=0x7f6204019040 comp rx=0 tx=0).stop 2023-06-23T18:34:11.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 -- 172.21.15.2:0/1775384859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62080ff640 msgr2=0x7f62081079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:11.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62080ff640 0x7f62081079a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f61f800e9c0 tx=0x7f61f800ecd0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 -- 172.21.15.2:0/1775384859 shutdown_connections 2023-06-23T18:34:11.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6208100040 0x7f6208107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62080fec90 0x7f6208109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f61f40a1150 0x7f61f40a3600 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 --2- 172.21.15.2:0/1775384859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62080ff640 0x7f62081079a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:11.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 -- 172.21.15.2:0/1775384859 >> 172.21.15.2:0/1775384859 conn(0x7f62080f9850 msgr2=0x7f6208102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:11.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 -- 172.21.15.2:0/1775384859 shutdown_connections 2023-06-23T18:34:11.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:11.030+0000 7f620fa91700 1 -- 172.21.15.2:0/1775384859 wait complete. 2023-06-23T18:34:11.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:11 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3087067892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:34:11.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:11 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3087067892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:34:11.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:11 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3087067892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:34:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 20.0 s, 401.8 tps, lat 247.818 ms stddev 132.183 2023-06-23T18:34:12.053 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:34:12.053 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:34:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/3450139812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c40fea80 msgr2=0x7fe3c40fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/3450139812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c40fea80 0x7fe3c40fee90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009990 tx=0x7fe3b4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/3450139812 shutdown_connections 2023-06-23T18:34:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/3450139812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3c4100600 0x7fe3c4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/3450139812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c40fea80 0x7fe3c40fee90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/3450139812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c40ffc70 0x7fe3c41000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/3450139812 >> 172.21.15.2:0/3450139812 conn(0x7fe3c40fa0e0 msgr2=0x7fe3c40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/3450139812 shutdown_connections 2023-06-23T18:34:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.238+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/3450139812 wait complete. 2023-06-23T18:34:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.239+0000 7fe3c9dcf700 1 Processor -- start 2023-06-23T18:34:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.239+0000 7fe3c9dcf700 1 -- start start 2023-06-23T18:34:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.239+0000 7fe3c9dcf700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c40fea80 0x7fe3c4068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c9dcf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c40ffc70 0x7fe3c419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c9dcf700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3c4100600 0x7fe3c419cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c9dcf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c419d380 con 0x7fe3c40ffc70 2023-06-23T18:34:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c9dcf700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c419d4f0 con 0x7fe3c40fea80 2023-06-23T18:34:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c9dcf700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c419d6b0 con 0x7fe3c4100600 2023-06-23T18:34:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c40ffc70 0x7fe3c419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c3fff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3c4100600 0x7fe3c419cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c40ffc70 0x7fe3c419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57008/0 (socket says 172.21.15.2:57008) 2023-06-23T18:34:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c2ffd700 1 -- 172.21.15.2:0/1568821492 learned_addr learned my addr 172.21.15.2:0/1568821492 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c2ffd700 1 -- 172.21.15.2:0/1568821492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3c4100600 msgr2=0x7fe3c419cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3c4100600 0x7fe3c419cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c2ffd700 1 -- 172.21.15.2:0/1568821492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c40fea80 msgr2=0x7fe3c4068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c40fea80 0x7fe3c4068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.240+0000 7fe3c2ffd700 1 -- 172.21.15.2:0/1568821492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b40095f0 con 0x7fe3c40ffc70 2023-06-23T18:34:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.241+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c40ffc70 0x7fe3c419a5a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80089e0 tx=0x7fe3b8008cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.241+0000 7fe3c0ff9700 1 -- 172.21.15.2:0/1568821492 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8016a50 con 0x7fe3c40ffc70 2023-06-23T18:34:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.241+0000 7fe3c0ff9700 1 -- 172.21.15.2:0/1568821492 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3b800d070 con 0x7fe3c40ffc70 2023-06-23T18:34:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.241+0000 7fe3c0ff9700 1 -- 172.21.15.2:0/1568821492 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b801e360 con 0x7fe3c40ffc70 2023-06-23T18:34:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.241+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/1568821492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c41943d0 con 0x7fe3c40ffc70 2023-06-23T18:34:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.241+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/1568821492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c4194910 con 0x7fe3c40ffc70 2023-06-23T18:34:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.246+0000 7fe3c0ff9700 1 -- 172.21.15.2:0/1568821492 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe3b8009630 con 0x7fe3c40ffc70 2023-06-23T18:34:12.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.246+0000 7fe3d5f54b80 1 Processor -- start 2023-06-23T18:34:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3d5f54b80 1 -- start start 2023-06-23T18:34:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3c0ff9700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3ac0a12a0 0x7fe3ac0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x5583175e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x5583175e7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x5583175e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583174ebd30 con 0x5583175e6d00 2023-06-23T18:34:12.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831753eac0 con 0x5583175e7650 2023-06-23T18:34:12.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558317537f50 con 0x5583175e8080 2023-06-23T18:34:12.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3c2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x5583175e7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3c2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x5583175e7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47606/0 (socket says 172.21.15.2:47606) 2023-06-23T18:34:12.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3c2ffd700 1 -- 172.21.15.2:0/3108482351 learned_addr learned my addr 172.21.15.2:0/3108482351 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x5583175e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x5583175e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.247+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3ac0a12a0 0x7fe3ac0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c0ff9700 1 -- 172.21.15.2:0/1568821492 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3b80d3c90 con 0x7fe3c40ffc70 2023-06-23T18:34:12.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c37fe700 1 -- 172.21.15.2:0/3108482351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 msgr2=0x5583175e84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x5583175e84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c37fe700 1 -- 172.21.15.2:0/3108482351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 msgr2=0x5583175e7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x5583175e7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c37fe700 1 -- 172.21.15.2:0/3108482351 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583173b96b0 con 0x5583175e6d00 2023-06-23T18:34:12.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x5583175e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3ac0a12a0 0x7fe3ac0a3750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009550 tx=0x7fe3bc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x5583175e7110 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003f60 tx=0x7fe3b4009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e8cf9fb98e5d7e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3a9ffb700 1 -- 172.21.15.2:0/3108482351 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b401d070 con 0x5583175e6d00 2023-06-23T18:34:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3a9ffb700 1 -- 172.21.15.2:0/3108482351 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3b4004cc0 con 0x5583175e6d00 2023-06-23T18:34:12.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.248+0000 7fe3a9ffb700 1 -- 172.21.15.2:0/3108482351 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4005480 con 0x5583175e6d00 2023-06-23T18:34:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/3108482351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 msgr2=0x5583175e7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x5583175e7110 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4003f60 tx=0x7fe3b4009550 comp rx=0 tx=0).stop 2023-06-23T18:34:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/3108482351 shutdown_connections 2023-06-23T18:34:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x5583175e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x5583175e7110 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/3108482351 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x5583175e7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/3108482351 >> 172.21.15.2:0/3108482351 conn(0x5583175dc470 msgr2=0x5583175e15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/3108482351 shutdown_connections 2023-06-23T18:34:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.249+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/3108482351 wait complete. 2023-06-23T18:34:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.250+0000 7fe3d5f54b80 1 -- start start 2023-06-23T18:34:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.250+0000 7fe3d5f54b80 1 Processor -- start 2023-06-23T18:34:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.250+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e6d00 0x558317601580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.250+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317601ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e8080 0x558317608b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583174d7b40 con 0x5583175e7650 2023-06-23T18:34:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583174d7840 con 0x5583175e8080 2023-06-23T18:34:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583173bb9a0 con 0x5583175e6d00 2023-06-23T18:34:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3d5f54b80 10 client.0 _ll_register_callbacks cb 0x5583174dd2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3c37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e6d00 0x558317601580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3c37fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e6d00 0x558317601580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57358/0 (socket says 172.21.15.2:57358) 2023-06-23T18:34:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3c37fe700 1 -- 172.21.15.2:0/2213732530 learned_addr learned my addr 172.21.15.2:0/2213732530 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3c37fe700 1 -- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e8080 msgr2=0x558317608b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317601ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.251+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e8080 0x558317608b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e8080 0x558317608b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3c37fe700 1 -- 172.21.15.2:0/2213732530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 msgr2=0x558317601ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317601ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3c37fe700 1 -- 172.21.15.2:0/2213732530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c4194910 con 0x5583175e6d00 2023-06-23T18:34:12.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317601ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:34:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e8080 0x558317608b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:34:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e6d00 0x558317601580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4006010 tx=0x7fe3b4004ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e241cf0e13fdbe44 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/2213732530 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b401d070 con 0x5583175e6d00 2023-06-23T18:34:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.252+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2213732530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583175d9c30 con 0x5583175e6d00 2023-06-23T18:34:12.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/2213732530 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3b4003680 con 0x5583175e6d00 2023-06-23T18:34:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/2213732530 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3b4005710 con 0x5583175e6d00 2023-06-23T18:34:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3d5f54b80 10 client.26341 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2213732530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fe3c41077f0 con 0x5583175e6d00 2023-06-23T18:34:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/2213732530 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5583175d9c30 con 0x5583175e6d00 2023-06-23T18:34:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/2213732530 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe3b4022470 con 0x5583175e6d00 2023-06-23T18:34:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3d5f54b80 10 client.26341 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:12.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3d5f54b80 10 client.26341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:12.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3d5f54b80 20 client.26341 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:12.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55831761dd80 0x558317620160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3d5f54b80 4 client.26341 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:12.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.253+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2213732530 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558317530f30 con 0x55831761dd80 2023-06-23T18:34:12.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.254+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55831761dd80 0x558317620160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.254+0000 7fe3a2ffd700 10 client.26341 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:12.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.254+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55831761dd80 0x558317620160 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb7df07324766ffe server_cookie=b6812d2edb78e4fb in_seq=0 out_seq=0 2023-06-23T18:34:12.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/2213732530 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558317530f30 con 0x55831761dd80 2023-06-23T18:34:12.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3a2ffd700 10 client.26341 handle_command_reply: tid=0 2023-06-23T18:34:12.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 client.26341 shutdown 2023-06-23T18:34:12.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2213732530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e6d00 msgr2=0x558317601580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e6d00 0x558317601580 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4006010 tx=0x7fe3b4004ac0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2213732530 shutdown_connections 2023-06-23T18:34:12.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55831761dd80 0x558317620160 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e8080 0x558317608b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3a2ffd700 0 client.26341 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:12.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e6d00 0x558317601580 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2213732530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317601ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2213732530 >> 172.21.15.2:0/2213732530 conn(0x5583175dc470 msgr2=0x5583175e0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2213732530 shutdown_connections 2023-06-23T18:34:12.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2213732530 wait complete. 2023-06-23T18:34:12.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.255+0000 7fe3d5f54b80 20 client.26341 trim_cache size 0 max 16384 2023-06-23T18:34:12.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3d5f54b80 1 Processor -- start 2023-06-23T18:34:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3d5f54b80 1 -- start start 2023-06-23T18:34:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e6d00 0x558317601a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317608ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x5583175e0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558317601150 con 0x5583175e7650 2023-06-23T18:34:12.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583175ffe90 con 0x5583175e6d00 2023-06-23T18:34:12.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558317530f30 con 0x5583175e8080 2023-06-23T18:34:12.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e6d00 0x558317601a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3c37fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e6d00 0x558317601a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47620/0 (socket says 172.21.15.2:47620) 2023-06-23T18:34:12.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3c37fe700 1 -- 172.21.15.2:0/2159942389 learned_addr learned my addr 172.21.15.2:0/2159942389 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x5583175e0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317608ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.288+0000 7fe3c37fe700 1 -- 172.21.15.2:0/2159942389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 msgr2=0x5583175e0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.289+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x5583175e0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.289+0000 7fe3c37fe700 1 -- 172.21.15.2:0/2159942389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 msgr2=0x558317608ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.289+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317608ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.289+0000 7fe3c37fe700 1 -- 172.21.15.2:0/2159942389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558317600500 con 0x5583175e6d00 2023-06-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.289+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e6d00 0x558317601a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4022800 tx=0x7fe3b40229a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2180b14d89ee8e91 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.289+0000 7fe3aa7fc700 1 -- 172.21.15.2:0/2159942389 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b401d070 con 0x5583175e6d00 2023-06-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.289+0000 7fe3aa7fc700 1 -- 172.21.15.2:0/2159942389 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3b401e040 con 0x5583175e6d00 2023-06-23T18:34:12.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.289+0000 7fe3aa7fc700 1 -- 172.21.15.2:0/2159942389 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4003d70 con 0x5583175e6d00 2023-06-23T18:34:12.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2159942389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e6d00 msgr2=0x558317601a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e6d00 0x558317601a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4022800 tx=0x7fe3b40229a0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2159942389 shutdown_connections 2023-06-23T18:34:12.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x5583175e0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e6d00 0x558317601a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/2159942389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e7650 0x558317608ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2159942389 >> 172.21.15.2:0/2159942389 conn(0x5583175dc470 msgr2=0x558317601330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2159942389 shutdown_connections 2023-06-23T18:34:12.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.290+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/2159942389 wait complete. 2023-06-23T18:34:12.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.291+0000 7fe3d5f54b80 1 -- start start 2023-06-23T18:34:12.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.291+0000 7fe3d5f54b80 1 Processor -- start 2023-06-23T18:34:12.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.291+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x558317605970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.291+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x558317605eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.291+0000 7fe3d5f54b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x55831760cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.291+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831760d430 con 0x5583175e6d00 2023-06-23T18:34:12.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.291+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831760d5a0 con 0x5583175e7650 2023-06-23T18:34:12.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.291+0000 7fe3d5f54b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831760d760 con 0x5583175e8080 2023-06-23T18:34:12.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3d5f54b80 10 client.0 _ll_register_callbacks cb 0x5583174dd2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:12.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x558317605970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c2ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x558317605eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x558317605970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57034/0 (socket says 172.21.15.2:57034) 2023-06-23T18:34:12.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 -- 172.21.15.2:0/969366076 learned_addr learned my addr 172.21.15.2:0/969366076 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x55831760cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 -- 172.21.15.2:0/969366076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 msgr2=0x55831760cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x55831760cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 -- 172.21.15.2:0/969366076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 msgr2=0x558317605eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x558317605eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 -- 172.21.15.2:0/969366076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c41077f0 con 0x5583175e6d00 2023-06-23T18:34:12.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c3fff700 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x55831760cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:12.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x558317605eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:12.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.292+0000 7fe3c37fe700 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x558317605970 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40093f0 tx=0x7fe3b400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b45064c9450251 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.293+0000 7fe3a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:34:12.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.293+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/969366076 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b401d070 con 0x5583175e6d00 2023-06-23T18:34:12.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.293+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/969366076 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3b4022a90 con 0x5583175e6d00 2023-06-23T18:34:12.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.293+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/969366076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3b4004cc0 con 0x5583175e6d00 2023-06-23T18:34:12.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.293+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/969366076 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3b4003880 con 0x5583175e6d00 2023-06-23T18:34:12.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.293+0000 7fe3d5f54b80 10 client.16629 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:12.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.293+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/969366076 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558317365e60 con 0x5583175e6d00 2023-06-23T18:34:12.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/969366076 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3b4004cc0 con 0x5583175e6d00 2023-06-23T18:34:12.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/969366076 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe3b4036460 con 0x5583175e6d00 2023-06-23T18:34:12.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3d5f54b80 10 client.16629 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:12.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3d5f54b80 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:12.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3d5f54b80 20 client.16629 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:12.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55831760ec10 0x558317610ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3d5f54b80 4 client.16629 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:34:12.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/969366076 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5583174f30d0 con 0x55831760ec10 2023-06-23T18:34:12.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.294+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55831760ec10 0x558317610ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.295+0000 7fe3a2ffd700 10 client.16629 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:12.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.295+0000 7fe3c2ffd700 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55831760ec10 0x558317610ff0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84cd922b953ad6d7 server_cookie=b739593e4f4190b3 in_seq=0 out_seq=0 2023-06-23T18:34:12.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.297+0000 7fe3a2ffd700 1 -- 172.21.15.2:0/969366076 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5583174f30d0 con 0x55831760ec10 2023-06-23T18:34:12.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.297+0000 7fe3a2ffd700 10 client.16629 handle_command_reply: tid=0 2023-06-23T18:34:12.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.297+0000 7fe3d5f54b80 1 client.16629 shutdown 2023-06-23T18:34:12.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.297+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/969366076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 msgr2=0x558317605970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.297+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x558317605970 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40093f0 tx=0x7fe3b400b390 comp rx=0 tx=0).stop 2023-06-23T18:34:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.297+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/969366076 shutdown_connections 2023-06-23T18:34:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.297+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55831760ec10 0x558317610ff0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.297+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583175e8080 0x55831760cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5583175e6d00 0x558317605970 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3d5f54b80 1 --2- 172.21.15.2:0/969366076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5583175e7650 0x558317605eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/969366076 >> 172.21.15.2:0/969366076 conn(0x5583175dc470 msgr2=0x5583175e5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3a2ffd700 0 client.16629 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:12.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/969366076 shutdown_connections 2023-06-23T18:34:12.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3d5f54b80 1 -- 172.21.15.2:0/969366076 wait complete. 2023-06-23T18:34:12.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3d5f54b80 20 client.16629 trim_cache size 0 max 16384 2023-06-23T18:34:12.312 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:34:12.313 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:34:12.313 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "a5d2a35b-01b9-4791-9b30-4724067f8dd6", 2023-06-23T18:34:12.313 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:34:12.313 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:34:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3ac0a12a0 msgr2=0x7fe3ac0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3ac0a12a0 0x7fe3ac0a3750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009550 tx=0x7fe3bc00b040 comp rx=0 tx=0).stop 2023-06-23T18:34:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.298+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c40ffc70 msgr2=0x7fe3c419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c40ffc70 0x7fe3c419a5a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe3b80089e0 tx=0x7fe3b8008cf0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/1568821492 shutdown_connections 2023-06-23T18:34:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3c4100600 0x7fe3c419cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3ac0a12a0 0x7fe3ac0a3750 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3c40fea80 0x7fe3c4068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 --2- 172.21.15.2:0/1568821492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3c40ffc70 0x7fe3c419a5a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/1568821492 >> 172.21.15.2:0/1568821492 conn(0x7fe3c40fa0e0 msgr2=0x7fe3c40fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/1568821492 shutdown_connections 2023-06-23T18:34:12.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.299+0000 7fe3c9dcf700 1 -- 172.21.15.2:0/1568821492 wait complete. 2023-06-23T18:34:12.321 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:34:12.352 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:12 smithi002 ceph-mon[95947]: pgmap v661: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 9.9 MiB/s wr, 519 op/s 2023-06-23T18:34:12.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 -- 172.21.15.2:0/2798115388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae740ffc70 msgr2=0x7fae741000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2798115388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae740ffc70 0x7fae741000c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fae60009990 tx=0x7fae60009ca0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 -- 172.21.15.2:0/2798115388 shutdown_connections 2023-06-23T18:34:12.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2798115388 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae74100600 0x7fae74106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2798115388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae740fea80 0x7fae740fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2798115388 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae740ffc70 0x7fae741000c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 -- 172.21.15.2:0/2798115388 >> 172.21.15.2:0/2798115388 conn(0x7fae740fa0e0 msgr2=0x7fae740fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 -- 172.21.15.2:0/2798115388 shutdown_connections 2023-06-23T18:34:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.507+0000 7fae7abf7700 1 -- 172.21.15.2:0/2798115388 wait complete. 2023-06-23T18:34:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae7abf7700 1 Processor -- start 2023-06-23T18:34:12.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae7abf7700 1 -- start start 2023-06-23T18:34:12.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae7abf7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae740fea80 0x7fae74195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae7abf7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae740ffc70 0x7fae74195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae7abf7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae74100600 0x7fae7419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae7abf7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7419d300 con 0x7fae74100600 2023-06-23T18:34:12.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae7abf7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7419d440 con 0x7fae740ffc70 2023-06-23T18:34:12.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae7abf7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7419d5d0 con 0x7fae740fea80 2023-06-23T18:34:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae74100600 0x7fae7419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae74100600 0x7fae7419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57052/0 (socket says 172.21.15.2:57052) 2023-06-23T18:34:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 -- 172.21.15.2:0/2719141918 learned_addr learned my addr 172.21.15.2:0/2719141918 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae78993700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae740fea80 0x7fae74195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae73fff700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae740ffc70 0x7fae74195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 -- 172.21.15.2:0/2719141918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae740fea80 msgr2=0x7fae74195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae740fea80 0x7fae74195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 -- 172.21.15.2:0/2719141918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae740ffc70 msgr2=0x7fae74195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae740ffc70 0x7fae74195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 -- 172.21.15.2:0/2719141918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae600095f0 con 0x7fae74100600 2023-06-23T18:34:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.508+0000 7fae79194700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae74100600 0x7fae7419cd30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fae6c007ee0 tx=0x7fae6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.509+0000 7fae71ffb700 1 -- 172.21.15.2:0/2719141918 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c017070 con 0x7fae74100600 2023-06-23T18:34:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.509+0000 7fae71ffb700 1 -- 172.21.15.2:0/2719141918 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae6c00f040 con 0x7fae74100600 2023-06-23T18:34:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.509+0000 7fae7abf7700 1 -- 172.21.15.2:0/2719141918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae7419d8b0 con 0x7fae74100600 2023-06-23T18:34:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.509+0000 7fae71ffb700 1 -- 172.21.15.2:0/2719141918 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c005570 con 0x7fae74100600 2023-06-23T18:34:12.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.509+0000 7fae7abf7700 1 -- 172.21.15.2:0/2719141918 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7419de10 con 0x7fae74100600 2023-06-23T18:34:12.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.511+0000 7fae71ffb700 1 -- 172.21.15.2:0/2719141918 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fae6c00db20 con 0x7fae74100600 2023-06-23T18:34:12.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.511+0000 7fae71ffb700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae640a11d0 0x7fae640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.512+0000 7fae78993700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae640a11d0 0x7fae640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.512+0000 7fae71ffb700 1 -- 172.21.15.2:0/2719141918 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae6c0cbfd0 con 0x7fae74100600 2023-06-23T18:34:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.512+0000 7fae78993700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae640a11d0 0x7fae640a3680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fae740ffad0 tx=0x7fae68009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.512+0000 7fae86d7cb80 1 Processor -- start 2023-06-23T18:34:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.512+0000 7fae86d7cb80 1 -- start start 2023-06-23T18:34:12.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20802c80 0x558b20803090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b20803a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20804000 0x558b20804470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b20707cb0 con 0x558b20804000 2023-06-23T18:34:12.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2075aa40 con 0x558b20802c80 2023-06-23T18:34:12.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b20753ed0 con 0x558b208035d0 2023-06-23T18:34:12.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20804000 0x558b20804470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20804000 0x558b20804470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57060/0 (socket says 172.21.15.2:57060) 2023-06-23T18:34:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae78993700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20802c80 0x558b20803090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 -- 172.21.15.2:0/2931114929 learned_addr learned my addr 172.21.15.2:0/2931114929 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae73fff700 1 --2- 172.21.15.2:0/2931114929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b20803a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 -- 172.21.15.2:0/2931114929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 msgr2=0x558b20803a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 --2- 172.21.15.2:0/2931114929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b20803a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 -- 172.21.15.2:0/2931114929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20802c80 msgr2=0x558b20803090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 --2- 172.21.15.2:0/2931114929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20802c80 0x558b20803090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 -- 172.21.15.2:0/2931114929 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b205d5630 con 0x558b20804000 2023-06-23T18:34:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae79194700 1 --2- 172.21.15.2:0/2931114929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20804000 0x558b20804470 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0958a0 tx=0x7fae6c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f78a58ef5dcb2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae5effd700 1 -- 172.21.15.2:0/2931114929 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c004c40 con 0x558b20804000 2023-06-23T18:34:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae5effd700 1 -- 172.21.15.2:0/2931114929 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae6c00ddd0 con 0x558b20804000 2023-06-23T18:34:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.513+0000 7fae5effd700 1 -- 172.21.15.2:0/2931114929 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c097740 con 0x558b20804000 2023-06-23T18:34:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 -- 172.21.15.2:0/2931114929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20804000 msgr2=0x558b20804470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/2931114929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20804000 0x558b20804470 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0958a0 tx=0x7fae6c004b50 comp rx=0 tx=0).stop 2023-06-23T18:34:12.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 -- 172.21.15.2:0/2931114929 shutdown_connections 2023-06-23T18:34:12.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/2931114929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20804000 0x558b20804470 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/2931114929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20802c80 0x558b20803090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/2931114929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b20803a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 -- 172.21.15.2:0/2931114929 >> 172.21.15.2:0/2931114929 conn(0x558b207f83f0 msgr2=0x558b207fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 -- 172.21.15.2:0/2931114929 shutdown_connections 2023-06-23T18:34:12.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.514+0000 7fae86d7cb80 1 -- 172.21.15.2:0/2931114929 wait complete. 2023-06-23T18:34:12.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 1 -- start start 2023-06-23T18:34:12.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 1 Processor -- start 2023-06-23T18:34:12.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b208035d0 0x558b2080cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20804000 0x558b20813b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b206f3ac0 con 0x558b20802c80 2023-06-23T18:34:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b206f37c0 con 0x558b208035d0 2023-06-23T18:34:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b205d7920 con 0x558b20804000 2023-06-23T18:34:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae86d7cb80 10 client.0 _ll_register_callbacks cb 0x558b206f9260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae78993700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae78993700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57062/0 (socket says 172.21.15.2:57062) 2023-06-23T18:34:12.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae78993700 1 -- 172.21.15.2:0/1683809745 learned_addr learned my addr 172.21.15.2:0/1683809745 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae79194700 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20804000 0x558b20813b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.515+0000 7fae73fff700 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b208035d0 0x558b2080cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae78993700 1 -- 172.21.15.2:0/1683809745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20804000 msgr2=0x558b20813b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae78993700 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20804000 0x558b20813b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae78993700 1 -- 172.21.15.2:0/1683809745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b208035d0 msgr2=0x558b2080cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae78993700 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b208035d0 0x558b2080cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae78993700 1 -- 172.21.15.2:0/1683809745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7419de10 con 0x558b20802c80 2023-06-23T18:34:12.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae79194700 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20804000 0x558b20813b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae78993700 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080c5a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fae6801dc00 tx=0x7fae680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5035c136b190cf1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae53fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:34:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae53fff700 1 -- 172.21.15.2:0/1683809745 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae68022070 con 0x558b20802c80 2023-06-23T18:34:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae53fff700 1 -- 172.21.15.2:0/1683809745 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae6801c610 con 0x558b20802c80 2023-06-23T18:34:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1683809745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b207f5bb0 con 0x558b20802c80 2023-06-23T18:34:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae53fff700 1 -- 172.21.15.2:0/1683809745 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae6801c7d0 con 0x558b20802c80 2023-06-23T18:34:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae86d7cb80 10 client.16641 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.516+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1683809745 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558b20581de0 con 0x558b20802c80 2023-06-23T18:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae53fff700 1 -- 172.21.15.2:0/1683809745 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558b207f5bb0 con 0x558b20802c80 2023-06-23T18:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae53fff700 1 -- 172.21.15.2:0/1683809745 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fae68031a30 con 0x558b20802c80 2023-06-23T18:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae86d7cb80 10 client.16641 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae86d7cb80 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae86d7cb80 20 client.16641 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558b20818330 0x558b2081a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae86d7cb80 4 client.16641 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1683809745 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b2074ceb0 con 0x558b20818330 2023-06-23T18:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae73fff700 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558b20818330 0x558b2081a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae53fff700 10 client.16641 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.517+0000 7fae73fff700 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558b20818330 0x558b2081a740 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d8e0b503b0eaa6 server_cookie=6090c5b77f07b8d7 in_seq=0 out_seq=0 2023-06-23T18:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae53fff700 1 -- 172.21.15.2:0/1683809745 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558b2074ceb0 con 0x558b20818330 2023-06-23T18:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae53fff700 10 client.16641 handle_command_reply: tid=0 2023-06-23T18:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 client.16641 shutdown 2023-06-23T18:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1683809745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 msgr2=0x558b2080c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080c5a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fae6801dc00 tx=0x7fae680047c0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1683809745 shutdown_connections 2023-06-23T18:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558b20818330 0x558b2081a740 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20804000 0x558b20813b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae53fff700 0 client.16641 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:12.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080c5a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1683809745 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b208035d0 0x558b2080cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1683809745 >> 172.21.15.2:0/1683809745 conn(0x558b207f83f0 msgr2=0x558b207fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1683809745 shutdown_connections 2023-06-23T18:34:12.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1683809745 wait complete. 2023-06-23T18:34:12.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.518+0000 7fae86d7cb80 20 client.16641 trim_cache size 0 max 16384 2023-06-23T18:34:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.544+0000 7fae86d7cb80 1 Processor -- start 2023-06-23T18:34:12.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.544+0000 7fae86d7cb80 1 -- start start 2023-06-23T18:34:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.544+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20802c80 0x558b2081f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.544+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b208035d0 0x558b20813ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.544+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b20814410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.544+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b20815810 con 0x558b208035d0 2023-06-23T18:34:12.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.544+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b20807b00 con 0x558b20804000 2023-06-23T18:34:12.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.544+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2081f5e0 con 0x558b20802c80 2023-06-23T18:34:12.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae73fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b208035d0 0x558b20813ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae78993700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20802c80 0x558b2081f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae73fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b208035d0 0x558b20813ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57070/0 (socket says 172.21.15.2:57070) 2023-06-23T18:34:12.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae78993700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20802c80 0x558b2081f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57408/0 (socket says 172.21.15.2:57408) 2023-06-23T18:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae73fff700 1 -- 172.21.15.2:0/1233091972 learned_addr learned my addr 172.21.15.2:0/1233091972 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae79194700 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b20814410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae79194700 1 -- 172.21.15.2:0/1233091972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20802c80 msgr2=0x558b2081f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae79194700 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20802c80 0x558b2081f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae79194700 1 -- 172.21.15.2:0/1233091972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b208035d0 msgr2=0x558b20813ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae79194700 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b208035d0 0x558b20813ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae79194700 1 -- 172.21.15.2:0/1233091972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b2056ac60 con 0x558b20804000 2023-06-23T18:34:12.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae73fff700 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b208035d0 0x558b20813ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:12.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae79194700 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b20814410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae6c099560 tx=0x7fae6c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a16b65489e04b1d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae5f7fe700 1 -- 172.21.15.2:0/1233091972 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0b5070 con 0x558b20804000 2023-06-23T18:34:12.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.545+0000 7fae5f7fe700 1 -- 172.21.15.2:0/1233091972 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae6c004c40 con 0x558b20804000 2023-06-23T18:34:12.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.546+0000 7fae5f7fe700 1 -- 172.21.15.2:0/1233091972 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0b1070 con 0x558b20804000 2023-06-23T18:34:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.546+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1233091972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 msgr2=0x558b20814410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.546+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b20814410 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae6c099560 tx=0x7fae6c000f40 comp rx=0 tx=0).stop 2023-06-23T18:34:12.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.546+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1233091972 shutdown_connections 2023-06-23T18:34:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.546+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b20814410 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.546+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b20802c80 0x558b2081f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.546+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/1233091972 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b208035d0 0x558b20813ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.546+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1233091972 >> 172.21.15.2:0/1233091972 conn(0x558b207f83f0 msgr2=0x558b207f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.547+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1233091972 shutdown_connections 2023-06-23T18:34:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.547+0000 7fae86d7cb80 1 -- 172.21.15.2:0/1233091972 wait complete. 2023-06-23T18:34:12.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.547+0000 7fae86d7cb80 1 -- start start 2023-06-23T18:34:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.547+0000 7fae86d7cb80 1 Processor -- start 2023-06-23T18:34:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b2080e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae86d7cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b208130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b20813630 con 0x558b20802c80 2023-06-23T18:34:12.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b208137a0 con 0x558b20804000 2023-06-23T18:34:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae86d7cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b20813960 con 0x558b208035d0 2023-06-23T18:34:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae86d7cb80 10 client.0 _ll_register_callbacks cb 0x558b206f9260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:12.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b208130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae78993700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b208130f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47670/0 (socket says 172.21.15.2:47670) 2023-06-23T18:34:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae78993700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57076/0 (socket says 172.21.15.2:57076) 2023-06-23T18:34:12.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 -- 172.21.15.2:0/618385630 learned_addr learned my addr 172.21.15.2:0/618385630 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae73fff700 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b2080e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 -- 172.21.15.2:0/618385630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 msgr2=0x558b2080e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b2080e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 -- 172.21.15.2:0/618385630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 msgr2=0x558b2080dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:12.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 -- 172.21.15.2:0/618385630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae741077f0 con 0x558b20804000 2023-06-23T18:34:12.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae73fff700 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b2080e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:12.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae79194700 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b208130f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae6c095120 tx=0x7fae6c007420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd51c1b28fc46cb9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:12.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.548+0000 7fae53fff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:34:12.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae53fff700 1 -- 172.21.15.2:0/618385630 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0b5030 con 0x558b20804000 2023-06-23T18:34:12.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae53fff700 1 -- 172.21.15.2:0/618385630 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae6c0b6070 con 0x558b20804000 2023-06-23T18:34:12.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 1 -- 172.21.15.2:0/618385630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fae6c00ddd0 con 0x558b20804000 2023-06-23T18:34:12.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae53fff700 1 -- 172.21.15.2:0/618385630 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae6c0b1070 con 0x558b20804000 2023-06-23T18:34:12.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 10 client.26153 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:12.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 1 -- 172.21.15.2:0/618385630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x558b205d5630 con 0x558b20804000 2023-06-23T18:34:12.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae53fff700 1 -- 172.21.15.2:0/618385630 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae6c00ddd0 con 0x558b20804000 2023-06-23T18:34:12.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae53fff700 1 -- 172.21.15.2:0/618385630 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fae6c0c3c60 con 0x558b20804000 2023-06-23T18:34:12.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 10 client.26153 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:12.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 10 client.26153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:12.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 20 client.26153 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:12.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558b2086a470 0x558b2086c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:12.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 4 client.26153 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:34:12.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.549+0000 7fae86d7cb80 1 -- 172.21.15.2:0/618385630 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b2070f050 con 0x558b2086a470 2023-06-23T18:34:12.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.550+0000 7fae78993700 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558b2086a470 0x558b2086c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:12.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.551+0000 7fae78993700 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558b2086a470 0x558b2086c850 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b6162eebfee801 server_cookie=c07db850f9abceb4 in_seq=0 out_seq=0 2023-06-23T18:34:12.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.551+0000 7fae53fff700 10 client.26153 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:12.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae53fff700 1 -- 172.21.15.2:0/618385630 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558b2070f050 con 0x558b2086a470 2023-06-23T18:34:12.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae53fff700 10 client.26153 handle_command_reply: tid=0 2023-06-23T18:34:12.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 client.26153 shutdown 2023-06-23T18:34:12.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 -- 172.21.15.2:0/618385630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 msgr2=0x558b208130f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b208130f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fae6c095120 tx=0x7fae6c007420 comp rx=0 tx=0).stop 2023-06-23T18:34:12.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 -- 172.21.15.2:0/618385630 shutdown_connections 2023-06-23T18:34:12.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558b2086a470 0x558b2086c850 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558b20804000 0x558b208130f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558b20802c80 0x558b2080dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae53fff700 0 client.26153 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:12.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 --2- 172.21.15.2:0/618385630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558b208035d0 0x558b2080e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 -- 172.21.15.2:0/618385630 >> 172.21.15.2:0/618385630 conn(0x558b207f83f0 msgr2=0x558b20814410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 -- 172.21.15.2:0/618385630 shutdown_connections 2023-06-23T18:34:12.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.552+0000 7fae86d7cb80 1 -- 172.21.15.2:0/618385630 wait complete. 2023-06-23T18:34:12.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.553+0000 7fae86d7cb80 20 client.26153 trim_cache size 0 max 16384 2023-06-23T18:34:12.573 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:34:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:34:12.573 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:34:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "a5d2a35b-01b9-4791-9b30-4724067f8dd6": { 2023-06-23T18:34:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:34:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "a5d2a35b-01b9-4791-9b30-4724067f8dd6", 2023-06-23T18:34:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:34:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:34:12.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:34:12.574 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:34:12.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.553+0000 7fae7abf7700 1 -- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae640a11d0 msgr2=0x7fae640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.553+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae640a11d0 0x7fae640a3680 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fae740ffad0 tx=0x7fae68009260 comp rx=0 tx=0).stop 2023-06-23T18:34:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.553+0000 7fae7abf7700 1 -- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae74100600 msgr2=0x7fae7419cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.553+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae74100600 0x7fae7419cd30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fae6c007ee0 tx=0x7fae6c00d3b0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.553+0000 7fae7abf7700 1 -- 172.21.15.2:0/2719141918 shutdown_connections 2023-06-23T18:34:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.554+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae74100600 0x7fae7419cd30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.554+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae740fea80 0x7fae74195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.554+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fae640a11d0 0x7fae640a3680 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.554+0000 7fae7abf7700 1 --2- 172.21.15.2:0/2719141918 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fae740ffc70 0x7fae74195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.554+0000 7fae7abf7700 1 -- 172.21.15.2:0/2719141918 >> 172.21.15.2:0/2719141918 conn(0x7fae740fa0e0 msgr2=0x7fae74067860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.554+0000 7fae7abf7700 1 -- 172.21.15.2:0/2719141918 shutdown_connections 2023-06-23T18:34:12.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:12.554+0000 7fae7abf7700 1 -- 172.21.15.2:0/2719141918 wait complete. 2023-06-23T18:34:12.579 INFO:tasks.cephfs.filesystem:scrub status for tag:a5d2a35b-01b9-4791-9b30-4724067f8dd6 - {'path': '/', 'tag': 'a5d2a35b-01b9-4791-9b30-4724067f8dd6', 'options': 'recursive,force'} 2023-06-23T18:34:12.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:12 smithi082 ceph-mon[97889]: pgmap v661: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 9.9 MiB/s wr, 519 op/s 2023-06-23T18:34:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:12 smithi017 ceph-mon[97472]: pgmap v661: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 732 KiB/s rd, 9.9 MiB/s wr, 519 op/s 2023-06-23T18:34:13.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:13 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:34:13.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:13 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:13 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:34:13.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:13 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:34:13.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:13 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:34:13.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:13 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:13 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.553 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:13 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:13 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:34:13.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:13 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:13 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:34:13.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:13 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:34:13.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:13 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:34:13.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:13 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:13 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.600 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:13 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:13 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:34:13.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:13 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:13 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:34:13.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:13 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:34:13.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:13 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:34:13.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:13 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:13 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:13.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:13 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:14.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:14 smithi002 ceph-mon[95947]: pgmap v662: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 9.3 MiB/s wr, 561 op/s 2023-06-23T18:34:14.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:14 smithi082 ceph-mon[97889]: pgmap v662: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 9.3 MiB/s wr, 561 op/s 2023-06-23T18:34:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:14 smithi017 ceph-mon[97472]: pgmap v662: 97 pgs: 97 active+clean; 8.3 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 9.3 MiB/s wr, 561 op/s 2023-06-23T18:34:16.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:16 smithi002 ceph-mon[95947]: pgmap v663: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 11 MiB/s wr, 638 op/s 2023-06-23T18:34:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:16 smithi082 ceph-mon[97889]: pgmap v663: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 11 MiB/s wr, 638 op/s 2023-06-23T18:34:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:16 smithi017 ceph-mon[97472]: pgmap v663: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 695 KiB/s rd, 11 MiB/s wr, 638 op/s 2023-06-23T18:34:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 25.0 s, 391.0 tps, lat 254.294 ms stddev 54.668 2023-06-23T18:34:18.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:18 smithi002 ceph-mon[95947]: pgmap v664: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 12 MiB/s wr, 748 op/s 2023-06-23T18:34:18.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:18 smithi082 ceph-mon[97889]: pgmap v664: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 12 MiB/s wr, 748 op/s 2023-06-23T18:34:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:18 smithi017 ceph-mon[97472]: pgmap v664: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 12 MiB/s wr, 748 op/s 2023-06-23T18:34:20.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:20 smithi002 ceph-mon[95947]: pgmap v665: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 13 MiB/s wr, 927 op/s 2023-06-23T18:34:20.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:20 smithi082 ceph-mon[97889]: pgmap v665: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 13 MiB/s wr, 927 op/s 2023-06-23T18:34:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:20 smithi017 ceph-mon[97472]: pgmap v665: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 13 MiB/s wr, 927 op/s 2023-06-23T18:34:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 30.0 s, 498.8 tps, lat 201.715 ms stddev 36.459 2023-06-23T18:34:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:22 smithi002 ceph-mon[95947]: pgmap v666: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:34:22.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:22 smithi082 ceph-mon[97889]: pgmap v666: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:34:22.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:22 smithi017 ceph-mon[97472]: pgmap v666: 97 pgs: 97 active+clean; 8.3 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:34:24.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:24 smithi002 ceph-mon[95947]: pgmap v667: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:34:24.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:24 smithi082 ceph-mon[97889]: pgmap v667: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:34:24.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:24 smithi017 ceph-mon[97472]: pgmap v667: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:34:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:26 smithi002 ceph-mon[95947]: pgmap v668: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-06-23T18:34:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:26 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:34:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:26 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:34:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:26 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:34:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:26 smithi082 ceph-mon[97889]: pgmap v668: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-06-23T18:34:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:26 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:34:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:26 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:34:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:26 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:34:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:26 smithi017 ceph-mon[97472]: pgmap v668: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 16 MiB/s wr, 1.23k op/s 2023-06-23T18:34:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:26 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:34:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:26 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:34:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:26 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:34:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 35.0 s, 465.2 tps, lat 214.408 ms stddev 38.581 2023-06-23T18:34:28.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:28 smithi002 ceph-mon[95947]: pgmap v669: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 15 MiB/s wr, 1.23k op/s 2023-06-23T18:34:28.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:28 smithi082 ceph-mon[97889]: pgmap v669: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 15 MiB/s wr, 1.23k op/s 2023-06-23T18:34:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:28 smithi017 ceph-mon[97472]: pgmap v669: 97 pgs: 97 active+clean; 8.3 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 15 MiB/s wr, 1.23k op/s 2023-06-23T18:34:30.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:30 smithi002 ceph-mon[95947]: pgmap v670: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2023-06-23T18:34:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:30 smithi082 ceph-mon[97889]: pgmap v670: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2023-06-23T18:34:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:30 smithi017 ceph-mon[97472]: pgmap v670: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 15 MiB/s wr, 1.24k op/s 2023-06-23T18:34:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 40.0 s, 372.6 tps, lat 268.467 ms stddev 52.994 2023-06-23T18:34:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:32 smithi002 ceph-mon[95947]: pgmap v671: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-06-23T18:34:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:32 smithi082 ceph-mon[97889]: pgmap v671: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-06-23T18:34:32.689 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:32 smithi017 ceph-mon[97472]: pgmap v671: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 16 MiB/s wr, 1.25k op/s 2023-06-23T18:34:34.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:34 smithi082 ceph-mon[97889]: pgmap v672: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-06-23T18:34:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:34 smithi017 ceph-mon[97472]: pgmap v672: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-06-23T18:34:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:34 smithi002 ceph-mon[95947]: pgmap v672: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 15 MiB/s wr, 1.20k op/s 2023-06-23T18:34:36.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:36 smithi082 ceph-mon[97889]: pgmap v673: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2023-06-23T18:34:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:36 smithi017 ceph-mon[97472]: pgmap v673: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2023-06-23T18:34:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:36 smithi002 ceph-mon[95947]: pgmap v673: 97 pgs: 97 active+clean; 8.3 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 666 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2023-06-23T18:34:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 45.0 s, 421.8 tps, lat 235.737 ms stddev 31.380 2023-06-23T18:34:38.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:38 smithi082 ceph-mon[97889]: pgmap v674: 97 pgs: 97 active+clean; 8.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2023-06-23T18:34:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:38 smithi017 ceph-mon[97472]: pgmap v674: 97 pgs: 97 active+clean; 8.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2023-06-23T18:34:38.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:38 smithi002 ceph-mon[95947]: pgmap v674: 97 pgs: 97 active+clean; 8.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 14 MiB/s wr, 1.19k op/s 2023-06-23T18:34:40.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:40 smithi082 ceph-mon[97889]: pgmap v675: 97 pgs: 97 active+clean; 8.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2023-06-23T18:34:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:40 smithi017 ceph-mon[97472]: pgmap v675: 97 pgs: 97 active+clean; 8.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2023-06-23T18:34:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:40 smithi002 ceph-mon[95947]: pgmap v675: 97 pgs: 97 active+clean; 8.3 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2023-06-23T18:34:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 50.0 s, 397.8 tps, lat 252.484 ms stddev 55.844 2023-06-23T18:34:42.580 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:34:42.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:42 smithi082 ceph-mon[97889]: pgmap v676: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-06-23T18:34:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:42 smithi017 ceph-mon[97472]: pgmap v676: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-06-23T18:34:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:42 smithi002 ceph-mon[95947]: pgmap v676: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 16 MiB/s wr, 1.37k op/s 2023-06-23T18:34:44.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:44 smithi082 ceph-mon[97889]: pgmap v677: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2023-06-23T18:34:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:44 smithi017 ceph-mon[97472]: pgmap v677: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2023-06-23T18:34:44.828 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:44 smithi002 ceph-mon[95947]: pgmap v677: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 15 MiB/s wr, 1.31k op/s 2023-06-23T18:34:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/3471387521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 msgr2=0x7f6e580fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:45.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/3471387521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 0x7f6e580fe630 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009990 tx=0x7f6e4c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/3471387521 shutdown_connections 2023-06-23T18:34:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/3471387521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e580ffda0 0x7f6e58106660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/3471387521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 0x7f6e580fe630 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/3471387521 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580ff410 0x7f6e580ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/3471387521 >> 172.21.15.2:0/3471387521 conn(0x7f6e580f9820 msgr2=0x7f6e580fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:45.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/3471387521 shutdown_connections 2023-06-23T18:34:45.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.696+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/3471387521 wait complete. 2023-06-23T18:34:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5f4cc700 1 Processor -- start 2023-06-23T18:34:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5f4cc700 1 -- start start 2023-06-23T18:34:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5f4cc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 0x7f6e5810c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5f4cc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580ff410 0x7f6e5810c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5f4cc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e580ffda0 0x7f6e58108fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5f4cc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58109770 con 0x7f6e580fe220 2023-06-23T18:34:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5f4cc700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e581098e0 con 0x7f6e580ff410 2023-06-23T18:34:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5f4cc700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58109aa0 con 0x7f6e580ffda0 2023-06-23T18:34:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5d268700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 0x7f6e5810c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5d268700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 0x7f6e5810c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57092/0 (socket says 172.21.15.2:57092) 2023-06-23T18:34:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5d268700 1 -- 172.21.15.2:0/1040915594 learned_addr learned my addr 172.21.15.2:0/1040915594 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:45.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.697+0000 7f6e5da69700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e580ffda0 0x7f6e58108fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e5da69700 1 -- 172.21.15.2:0/1040915594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580ff410 msgr2=0x7f6e5810c980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:34:45.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e5da69700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580ff410 0x7f6e5810c980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e5da69700 1 -- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 msgr2=0x7f6e5810c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:45.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e5da69700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 0x7f6e5810c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e5da69700 1 -- 172.21.15.2:0/1040915594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009c40 con 0x7f6e580ffda0 2023-06-23T18:34:45.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e5da69700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e580ffda0 0x7f6e58108fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e970 tx=0x7f6e5000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:45.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e567fc700 1 -- 172.21.15.2:0/1040915594 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000cbd0 con 0x7f6e580ffda0 2023-06-23T18:34:45.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e567fc700 1 -- 172.21.15.2:0/1040915594 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e5000ee60 con 0x7f6e580ffda0 2023-06-23T18:34:45.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.698+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/1040915594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c0095f0 con 0x7f6e580ffda0 2023-06-23T18:34:45.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.699+0000 7f6e567fc700 1 -- 172.21.15.2:0/1040915594 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50005020 con 0x7f6e580ffda0 2023-06-23T18:34:45.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.699+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/1040915594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e581a4230 con 0x7f6e580ffda0 2023-06-23T18:34:45.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.701+0000 7f6e567fc700 1 -- 172.21.15.2:0/1040915594 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6e50005180 con 0x7f6e580ffda0 2023-06-23T18:34:45.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.702+0000 7f6e567fc700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6e400a1130 0x7f6e400a35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.702+0000 7f6e567fc700 1 -- 172.21.15.2:0/1040915594 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e50014070 con 0x7f6e580ffda0 2023-06-23T18:34:45.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.702+0000 7f6e5d268700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6e400a1130 0x7f6e400a35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.702+0000 7f6e5d268700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6e400a1130 0x7f6e400a35e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c005b40 tx=0x7f6e4c005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:45.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.925+0000 7f6e6b651b80 1 Processor -- start 2023-06-23T18:34:45.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e6b651b80 1 -- start start 2023-06-23T18:34:45.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6ae5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6ae5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6ae6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc69e9cb0 con 0x562bc6ae55d0 2023-06-23T18:34:45.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6a3ca40 con 0x562bc6ae6000 2023-06-23T18:34:45.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6a35ed0 con 0x562bc6ae4c80 2023-06-23T18:34:45.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e5ca67700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6ae5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e5ca67700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6ae5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57098/0 (socket says 172.21.15.2:57098) 2023-06-23T18:34:45.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e5d268700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6ae5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e5ca67700 1 -- 172.21.15.2:0/3572790992 learned_addr learned my addr 172.21.15.2:0/3572790992 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:45.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.926+0000 7f6e5da69700 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6ae6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e5d268700 1 -- 172.21.15.2:0/3572790992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 msgr2=0x562bc6ae6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:45.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e5d268700 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6ae6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e5d268700 1 -- 172.21.15.2:0/3572790992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 msgr2=0x562bc6ae5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:45.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e5d268700 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6ae5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e5d268700 1 -- 172.21.15.2:0/3572790992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc68b7630 con 0x562bc6ae4c80 2023-06-23T18:34:45.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e5da69700 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6ae6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:34:45.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e5d268700 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6ae5090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006010 tx=0x7f6e4c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b19d49ded45a34e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:45.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e3f7fe700 1 -- 172.21.15.2:0/3572790992 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c02b070 con 0x562bc6ae4c80 2023-06-23T18:34:45.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e3f7fe700 1 -- 172.21.15.2:0/3572790992 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e4c01e040 con 0x562bc6ae4c80 2023-06-23T18:34:45.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.927+0000 7f6e3f7fe700 1 -- 172.21.15.2:0/3572790992 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c004760 con 0x562bc6ae4c80 2023-06-23T18:34:45.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3572790992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 msgr2=0x562bc6ae5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:45.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6ae5090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006010 tx=0x7f6e4c020000 comp rx=0 tx=0).stop 2023-06-23T18:34:45.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3572790992 shutdown_connections 2023-06-23T18:34:45.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6ae6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6ae5090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3572790992 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6ae5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3572790992 >> 172.21.15.2:0/3572790992 conn(0x562bc6ada3f0 msgr2=0x562bc6adf520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:45.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3572790992 shutdown_connections 2023-06-23T18:34:45.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.928+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3572790992 wait complete. 2023-06-23T18:34:45.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.929+0000 7f6e6b651b80 1 -- start start 2023-06-23T18:34:45.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.929+0000 7f6e6b651b80 1 Processor -- start 2023-06-23T18:34:45.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6aee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae55d0 0x562bc6aeea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae6000 0x562bc6af5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc69d5ac0 con 0x562bc6ae6000 2023-06-23T18:34:45.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc69d57c0 con 0x562bc6ae4c80 2023-06-23T18:34:45.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc68b9920 con 0x562bc6ae55d0 2023-06-23T18:34:45.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e6b651b80 10 client.0 _ll_register_callbacks cb 0x562bc69db260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:45.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5d268700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6aee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5d268700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6aee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47696/0 (socket says 172.21.15.2:47696) 2023-06-23T18:34:45.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5d268700 1 -- 172.21.15.2:0/3341457134 learned_addr learned my addr 172.21.15.2:0/3341457134 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:45.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5ca67700 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae55d0 0x562bc6aeea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5da69700 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae6000 0x562bc6af5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5d268700 1 -- 172.21.15.2:0/3341457134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae55d0 msgr2=0x562bc6aeea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:45.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5d268700 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae55d0 0x562bc6aeea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5d268700 1 -- 172.21.15.2:0/3341457134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae6000 msgr2=0x562bc6af5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:45.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5d268700 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae6000 0x562bc6af5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5d268700 1 -- 172.21.15.2:0/3341457134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e58106f90 con 0x562bc6ae4c80 2023-06-23T18:34:45.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.930+0000 7f6e5da69700 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae6000 0x562bc6af5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:34:45.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e5d268700 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6aee510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0041d0 tx=0x7f6e4c004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923653aa9086d04e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:45.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:34:45.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3341457134 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c02b040 con 0x562bc6ae4c80 2023-06-23T18:34:45.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3341457134 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e4c004a20 con 0x562bc6ae4c80 2023-06-23T18:34:45.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3341457134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bc6ad7bb0 con 0x562bc6ae4c80 2023-06-23T18:34:45.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3341457134 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e4c00bac0 con 0x562bc6ae4c80 2023-06-23T18:34:45.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e6b651b80 10 client.26162 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:45.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3341457134 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x562bc6863de0 con 0x562bc6ae4c80 2023-06-23T18:34:45.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3341457134 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562bc6ad7bb0 con 0x562bc6ae4c80 2023-06-23T18:34:45.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3341457134 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6e4c00fc50 con 0x562bc6ae4c80 2023-06-23T18:34:45.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e6b651b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:45.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e6b651b80 10 client.26162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:45.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e6b651b80 20 client.26162 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:45.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562bc6af9da0 0x562bc6afc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.931+0000 7f6e6b651b80 4 client.26162 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:45.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.932+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3341457134 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bc6a2eeb0 con 0x562bc6af9da0 2023-06-23T18:34:45.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.932+0000 7f6e5ca67700 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562bc6af9da0 0x562bc6afc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.932+0000 7f6e5ca67700 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562bc6af9da0 0x562bc6afc180 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879ac6e2e685fd42 server_cookie=387c7f08090224b in_seq=0 out_seq=0 2023-06-23T18:34:45.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.932+0000 7f6e3cff9700 10 client.26162 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:45.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3341457134 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562bc6a2eeb0 con 0x562bc6af9da0 2023-06-23T18:34:45.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e3cff9700 10 client.26162 handle_command_reply: tid=0 2023-06-23T18:34:45.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 client.26162 shutdown 2023-06-23T18:34:45.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3341457134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 msgr2=0x562bc6aee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:45.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6aee510 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0041d0 tx=0x7f6e4c004860 comp rx=0 tx=0).stop 2023-06-23T18:34:45.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3341457134 shutdown_connections 2023-06-23T18:34:45.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562bc6af9da0 0x562bc6afc180 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae6000 0x562bc6af5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6aee510 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3341457134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae55d0 0x562bc6aeea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3341457134 >> 172.21.15.2:0/3341457134 conn(0x562bc6ada3f0 msgr2=0x562bc6adec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:45.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e3cff9700 0 client.26162 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:45.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3341457134 shutdown_connections 2023-06-23T18:34:45.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3341457134 wait complete. 2023-06-23T18:34:45.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.933+0000 7f6e6b651b80 20 client.26162 trim_cache size 0 max 16384 2023-06-23T18:34:45.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.959+0000 7f6e6b651b80 1 Processor -- start 2023-06-23T18:34:45.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.959+0000 7f6e6b651b80 1 -- start start 2023-06-23T18:34:45.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6ae8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6af5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae6000 0x562bc6ae94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6adfe60 con 0x562bc6ae55d0 2023-06-23T18:34:45.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6aed9f0 con 0x562bc6ae4c80 2023-06-23T18:34:45.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6ae0450 con 0x562bc6ae6000 2023-06-23T18:34:45.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5d268700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6ae8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5d268700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6ae8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47704/0 (socket says 172.21.15.2:47704) 2023-06-23T18:34:45.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5d268700 1 -- 172.21.15.2:0/2903236692 learned_addr learned my addr 172.21.15.2:0/2903236692 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:45.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5ca67700 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6af5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5da69700 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae6000 0x562bc6ae94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5d268700 1 -- 172.21.15.2:0/2903236692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae6000 msgr2=0x562bc6ae94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:45.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5d268700 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae6000 0x562bc6ae94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5d268700 1 -- 172.21.15.2:0/2903236692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 msgr2=0x562bc6af5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:45.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5d268700 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6af5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.960+0000 7f6e5d268700 1 -- 172.21.15.2:0/2903236692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bc684cc60 con 0x562bc6ae4c80 2023-06-23T18:34:45.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.961+0000 7f6e5d268700 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6ae8f60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c004200 tx=0x7f6e4c0053a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bcb42ae8be2fd4a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:45.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.961+0000 7f6e3ffff700 1 -- 172.21.15.2:0/2903236692 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c028070 con 0x562bc6ae4c80 2023-06-23T18:34:45.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.961+0000 7f6e3ffff700 1 -- 172.21.15.2:0/2903236692 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e4c00bd50 con 0x562bc6ae4c80 2023-06-23T18:34:45.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.961+0000 7f6e3ffff700 1 -- 172.21.15.2:0/2903236692 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c02b070 con 0x562bc6ae4c80 2023-06-23T18:34:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 -- 172.21.15.2:0/2903236692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 msgr2=0x562bc6ae8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:45.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6ae8f60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c004200 tx=0x7f6e4c0053a0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 -- 172.21.15.2:0/2903236692 shutdown_connections 2023-06-23T18:34:45.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae6000 0x562bc6ae94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae4c80 0x562bc6ae8f60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/2903236692 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6af5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 -- 172.21.15.2:0/2903236692 >> 172.21.15.2:0/2903236692 conn(0x562bc6ada3f0 msgr2=0x562bc6aed200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:45.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 -- 172.21.15.2:0/2903236692 shutdown_connections 2023-06-23T18:34:45.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 -- 172.21.15.2:0/2903236692 wait complete. 2023-06-23T18:34:45.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 -- start start 2023-06-23T18:34:45.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.962+0000 7f6e6b651b80 1 Processor -- start 2023-06-23T18:34:45.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6af1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6af20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e6b651b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6af50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6af5610 con 0x562bc6ae55d0 2023-06-23T18:34:45.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6af5780 con 0x562bc6ae6000 2023-06-23T18:34:45.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e6b651b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bc6af5940 con 0x562bc6ae4c80 2023-06-23T18:34:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e6b651b80 10 client.0 _ll_register_callbacks cb 0x562bc69db260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6af50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5d268700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6af1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6af50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47714/0 (socket says 172.21.15.2:47714) 2023-06-23T18:34:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 -- 172.21.15.2:0/3721637735 learned_addr learned my addr 172.21.15.2:0/3721637735 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 -- 172.21.15.2:0/3721637735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 msgr2=0x562bc6af1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6af1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 -- 172.21.15.2:0/3721637735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 msgr2=0x562bc6af20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:45.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6af20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 -- 172.21.15.2:0/3721637735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bc6863de0 con 0x562bc6ae6000 2023-06-23T18:34:45.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e5da69700 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6af50d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6e50096580 tx=0x7f6e5009a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b46a786ea62d10 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:45.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.963+0000 7f6e3cff9700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:34:45.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3721637735 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50099b10 con 0x562bc6ae6000 2023-06-23T18:34:45.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3721637735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6e4c01e040 con 0x562bc6ae6000 2023-06-23T18:34:45.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3721637735 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6e5009db40 con 0x562bc6ae6000 2023-06-23T18:34:45.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3721637735 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e500a1f60 con 0x562bc6ae6000 2023-06-23T18:34:45.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 10 client.26168 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:45.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3721637735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x562bc688a250 con 0x562bc6ae6000 2023-06-23T18:34:45.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3721637735 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6e4c01e040 con 0x562bc6ae6000 2023-06-23T18:34:45.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3721637735 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6e50095970 con 0x562bc6ae6000 2023-06-23T18:34:45.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 10 client.26168 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:45.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 10 client.26168 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:45.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 20 client.26168 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:45.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562bc6b4efa0 0x562bc6b513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:45.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 4 client.26168 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:34:45.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.964+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3721637735 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562bc69f1050 con 0x562bc6b4efa0 2023-06-23T18:34:45.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.965+0000 7f6e5d268700 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562bc6b4efa0 0x562bc6b513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:45.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.965+0000 7f6e5d268700 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562bc6b4efa0 0x562bc6b513b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e07d768718a45e server_cookie=8604e21b3d2046d in_seq=0 out_seq=0 2023-06-23T18:34:45.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.965+0000 7f6e3cff9700 10 client.26168 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:45.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e3cff9700 1 -- 172.21.15.2:0/3721637735 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562bc69f1050 con 0x562bc6b4efa0 2023-06-23T18:34:45.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e3cff9700 10 client.26168 handle_command_reply: tid=0 2023-06-23T18:34:45.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e6b651b80 1 client.26168 shutdown 2023-06-23T18:34:45.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3721637735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 msgr2=0x562bc6af50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:45.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6af50d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6e50096580 tx=0x7f6e5009a050 comp rx=0 tx=0).stop 2023-06-23T18:34:45.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3721637735 shutdown_connections 2023-06-23T18:34:45.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562bc6b4efa0 0x562bc6b513b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562bc6ae6000 0x562bc6af50d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562bc6ae4c80 0x562bc6af1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e6b651b80 1 --2- 172.21.15.2:0/3721637735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562bc6ae55d0 0x562bc6af20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.966+0000 7f6e3cff9700 0 client.26168 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:45.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.967+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3721637735 >> 172.21.15.2:0/3721637735 conn(0x562bc6ada3f0 msgr2=0x562bc6ae8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:45.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.967+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3721637735 shutdown_connections 2023-06-23T18:34:45.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.967+0000 7f6e6b651b80 1 -- 172.21.15.2:0/3721637735 wait complete. 2023-06-23T18:34:45.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.967+0000 7f6e6b651b80 20 client.26168 trim_cache size 0 max 16384 2023-06-23T18:34:45.979 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:34:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "no active scrubs running", 2023-06-23T18:34:45.979 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": {} 2023-06-23T18:34:45.979 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:34:45.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6e400a1130 msgr2=0x7f6e400a35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:45.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6e400a1130 0x7f6e400a35e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c005b40 tx=0x7f6e4c005a90 comp rx=0 tx=0).stop 2023-06-23T18:34:45.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/1040915594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e580ffda0 msgr2=0x7f6e58108fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:45.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e580ffda0 0x7f6e58108fd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e970 tx=0x7f6e5000ec80 comp rx=0 tx=0).stop 2023-06-23T18:34:45.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/1040915594 shutdown_connections 2023-06-23T18:34:45.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e580ffda0 0x7f6e58108fd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e580fe220 0x7f6e5810c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6e400a1130 0x7f6e400a35e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 --2- 172.21.15.2:0/1040915594 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6e580ff410 0x7f6e5810c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/1040915594 >> 172.21.15.2:0/1040915594 conn(0x7f6e580f9820 msgr2=0x7f6e581029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/1040915594 shutdown_connections 2023-06-23T18:34:45.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:45.968+0000 7f6e5f4cc700 1 -- 172.21.15.2:0/1040915594 wait complete. 2023-06-23T18:34:46.140 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-06-23T18:34:46.141 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-06-23T18:34:46.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:46 smithi002 ceph-mon[95947]: pgmap v678: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2023-06-23T18:34:46.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:46 smithi082 ceph-mon[97889]: pgmap v678: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2023-06-23T18:34:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:46 smithi017 ceph-mon[97472]: pgmap v678: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 15 MiB/s wr, 1.30k op/s 2023-06-23T18:34:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 55.0 s, 419.6 tps, lat 238.588 ms stddev 49.709 2023-06-23T18:34:47.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 -- 172.21.15.2:0/1964486052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc0ffb90 msgr2=0x7f08dc0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:47.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 --2- 172.21.15.2:0/1964486052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc0ffb90 0x7f08dc0fffa0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f08d8009990 tx=0x7f08d8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:34:47.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 -- 172.21.15.2:0/1964486052 shutdown_connections 2023-06-23T18:34:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 --2- 172.21.15.2:0/1964486052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08dc101710 0x7f08dc107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 --2- 172.21.15.2:0/1964486052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc0ffb90 0x7f08dc0fffa0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 --2- 172.21.15.2:0/1964486052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08dc100d80 0x7f08dc1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 -- 172.21.15.2:0/1964486052 >> 172.21.15.2:0/1964486052 conn(0x7f08dc0fb190 msgr2=0x7f08dc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 -- 172.21.15.2:0/1964486052 shutdown_connections 2023-06-23T18:34:47.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.651+0000 7f08e41ab700 1 -- 172.21.15.2:0/1964486052 wait complete. 2023-06-23T18:34:47.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e41ab700 1 Processor -- start 2023-06-23T18:34:47.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e41ab700 1 -- start start 2023-06-23T18:34:47.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e41ab700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08dc0ffb90 0x7f08dc1957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:47.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e41ab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc100d80 0x7f08dc195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:47.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e41ab700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08dc101710 0x7f08dc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:47.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e41ab700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc19d360 con 0x7f08dc100d80 2023-06-23T18:34:47.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e41ab700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc19d4a0 con 0x7f08dc101710 2023-06-23T18:34:47.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e41ab700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc19d630 con 0x7f08dc0ffb90 2023-06-23T18:34:47.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e1746700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc100d80 0x7f08dc195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:47.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.652+0000 7f08e1746700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc100d80 0x7f08dc195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57132/0 (socket says 172.21.15.2:57132) 2023-06-23T18:34:47.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e1746700 1 -- 172.21.15.2:0/3775057551 learned_addr learned my addr 172.21.15.2:0/3775057551 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:47.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e2748700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08dc101710 0x7f08dc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:47.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e1f47700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08dc0ffb90 0x7f08dc1957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:47.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e2748700 1 -- 172.21.15.2:0/3775057551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08dc0ffb90 msgr2=0x7f08dc1957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:47.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e2748700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08dc0ffb90 0x7f08dc1957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:47.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e2748700 1 -- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc100d80 msgr2=0x7f08dc195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:47.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e2748700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc100d80 0x7f08dc195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:47.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e2748700 1 -- 172.21.15.2:0/3775057551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d80095f0 con 0x7f08dc101710 2023-06-23T18:34:47.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e1746700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc100d80 0x7f08dc195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:47.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e2748700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08dc101710 0x7f08dc19cd90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f08d400efc0 tx=0x7f08d400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:47.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08d2ffd700 1 -- 172.21.15.2:0/3775057551 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4018070 con 0x7f08dc101710 2023-06-23T18:34:47.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.653+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc19d8b0 con 0x7f08dc101710 2023-06-23T18:34:47.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.654+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc19ddc0 con 0x7f08dc101710 2023-06-23T18:34:47.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.654+0000 7f08d2ffd700 1 -- 172.21.15.2:0/3775057551 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f08d400f040 con 0x7f08dc101710 2023-06-23T18:34:47.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.654+0000 7f08d2ffd700 1 -- 172.21.15.2:0/3775057551 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d400cba0 con 0x7f08dc101710 2023-06-23T18:34:47.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.655+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c0005320 con 0x7f08dc101710 2023-06-23T18:34:47.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.656+0000 7f08d2ffd700 1 -- 172.21.15.2:0/3775057551 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f08d4003a00 con 0x7f08dc101710 2023-06-23T18:34:47.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.657+0000 7f08d2ffd700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f08c80a5560 0x7f08c80a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:47.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.657+0000 7f08d2ffd700 1 -- 172.21.15.2:0/3775057551 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08d40cc3e0 con 0x7f08dc101710 2023-06-23T18:34:47.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.657+0000 7f08e1f47700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f08c80a5560 0x7f08c80a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:47.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.658+0000 7f08e1f47700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f08c80a5560 0x7f08c80a7a10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f08d80098e0 tx=0x7f08d800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:47.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:47.660+0000 7f08d2ffd700 1 -- 172.21.15.2:0/3775057551 <== mon.2 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f08d4095bc0 con 0x7f08dc101710 2023-06-23T18:34:48.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.073+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08c0005190 con 0x7f08dc101710 2023-06-23T18:34:48.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.074+0000 7f08d2ffd700 1 -- 172.21.15.2:0/3775057551 <== mon.2 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10783 (secure 0 0 0) 0x7f08d4003cb0 con 0x7f08dc101710 2023-06-23T18:34:48.078 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:34:48.079 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/1508985058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":1508985058},{"type":"v1","addr":"172.21.15.2:6835","nonce":1508985058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/3317629497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":3317629497},{"type":"v1","addr":"172.21.15.2:6837","nonce":3317629497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6839/3063991639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6838","nonce":3063991639},{"type":"v1","addr":"172.21.15.2:6839","nonce":3063991639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6841/2758814728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6840","nonce":2758814728},{"type":"v1","addr":"172.21.15.2:6841","nonce":2758814728}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6833/2827087691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":2827087691},{"type":"v1","addr":"172.21.15.82:6833","nonce":2827087691}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/719153877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":719153877},{"type":"v1","addr":"172.21.15.17:6833","nonce":719153877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-06-23T18:20:44.434596+0000","modified":"2023-06-23T18:33:28.815886+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24533,"mds_1":24526,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24467":{"gid":24467,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6835/2425660103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":2425660103},{"type":"v1","addr":"172.21.15.17:6835","nonce":2425660103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24493":{"gid":24493,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.82:6835/2287539900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2287539900},{"type":"v1","addr":"172.21.15.82:6835","nonce":2287539900}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24497":{"gid":24497,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.17:6837/2388271800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":2388271800},{"type":"v1","addr":"172.21.15.17:6837","nonce":2388271800}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24505":{"gid":24505,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/2740251056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2740251056},{"type":"v1","addr":"172.21.15.82:6837","nonce":2740251056}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24526":{"gid":24526,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":5,"addr":"172.21.15.82:6839/2781114494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6838","nonce":2781114494},{"type":"v1","addr":"172.21.15.82:6839","nonce":2781114494}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24533":{"gid":24533,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2,"addr":"172.21.15.17:6839/4283937693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6838","nonce":4283937693},{"type":"v1","addr":"172.21.15.17:6839","nonce":4283937693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-06-23T18:34:48.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f08c80a5560 msgr2=0x7f08c80a7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f08c80a5560 0x7f08c80a7a10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f08d80098e0 tx=0x7f08d800b260 comp rx=0 tx=0).stop 2023-06-23T18:34:48.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08dc101710 msgr2=0x7f08dc19cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08dc101710 0x7f08dc19cd90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f08d400efc0 tx=0x7f08d400c3a0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 shutdown_connections 2023-06-23T18:34:48.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f08c80a5560 0x7f08c80a7a10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08dc101710 0x7f08dc19cd90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08dc0ffb90 0x7f08dc1957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 --2- 172.21.15.2:0/3775057551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08dc100d80 0x7f08dc195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 >> 172.21.15.2:0/3775057551 conn(0x7f08dc0fb190 msgr2=0x7f08dc1042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 shutdown_connections 2023-06-23T18:34:48.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.077+0000 7f08e41ab700 1 -- 172.21.15.2:0/3775057551 wait complete. 2023-06-23T18:34:48.084 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 38 2023-06-23T18:34:48.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-06-23T18:34:48.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.304+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/44183334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d00ffc70 msgr2=0x7ff4d01000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.304+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/44183334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d00ffc70 0x7ff4d01000c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00faf0 tx=0x7ff4bc00fe00 comp rx=0 tx=0).stop 2023-06-23T18:34:48.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.304+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/44183334 shutdown_connections 2023-06-23T18:34:48.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.304+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/44183334 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d0100600 0x7ff4d0106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/44183334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d00fea80 0x7ff4d00fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/44183334 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d00ffc70 0x7ff4d01000c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/44183334 >> 172.21.15.2:0/44183334 conn(0x7ff4d00fa0e0 msgr2=0x7ff4d00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/44183334 shutdown_connections 2023-06-23T18:34:48.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/44183334 wait complete. 2023-06-23T18:34:48.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 Processor -- start 2023-06-23T18:34:48.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 -- start start 2023-06-23T18:34:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d00fea80 0x7ff4d01935f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d00ffc70 0x7ff4d0193b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d0100600 0x7ff4d019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019b170 con 0x7ff4d00fea80 2023-06-23T18:34:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019b2b0 con 0x7ff4d0100600 2023-06-23T18:34:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.305+0000 7ff4d6ff5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d019b440 con 0x7ff4d00ffc70 2023-06-23T18:34:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d4d91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d00fea80 0x7ff4d01935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d00ffc70 0x7ff4d0193b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d00ffc70 0x7ff4d0193b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57484/0 (socket says 172.21.15.2:57484) 2023-06-23T18:34:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d4d91700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d00fea80 0x7ff4d01935f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57144/0 (socket says 172.21.15.2:57144) 2023-06-23T18:34:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4cffff700 1 -- 172.21.15.2:0/1217871153 learned_addr learned my addr 172.21.15.2:0/1217871153 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d5592700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d0100600 0x7ff4d019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d5592700 1 -- 172.21.15.2:0/1217871153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d00ffc70 msgr2=0x7ff4d0193b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d5592700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d00ffc70 0x7ff4d0193b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d5592700 1 -- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d00fea80 msgr2=0x7ff4d01935f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d5592700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d00fea80 0x7ff4d01935f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d5592700 1 -- 172.21.15.2:0/1217871153 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4bc00f750 con 0x7ff4d0100600 2023-06-23T18:34:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d00fea80 0x7ff4d01935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4cffff700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d00ffc70 0x7ff4d0193b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4d5592700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d0100600 0x7ff4d019aba0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800efc0 tx=0x7ff4c800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.306+0000 7ff4cdffb700 1 -- 172.21.15.2:0/1217871153 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8018070 con 0x7ff4d0100600 2023-06-23T18:34:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.307+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/1217871153 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d019b6c0 con 0x7ff4d0100600 2023-06-23T18:34:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.307+0000 7ff4cdffb700 1 -- 172.21.15.2:0/1217871153 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c800f040 con 0x7ff4d0100600 2023-06-23T18:34:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.307+0000 7ff4cdffb700 1 -- 172.21.15.2:0/1217871153 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c800cba0 con 0x7ff4d0100600 2023-06-23T18:34:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.307+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/1217871153 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d019bc20 con 0x7ff4d0100600 2023-06-23T18:34:48.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.310+0000 7ff4cdffb700 1 -- 172.21.15.2:0/1217871153 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff4c8003a00 con 0x7ff4d0100600 2023-06-23T18:34:48.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.311+0000 7ff4cdffb700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4c00a11d0 0x7ff4c00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.311+0000 7ff4cdffb700 1 -- 172.21.15.2:0/1217871153 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff4c80cc330 con 0x7ff4d0100600 2023-06-23T18:34:48.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.311+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4c00a11d0 0x7ff4c00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.312+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4c00a11d0 0x7ff4c00a3680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00ffad0 tx=0x7ff4c400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4e317ab80 1 Processor -- start 2023-06-23T18:34:48.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4e317ab80 1 -- start start 2023-06-23T18:34:48.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d42a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d43470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56c46cb0 con 0x55cb56d43000 2023-06-23T18:34:48.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56c99a40 con 0x55cb56d425d0 2023-06-23T18:34:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56c92ed0 con 0x55cb56d41c80 2023-06-23T18:34:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4d4d91700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4d4d91700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d42090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57490/0 (socket says 172.21.15.2:57490) 2023-06-23T18:34:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4d4d91700 1 -- 172.21.15.2:0/3392805309 learned_addr learned my addr 172.21.15.2:0/3392805309 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4d5592700 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d43470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.314+0000 7ff4cffff700 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d42a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4d4d91700 1 -- 172.21.15.2:0/3392805309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 msgr2=0x55cb56d42a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d42a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4d4d91700 1 -- 172.21.15.2:0/3392805309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 msgr2=0x55cb56d43470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d43470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4d4d91700 1 -- 172.21.15.2:0/3392805309 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb56b14630 con 0x55cb56d41c80 2023-06-23T18:34:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d42090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004170 tx=0x7ff4c4015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd251c892a8510e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4baffd700 1 -- 172.21.15.2:0/3392805309 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c401d040 con 0x55cb56d41c80 2023-06-23T18:34:48.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4baffd700 1 -- 172.21.15.2:0/3392805309 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c4003d10 con 0x55cb56d41c80 2023-06-23T18:34:48.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.315+0000 7ff4baffd700 1 -- 172.21.15.2:0/3392805309 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4010040 con 0x55cb56d41c80 2023-06-23T18:34:48.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.316+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3392805309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 msgr2=0x55cb56d42090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.316+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d42090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4004170 tx=0x7ff4c4015000 comp rx=0 tx=0).stop 2023-06-23T18:34:48.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.316+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3392805309 shutdown_connections 2023-06-23T18:34:48.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.316+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d43470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.316+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d42090 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.316+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3392805309 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d42a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.316+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3392805309 >> 172.21.15.2:0/3392805309 conn(0x55cb56d373f0 msgr2=0x55cb56d3c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.316+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3392805309 shutdown_connections 2023-06-23T18:34:48.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.317+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3392805309 wait complete. 2023-06-23T18:34:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.317+0000 7ff4e317ab80 1 -- start start 2023-06-23T18:34:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.317+0000 7ff4e317ab80 1 Processor -- start 2023-06-23T18:34:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d41c80 0x55cb56d4b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d425d0 0x55cb56d4bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d52b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56c32ac0 con 0x55cb56d43000 2023-06-23T18:34:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56c327c0 con 0x55cb56d41c80 2023-06-23T18:34:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56b16920 con 0x55cb56d425d0 2023-06-23T18:34:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4e317ab80 10 client.0 _ll_register_callbacks cb 0x55cb56c38260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d4d91700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d41c80 0x55cb56d4b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d4d91700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d41c80 0x55cb56d4b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47750/0 (socket says 172.21.15.2:47750) 2023-06-23T18:34:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d4d91700 1 -- 172.21.15.2:0/778950326 learned_addr learned my addr 172.21.15.2:0/778950326 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d5592700 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d52b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4cffff700 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d425d0 0x55cb56d4bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d5592700 1 -- 172.21.15.2:0/778950326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d425d0 msgr2=0x55cb56d4bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d5592700 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d425d0 0x55cb56d4bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d5592700 1 -- 172.21.15.2:0/778950326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d41c80 msgr2=0x55cb56d4b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d5592700 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d41c80 0x55cb56d4b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.318+0000 7ff4d5592700 1 -- 172.21.15.2:0/778950326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c4012e10 con 0x55cb56d43000 2023-06-23T18:34:48.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.319+0000 7ff4d5592700 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d52b60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80c9e30 tx=0x7ff4c8099130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadb89e31b08b73f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.319+0000 7ff4affff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:34:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.320+0000 7ff4affff700 1 -- 172.21.15.2:0/778950326 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c809d800 con 0x55cb56d43000 2023-06-23T18:34:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.320+0000 7ff4affff700 1 -- 172.21.15.2:0/778950326 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff4c80a1b90 con 0x55cb56d43000 2023-06-23T18:34:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.320+0000 7ff4e317ab80 1 -- 172.21.15.2:0/778950326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb56d34bb0 con 0x55cb56d43000 2023-06-23T18:34:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.320+0000 7ff4affff700 1 -- 172.21.15.2:0/778950326 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c80b9070 con 0x55cb56d43000 2023-06-23T18:34:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.320+0000 7ff4e317ab80 10 client.16680 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:48.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.320+0000 7ff4e317ab80 1 -- 172.21.15.2:0/778950326 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cb56ac0de0 con 0x55cb56d43000 2023-06-23T18:34:48.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.320+0000 7ff4affff700 1 -- 172.21.15.2:0/778950326 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cb56d34bb0 con 0x55cb56d43000 2023-06-23T18:34:48.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4affff700 1 -- 172.21.15.2:0/778950326 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff4c80a1700 con 0x55cb56d43000 2023-06-23T18:34:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4e317ab80 10 client.16680 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4e317ab80 10 client.16680 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:34:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4e317ab80 20 client.16680 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55cb56d57430 0x55cb56d59840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4e317ab80 4 client.16680 mds_command: new command op to 24505 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:48.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4e317ab80 1 -- 172.21.15.2:0/778950326 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb56c8beb0 con 0x55cb56d57430 2023-06-23T18:34:48.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55cb56d57430 0x55cb56d59840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55cb56d57430 0x55cb56d59840 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36c6c1c58a9ccf13 server_cookie=cf9e5ce20d51e4c8 in_seq=0 out_seq=0 2023-06-23T18:34:48.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.321+0000 7ff4affff700 10 client.16680 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:34:48.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.322+0000 7ff4affff700 1 -- 172.21.15.2:0/778950326 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb56c8beb0 con 0x55cb56d57430 2023-06-23T18:34:48.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.322+0000 7ff4affff700 10 client.16680 handle_command_reply: tid=0 2023-06-23T18:34:48.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.322+0000 7ff4e317ab80 1 client.16680 shutdown 2023-06-23T18:34:48.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.322+0000 7ff4e317ab80 1 -- 172.21.15.2:0/778950326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 msgr2=0x55cb56d52b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.322+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d52b60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80c9e30 tx=0x7ff4c8099130 comp rx=0 tx=0).stop 2023-06-23T18:34:48.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 1 -- 172.21.15.2:0/778950326 shutdown_connections 2023-06-23T18:34:48.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55cb56d57430 0x55cb56d59840 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d52b60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d41c80 0x55cb56d4b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/778950326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d425d0 0x55cb56d4bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 1 -- 172.21.15.2:0/778950326 >> 172.21.15.2:0/778950326 conn(0x55cb56d373f0 msgr2=0x55cb56d3bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4affff700 0 client.16680 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:34:48.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 1 -- 172.21.15.2:0/778950326 shutdown_connections 2023-06-23T18:34:48.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 1 -- 172.21.15.2:0/778950326 wait complete. 2023-06-23T18:34:48.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.323+0000 7ff4e317ab80 20 client.16680 trim_cache size 0 max 16384 2023-06-23T18:34:48.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4e317ab80 1 Processor -- start 2023-06-23T18:34:48.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4e317ab80 1 -- start start 2023-06-23T18:34:48.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d45ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d52b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d46400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56c8beb0 con 0x55cb56d43000 2023-06-23T18:34:48.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56d4c7d0 con 0x55cb56d425d0 2023-06-23T18:34:48.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56d3dff0 con 0x55cb56d41c80 2023-06-23T18:34:48.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d52b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d52b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47760/0 (socket says 172.21.15.2:47760) 2023-06-23T18:34:48.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4cffff700 1 -- 172.21.15.2:0/1571016093 learned_addr learned my addr 172.21.15.2:0/1571016093 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4d5592700 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d46400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d45ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4cffff700 1 -- 172.21.15.2:0/1571016093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 msgr2=0x55cb56d45ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.345+0000 7ff4cffff700 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d45ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.346+0000 7ff4cffff700 1 -- 172.21.15.2:0/1571016093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 msgr2=0x55cb56d46400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.346+0000 7ff4cffff700 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d46400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.346+0000 7ff4cffff700 1 -- 172.21.15.2:0/1571016093 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb56d4a560 con 0x55cb56d425d0 2023-06-23T18:34:48.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.346+0000 7ff4cffff700 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d52b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00fa40 tx=0x7ff4bc00ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c24740729eb7a51 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.346+0000 7ff4bb7fe700 1 -- 172.21.15.2:0/1571016093 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc009a20 con 0x55cb56d425d0 2023-06-23T18:34:48.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.346+0000 7ff4bb7fe700 1 -- 172.21.15.2:0/1571016093 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4bc005050 con 0x55cb56d425d0 2023-06-23T18:34:48.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.346+0000 7ff4bb7fe700 1 -- 172.21.15.2:0/1571016093 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc003de0 con 0x55cb56d425d0 2023-06-23T18:34:48.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 -- 172.21.15.2:0/1571016093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 msgr2=0x55cb56d52b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d52b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00fa40 tx=0x7ff4bc00ff90 comp rx=0 tx=0).stop 2023-06-23T18:34:48.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 -- 172.21.15.2:0/1571016093 shutdown_connections 2023-06-23T18:34:48.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d46400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d45ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/1571016093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d52b20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 -- 172.21.15.2:0/1571016093 >> 172.21.15.2:0/1571016093 conn(0x55cb56d373f0 msgr2=0x55cb56d59440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 -- 172.21.15.2:0/1571016093 shutdown_connections 2023-06-23T18:34:48.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.347+0000 7ff4e317ab80 1 -- 172.21.15.2:0/1571016093 wait complete. 2023-06-23T18:34:48.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 1 -- start start 2023-06-23T18:34:48.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 1 Processor -- start 2023-06-23T18:34:48.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d4cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d4d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d51db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56d522f0 con 0x55cb56d43000 2023-06-23T18:34:48.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56d52460 con 0x55cb56d425d0 2023-06-23T18:34:48.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb56d52620 con 0x55cb56d41c80 2023-06-23T18:34:48.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4e317ab80 10 client.0 _ll_register_callbacks cb 0x55cb56c38260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:48.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d4d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4d4d91700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d4cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4cffff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d4d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47764/0 (socket says 172.21.15.2:47764) 2023-06-23T18:34:48.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4d4d91700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d4cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57512/0 (socket says 172.21.15.2:57512) 2023-06-23T18:34:48.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.348+0000 7ff4cffff700 1 -- 172.21.15.2:0/3941942523 learned_addr learned my addr 172.21.15.2:0/3941942523 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4d5592700 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d51db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4d4d91700 1 -- 172.21.15.2:0/3941942523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 msgr2=0x55cb56d4d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d4d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4d4d91700 1 -- 172.21.15.2:0/3941942523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 msgr2=0x55cb56d51db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d51db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4d4d91700 1 -- 172.21.15.2:0/3941942523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d01077f0 con 0x55cb56d41c80 2023-06-23T18:34:48.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4cffff700 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d4d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:48.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4d4d91700 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d4cca0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55cb56d5a0f0 tx=0x7ff4c4003e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=173944320473a5ed server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4affff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:48.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4affff700 1 -- 172.21.15.2:0/3941942523 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4032040 con 0x55cb56d41c80 2023-06-23T18:34:48.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4affff700 1 -- 172.21.15.2:0/3941942523 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff4c40342b0 con 0x55cb56d41c80 2023-06-23T18:34:48.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.349+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3941942523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4c4003d10 con 0x55cb56d41c80 2023-06-23T18:34:48.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.350+0000 7ff4affff700 1 -- 172.21.15.2:0/3941942523 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4c4003b70 con 0x55cb56d41c80 2023-06-23T18:34:48.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.350+0000 7ff4e317ab80 10 client.26404 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:48.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.350+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3941942523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cb56d438f0 con 0x55cb56d41c80 2023-06-23T18:34:48.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.350+0000 7ff4affff700 1 -- 172.21.15.2:0/3941942523 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff4c4003d10 con 0x55cb56d41c80 2023-06-23T18:34:48.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.350+0000 7ff4affff700 1 -- 172.21.15.2:0/3941942523 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff4c4005040 con 0x55cb56d41c80 2023-06-23T18:34:48.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4e317ab80 10 client.26404 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:48.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4e317ab80 10 client.26404 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2023-06-23T18:34:48.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4e317ab80 20 client.26404 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:48.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55cb56dc5880 0x55cb56dc7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4e317ab80 4 client.26404 mds_command: new command op to 24505 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:34:48.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3941942523 --> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb56c4e050 con 0x55cb56dc5880 2023-06-23T18:34:48.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4cffff700 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55cb56dc5880 0x55cb56dc7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4affff700 10 client.26404 ms_handle_connect on v2:172.21.15.82:6836/2740251056 2023-06-23T18:34:48.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.351+0000 7ff4cffff700 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55cb56dc5880 0x55cb56dc7c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=43a13d470efbeea server_cookie=9a9a6429b1e6249 in_seq=0 out_seq=0 2023-06-23T18:34:48.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4affff700 1 -- 172.21.15.2:0/3941942523 <== mds.2 v2:172.21.15.82:6836/2740251056 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb56c4e050 con 0x55cb56dc5880 2023-06-23T18:34:48.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4affff700 10 client.26404 handle_command_reply: tid=0 2023-06-23T18:34:48.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 client.26404 shutdown 2023-06-23T18:34:48.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3941942523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 msgr2=0x55cb56d4cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d4cca0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55cb56d5a0f0 tx=0x7ff4c4003e80 comp rx=0 tx=0).stop 2023-06-23T18:34:48.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3941942523 shutdown_connections 2023-06-23T18:34:48.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.82:6836/2740251056,v1:172.21.15.82:6837/2740251056] conn(0x55cb56dc5880 0x55cb56dc7c90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55cb56d43000 0x55cb56d51db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cb56d41c80 0x55cb56d4cca0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 --2- 172.21.15.2:0/3941942523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55cb56d425d0 0x55cb56d4d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3941942523 >> 172.21.15.2:0/3941942523 conn(0x55cb56d373f0 msgr2=0x55cb56d45ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4affff700 0 client.26404 ms_handle_reset on v2:172.21.15.82:6836/2740251056 2023-06-23T18:34:48.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3941942523 shutdown_connections 2023-06-23T18:34:48.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 1 -- 172.21.15.2:0/3941942523 wait complete. 2023-06-23T18:34:48.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.352+0000 7ff4e317ab80 20 client.26404 trim_cache size 0 max 16384 2023-06-23T18:34:48.365 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:34:48.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4c00a11d0 msgr2=0x7ff4c00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4c00a11d0 0x7ff4c00a3680 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00ffad0 tx=0x7ff4c400d040 comp rx=0 tx=0).stop 2023-06-23T18:34:48.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/1217871153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d0100600 msgr2=0x7ff4d019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d0100600 0x7ff4d019aba0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800efc0 tx=0x7ff4c800c3a0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/1217871153 shutdown_connections 2023-06-23T18:34:48.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff4c00a11d0 0x7ff4c00a3680 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff4d0100600 0x7ff4d019aba0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff4d00fea80 0x7ff4d01935f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 --2- 172.21.15.2:0/1217871153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4d00ffc70 0x7ff4d0193b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.353+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/1217871153 >> 172.21.15.2:0/1217871153 conn(0x7ff4d00fa0e0 msgr2=0x7ff4d0103100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.354+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/1217871153 shutdown_connections 2023-06-23T18:34:48.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.354+0000 7ff4d6ff5700 1 -- 172.21.15.2:0/1217871153 wait complete. 2023-06-23T18:34:48.376 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:48 smithi002 ceph-mon[95947]: pgmap v679: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2023-06-23T18:34:48.377 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:48 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3775057551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:34:48.377 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-06-23T18:34:48.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 -- 172.21.15.2:0/2259796323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978071790 msgr2=0x7f397810f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 --2- 172.21.15.2:0/2259796323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978071790 0x7f397810f830 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f397000b0a0 tx=0x7f397000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 -- 172.21.15.2:0/2259796323 shutdown_connections 2023-06-23T18:34:48.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 --2- 172.21.15.2:0/2259796323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978071790 0x7f397810f830 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 --2- 172.21.15.2:0/2259796323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3978108850 0x7f3978108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 --2- 172.21.15.2:0/2259796323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39781091a0 0x7f3978071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 -- 172.21.15.2:0/2259796323 >> 172.21.15.2:0/2259796323 conn(0x7f397806cb20 msgr2=0x7f397806ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 -- 172.21.15.2:0/2259796323 shutdown_connections 2023-06-23T18:34:48.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.561+0000 7f397d22d700 1 -- 172.21.15.2:0/2259796323 wait complete. 2023-06-23T18:34:48.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f397d22d700 1 Processor -- start 2023-06-23T18:34:48.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f397d22d700 1 -- start start 2023-06-23T18:34:48.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f397d22d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3978071790 0x7f397810b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f397d22d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978108850 0x7f39781099c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f397d22d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39781091a0 0x7f3978109f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f397d22d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397810a6a0 con 0x7f3978108850 2023-06-23T18:34:48.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f397d22d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f397810a810 con 0x7f39781091a0 2023-06-23T18:34:48.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f397d22d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39781a8150 con 0x7f3978071790 2023-06-23T18:34:48.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.562+0000 7f3976d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3978071790 0x7f397810b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978108850 0x7f39781099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f3976d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3978071790 0x7f397810b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57524/0 (socket says 172.21.15.2:57524) 2023-06-23T18:34:48.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f3976d9d700 1 -- 172.21.15.2:0/2702497435 learned_addr learned my addr 172.21.15.2:0/2702497435 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397759e700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39781091a0 0x7f3978109f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397659c700 1 -- 172.21.15.2:0/2702497435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3978071790 msgr2=0x7f397810b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397659c700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3978071790 0x7f397810b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397659c700 1 -- 172.21.15.2:0/2702497435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39781091a0 msgr2=0x7f3978109f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397659c700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39781091a0 0x7f3978109f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397659c700 1 -- 172.21.15.2:0/2702497435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3968009520 con 0x7f3978108850 2023-06-23T18:34:48.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f3976d9d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3978071790 0x7f397810b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:48.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397659c700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978108850 0x7f39781099c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f396c00e9c0 tx=0x7f396c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f395ffff700 1 -- 172.21.15.2:0/2702497435 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c00cb70 con 0x7f3978108850 2023-06-23T18:34:48.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397d22d700 1 -- 172.21.15.2:0/2702497435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c009520 con 0x7f3978108850 2023-06-23T18:34:48.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.563+0000 7f397d22d700 1 -- 172.21.15.2:0/2702497435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3970009d00 con 0x7f3978108850 2023-06-23T18:34:48.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.564+0000 7f395ffff700 1 -- 172.21.15.2:0/2702497435 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f396c00ee60 con 0x7f3978108850 2023-06-23T18:34:48.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.564+0000 7f395ffff700 1 -- 172.21.15.2:0/2702497435 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c003c50 con 0x7f3978108850 2023-06-23T18:34:48.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.566+0000 7f395ffff700 1 -- 172.21.15.2:0/2702497435 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f396c003db0 con 0x7f3978108850 2023-06-23T18:34:48.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.567+0000 7f395ffff700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f39600a11d0 0x7f39600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.567+0000 7f3976d9d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f39600a11d0 0x7f39600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.567+0000 7f395ffff700 1 -- 172.21.15.2:0/2702497435 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f396c014070 con 0x7f3978108850 2023-06-23T18:34:48.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f39893b2b80 1 Processor -- start 2023-06-23T18:34:48.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f39893b2b80 1 -- start start 2023-06-23T18:34:48.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f3976d9d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f39600a11d0 0x7f39600a3680 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3968009fd0 tx=0x7f3968009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78324c80 0x55ec78325090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78325a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec78326470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f39893b2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec78229cb0 con 0x55ec78326000 2023-06-23T18:34:48.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f39893b2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec7827ca40 con 0x55ec783255d0 2023-06-23T18:34:48.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f39893b2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec78275ed0 con 0x55ec78324c80 2023-06-23T18:34:48.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f3976d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78324c80 0x55ec78325090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f3976d9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78324c80 0x55ec78325090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57532/0 (socket says 172.21.15.2:57532) 2023-06-23T18:34:48.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f3976d9d700 1 -- 172.21.15.2:0/1284699494 learned_addr learned my addr 172.21.15.2:0/1284699494 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.568+0000 7f397659c700 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78325a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f397759e700 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec78326470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f3976d9d700 1 -- 172.21.15.2:0/1284699494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 msgr2=0x55ec78325a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f3976d9d700 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78325a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f3976d9d700 1 -- 172.21.15.2:0/1284699494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 msgr2=0x55ec78326470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f3976d9d700 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec78326470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f3976d9d700 1 -- 172.21.15.2:0/1284699494 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec780f7630 con 0x55ec78324c80 2023-06-23T18:34:48.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f3976d9d700 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78324c80 0x55ec78325090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f396800f780 tx=0x7f3968014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d225ab4f85909b87 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f395d7fa700 1 -- 172.21.15.2:0/1284699494 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968004220 con 0x55ec78324c80 2023-06-23T18:34:48.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f395d7fa700 1 -- 172.21.15.2:0/1284699494 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3968004380 con 0x55ec78324c80 2023-06-23T18:34:48.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.569+0000 7f395d7fa700 1 -- 172.21.15.2:0/1284699494 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396801b6a0 con 0x55ec78324c80 2023-06-23T18:34:48.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.570+0000 7f39893b2b80 1 -- 172.21.15.2:0/1284699494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78324c80 msgr2=0x55ec78325090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.570+0000 7f39893b2b80 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78324c80 0x55ec78325090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f396800f780 tx=0x7f3968014000 comp rx=0 tx=0).stop 2023-06-23T18:34:48.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.570+0000 7f39893b2b80 1 -- 172.21.15.2:0/1284699494 shutdown_connections 2023-06-23T18:34:48.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.570+0000 7f39893b2b80 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec78326470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.570+0000 7f39893b2b80 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78324c80 0x55ec78325090 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f396800f780 tx=0x7f3968014000 comp rx=0 tx=0).stop 2023-06-23T18:34:48.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.570+0000 7f39893b2b80 1 --2- 172.21.15.2:0/1284699494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78325a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.570+0000 7f39893b2b80 1 -- 172.21.15.2:0/1284699494 >> 172.21.15.2:0/1284699494 conn(0x55ec7831a3f0 msgr2=0x55ec7831f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.571+0000 7f39893b2b80 1 -- 172.21.15.2:0/1284699494 shutdown_connections 2023-06-23T18:34:48.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.571+0000 7f39893b2b80 1 -- 172.21.15.2:0/1284699494 wait complete. 2023-06-23T18:34:48.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.571+0000 7f39893b2b80 1 -- start start 2023-06-23T18:34:48.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.571+0000 7f39893b2b80 1 Processor -- start 2023-06-23T18:34:48.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec783255d0 0x55ec78336f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec7833e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f39893b2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec78215ac0 con 0x55ec78326000 2023-06-23T18:34:48.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f39893b2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec782157c0 con 0x55ec7833c4f0 2023-06-23T18:34:48.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f39893b2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec780f9920 con 0x55ec783255d0 2023-06-23T18:34:48.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f39893b2b80 10 client.0 _ll_register_callbacks cb 0x55ec7821b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:48.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f397759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec7833e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f397759e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec7833e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47792/0 (socket says 172.21.15.2:47792) 2023-06-23T18:34:48.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f397759e700 1 -- 172.21.15.2:0/553314617 learned_addr learned my addr 172.21.15.2:0/553314617 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f3976d9d700 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec783255d0 0x55ec78336f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.572+0000 7f397659c700 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f3976d9d700 1 -- 172.21.15.2:0/553314617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 msgr2=0x55ec7833e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f3976d9d700 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec7833e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f3976d9d700 1 -- 172.21.15.2:0/553314617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 msgr2=0x55ec783374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f3976d9d700 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f3976d9d700 1 -- 172.21.15.2:0/553314617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3968017050 con 0x55ec783255d0 2023-06-23T18:34:48.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f3976d9d700 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec783255d0 0x55ec78336f80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3968020000 tx=0x7f396800ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a5cd3ed7f5a0ae server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f395aff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:48.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f395aff5700 1 -- 172.21.15.2:0/553314617 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3968003880 con 0x55ec783255d0 2023-06-23T18:34:48.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f395aff5700 1 -- 172.21.15.2:0/553314617 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f39680039e0 con 0x55ec783255d0 2023-06-23T18:34:48.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.573+0000 7f39893b2b80 1 -- 172.21.15.2:0/553314617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec78317bb0 con 0x55ec783255d0 2023-06-23T18:34:48.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f395aff5700 1 -- 172.21.15.2:0/553314617 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3968005040 con 0x55ec783255d0 2023-06-23T18:34:48.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f39893b2b80 10 client.26416 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:48.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f39893b2b80 1 -- 172.21.15.2:0/553314617 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ec780a3de0 con 0x55ec783255d0 2023-06-23T18:34:48.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f395aff5700 1 -- 172.21.15.2:0/553314617 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ec78317bb0 con 0x55ec783255d0 2023-06-23T18:34:48.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f395aff5700 1 -- 172.21.15.2:0/553314617 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3968004b60 con 0x55ec783255d0 2023-06-23T18:34:48.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f39893b2b80 10 client.26416 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:48.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f39893b2b80 10 client.26416 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:34:48.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f39893b2b80 20 client.26416 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:48.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f39893b2b80 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55ec783326d0 0x55ec78334ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f39893b2b80 4 client.26416 mds_command: new command op to 24526 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:48.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f39893b2b80 1 -- 172.21.15.2:0/553314617 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec7826eeb0 con 0x55ec783326d0 2023-06-23T18:34:48.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.574+0000 7f397659c700 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55ec783326d0 0x55ec78334ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.575+0000 7f395aff5700 10 client.26416 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:34:48.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.575+0000 7f397659c700 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55ec783326d0 0x55ec78334ae0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e82de60b9a00221e server_cookie=c45812b25309d757 in_seq=0 out_seq=0 2023-06-23T18:34:48.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.575+0000 7f395aff5700 1 -- 172.21.15.2:0/553314617 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec7826eeb0 con 0x55ec783326d0 2023-06-23T18:34:48.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.575+0000 7f395aff5700 10 client.26416 handle_command_reply: tid=0 2023-06-23T18:34:48.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.575+0000 7f39893b2b80 1 client.26416 shutdown 2023-06-23T18:34:48.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 -- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec783255d0 msgr2=0x55ec78336f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec783255d0 0x55ec78336f80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3968020000 tx=0x7f396800ffd0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 -- 172.21.15.2:0/553314617 shutdown_connections 2023-06-23T18:34:48.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55ec783326d0 0x55ec78334ae0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec7833e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec783255d0 0x55ec78336f80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f395aff5700 0 client.26416 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:34:48.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 --2- 172.21.15.2:0/553314617 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 -- 172.21.15.2:0/553314617 >> 172.21.15.2:0/553314617 conn(0x55ec7831a3f0 msgr2=0x55ec7831ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 -- 172.21.15.2:0/553314617 shutdown_connections 2023-06-23T18:34:48.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 1 -- 172.21.15.2:0/553314617 wait complete. 2023-06-23T18:34:48.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.576+0000 7f39893b2b80 20 client.26416 trim_cache size 0 max 16384 2023-06-23T18:34:48.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f39893b2b80 1 Processor -- start 2023-06-23T18:34:48.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f39893b2b80 1 -- start start 2023-06-23T18:34:48.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec783255d0 0x55ec7831ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78326000 0x55ec7831b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec78353e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f39893b2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec7832c3c0 con 0x55ec783255d0 2023-06-23T18:34:48.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f39893b2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec7831be00 con 0x55ec7833c4f0 2023-06-23T18:34:48.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f39893b2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec78329c10 con 0x55ec78326000 2023-06-23T18:34:48.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f3976d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec783255d0 0x55ec7831ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f397659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78326000 0x55ec7831b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f3976d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec783255d0 0x55ec7831ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57208/0 (socket says 172.21.15.2:57208) 2023-06-23T18:34:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f397659c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78326000 0x55ec7831b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57546/0 (socket says 172.21.15.2:57546) 2023-06-23T18:34:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f3976d9d700 1 -- 172.21.15.2:0/4074749695 learned_addr learned my addr 172.21.15.2:0/4074749695 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.591+0000 7f397759e700 1 --2- 172.21.15.2:0/4074749695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec78353e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f397659c700 1 -- 172.21.15.2:0/4074749695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 msgr2=0x55ec78353e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f397659c700 1 --2- 172.21.15.2:0/4074749695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec78353e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f397659c700 1 -- 172.21.15.2:0/4074749695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec783255d0 msgr2=0x55ec7831ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f397659c700 1 --2- 172.21.15.2:0/4074749695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec783255d0 0x55ec7831ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f397659c700 1 -- 172.21.15.2:0/4074749695 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec7808cc60 con 0x55ec78326000 2023-06-23T18:34:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f397659c700 1 --2- 172.21.15.2:0/4074749695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78326000 0x55ec7831b8c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f396c0963b0 tx=0x7f396c0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b7428af6065ef0b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f395dffb700 1 -- 172.21.15.2:0/4074749695 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0b82d0 con 0x55ec78326000 2023-06-23T18:34:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f395dffb700 1 -- 172.21.15.2:0/4074749695 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f396c09dee0 con 0x55ec78326000 2023-06-23T18:34:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.592+0000 7f395dffb700 1 -- 172.21.15.2:0/4074749695 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0993a0 con 0x55ec78326000 2023-06-23T18:34:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 -- 172.21.15.2:0/4074749695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78326000 msgr2=0x55ec7831b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 --2- 172.21.15.2:0/4074749695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78326000 0x55ec7831b8c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f396c0963b0 tx=0x7f396c0cc040 comp rx=0 tx=0).stop 2023-06-23T18:34:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 -- 172.21.15.2:0/4074749695 shutdown_connections 2023-06-23T18:34:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 --2- 172.21.15.2:0/4074749695 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec7833c4f0 0x55ec78353e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 --2- 172.21.15.2:0/4074749695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec783255d0 0x55ec7831ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 --2- 172.21.15.2:0/4074749695 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec78326000 0x55ec7831b8c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 -- 172.21.15.2:0/4074749695 >> 172.21.15.2:0/4074749695 conn(0x55ec7831a3f0 msgr2=0x55ec7831f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 -- 172.21.15.2:0/4074749695 shutdown_connections 2023-06-23T18:34:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 -- 172.21.15.2:0/4074749695 wait complete. 2023-06-23T18:34:48.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 -- start start 2023-06-23T18:34:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.593+0000 7f39893b2b80 1 Processor -- start 2023-06-23T18:34:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78358f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f39893b2b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec7833c4f0 0x55ec7835c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f39893b2b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec7835c8d0 con 0x55ec78326000 2023-06-23T18:34:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f39893b2b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec7835ca40 con 0x55ec783255d0 2023-06-23T18:34:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f39893b2b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec7835cc00 con 0x55ec7833c4f0 2023-06-23T18:34:48.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f39893b2b80 10 client.0 _ll_register_callbacks cb 0x55ec7821b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f3976d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78358f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57214/0 (socket says 172.21.15.2:57214) 2023-06-23T18:34:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 -- 172.21.15.2:0/2431310310 learned_addr learned my addr 172.21.15.2:0/2431310310 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397759e700 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec7833c4f0 0x55ec7835c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 -- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec7833c4f0 msgr2=0x55ec7835c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec7833c4f0 0x55ec7835c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 -- 172.21.15.2:0/2431310310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 msgr2=0x55ec78358f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78358f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 -- 172.21.15.2:0/2431310310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3970009d00 con 0x55ec78326000 2023-06-23T18:34:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397759e700 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec7833c4f0 0x55ec7835c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f3976d9d700 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78358f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f397659c700 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783594d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f396c0a1dd0 tx=0x7f396c098460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57888ac22c7b041c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f395aff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:34:48.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f395aff5700 1 -- 172.21.15.2:0/2431310310 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c0b82d0 con 0x55ec78326000 2023-06-23T18:34:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f395aff5700 1 -- 172.21.15.2:0/2431310310 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f396c0a2300 con 0x55ec78326000 2023-06-23T18:34:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.594+0000 7f39893b2b80 1 -- 172.21.15.2:0/2431310310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3968004380 con 0x55ec78326000 2023-06-23T18:34:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f395aff5700 1 -- 172.21.15.2:0/2431310310 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f396c0ca300 con 0x55ec78326000 2023-06-23T18:34:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f39893b2b80 10 client.16704 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f39893b2b80 1 -- 172.21.15.2:0/2431310310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ec780ca250 con 0x55ec78326000 2023-06-23T18:34:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f395aff5700 1 -- 172.21.15.2:0/2431310310 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3968004380 con 0x55ec78326000 2023-06-23T18:34:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f395aff5700 1 -- 172.21.15.2:0/2431310310 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f396c0a1f90 con 0x55ec78326000 2023-06-23T18:34:48.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f39893b2b80 10 client.16704 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f39893b2b80 10 client.16704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2023-06-23T18:34:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f39893b2b80 20 client.16704 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f39893b2b80 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55ec783a0570 0x55ec783a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f39893b2b80 4 client.16704 mds_command: new command op to 24526 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:34:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f39893b2b80 1 -- 172.21.15.2:0/2431310310 --> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ec78231050 con 0x55ec783a0570 2023-06-23T18:34:48.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.595+0000 7f3976d9d700 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55ec783a0570 0x55ec783a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:48 smithi082 ceph-mon[97889]: pgmap v679: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2023-06-23T18:34:48.609 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:48 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3775057551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:34:48.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f395aff5700 10 client.16704 ms_handle_connect on v2:172.21.15.82:6838/2781114494 2023-06-23T18:34:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f3976d9d700 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55ec783a0570 0x55ec783a2980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec3775f22966b263 server_cookie=83bc5f0ff91afd4c in_seq=0 out_seq=0 2023-06-23T18:34:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f395aff5700 1 -- 172.21.15.2:0/2431310310 <== mds.1 v2:172.21.15.82:6838/2781114494 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ec78231050 con 0x55ec783a0570 2023-06-23T18:34:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f395aff5700 10 client.16704 handle_command_reply: tid=0 2023-06-23T18:34:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 client.16704 shutdown 2023-06-23T18:34:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 -- 172.21.15.2:0/2431310310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 msgr2=0x55ec783594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783594d0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f396c0a1dd0 tx=0x7f396c098460 comp rx=0 tx=0).stop 2023-06-23T18:34:48.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 -- 172.21.15.2:0/2431310310 shutdown_connections 2023-06-23T18:34:48.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:6838/2781114494,v1:172.21.15.82:6839/2781114494] conn(0x55ec783a0570 0x55ec783a2980 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ec7833c4f0 0x55ec7835c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f395aff5700 0 client.16704 ms_handle_reset on v2:172.21.15.82:6838/2781114494 2023-06-23T18:34:48.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ec783255d0 0x55ec78358f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 --2- 172.21.15.2:0/2431310310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ec78326000 0x55ec783594d0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 -- 172.21.15.2:0/2431310310 >> 172.21.15.2:0/2431310310 conn(0x55ec7831a3f0 msgr2=0x55ec7832c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 -- 172.21.15.2:0/2431310310 shutdown_connections 2023-06-23T18:34:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 1 -- 172.21.15.2:0/2431310310 wait complete. 2023-06-23T18:34:48.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.596+0000 7f39893b2b80 20 client.16704 trim_cache size 0 max 16384 2023-06-23T18:34:48.613 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:34:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 -- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f39600a11d0 msgr2=0x7f39600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f39600a11d0 0x7f39600a3680 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3968009fd0 tx=0x7f3968009260 comp rx=0 tx=0).stop 2023-06-23T18:34:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 -- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978108850 msgr2=0x7f39781099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978108850 0x7f39781099c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f396c00e9c0 tx=0x7f396c00ecd0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 -- 172.21.15.2:0/2702497435 shutdown_connections 2023-06-23T18:34:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f39600a11d0 0x7f39600a3680 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39781091a0 0x7f3978109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3978071790 0x7f397810b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 --2- 172.21.15.2:0/2702497435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3978108850 0x7f39781099c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 -- 172.21.15.2:0/2702497435 >> 172.21.15.2:0/2702497435 conn(0x7f397806cb20 msgr2=0x7f39781074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 -- 172.21.15.2:0/2702497435 shutdown_connections 2023-06-23T18:34:48.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.597+0000 7f397d22d700 1 -- 172.21.15.2:0/2702497435 wait complete. 2023-06-23T18:34:48.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-06-23T18:34:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:48 smithi017 ceph-mon[97472]: pgmap v679: 97 pgs: 97 active+clean; 8.3 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2023-06-23T18:34:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:48 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3775057551' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-06-23T18:34:48.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 -- 172.21.15.2:0/4099495220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc101710 msgr2=0x7f67fc107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 --2- 172.21.15.2:0/4099495220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc101710 0x7f67fc107fd0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f67f400b0a0 tx=0x7f67f400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 -- 172.21.15.2:0/4099495220 shutdown_connections 2023-06-23T18:34:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 --2- 172.21.15.2:0/4099495220 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc101710 0x7f67fc107fd0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 --2- 172.21.15.2:0/4099495220 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67fc0ffb90 0x7f67fc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 --2- 172.21.15.2:0/4099495220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67fc100d80 0x7f67fc1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 -- 172.21.15.2:0/4099495220 >> 172.21.15.2:0/4099495220 conn(0x7f67fc0fb1b0 msgr2=0x7f67fc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 -- 172.21.15.2:0/4099495220 shutdown_connections 2023-06-23T18:34:48.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 -- 172.21.15.2:0/4099495220 wait complete. 2023-06-23T18:34:48.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.757+0000 7f68030f6700 1 Processor -- start 2023-06-23T18:34:48.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f68030f6700 1 -- start start 2023-06-23T18:34:48.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f68030f6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc0ffb90 0x7f67fc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f68030f6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67fc100d80 0x7f67fc198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f68030f6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67fc101710 0x7f67fc19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f68030f6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc19f6a0 con 0x7f67fc0ffb90 2023-06-23T18:34:48.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f68030f6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc19f7e0 con 0x7f67fc100d80 2023-06-23T18:34:48.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f68030f6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc19f970 con 0x7f67fc101710 2023-06-23T18:34:48.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc0ffb90 0x7f67fc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc0ffb90 0x7f67fc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57226/0 (socket says 172.21.15.2:57226) 2023-06-23T18:34:48.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 -- 172.21.15.2:0/453159401 learned_addr learned my addr 172.21.15.2:0/453159401 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f67fbfff700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67fc100d80 0x7f67fc198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6801693700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67fc101710 0x7f67fc19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 -- 172.21.15.2:0/453159401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67fc101710 msgr2=0x7f67fc19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67fc101710 0x7f67fc19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 -- 172.21.15.2:0/453159401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67fc100d80 msgr2=0x7f67fc198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67fc100d80 0x7f67fc198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 -- 172.21.15.2:0/453159401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f4009d00 con 0x7f67fc0ffb90 2023-06-23T18:34:48.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6801693700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67fc101710 0x7f67fc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:34:48.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f67fbfff700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67fc100d80 0x7f67fc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:34:48.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f6800e92700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc0ffb90 0x7f67fc197b20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f67f000d980 tx=0x7f67f000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f67f9ffb700 1 -- 172.21.15.2:0/453159401 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f000de60 con 0x7f67fc0ffb90 2023-06-23T18:34:48.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f67f9ffb700 1 -- 172.21.15.2:0/453159401 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67f00044b0 con 0x7f67fc0ffb90 2023-06-23T18:34:48.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.758+0000 7f68030f6700 1 -- 172.21.15.2:0/453159401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc19fc50 con 0x7f67fc0ffb90 2023-06-23T18:34:48.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.759+0000 7f67f9ffb700 1 -- 172.21.15.2:0/453159401 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0003b40 con 0x7f67fc0ffb90 2023-06-23T18:34:48.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.759+0000 7f68030f6700 1 -- 172.21.15.2:0/453159401 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc1a0160 con 0x7f67fc0ffb90 2023-06-23T18:34:48.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.761+0000 7f67f9ffb700 1 -- 172.21.15.2:0/453159401 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f67f0003ca0 con 0x7f67fc0ffb90 2023-06-23T18:34:48.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.761+0000 7f67f9ffb700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67ec0a1150 0x7f67ec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.761+0000 7f67f9ffb700 1 -- 172.21.15.2:0/453159401 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67f00cc640 con 0x7f67fc0ffb90 2023-06-23T18:34:48.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.761+0000 7f67fbfff700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67ec0a1150 0x7f67ec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.762+0000 7f67fbfff700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67ec0a1150 0x7f67ec0a3600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009cb0 tx=0x7f67e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.762+0000 7f680f27bb80 1 Processor -- start 2023-06-23T18:34:48.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f680f27bb80 1 -- start start 2023-06-23T18:34:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f680f27bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d4c4cb0 con 0x55d65d5c05d0 2023-06-23T18:34:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f680f27bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d517a40 con 0x55d65d5c1000 2023-06-23T18:34:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f680f27bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d510ed0 con 0x55d65d5bfc80 2023-06-23T18:34:48.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6801693700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5c0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57236/0 (socket says 172.21.15.2:57236) 2023-06-23T18:34:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6801693700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5c1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47830/0 (socket says 172.21.15.2:47830) 2023-06-23T18:34:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f67fbfff700 1 -- 172.21.15.2:0/1133886465 learned_addr learned my addr 172.21.15.2:0/1133886465 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6800e92700 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6801693700 1 -- 172.21.15.2:0/1133886465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 msgr2=0x55d65d5c0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6801693700 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5c0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6801693700 1 -- 172.21.15.2:0/1133886465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 msgr2=0x55d65d5c0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6801693700 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5c0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6801693700 1 -- 172.21.15.2:0/1133886465 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d65d392630 con 0x55d65d5c1000 2023-06-23T18:34:48.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6800e92700 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:34:48.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.763+0000 7f6801693700 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5c1470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009fd0 tx=0x7f67f40092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0453fe0e3b862a2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f67e6ffd700 1 -- 172.21.15.2:0/1133886465 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f400e050 con 0x55d65d5c1000 2023-06-23T18:34:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f67e6ffd700 1 -- 172.21.15.2:0/1133886465 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67f40094d0 con 0x55d65d5c1000 2023-06-23T18:34:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f67e6ffd700 1 -- 172.21.15.2:0/1133886465 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4004d40 con 0x55d65d5c1000 2023-06-23T18:34:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 -- 172.21.15.2:0/1133886465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 msgr2=0x55d65d5c1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5c1470 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009fd0 tx=0x7f67f40092e0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 -- 172.21.15.2:0/1133886465 shutdown_connections 2023-06-23T18:34:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5c1470 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5c0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1133886465 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5c0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 -- 172.21.15.2:0/1133886465 >> 172.21.15.2:0/1133886465 conn(0x55d65d5b53f0 msgr2=0x55d65d5ba520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 -- 172.21.15.2:0/1133886465 shutdown_connections 2023-06-23T18:34:48.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.764+0000 7f680f27bb80 1 -- 172.21.15.2:0/1133886465 wait complete. 2023-06-23T18:34:48.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 1 -- start start 2023-06-23T18:34:48.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 1 Processor -- start 2023-06-23T18:34:48.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5bfc80 0x55d65d5c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c05d0 0x55d65d5c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5c1000 0x55d65d5d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d4b0ac0 con 0x55d65d5bfc80 2023-06-23T18:34:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d4b07c0 con 0x55d65d5c05d0 2023-06-23T18:34:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d394920 con 0x55d65d5c1000 2023-06-23T18:34:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f680f27bb80 10 client.0 _ll_register_callbacks cb 0x55d65d4b6260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f6800e92700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5bfc80 0x55d65d5c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c05d0 0x55d65d5c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f6800e92700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5bfc80 0x55d65d5c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57242/0 (socket says 172.21.15.2:57242) 2023-06-23T18:34:48.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c05d0 0x55d65d5c9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47832/0 (socket says 172.21.15.2:47832) 2023-06-23T18:34:48.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.765+0000 7f6800e92700 1 -- 172.21.15.2:0/2068685920 learned_addr learned my addr 172.21.15.2:0/2068685920 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f6801693700 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5c1000 0x55d65d5d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67fbfff700 1 -- 172.21.15.2:0/2068685920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5c1000 msgr2=0x55d65d5d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67fbfff700 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5c1000 0x55d65d5d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67fbfff700 1 -- 172.21.15.2:0/2068685920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5bfc80 msgr2=0x55d65d5c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67fbfff700 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5bfc80 0x55d65d5c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67fbfff700 1 -- 172.21.15.2:0/2068685920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc108900 con 0x55d65d5c05d0 2023-06-23T18:34:48.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f6800e92700 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5bfc80 0x55d65d5c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:48.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67fbfff700 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c05d0 0x55d65d5c9a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55d65d5c09e0 tx=0x7f67e80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d89d748b3040d0f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:34:48.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67dbfff700 1 -- 172.21.15.2:0/2068685920 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8012450 con 0x55d65d5c05d0 2023-06-23T18:34:48.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67dbfff700 1 -- 172.21.15.2:0/2068685920 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67e8018070 con 0x55d65d5c05d0 2023-06-23T18:34:48.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f680f27bb80 1 -- 172.21.15.2:0/2068685920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d65d5b2bb0 con 0x55d65d5c05d0 2023-06-23T18:34:48.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67dbfff700 1 -- 172.21.15.2:0/2068685920 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67e80125b0 con 0x55d65d5c05d0 2023-06-23T18:34:48.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f680f27bb80 10 client.26222 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:48.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f680f27bb80 1 -- 172.21.15.2:0/2068685920 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d65d33ede0 con 0x55d65d5c05d0 2023-06-23T18:34:48.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.766+0000 7f67dbfff700 1 -- 172.21.15.2:0/2068685920 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d65d5b2bb0 con 0x55d65d5c05d0 2023-06-23T18:34:48.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.767+0000 7f67dbfff700 1 -- 172.21.15.2:0/2068685920 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f67e8021640 con 0x55d65d5c05d0 2023-06-23T18:34:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.767+0000 7f680f27bb80 10 client.26222 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.767+0000 7f680f27bb80 10 client.26222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.767+0000 7f680f27bb80 20 client.26222 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.767+0000 7f680f27bb80 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d65d5d5240 0x55d65d5d7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.767+0000 7f680f27bb80 4 client.26222 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:48.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.767+0000 7f680f27bb80 1 -- 172.21.15.2:0/2068685920 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d65d509eb0 con 0x55d65d5d5240 2023-06-23T18:34:48.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.779+0000 7f6800e92700 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d65d5d5240 0x55d65d5d7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.780+0000 7f6800e92700 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d65d5d5240 0x55d65d5d7650 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0edf0b426f716d4 server_cookie=aec3526577a2824b in_seq=0 out_seq=0 2023-06-23T18:34:48.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.780+0000 7f67dbfff700 10 client.26222 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:48.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f67dbfff700 1 -- 172.21.15.2:0/2068685920 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d65d509eb0 con 0x55d65d5d5240 2023-06-23T18:34:48.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f67dbfff700 10 client.26222 handle_command_reply: tid=0 2023-06-23T18:34:48.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 client.26222 shutdown 2023-06-23T18:34:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 -- 172.21.15.2:0/2068685920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c05d0 msgr2=0x55d65d5c9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c05d0 0x55d65d5c9a00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x55d65d5c09e0 tx=0x7f67e80047c0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 -- 172.21.15.2:0/2068685920 shutdown_connections 2023-06-23T18:34:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d65d5d5240 0x55d65d5d7650 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5c1000 0x55d65d5d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5bfc80 0x55d65d5c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 --2- 172.21.15.2:0/2068685920 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c05d0 0x55d65d5c9a00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f680f27bb80 1 -- 172.21.15.2:0/2068685920 >> 172.21.15.2:0/2068685920 conn(0x55d65d5b53f0 msgr2=0x55d65d5b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.781+0000 7f67dbfff700 0 client.26222 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.782+0000 7f680f27bb80 1 -- 172.21.15.2:0/2068685920 shutdown_connections 2023-06-23T18:34:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.782+0000 7f680f27bb80 1 -- 172.21.15.2:0/2068685920 wait complete. 2023-06-23T18:34:48.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.782+0000 7f680f27bb80 20 client.26222 trim_cache size 0 max 16384 2023-06-23T18:34:48.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.800+0000 7f680f27bb80 1 Processor -- start 2023-06-23T18:34:48.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.800+0000 7f680f27bb80 1 -- start start 2023-06-23T18:34:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.800+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.800+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.800+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5dbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.800+0000 7f680f27bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d5bae20 con 0x55d65d5c05d0 2023-06-23T18:34:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.800+0000 7f680f27bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d5d4db0 con 0x55d65d5c1000 2023-06-23T18:34:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.800+0000 7f680f27bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d5bc010 con 0x55d65d5bfc80 2023-06-23T18:34:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57248/0 (socket says 172.21.15.2:57248) 2023-06-23T18:34:48.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 -- 172.21.15.2:0/1172258743 learned_addr learned my addr 172.21.15.2:0/1172258743 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f6801693700 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5dbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f6800e92700 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 -- 172.21.15.2:0/1172258743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 msgr2=0x55d65d5b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 -- 172.21.15.2:0/1172258743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 msgr2=0x55d65d5dbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5dbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 -- 172.21.15.2:0/1172258743 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d65d327c60 con 0x55d65d5c05d0 2023-06-23T18:34:48.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67fbfff700 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5d0a00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f67e8000c00 tx=0x7f67e8021bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc41560079e9c7f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67e77fe700 1 -- 172.21.15.2:0/1172258743 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8011e60 con 0x55d65d5c05d0 2023-06-23T18:34:48.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67e77fe700 1 -- 172.21.15.2:0/1172258743 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67e8004630 con 0x55d65d5c05d0 2023-06-23T18:34:48.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.801+0000 7f67e77fe700 1 -- 172.21.15.2:0/1172258743 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8012970 con 0x55d65d5c05d0 2023-06-23T18:34:48.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 -- 172.21.15.2:0/1172258743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 msgr2=0x55d65d5d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5d0a00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f67e8000c00 tx=0x7f67e8021bf0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 -- 172.21.15.2:0/1172258743 shutdown_connections 2023-06-23T18:34:48.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5dbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1172258743 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5d0a00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 -- 172.21.15.2:0/1172258743 >> 172.21.15.2:0/1172258743 conn(0x55d65d5b53f0 msgr2=0x55d65d5ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 -- 172.21.15.2:0/1172258743 shutdown_connections 2023-06-23T18:34:48.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 -- 172.21.15.2:0/1172258743 wait complete. 2023-06-23T18:34:48.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.802+0000 7f680f27bb80 1 -- start start 2023-06-23T18:34:48.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f680f27bb80 1 Processor -- start 2023-06-23T18:34:48.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5cc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5cc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f680f27bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f680f27bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d5cfcb0 con 0x55d65d5c05d0 2023-06-23T18:34:48.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f680f27bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d5cfe20 con 0x55d65d5c1000 2023-06-23T18:34:48.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f680f27bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65d5cffe0 con 0x55d65d5bfc80 2023-06-23T18:34:48.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f680f27bb80 10 client.0 _ll_register_callbacks cb 0x55d65d4b6260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:48.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f6800e92700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f67fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57256/0 (socket says 172.21.15.2:57256) 2023-06-23T18:34:48.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f67fbfff700 1 -- 172.21.15.2:0/1335702933 learned_addr learned my addr 172.21.15.2:0/1335702933 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:48.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f6801693700 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f6800e92700 1 -- 172.21.15.2:0/1335702933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 msgr2=0x55d65d5cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f6800e92700 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5cf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f6800e92700 1 -- 172.21.15.2:0/1335702933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 msgr2=0x55d65d5cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:48.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.803+0000 7f6800e92700 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f6800e92700 1 -- 172.21.15.2:0/1335702933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d65d33ede0 con 0x55d65d5bfc80 2023-06-23T18:34:48.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f6800e92700 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5cc210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67f00ca140 tx=0x7f67f0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3abdf5c2308d37 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:48.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f67dbfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:48.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f67dbfff700 1 -- 172.21.15.2:0/1335702933 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f0099a50 con 0x55d65d5bfc80 2023-06-23T18:34:48.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f67dbfff700 1 -- 172.21.15.2:0/1335702933 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67f009da80 con 0x55d65d5bfc80 2023-06-23T18:34:48.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f680f27bb80 1 -- 172.21.15.2:0/1335702933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f67f40094d0 con 0x55d65d5bfc80 2023-06-23T18:34:48.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f67dbfff700 1 -- 172.21.15.2:0/1335702933 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67f00a1ea0 con 0x55d65d5bfc80 2023-06-23T18:34:48.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f680f27bb80 10 client.26440 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:48.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f680f27bb80 1 -- 172.21.15.2:0/1335702933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55d65d365250 con 0x55d65d5bfc80 2023-06-23T18:34:48.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f67dbfff700 1 -- 172.21.15.2:0/1335702933 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67f40094d0 con 0x55d65d5bfc80 2023-06-23T18:34:48.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f67dbfff700 1 -- 172.21.15.2:0/1335702933 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f67f00a1a10 con 0x55d65d5bfc80 2023-06-23T18:34:48.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f680f27bb80 10 client.26440 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:48.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f680f27bb80 10 client.26440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:48.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f680f27bb80 20 client.26440 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:48.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.804+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d65d625a80 0x55d65d627e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:48.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.805+0000 7f680f27bb80 4 client.26440 mds_command: new command op to 24533 tid=0[{"prefix": "damage ls"}] 2023-06-23T18:34:48.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.805+0000 7f680f27bb80 1 -- 172.21.15.2:0/1335702933 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d65d4cc050 con 0x55d65d625a80 2023-06-23T18:34:48.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.805+0000 7f67fbfff700 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d65d625a80 0x55d65d627e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:48.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.805+0000 7f67dbfff700 10 client.26440 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:48.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.805+0000 7f67fbfff700 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d65d625a80 0x55d65d627e60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c473740dddc5ce66 server_cookie=9c6782772db14905 in_seq=0 out_seq=0 2023-06-23T18:34:48.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f67dbfff700 1 -- 172.21.15.2:0/1335702933 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d65d4cc050 con 0x55d65d625a80 2023-06-23T18:34:48.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f67dbfff700 10 client.26440 handle_command_reply: tid=0 2023-06-23T18:34:48.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f680f27bb80 1 client.26440 shutdown 2023-06-23T18:34:48.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f680f27bb80 1 -- 172.21.15.2:0/1335702933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 msgr2=0x55d65d5cc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5cc210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f67f00ca140 tx=0x7f67f0000bc0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f680f27bb80 1 -- 172.21.15.2:0/1335702933 shutdown_connections 2023-06-23T18:34:48.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55d65d625a80 0x55d65d627e60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55d65d5c1000 0x55d65d5cf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f67dbfff700 0 client.26440 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:48.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.806+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d65d5bfc80 0x55d65d5cc210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f680f27bb80 1 --2- 172.21.15.2:0/1335702933 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55d65d5c05d0 0x55d65d5cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f680f27bb80 1 -- 172.21.15.2:0/1335702933 >> 172.21.15.2:0/1335702933 conn(0x55d65d5b53f0 msgr2=0x55d65d5dbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f680f27bb80 1 -- 172.21.15.2:0/1335702933 shutdown_connections 2023-06-23T18:34:48.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f680f27bb80 1 -- 172.21.15.2:0/1335702933 wait complete. 2023-06-23T18:34:48.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f680f27bb80 20 client.26440 trim_cache size 0 max 16384 2023-06-23T18:34:48.828 INFO:teuthology.orchestra.run.smithi002.stdout:[] 2023-06-23T18:34:48.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f68030f6700 1 -- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67ec0a1150 msgr2=0x7f67ec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f68030f6700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67ec0a1150 0x7f67ec0a3600 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009cb0 tx=0x7f67e8009260 comp rx=0 tx=0).stop 2023-06-23T18:34:48.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f68030f6700 1 -- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc0ffb90 msgr2=0x7f67fc197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:48.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.807+0000 7f68030f6700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc0ffb90 0x7f67fc197b20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f67f000d980 tx=0x7f67f000dd40 comp rx=0 tx=0).stop 2023-06-23T18:34:48.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.808+0000 7f68030f6700 1 -- 172.21.15.2:0/453159401 shutdown_connections 2023-06-23T18:34:48.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.808+0000 7f68030f6700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67fc101710 0x7f67fc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.808+0000 7f68030f6700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67fc0ffb90 0x7f67fc197b20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.808+0000 7f68030f6700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f67ec0a1150 0x7f67ec0a3600 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.808+0000 7f68030f6700 1 --2- 172.21.15.2:0/453159401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67fc100d80 0x7f67fc198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:48.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.808+0000 7f68030f6700 1 -- 172.21.15.2:0/453159401 >> 172.21.15.2:0/453159401 conn(0x7f67fc0fb1b0 msgr2=0x7f67fc106660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:48.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.808+0000 7f68030f6700 1 -- 172.21.15.2:0/453159401 shutdown_connections 2023-06-23T18:34:48.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:48.808+0000 7f68030f6700 1 -- 172.21.15.2:0/453159401 wait complete. 2023-06-23T18:34:49.832 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-06-23T18:34:49.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-06-23T18:34:50.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 -- 172.21.15.2:0/1165419959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c0fea60 msgr2=0x7fd91c0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 --2- 172.21.15.2:0/1165419959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c0fea60 0x7fd91c0fee70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd910009990 tx=0x7fd910009ca0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 -- 172.21.15.2:0/1165419959 shutdown_connections 2023-06-23T18:34:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 --2- 172.21.15.2:0/1165419959 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd91c1006a0 0x7fd91c106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 --2- 172.21.15.2:0/1165419959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c0fea60 0x7fd91c0fee70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 --2- 172.21.15.2:0/1165419959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd91c0ffcf0 0x7fd91c100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 -- 172.21.15.2:0/1165419959 >> 172.21.15.2:0/1165419959 conn(0x7fd91c0fa080 msgr2=0x7fd91c0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 -- 172.21.15.2:0/1165419959 shutdown_connections 2023-06-23T18:34:50.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.022+0000 7fd922e0b700 1 -- 172.21.15.2:0/1165419959 wait complete. 2023-06-23T18:34:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd922e0b700 1 Processor -- start 2023-06-23T18:34:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd922e0b700 1 -- start start 2023-06-23T18:34:50.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd922e0b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd91c0fea60 0x7fd91c194700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd922e0b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd91c0ffcf0 0x7fd91c194c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd922e0b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c1006a0 0x7fd91c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd922e0b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19c280 con 0x7fd91c0ffcf0 2023-06-23T18:34:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd922e0b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19c3c0 con 0x7fd91c0fea60 2023-06-23T18:34:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd922e0b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19c550 con 0x7fd91c1006a0 2023-06-23T18:34:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd91c0fea60 0x7fd91c194700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd91c0ffcf0 0x7fd91c194c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd9213a8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c1006a0 0x7fd91c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd91c0fea60 0x7fd91c194700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47860/0 (socket says 172.21.15.2:47860) 2023-06-23T18:34:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd9213a8700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c1006a0 0x7fd91c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57610/0 (socket says 172.21.15.2:57610) 2023-06-23T18:34:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.023+0000 7fd920ba7700 1 -- 172.21.15.2:0/2221019735 learned_addr learned my addr 172.21.15.2:0/2221019735 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd920ba7700 1 -- 172.21.15.2:0/2221019735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c1006a0 msgr2=0x7fd91c19bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c1006a0 0x7fd91c19bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd920ba7700 1 -- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd91c0ffcf0 msgr2=0x7fd91c194c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd91c0ffcf0 0x7fd91c194c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd920ba7700 1 -- 172.21.15.2:0/2221019735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914009c40 con 0x7fd91c0fea60 2023-06-23T18:34:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd91c0fea60 0x7fd91c194700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd910005910 tx=0x7fd910005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd919ffb700 1 -- 172.21.15.2:0/2221019735 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91001d070 con 0x7fd91c0fea60 2023-06-23T18:34:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd919ffb700 1 -- 172.21.15.2:0/2221019735 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd91000bdb0 con 0x7fd91c0fea60 2023-06-23T18:34:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd922e0b700 1 -- 172.21.15.2:0/2221019735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9100095f0 con 0x7fd91c0fea60 2023-06-23T18:34:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd919ffb700 1 -- 172.21.15.2:0/2221019735 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910022dd0 con 0x7fd91c0fea60 2023-06-23T18:34:50.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.024+0000 7fd922e0b700 1 -- 172.21.15.2:0/2221019735 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c068d80 con 0x7fd91c0fea60 2023-06-23T18:34:50.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.028+0000 7fd919ffb700 1 -- 172.21.15.2:0/2221019735 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd910022570 con 0x7fd91c0fea60 2023-06-23T18:34:50.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.029+0000 7fd919ffb700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd90c0a11d0 0x7fd90c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.029+0000 7fd919ffb700 1 -- 172.21.15.2:0/2221019735 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd9100cd7a0 con 0x7fd91c0fea60 2023-06-23T18:34:50.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.029+0000 7fd91bfff700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd90c0a11d0 0x7fd90c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd92ef90b80 1 Processor -- start 2023-06-23T18:34:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd91bfff700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd90c0a11d0 0x7fd90c0a3680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd908009c60 tx=0x7fd908009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd92ef90b80 1 -- start start 2023-06-23T18:34:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafde110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafdeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 0x5566eafdf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eaee2d30 con 0x5566eafddd00 2023-06-23T18:34:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eaf35ac0 con 0x5566eafde650 2023-06-23T18:34:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eaf2ef50 con 0x5566eafdf080 2023-06-23T18:34:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafde110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafde110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57276/0 (socket says 172.21.15.2:57276) 2023-06-23T18:34:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.030+0000 7fd920ba7700 1 -- 172.21.15.2:0/2971326851 learned_addr learned my addr 172.21.15.2:0/2971326851 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd9213a8700 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 0x5566eafdf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd91bfff700 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafdeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd920ba7700 1 -- 172.21.15.2:0/2971326851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 msgr2=0x5566eafdf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 0x5566eafdf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd920ba7700 1 -- 172.21.15.2:0/2971326851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 msgr2=0x5566eafdeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafdeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd920ba7700 1 -- 172.21.15.2:0/2971326851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566eadb06b0 con 0x5566eafddd00 2023-06-23T18:34:50.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafde110 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd910097260 tx=0x7fd91009abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158f318f81c119bf server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd906ffd700 1 -- 172.21.15.2:0/2971326851 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91009ad60 con 0x5566eafddd00 2023-06-23T18:34:50.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd906ffd700 1 -- 172.21.15.2:0/2971326851 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd91009ed90 con 0x5566eafddd00 2023-06-23T18:34:50.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.031+0000 7fd906ffd700 1 -- 172.21.15.2:0/2971326851 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100a2c90 con 0x5566eafddd00 2023-06-23T18:34:50.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.032+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2971326851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 msgr2=0x5566eafde110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.032+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafde110 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd910097260 tx=0x7fd91009abb0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.032+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2971326851 shutdown_connections 2023-06-23T18:34:50.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.032+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 0x5566eafdf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.032+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafde110 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.032+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2971326851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafdeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.032+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2971326851 >> 172.21.15.2:0/2971326851 conn(0x5566eafd3470 msgr2=0x5566eafd85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.033+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2971326851 shutdown_connections 2023-06-23T18:34:50.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.033+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2971326851 wait complete. 2023-06-23T18:34:50.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.033+0000 7fd92ef90b80 1 -- start start 2023-06-23T18:34:50.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.033+0000 7fd92ef90b80 1 Processor -- start 2023-06-23T18:34:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafde650 0x5566eafe7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafee850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eaeceb40 con 0x5566eafddd00 2023-06-23T18:34:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eaece840 con 0x5566eafdf080 2023-06-23T18:34:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eadb29a0 con 0x5566eafde650 2023-06-23T18:34:50.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd92ef90b80 10 client.0 _ll_register_callbacks cb 0x5566eaed42e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd9213a8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafee850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafde650 0x5566eafe7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57282/0 (socket says 172.21.15.2:57282) 2023-06-23T18:34:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd920ba7700 1 -- 172.21.15.2:0/2711259644 learned_addr learned my addr 172.21.15.2:0/2711259644 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd9213a8700 1 -- 172.21.15.2:0/2711259644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafde650 msgr2=0x5566eafe7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.034+0000 7fd9213a8700 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafde650 0x5566eafe7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd9213a8700 1 -- 172.21.15.2:0/2711259644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 msgr2=0x5566eafe72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd9213a8700 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd9213a8700 1 -- 172.21.15.2:0/2711259644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c068d80 con 0x5566eafdf080 2023-06-23T18:34:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd91bfff700 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafde650 0x5566eafe7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd9213a8700 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafee850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd91400eee0 tx=0x7fd914014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a81f69beac5a80 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd8fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:34:50.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2711259644 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91400e040 con 0x5566eafdf080 2023-06-23T18:34:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2711259644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566eafd0c30 con 0x5566eafdf080 2023-06-23T18:34:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2711259644 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd91401a070 con 0x5566eafdf080 2023-06-23T18:34:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2711259644 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd914014b90 con 0x5566eafdf080 2023-06-23T18:34:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd92ef90b80 10 client.26240 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.035+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2711259644 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5566ead5ce60 con 0x5566eafdf080 2023-06-23T18:34:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2711259644 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5566eafd0c30 con 0x5566eafdf080 2023-06-23T18:34:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2711259644 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd914007d00 con 0x5566eafdf080 2023-06-23T18:34:50.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd92ef90b80 10 client.26240 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd92ef90b80 10 client.26240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd92ef90b80 20 client.26240 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566eb004090 0x5566eb0064a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd92ef90b80 4 client.26240 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2711259644 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566eaf27f30 con 0x5566eb004090 2023-06-23T18:34:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566eb004090 0x5566eb0064a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.036+0000 7fd8fbfff700 10 client.26240 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:50.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.037+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566eb004090 0x5566eb0064a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=762bd2e41597635d server_cookie=3b834f4f658b5bd3 in_seq=0 out_seq=0 2023-06-23T18:34:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2711259644 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5566eaf27f30 con 0x5566eb004090 2023-06-23T18:34:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd8fbfff700 10 client.26240 handle_command_reply: tid=0 2023-06-23T18:34:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 client.26240 shutdown 2023-06-23T18:34:50.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2711259644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 msgr2=0x5566eafee850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafee850 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd91400eee0 tx=0x7fd914014600 comp rx=0 tx=0).stop 2023-06-23T18:34:50.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2711259644 shutdown_connections 2023-06-23T18:34:50.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566eb004090 0x5566eb0064a0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafee850 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2711259644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafde650 0x5566eafe7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2711259644 >> 172.21.15.2:0/2711259644 conn(0x5566eafd3470 msgr2=0x5566eafd7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd8fbfff700 0 client.26240 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:50.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2711259644 shutdown_connections 2023-06-23T18:34:50.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.038+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2711259644 wait complete. 2023-06-23T18:34:50.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.039+0000 7fd92ef90b80 20 client.26240 trim_cache size 0 max 16384 2023-06-23T18:34:50.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.067+0000 7fd92ef90b80 1 Processor -- start 2023-06-23T18:34:50.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.067+0000 7fd92ef90b80 1 -- start start 2023-06-23T18:34:50.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.067+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.067+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafee810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.067+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 0x5566eafe2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eafe84c0 con 0x5566eafddd00 2023-06-23T18:34:50.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eafd8fd0 con 0x5566eafde650 2023-06-23T18:34:50.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eafd91c0 con 0x5566eafdf080 2023-06-23T18:34:50.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd91bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57292/0 (socket says 172.21.15.2:57292) 2023-06-23T18:34:50.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd920ba7700 1 -- 172.21.15.2:0/2846230063 learned_addr learned my addr 172.21.15.2:0/2846230063 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd9213a8700 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 0x5566eafe2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd920ba7700 1 -- 172.21.15.2:0/2846230063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 msgr2=0x5566eafe2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 0x5566eafe2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd920ba7700 1 -- 172.21.15.2:0/2846230063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 msgr2=0x5566eafee810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafee810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd920ba7700 1 -- 172.21.15.2:0/2846230063 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566eafdf720 con 0x5566eafddd00 2023-06-23T18:34:50.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.068+0000 7fd91bfff700 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:50.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.069+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe1f40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd91009aca0 tx=0x7fd910005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51bf738899b1cfa server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.069+0000 7fd9077fe700 1 -- 172.21.15.2:0/2846230063 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91009ad60 con 0x5566eafddd00 2023-06-23T18:34:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.069+0000 7fd9077fe700 1 -- 172.21.15.2:0/2846230063 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd910099100 con 0x5566eafddd00 2023-06-23T18:34:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.069+0000 7fd9077fe700 1 -- 172.21.15.2:0/2846230063 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100a2ef0 con 0x5566eafddd00 2023-06-23T18:34:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2846230063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 msgr2=0x5566eafe1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe1f40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fd91009aca0 tx=0x7fd910005c00 comp rx=0 tx=0).stop 2023-06-23T18:34:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2846230063 shutdown_connections 2023-06-23T18:34:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafdf080 0x5566eafe2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafddd00 0x5566eafe1f40 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2846230063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafde650 0x5566eafee810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2846230063 >> 172.21.15.2:0/2846230063 conn(0x5566eafd3470 msgr2=0x5566eb0060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2846230063 shutdown_connections 2023-06-23T18:34:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2846230063 wait complete. 2023-06-23T18:34:50.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 -- start start 2023-06-23T18:34:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.070+0000 7fd92ef90b80 1 Processor -- start 2023-06-23T18:34:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafddd00 0x5566eafea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafde650 0x5566eafeac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd92ef90b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafedc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eafee160 con 0x5566eafde650 2023-06-23T18:34:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eafee2d0 con 0x5566eafdf080 2023-06-23T18:34:50.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd92ef90b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566eafee490 con 0x5566eafddd00 2023-06-23T18:34:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd92ef90b80 10 client.0 _ll_register_callbacks cb 0x5566eaed42e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafddd00 0x5566eafea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd920ba7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafddd00 0x5566eafea6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57634/0 (socket says 172.21.15.2:57634) 2023-06-23T18:34:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd920ba7700 1 -- 172.21.15.2:0/2707274205 learned_addr learned my addr 172.21.15.2:0/2707274205 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd9213a8700 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafedc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.071+0000 7fd91bfff700 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafde650 0x5566eafeac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd920ba7700 1 -- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 msgr2=0x5566eafedc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafedc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd920ba7700 1 -- 172.21.15.2:0/2707274205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafde650 msgr2=0x5566eafeac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafde650 0x5566eafeac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd920ba7700 1 -- 172.21.15.2:0/2707274205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd908017070 con 0x5566eafddd00 2023-06-23T18:34:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd9213a8700 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafedc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:34:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd920ba7700 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafddd00 0x5566eafea6c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd910000c00 tx=0x7fd91009ab30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9402576a7b6b27a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd8fbfff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:34:50.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2707274205 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100a2e90 con 0x5566eafddd00 2023-06-23T18:34:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2707274205 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd91009ad60 con 0x5566eafddd00 2023-06-23T18:34:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2707274205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd91009ed90 con 0x5566eafddd00 2023-06-23T18:34:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2707274205 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd91009a4f0 con 0x5566eafddd00 2023-06-23T18:34:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd92ef90b80 10 client.26458 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.072+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2707274205 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5566eadb06b0 con 0x5566eafddd00 2023-06-23T18:34:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2707274205 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd91009ed90 con 0x5566eafddd00 2023-06-23T18:34:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2707274205 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd910096ce0 con 0x5566eafddd00 2023-06-23T18:34:50.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd92ef90b80 10 client.26458 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:50.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd92ef90b80 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:50.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd92ef90b80 20 client.26458 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:50.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566eb061370 0x5566eb063780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd92ef90b80 4 client.26458 mds_command: new command op to 24533 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-06-23T18:34:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2707274205 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5566eaeea0d0 con 0x5566eb061370 2023-06-23T18:34:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.073+0000 7fd91bfff700 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566eb061370 0x5566eb063780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.074+0000 7fd91bfff700 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566eb061370 0x5566eb063780 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1879a820543395a7 server_cookie=bb1bb2cc428321bb in_seq=0 out_seq=0 2023-06-23T18:34:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.074+0000 7fd8fbfff700 10 client.26458 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:50.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd8fbfff700 1 -- 172.21.15.2:0/2707274205 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5566eaeea0d0 con 0x5566eb061370 2023-06-23T18:34:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd8fbfff700 10 client.26458 handle_command_reply: tid=0 2023-06-23T18:34:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 client.26458 shutdown 2023-06-23T18:34:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2707274205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafddd00 msgr2=0x5566eafea6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafddd00 0x5566eafea6c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd910000c00 tx=0x7fd91009ab30 comp rx=0 tx=0).stop 2023-06-23T18:34:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2707274205 shutdown_connections 2023-06-23T18:34:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5566eb061370 0x5566eb063780 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5566eafdf080 0x5566eafedc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5566eafddd00 0x5566eafea6c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 --2- 172.21.15.2:0/2707274205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5566eafde650 0x5566eafeac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2707274205 >> 172.21.15.2:0/2707274205 conn(0x5566eafd3470 msgr2=0x5566eafe1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.075+0000 7fd8fbfff700 0 client.26458 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:50.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.076+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2707274205 shutdown_connections 2023-06-23T18:34:50.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.076+0000 7fd92ef90b80 1 -- 172.21.15.2:0/2707274205 wait complete. 2023-06-23T18:34:50.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.076+0000 7fd92ef90b80 20 client.26458 trim_cache size 0 max 16384 2023-06-23T18:34:50.087 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:34:50.087 INFO:teuthology.orchestra.run.smithi002.stdout: "return_code": 0, 2023-06-23T18:34:50.087 INFO:teuthology.orchestra.run.smithi002.stdout: "scrub_tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:34:50.087 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": "asynchronous" 2023-06-23T18:34:50.087 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:34:50.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.076+0000 7fd922e0b700 1 -- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd90c0a11d0 msgr2=0x7fd90c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.076+0000 7fd922e0b700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd90c0a11d0 0x7fd90c0a3680 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd908009c60 tx=0x7fd908009260 comp rx=0 tx=0).stop 2023-06-23T18:34:50.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.076+0000 7fd922e0b700 1 -- 172.21.15.2:0/2221019735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd91c0fea60 msgr2=0x7fd91c194700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.076+0000 7fd922e0b700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd91c0fea60 0x7fd91c194700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd910005910 tx=0x7fd910005740 comp rx=0 tx=0).stop 2023-06-23T18:34:50.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.077+0000 7fd922e0b700 1 -- 172.21.15.2:0/2221019735 shutdown_connections 2023-06-23T18:34:50.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.077+0000 7fd922e0b700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd91c1006a0 0x7fd91c19bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.077+0000 7fd922e0b700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fd90c0a11d0 0x7fd90c0a3680 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.077+0000 7fd922e0b700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd91c0fea60 0x7fd91c194700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.077+0000 7fd922e0b700 1 --2- 172.21.15.2:0/2221019735 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd91c0ffcf0 0x7fd91c194c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.077+0000 7fd922e0b700 1 -- 172.21.15.2:0/2221019735 >> 172.21.15.2:0/2221019735 conn(0x7fd91c0fa080 msgr2=0x7fd91c0fc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.077+0000 7fd922e0b700 1 -- 172.21.15.2:0/2221019735 shutdown_connections 2023-06-23T18:34:50.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.077+0000 7fd922e0b700 1 -- 172.21.15.2:0/2221019735 wait complete. 2023-06-23T18:34:50.097 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:34:50.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.287+0000 7fe3219e2700 1 -- 172.21.15.2:0/1367964524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe31c100de0 msgr2=0x7fe31c101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.287+0000 7fe3219e2700 1 --2- 172.21.15.2:0/1367964524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe31c100de0 0x7fe31c101250 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe310009990 tx=0x7fe310009ca0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.288+0000 7fe3219e2700 1 -- 172.21.15.2:0/1367964524 shutdown_connections 2023-06-23T18:34:50.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.288+0000 7fe3219e2700 1 --2- 172.21.15.2:0/1367964524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c101790 0x7fe31c108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.288+0000 7fe3219e2700 1 --2- 172.21.15.2:0/1367964524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe31c0ffb50 0x7fe31c0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.288+0000 7fe3219e2700 1 --2- 172.21.15.2:0/1367964524 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe31c100de0 0x7fe31c101250 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.288+0000 7fe3219e2700 1 -- 172.21.15.2:0/1367964524 >> 172.21.15.2:0/1367964524 conn(0x7fe31c0fb150 msgr2=0x7fe31c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.288+0000 7fe3219e2700 1 -- 172.21.15.2:0/1367964524 shutdown_connections 2023-06-23T18:34:50.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.289+0000 7fe3219e2700 1 -- 172.21.15.2:0/1367964524 wait complete. 2023-06-23T18:34:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.289+0000 7fe3219e2700 1 Processor -- start 2023-06-23T18:34:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.289+0000 7fe3219e2700 1 -- start start 2023-06-23T18:34:50.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe3219e2700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c0ffb50 0x7fe31c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe3219e2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe31c100de0 0x7fe31c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe3219e2700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe31c101790 0x7fe31c1a3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe3219e2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1a3930 con 0x7fe31c100de0 2023-06-23T18:34:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe3219e2700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1a3a70 con 0x7fe31c0ffb50 2023-06-23T18:34:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe3219e2700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31c1a3c00 con 0x7fe31c101790 2023-06-23T18:34:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe31c101790 0x7fe31c1a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c0ffb50 0x7fe31c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe31c100de0 0x7fe31c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe31c101790 0x7fe31c1a3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57648/0 (socket says 172.21.15.2:57648) 2023-06-23T18:34:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31b7fe700 1 -- 172.21.15.2:0/852947485 learned_addr learned my addr 172.21.15.2:0/852947485 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31b7fe700 1 -- 172.21.15.2:0/852947485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c0ffb50 msgr2=0x7fe31c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c0ffb50 0x7fe31c19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31b7fe700 1 -- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe31c100de0 msgr2=0x7fe31c19c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe31c100de0 0x7fe31c19c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.290+0000 7fe31b7fe700 1 -- 172.21.15.2:0/852947485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100095f0 con 0x7fe31c101790 2023-06-23T18:34:50.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.291+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe31c101790 0x7fe31c1a3360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe31400ef30 tx=0x7fe31400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.291+0000 7fe3209e0700 1 -- 172.21.15.2:0/852947485 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314018070 con 0x7fe31c101790 2023-06-23T18:34:50.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.291+0000 7fe3209e0700 1 -- 172.21.15.2:0/852947485 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe31400f040 con 0x7fe31c101790 2023-06-23T18:34:50.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.291+0000 7fe3209e0700 1 -- 172.21.15.2:0/852947485 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400cbc0 con 0x7fe31c101790 2023-06-23T18:34:50.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.291+0000 7fe3219e2700 1 -- 172.21.15.2:0/852947485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31c1a3e80 con 0x7fe31c101790 2023-06-23T18:34:50.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.291+0000 7fe3219e2700 1 -- 172.21.15.2:0/852947485 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c1a4250 con 0x7fe31c101790 2023-06-23T18:34:50.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.294+0000 7fe3209e0700 1 -- 172.21.15.2:0/852947485 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe314007500 con 0x7fe31c101790 2023-06-23T18:34:50.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.295+0000 7fe3209e0700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3040a1150 0x7fe3040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.295+0000 7fe3209e0700 1 -- 172.21.15.2:0/852947485 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe3140cc3d0 con 0x7fe31c101790 2023-06-23T18:34:50.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.295+0000 7fe31affd700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3040a1150 0x7fe3040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.295+0000 7fe32db67b80 1 Processor -- start 2023-06-23T18:34:50.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe32db67b80 1 -- start start 2023-06-23T18:34:50.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e197090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e197a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 0x564c0e198470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe32db67b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e09bcb0 con 0x564c0e1975d0 2023-06-23T18:34:50.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe32db67b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e0eea40 con 0x564c0e196c80 2023-06-23T18:34:50.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe32db67b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e0e7ed0 con 0x564c0e198000 2023-06-23T18:34:50.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57318/0 (socket says 172.21.15.2:57318) 2023-06-23T18:34:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31a7fc700 1 -- 172.21.15.2:0/1593458639 learned_addr learned my addr 172.21.15.2:0/1593458639 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31affd700 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31affd700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3040a1150 0x7fe3040a3600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00ddd0 tx=0x7fe30c00f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31a7fc700 1 -- 172.21.15.2:0/1593458639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 msgr2=0x564c0e198470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:34:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31a7fc700 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 0x564c0e198470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31a7fc700 1 -- 172.21.15.2:0/1593458639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 msgr2=0x564c0e197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.296+0000 7fe31a7fc700 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.297+0000 7fe31a7fc700 1 -- 172.21.15.2:0/1593458639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0df69630 con 0x564c0e1975d0 2023-06-23T18:34:50.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.297+0000 7fe31affd700 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:34:50.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.297+0000 7fe31a7fc700 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e197a70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe3100059d0 tx=0x7fe310005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33fdd8b3ed7d601 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.297+0000 7fe301ffb700 1 -- 172.21.15.2:0/1593458639 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31001d070 con 0x564c0e1975d0 2023-06-23T18:34:50.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.297+0000 7fe301ffb700 1 -- 172.21.15.2:0/1593458639 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe31001e040 con 0x564c0e1975d0 2023-06-23T18:34:50.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.297+0000 7fe301ffb700 1 -- 172.21.15.2:0/1593458639 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31000b730 con 0x564c0e1975d0 2023-06-23T18:34:50.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 -- 172.21.15.2:0/1593458639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 msgr2=0x564c0e197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e197a70 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe3100059d0 tx=0x7fe310005bc0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 -- 172.21.15.2:0/1593458639 shutdown_connections 2023-06-23T18:34:50.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 0x564c0e198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 --2- 172.21.15.2:0/1593458639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e197a70 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 -- 172.21.15.2:0/1593458639 >> 172.21.15.2:0/1593458639 conn(0x564c0e18c3f0 msgr2=0x564c0e191520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 -- 172.21.15.2:0/1593458639 shutdown_connections 2023-06-23T18:34:50.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.298+0000 7fe32db67b80 1 -- 172.21.15.2:0/1593458639 wait complete. 2023-06-23T18:34:50.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 1 -- start start 2023-06-23T18:34:50.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 1 Processor -- start 2023-06-23T18:34:50.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e1975d0 0x564c0e1a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e087ac0 con 0x564c0e198000 2023-06-23T18:34:50.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e0877c0 con 0x564c0e196c80 2023-06-23T18:34:50.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0df6b920 con 0x564c0e1975d0 2023-06-23T18:34:50.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.299+0000 7fe32db67b80 10 client.0 _ll_register_callbacks cb 0x564c0e08d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:50.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47916/0 (socket says 172.21.15.2:47916) 2023-06-23T18:34:50.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 -- 172.21.15.2:0/2178397954 learned_addr learned my addr 172.21.15.2:0/2178397954 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31a7fc700 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e1975d0 0x564c0e1a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 -- 172.21.15.2:0/2178397954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e1975d0 msgr2=0x564c0e1a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e1975d0 0x564c0e1a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 -- 172.21.15.2:0/2178397954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 msgr2=0x564c0e1a7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 -- 172.21.15.2:0/2178397954 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c1088a0 con 0x564c0e196c80 2023-06-23T18:34:50.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe31affd700 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a0640 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe30c019970 tx=0x7fe30c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb52f1f417cde21c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe2faffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:34:50.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.300+0000 7fe2faffd700 1 -- 172.21.15.2:0/2178397954 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c020040 con 0x564c0e196c80 2023-06-23T18:34:50.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe2faffd700 1 -- 172.21.15.2:0/2178397954 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe30c019e30 con 0x564c0e196c80 2023-06-23T18:34:50.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 1 -- 172.21.15.2:0/2178397954 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c0e189bb0 con 0x564c0e196c80 2023-06-23T18:34:50.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe2faffd700 1 -- 172.21.15.2:0/2178397954 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30c004830 con 0x564c0e196c80 2023-06-23T18:34:50.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 10 client.26258 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:50.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 1 -- 172.21.15.2:0/2178397954 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x564c0df15de0 con 0x564c0e196c80 2023-06-23T18:34:50.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe2faffd700 1 -- 172.21.15.2:0/2178397954 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564c0e189bb0 con 0x564c0e196c80 2023-06-23T18:34:50.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe2faffd700 1 -- 172.21.15.2:0/2178397954 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe30c0257b0 con 0x564c0e196c80 2023-06-23T18:34:50.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 10 client.26258 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:50.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 10 client.26258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:50.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 20 client.26258 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:50.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564c0e1ac4d0 0x564c0e1ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 4 client.26258 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:34:50.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.301+0000 7fe32db67b80 1 -- 172.21.15.2:0/2178397954 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c0e0e0eb0 con 0x564c0e1ac4d0 2023-06-23T18:34:50.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.302+0000 7fe31a7fc700 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564c0e1ac4d0 0x564c0e1ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.302+0000 7fe31a7fc700 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564c0e1ac4d0 0x564c0e1ae8e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da366e666e3dcd73 server_cookie=70b6d1de223ebe7 in_seq=0 out_seq=0 2023-06-23T18:34:50.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.302+0000 7fe2faffd700 10 client.26258 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:50.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.303+0000 7fe2faffd700 1 -- 172.21.15.2:0/2178397954 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c0e0e0eb0 con 0x564c0e1ac4d0 2023-06-23T18:34:50.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.303+0000 7fe2faffd700 10 client.26258 handle_command_reply: tid=0 2023-06-23T18:34:50.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.303+0000 7fe32db67b80 1 client.26258 shutdown 2023-06-23T18:34:50.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.303+0000 7fe32db67b80 1 -- 172.21.15.2:0/2178397954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 msgr2=0x564c0e1a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.303+0000 7fe32db67b80 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a0640 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe30c019970 tx=0x7fe30c0047c0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.303+0000 7fe32db67b80 1 -- 172.21.15.2:0/2178397954 shutdown_connections 2023-06-23T18:34:50.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe32db67b80 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564c0e1ac4d0 0x564c0e1ae8e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe32db67b80 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe2faffd700 0 client.26258 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:50.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe32db67b80 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a0640 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe32db67b80 1 --2- 172.21.15.2:0/2178397954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e1975d0 0x564c0e1a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe32db67b80 1 -- 172.21.15.2:0/2178397954 >> 172.21.15.2:0/2178397954 conn(0x564c0e18c3f0 msgr2=0x564c0e190c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe32db67b80 1 -- 172.21.15.2:0/2178397954 shutdown_connections 2023-06-23T18:34:50.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe32db67b80 1 -- 172.21.15.2:0/2178397954 wait complete. 2023-06-23T18:34:50.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.304+0000 7fe32db67b80 20 client.26258 trim_cache size 0 max 16384 2023-06-23T18:34:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe32db67b80 1 Processor -- start 2023-06-23T18:34:50.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe32db67b80 1 -- start start 2023-06-23T18:34:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e196c80 0x564c0e19af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e1975d0 0x564c0e19b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe32db67b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e0e0eb0 con 0x564c0e198000 2023-06-23T18:34:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe32db67b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e191e20 con 0x564c0e1975d0 2023-06-23T18:34:50.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe32db67b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e1a98b0 con 0x564c0e196c80 2023-06-23T18:34:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe31a7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e1975d0 0x564c0e19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe31b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57334/0 (socket says 172.21.15.2:57334) 2023-06-23T18:34:50.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.332+0000 7fe31b7fe700 1 -- 172.21.15.2:0/1362267435 learned_addr learned my addr 172.21.15.2:0/1362267435 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.333+0000 7fe31affd700 1 --2- 172.21.15.2:0/1362267435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e196c80 0x564c0e19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.333+0000 7fe31b7fe700 1 -- 172.21.15.2:0/1362267435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e196c80 msgr2=0x564c0e19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.333+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/1362267435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e196c80 0x564c0e19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.333+0000 7fe31b7fe700 1 -- 172.21.15.2:0/1362267435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e1975d0 msgr2=0x564c0e19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.333+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/1362267435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e1975d0 0x564c0e19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.333+0000 7fe31b7fe700 1 -- 172.21.15.2:0/1362267435 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0defec60 con 0x564c0e198000 2023-06-23T18:34:50.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.333+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/1362267435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7b50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe3140090a0 tx=0x7fe3140994d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f9a4a3f16ade17 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.333+0000 7fe3027fc700 1 -- 172.21.15.2:0/1362267435 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31409d8a0 con 0x564c0e198000 2023-06-23T18:34:50.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.334+0000 7fe3027fc700 1 -- 172.21.15.2:0/1362267435 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3140a1c30 con 0x564c0e198000 2023-06-23T18:34:50.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.334+0000 7fe3027fc700 1 -- 172.21.15.2:0/1362267435 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3140b9310 con 0x564c0e198000 2023-06-23T18:34:50.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.334+0000 7fe32db67b80 1 -- 172.21.15.2:0/1362267435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 msgr2=0x564c0e1a7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.334+0000 7fe32db67b80 1 --2- 172.21.15.2:0/1362267435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7b50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe3140090a0 tx=0x7fe3140994d0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 -- 172.21.15.2:0/1362267435 shutdown_connections 2023-06-23T18:34:50.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 --2- 172.21.15.2:0/1362267435 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e198000 0x564c0e1a7b50 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 --2- 172.21.15.2:0/1362267435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e196c80 0x564c0e19af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 --2- 172.21.15.2:0/1362267435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e1975d0 0x564c0e19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 -- 172.21.15.2:0/1362267435 >> 172.21.15.2:0/1362267435 conn(0x564c0e18c3f0 msgr2=0x564c0e19f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 -- 172.21.15.2:0/1362267435 shutdown_connections 2023-06-23T18:34:50.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 -- 172.21.15.2:0/1362267435 wait complete. 2023-06-23T18:34:50.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 -- start start 2023-06-23T18:34:50.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.335+0000 7fe32db67b80 1 Processor -- start 2023-06-23T18:34:50.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e1a4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe32db67b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 0x564c0e1a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe32db67b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e1a4ea0 con 0x564c0e1975d0 2023-06-23T18:34:50.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe32db67b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e1a7e60 con 0x564c0e196c80 2023-06-23T18:34:50.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe32db67b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0e1a7ff0 con 0x564c0e198000 2023-06-23T18:34:50.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe32db67b80 10 client.0 _ll_register_callbacks cb 0x7fe30c0194a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:34:50.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e1a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e1a4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57338/0 (socket says 172.21.15.2:57338) 2023-06-23T18:34:50.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31affd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47928/0 (socket says 172.21.15.2:47928) 2023-06-23T18:34:50.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31a7fc700 1 -- 172.21.15.2:0/3972284753 learned_addr learned my addr 172.21.15.2:0/3972284753 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:34:50.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 0x564c0e1a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31affd700 1 -- 172.21.15.2:0/3972284753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 msgr2=0x564c0e1a47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31affd700 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 0x564c0e1a47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31affd700 1 -- 172.21.15.2:0/3972284753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 msgr2=0x564c0e1a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:34:50.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31affd700 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e1a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31affd700 1 -- 172.21.15.2:0/3972284753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31c1a4250 con 0x564c0e196c80 2023-06-23T18:34:50.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.336+0000 7fe31b7fe700 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 0x564c0e1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:34:50.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.337+0000 7fe31affd700 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a77e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe30c025c60 tx=0x7fe30c0187e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98297141b0f1eab4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:34:50.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.337+0000 7fe2faffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:34:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.337+0000 7fe2faffd700 1 -- 172.21.15.2:0/3972284753 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c020040 con 0x564c0e196c80 2023-06-23T18:34:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.337+0000 7fe32db67b80 1 -- 172.21.15.2:0/3972284753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe31001e040 con 0x564c0e196c80 2023-06-23T18:34:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.337+0000 7fe2faffd700 1 -- 172.21.15.2:0/3972284753 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe30c018c80 con 0x564c0e196c80 2023-06-23T18:34:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.338+0000 7fe2faffd700 1 -- 172.21.15.2:0/3972284753 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30c018e40 con 0x564c0e196c80 2023-06-23T18:34:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.338+0000 7fe32db67b80 10 client.26264 fetch_fsmap learned FSMap version 38 2023-06-23T18:34:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.338+0000 7fe32db67b80 1 -- 172.21.15.2:0/3972284753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x564c0df3c250 con 0x564c0e196c80 2023-06-23T18:34:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.338+0000 7fe2faffd700 1 -- 172.21.15.2:0/3972284753 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe31001e040 con 0x564c0e196c80 2023-06-23T18:34:50.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe2faffd700 1 -- 172.21.15.2:0/3972284753 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 38) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe30c005220 con 0x564c0e196c80 2023-06-23T18:34:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe32db67b80 10 client.26264 fetch_fsmap finished waiting for FSMap version 38 2023-06-23T18:34:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe32db67b80 10 client.26264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:34:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe32db67b80 20 client.26264 populate_metadata read hostname 'smithi002' 2023-06-23T18:34:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe32db67b80 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564c0e201b40 0x564c0e203f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:34:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe32db67b80 4 client.26264 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:34:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe32db67b80 1 -- 172.21.15.2:0/3972284753 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c0e0a3050 con 0x564c0e201b40 2023-06-23T18:34:50.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe31a7fc700 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564c0e201b40 0x564c0e203f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:34:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe31a7fc700 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564c0e201b40 0x564c0e203f50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bca48b70bb1153e server_cookie=6bc4541a9dcb95f2 in_seq=0 out_seq=0 2023-06-23T18:34:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe2faffd700 10 client.26264 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe2faffd700 1 -- 172.21.15.2:0/3972284753 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c0e0a3050 con 0x564c0e201b40 2023-06-23T18:34:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe2faffd700 10 client.26264 handle_command_reply: tid=0 2023-06-23T18:34:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.339+0000 7fe32db67b80 1 client.26264 shutdown 2023-06-23T18:34:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 -- 172.21.15.2:0/3972284753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 msgr2=0x564c0e1a77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a77e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe30c025c60 tx=0x7fe30c0187e0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 -- 172.21.15.2:0/3972284753 shutdown_connections 2023-06-23T18:34:50.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564c0e201b40 0x564c0e203f50 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564c0e198000 0x564c0e1a47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564c0e196c80 0x564c0e1a77e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 --2- 172.21.15.2:0/3972284753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564c0e1975d0 0x564c0e1a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe2faffd700 0 client.26264 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:34:50.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 -- 172.21.15.2:0/3972284753 >> 172.21.15.2:0/3972284753 conn(0x564c0e18c3f0 msgr2=0x564c0e19af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 -- 172.21.15.2:0/3972284753 shutdown_connections 2023-06-23T18:34:50.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 1 -- 172.21.15.2:0/3972284753 wait complete. 2023-06-23T18:34:50.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.340+0000 7fe32db67b80 20 client.26264 trim_cache size 0 max 16384 2023-06-23T18:34:50.352 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:34:50.352 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:34:50.352 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:34:50.352 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:34:50.353 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:34:50.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:34:50.353 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:34:50.353 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:34:50.353 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:34:50.353 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:34:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 -- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3040a1150 msgr2=0x7fe3040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3040a1150 0x7fe3040a3600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00ddd0 tx=0x7fe30c00f040 comp rx=0 tx=0).stop 2023-06-23T18:34:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 -- 172.21.15.2:0/852947485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe31c101790 msgr2=0x7fe31c1a3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:34:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe31c101790 0x7fe31c1a3360 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe31400ef30 tx=0x7fe31400c3a0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 -- 172.21.15.2:0/852947485 shutdown_connections 2023-06-23T18:34:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe31c101790 0x7fe31c1a3360 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe3040a1150 0x7fe3040a3600 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c0ffb50 0x7fe31c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 --2- 172.21.15.2:0/852947485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe31c100de0 0x7fe31c19c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:34:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 -- 172.21.15.2:0/852947485 >> 172.21.15.2:0/852947485 conn(0x7fe31c0fb150 msgr2=0x7fe31c104310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:34:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 -- 172.21.15.2:0/852947485 shutdown_connections 2023-06-23T18:34:50.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:34:50.341+0000 7fe3219e2700 1 -- 172.21.15.2:0/852947485 wait complete. 2023-06-23T18:34:50.363 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:34:50.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:50 smithi002 ceph-mon[95947]: pgmap v680: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2023-06-23T18:34:50.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:50 smithi082 ceph-mon[97889]: pgmap v680: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2023-06-23T18:34:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:50 smithi017 ceph-mon[97472]: pgmap v680: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 15 MiB/s wr, 1.27k op/s 2023-06-23T18:34:51.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:51 smithi082 ceph-mon[97889]: scrub queued for path: / 2023-06-23T18:34:51.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:51 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:51.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:51 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:34:51.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:51 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:34:51.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:51 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:34:51.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:51 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:51.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:51 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:51.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:51 smithi017 ceph-mon[97472]: scrub queued for path: / 2023-06-23T18:34:51.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:51 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:51.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:51 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:34:51.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:51 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:34:51.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:51 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:34:51.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:51 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:51.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:51 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:51 smithi002 ceph-mon[95947]: scrub queued for path: / 2023-06-23T18:34:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:51 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:51 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:34:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:51 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:34:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:51 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:34:51.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:51 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:51.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:51 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:34:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 60.0 s, 411.0 tps, lat 243.639 ms stddev 50.285 2023-06-23T18:34:52.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:52 smithi082 ceph-mon[97889]: pgmap v681: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-06-23T18:34:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:52 smithi017 ceph-mon[97472]: pgmap v681: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-06-23T18:34:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:52 smithi002 ceph-mon[95947]: pgmap v681: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2023-06-23T18:34:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:54 smithi082 ceph-mon[97889]: pgmap v682: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:34:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:54 smithi017 ceph-mon[97472]: pgmap v682: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:34:54.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:54 smithi002 ceph-mon[95947]: pgmap v682: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:34:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:56 smithi082 ceph-mon[97889]: pgmap v683: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-06-23T18:34:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:56 smithi017 ceph-mon[97472]: pgmap v683: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-06-23T18:34:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:56 smithi002 ceph-mon[95947]: pgmap v683: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-06-23T18:34:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 65.0 s, 421.0 tps, lat 236.831 ms stddev 29.379 2023-06-23T18:34:58.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:34:58 smithi082 ceph-mon[97889]: pgmap v684: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-06-23T18:34:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:34:58 smithi017 ceph-mon[97472]: pgmap v684: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-06-23T18:34:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:34:58 smithi002 ceph-mon[95947]: pgmap v684: 97 pgs: 97 active+clean; 8.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 13 MiB/s wr, 1.00k op/s 2023-06-23T18:35:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:00 smithi082 ceph-mon[97889]: pgmap v685: 97 pgs: 97 active+clean; 8.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:35:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:00 smithi017 ceph-mon[97472]: pgmap v685: 97 pgs: 97 active+clean; 8.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:35:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:00 smithi002 ceph-mon[95947]: pgmap v685: 97 pgs: 97 active+clean; 8.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:35:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 70.0 s, 386.4 tps, lat 259.269 ms stddev 70.862 2023-06-23T18:35:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:02 smithi082 ceph-mon[97889]: pgmap v686: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-06-23T18:35:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:02 smithi017 ceph-mon[97472]: pgmap v686: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-06-23T18:35:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:02 smithi002 ceph-mon[95947]: pgmap v686: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-06-23T18:35:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:04 smithi082 ceph-mon[97889]: pgmap v687: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-06-23T18:35:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:04 smithi017 ceph-mon[97472]: pgmap v687: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-06-23T18:35:04.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:04 smithi002 ceph-mon[95947]: pgmap v687: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-06-23T18:35:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:06 smithi017 ceph-mon[97472]: pgmap v688: 97 pgs: 97 active+clean; 8.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:35:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:06 smithi002 ceph-mon[95947]: pgmap v688: 97 pgs: 97 active+clean; 8.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:35:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:06 smithi082 ceph-mon[97889]: pgmap v688: 97 pgs: 97 active+clean; 8.3 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:35:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 75.0 s, 414.8 tps, lat 239.910 ms stddev 33.818 2023-06-23T18:35:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:08 smithi017 ceph-mon[97472]: pgmap v689: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:08 smithi002 ceph-mon[95947]: pgmap v689: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:08 smithi082 ceph-mon[97889]: pgmap v689: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:10 smithi017 ceph-mon[97472]: pgmap v690: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-06-23T18:35:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:10 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:35:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:10 smithi002 ceph-mon[95947]: pgmap v690: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-06-23T18:35:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:10 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:35:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:10 smithi082 ceph-mon[97889]: pgmap v690: 97 pgs: 97 active+clean; 8.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-06-23T18:35:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:10 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:35:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 80.0 s, 359.0 tps, lat 278.708 ms stddev 160.458 2023-06-23T18:35:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:12 smithi017 ceph-mon[97472]: pgmap v691: 97 pgs: 97 active+clean; 8.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:35:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:12 smithi002 ceph-mon[95947]: pgmap v691: 97 pgs: 97 active+clean; 8.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:35:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:12 smithi082 ceph-mon[97889]: pgmap v691: 97 pgs: 97 active+clean; 8.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:35:14.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:14 smithi082 ceph-mon[97889]: pgmap v692: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-06-23T18:35:14.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:14 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:14.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:14 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:14.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:14 smithi017 ceph-mon[97472]: pgmap v692: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-06-23T18:35:14.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:14 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:14.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:14 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:14.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:14 smithi002 ceph-mon[95947]: pgmap v692: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 15 MiB/s wr, 1.13k op/s 2023-06-23T18:35:14.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:14.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:14 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:15.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:35:15.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:35:15.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:15.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:35:15.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:35:15.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:15.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:35:15.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:35:15.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:35:16.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:16 smithi002 ceph-mon[95947]: pgmap v693: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-06-23T18:35:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:16 smithi082 ceph-mon[97889]: pgmap v693: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-06-23T18:35:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:16 smithi017 ceph-mon[97472]: pgmap v693: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 14 MiB/s wr, 1.08k op/s 2023-06-23T18:35:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 85.0 s, 369.8 tps, lat 121.045 ms stddev 55.107 2023-06-23T18:35:18.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:18 smithi002 ceph-mon[95947]: pgmap v694: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-06-23T18:35:18.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:18 smithi082 ceph-mon[97889]: pgmap v694: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-06-23T18:35:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:18 smithi017 ceph-mon[97472]: pgmap v694: 97 pgs: 97 active+clean; 8.4 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-06-23T18:35:20.364 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:35:20.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:20 smithi002 ceph-mon[95947]: pgmap v695: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-06-23T18:35:20.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:20 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:35:20.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 -- 172.21.15.2:0/3918783576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0ffc70 msgr2=0x7f833c1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 --2- 172.21.15.2:0/3918783576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0ffc70 0x7f833c1000c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f832c009990 tx=0x7f832c009ca0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 -- 172.21.15.2:0/3918783576 shutdown_connections 2023-06-23T18:35:20.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 --2- 172.21.15.2:0/3918783576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f833c100600 0x7f833c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 --2- 172.21.15.2:0/3918783576 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c0fea80 0x7f833c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 --2- 172.21.15.2:0/3918783576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0ffc70 0x7f833c1000c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 -- 172.21.15.2:0/3918783576 >> 172.21.15.2:0/3918783576 conn(0x7f833c0fa0e0 msgr2=0x7f833c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:20.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 -- 172.21.15.2:0/3918783576 shutdown_connections 2023-06-23T18:35:20.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.591+0000 7f83447c5700 1 -- 172.21.15.2:0/3918783576 wait complete. 2023-06-23T18:35:20.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f83447c5700 1 Processor -- start 2023-06-23T18:35:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f83447c5700 1 -- start start 2023-06-23T18:35:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f83447c5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0fea80 0x7f833c1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f83447c5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c0ffc70 0x7f833c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f83447c5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f833c100600 0x7f833c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f83447c5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c19f530 con 0x7f833c0ffc70 2023-06-23T18:35:20.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f83447c5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c19f670 con 0x7f833c100600 2023-06-23T18:35:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f83447c5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c19f800 con 0x7f833c0fea80 2023-06-23T18:35:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f8342561700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0fea80 0x7f833c1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f8342561700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0fea80 0x7f833c1979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57690/0 (socket says 172.21.15.2:57690) 2023-06-23T18:35:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f8342561700 1 -- 172.21.15.2:0/2584281338 learned_addr learned my addr 172.21.15.2:0/2584281338 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f8342d62700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f833c100600 0x7f833c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.592+0000 7f8341d60700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c0ffc70 0x7f833c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f8342561700 1 -- 172.21.15.2:0/2584281338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f833c100600 msgr2=0x7f833c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f8342561700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f833c100600 0x7f833c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f8342561700 1 -- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c0ffc70 msgr2=0x7f833c197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f8342561700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c0ffc70 0x7f833c197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f8342561700 1 -- 172.21.15.2:0/2584281338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c0095f0 con 0x7f833c0fea80 2023-06-23T18:35:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f8341d60700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c0ffc70 0x7f833c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f8342561700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0fea80 0x7f833c1979b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f833800b3c0 tx=0x7f8338009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f83337fe700 1 -- 172.21.15.2:0/2584281338 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338018070 con 0x7f833c0fea80 2023-06-23T18:35:20.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f83447c5700 1 -- 172.21.15.2:0/2584281338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c19fae0 con 0x7f833c0fea80 2023-06-23T18:35:20.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f83337fe700 1 -- 172.21.15.2:0/2584281338 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f83380044b0 con 0x7f833c0fea80 2023-06-23T18:35:20.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f83337fe700 1 -- 172.21.15.2:0/2584281338 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338010040 con 0x7f833c0fea80 2023-06-23T18:35:20.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.593+0000 7f83447c5700 1 -- 172.21.15.2:0/2584281338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c1a0040 con 0x7f833c0fea80 2023-06-23T18:35:20.601 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:20 smithi082 ceph-mon[97889]: pgmap v695: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-06-23T18:35:20.602 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:20 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:35:20.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.596+0000 7f83337fe700 1 -- 172.21.15.2:0/2584281338 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8338003eb0 con 0x7f833c0fea80 2023-06-23T18:35:20.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.597+0000 7f83337fe700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f83280a1150 0x7f83280a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.597+0000 7f83337fe700 1 -- 172.21.15.2:0/2584281338 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f83380cc010 con 0x7f833c0fea80 2023-06-23T18:35:20.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.597+0000 7f8341d60700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f83280a1150 0x7f83280a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.597+0000 7f8341d60700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f83280a1150 0x7f83280a3600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b2e0 tx=0x7f832c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:20.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f835094ab80 1 Processor -- start 2023-06-23T18:35:20.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f835094ab80 1 -- start start 2023-06-23T18:35:20.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f835094ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f835094ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b17a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f835094ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b18470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f835094ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866a1bcb0 con 0x560866b16c80 2023-06-23T18:35:20.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f835094ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866a6ea40 con 0x560866b18000 2023-06-23T18:35:20.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f835094ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866a67ed0 con 0x560866b175d0 2023-06-23T18:35:20.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f8342561700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f8342561700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b17090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57360/0 (socket says 172.21.15.2:57360) 2023-06-23T18:35:20.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f8342561700 1 -- 172.21.15.2:0/1251388170 learned_addr learned my addr 172.21.15.2:0/1251388170 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:20.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f8342d62700 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b18470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.598+0000 7f8341d60700 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b17a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8342561700 1 -- 172.21.15.2:0/1251388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 msgr2=0x560866b17a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:20.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8342561700 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b17a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8342561700 1 -- 172.21.15.2:0/1251388170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 msgr2=0x560866b18470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:20.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8342561700 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b18470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8342561700 1 -- 172.21.15.2:0/1251388170 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608668e9630 con 0x560866b16c80 2023-06-23T18:35:20.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8341d60700 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b17a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:20.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8342d62700 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b18470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:35:20.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8342561700 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b17090 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8338095e90 tx=0x7f83380957e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57be54614233a949 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:20.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8330ff9700 1 -- 172.21.15.2:0/1251388170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338098f60 con 0x560866b16c80 2023-06-23T18:35:20.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8330ff9700 1 -- 172.21.15.2:0/1251388170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8338098d80 con 0x560866b16c80 2023-06-23T18:35:20.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.599+0000 7f8330ff9700 1 -- 172.21.15.2:0/1251388170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83380977c0 con 0x560866b16c80 2023-06-23T18:35:20.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.600+0000 7f835094ab80 1 -- 172.21.15.2:0/1251388170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 msgr2=0x560866b17090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.600+0000 7f835094ab80 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b17090 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8338095e90 tx=0x7f83380957e0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.600+0000 7f835094ab80 1 -- 172.21.15.2:0/1251388170 shutdown_connections 2023-06-23T18:35:20.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.600+0000 7f835094ab80 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b18470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.600+0000 7f835094ab80 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b17090 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.600+0000 7f835094ab80 1 --2- 172.21.15.2:0/1251388170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b17a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.600+0000 7f835094ab80 1 -- 172.21.15.2:0/1251388170 >> 172.21.15.2:0/1251388170 conn(0x560866b0c3f0 msgr2=0x560866b11520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:20.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.600+0000 7f835094ab80 1 -- 172.21.15.2:0/1251388170 shutdown_connections 2023-06-23T18:35:20.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.601+0000 7f835094ab80 1 -- 172.21.15.2:0/1251388170 wait complete. 2023-06-23T18:35:20.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.601+0000 7f835094ab80 1 -- start start 2023-06-23T18:35:20.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.601+0000 7f835094ab80 1 Processor -- start 2023-06-23T18:35:20.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.601+0000 7f835094ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b16c80 0x560866b205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f835094ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b175d0 0x560866b20b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f835094ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b27b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f835094ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866a07ac0 con 0x560866b18000 2023-06-23T18:35:20.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f835094ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866a077c0 con 0x560866b175d0 2023-06-23T18:35:20.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f835094ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608668eb920 con 0x560866b16c80 2023-06-23T18:35:20.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f835094ab80 10 client.0 _ll_register_callbacks cb 0x560866a0d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:35:20.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b27b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8341d60700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b175d0 0x560866b20b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b27b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57368/0 (socket says 172.21.15.2:57368) 2023-06-23T18:35:20.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 -- 172.21.15.2:0/3973426890 learned_addr learned my addr 172.21.15.2:0/3973426890 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:20.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342561700 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b16c80 0x560866b205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 -- 172.21.15.2:0/3973426890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b16c80 msgr2=0x560866b205f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:20.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b16c80 0x560866b205f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 -- 172.21.15.2:0/3973426890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b175d0 msgr2=0x560866b20b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:20.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b175d0 0x560866b20b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 -- 172.21.15.2:0/3973426890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c1077f0 con 0x560866b18000 2023-06-23T18:35:20.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8341d60700 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b175d0 0x560866b20b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:20.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8342d62700 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b27b70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8334007c60 tx=0x7f8334007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92834cef0a54ebea server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:20.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.602+0000 7f8325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:35:20.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.603+0000 7f8325ffb700 1 -- 172.21.15.2:0/3973426890 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400e040 con 0x560866b18000 2023-06-23T18:35:20.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.603+0000 7f8325ffb700 1 -- 172.21.15.2:0/3973426890 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8334013970 con 0x560866b18000 2023-06-23T18:35:20.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.603+0000 7f835094ab80 1 -- 172.21.15.2:0/3973426890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560866b09bb0 con 0x560866b18000 2023-06-23T18:35:20.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.603+0000 7f8325ffb700 1 -- 172.21.15.2:0/3973426890 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8334013b30 con 0x560866b18000 2023-06-23T18:35:20.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f835094ab80 10 client.16770 fetch_fsmap learned FSMap version 39 2023-06-23T18:35:20.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f835094ab80 1 -- 172.21.15.2:0/3973426890 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f833c1a0040 con 0x560866b18000 2023-06-23T18:35:20.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f8325ffb700 1 -- 172.21.15.2:0/3973426890 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560866b09bb0 con 0x560866b18000 2023-06-23T18:35:20.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f8325ffb700 1 -- 172.21.15.2:0/3973426890 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 39) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f833401e3b0 con 0x560866b18000 2023-06-23T18:35:20.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f835094ab80 10 client.16770 fetch_fsmap finished waiting for FSMap version 39 2023-06-23T18:35:20.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f835094ab80 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:35:20.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f835094ab80 20 client.16770 populate_metadata read hostname 'smithi002' 2023-06-23T18:35:20.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f835094ab80 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560866b2c420 0x560866b2e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f835094ab80 4 client.16770 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:35:20.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.604+0000 7f835094ab80 1 -- 172.21.15.2:0/3973426890 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560866a60eb0 con 0x560866b2c420 2023-06-23T18:35:20.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.605+0000 7f8342561700 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560866b2c420 0x560866b2e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.605+0000 7f8342561700 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560866b2c420 0x560866b2e830 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9019fe1274894834 server_cookie=9417abdd6bcc69fa in_seq=0 out_seq=0 2023-06-23T18:35:20.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.605+0000 7f8325ffb700 10 client.16770 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:35:20.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.606+0000 7f8325ffb700 1 -- 172.21.15.2:0/3973426890 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560866a60eb0 con 0x560866b2c420 2023-06-23T18:35:20.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.606+0000 7f8325ffb700 10 client.16770 handle_command_reply: tid=0 2023-06-23T18:35:20.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.606+0000 7f835094ab80 1 client.16770 shutdown 2023-06-23T18:35:20.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 -- 172.21.15.2:0/3973426890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 msgr2=0x560866b27b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b27b70 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8334007c60 tx=0x7f8334007c90 comp rx=0 tx=0).stop 2023-06-23T18:35:20.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 -- 172.21.15.2:0/3973426890 shutdown_connections 2023-06-23T18:35:20.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560866b2c420 0x560866b2e830 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b27b70 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b16c80 0x560866b205f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 --2- 172.21.15.2:0/3973426890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b175d0 0x560866b20b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f8325ffb700 0 client.16770 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:35:20.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 -- 172.21.15.2:0/3973426890 >> 172.21.15.2:0/3973426890 conn(0x560866b0c3f0 msgr2=0x560866b10c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:20.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 -- 172.21.15.2:0/3973426890 shutdown_connections 2023-06-23T18:35:20.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 1 -- 172.21.15.2:0/3973426890 wait complete. 2023-06-23T18:35:20.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.607+0000 7f835094ab80 20 client.16770 trim_cache size 0 max 16384 2023-06-23T18:35:20.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f835094ab80 1 Processor -- start 2023-06-23T18:35:20.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f835094ab80 1 -- start start 2023-06-23T18:35:20.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f835094ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b1af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f835094ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b1b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f835094ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b27a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f835094ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866b0e470 con 0x560866b16c80 2023-06-23T18:35:20.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f835094ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866b29860 con 0x560866b18000 2023-06-23T18:35:20.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f835094ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866b12810 con 0x560866b175d0 2023-06-23T18:35:20.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f8342d62700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b27a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f8342d62700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b27a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47966/0 (socket says 172.21.15.2:47966) 2023-06-23T18:35:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.635+0000 7f8342d62700 1 -- 172.21.15.2:0/2512647964 learned_addr learned my addr 172.21.15.2:0/2512647964 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.636+0000 7f8342d62700 1 -- 172.21.15.2:0/2512647964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 msgr2=0x560866b1b440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:35:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.636+0000 7f8342d62700 1 --2- 172.21.15.2:0/2512647964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b1b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.636+0000 7f8342d62700 1 -- 172.21.15.2:0/2512647964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 msgr2=0x560866b1af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.636+0000 7f8342d62700 1 --2- 172.21.15.2:0/2512647964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b1af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.636+0000 7f8342d62700 1 -- 172.21.15.2:0/2512647964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560866895de0 con 0x560866b18000 2023-06-23T18:35:20.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.636+0000 7f8342d62700 1 --2- 172.21.15.2:0/2512647964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b27a90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8334002230 tx=0x7f8334013330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5dc11f02f07386 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:20.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.636+0000 7f83317fa700 1 -- 172.21.15.2:0/2512647964 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400e040 con 0x560866b18000 2023-06-23T18:35:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.637+0000 7f83317fa700 1 -- 172.21.15.2:0/2512647964 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f833400f040 con 0x560866b18000 2023-06-23T18:35:20.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.637+0000 7f83317fa700 1 -- 172.21.15.2:0/2512647964 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833401eea0 con 0x560866b18000 2023-06-23T18:35:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.637+0000 7f835094ab80 1 -- 172.21.15.2:0/2512647964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 msgr2=0x560866b27a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 --2- 172.21.15.2:0/2512647964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b27a90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8334002230 tx=0x7f8334013330 comp rx=0 tx=0).stop 2023-06-23T18:35:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 -- 172.21.15.2:0/2512647964 shutdown_connections 2023-06-23T18:35:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 --2- 172.21.15.2:0/2512647964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b18000 0x560866b27a90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 --2- 172.21.15.2:0/2512647964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b16c80 0x560866b1af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 --2- 172.21.15.2:0/2512647964 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b1b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 -- 172.21.15.2:0/2512647964 >> 172.21.15.2:0/2512647964 conn(0x560866b0c3f0 msgr2=0x560866b1f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 -- 172.21.15.2:0/2512647964 shutdown_connections 2023-06-23T18:35:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 -- 172.21.15.2:0/2512647964 wait complete. 2023-06-23T18:35:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 -- start start 2023-06-23T18:35:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.638+0000 7f835094ab80 1 Processor -- start 2023-06-23T18:35:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f835094ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b16c80 0x560866b27310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f835094ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b27850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f835094ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b23db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f835094ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866b24390 con 0x560866b18000 2023-06-23T18:35:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f835094ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866b24500 con 0x560866b16c80 2023-06-23T18:35:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f835094ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560866b246c0 con 0x560866b175d0 2023-06-23T18:35:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f835094ab80 10 client.0 _ll_register_callbacks cb 0x560866a0d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:35:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342561700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b16c80 0x560866b27310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342561700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b16c80 0x560866b27310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47972/0 (socket says 172.21.15.2:47972) 2023-06-23T18:35:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342561700 1 -- 172.21.15.2:0/239824528 learned_addr learned my addr 172.21.15.2:0/239824528 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:20.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342d62700 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b23db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8341d60700 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b27850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342561700 1 -- 172.21.15.2:0/239824528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 msgr2=0x560866b27850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342561700 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b27850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342561700 1 -- 172.21.15.2:0/239824528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 msgr2=0x560866b23db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342561700 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b23db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.639+0000 7f8342561700 1 -- 172.21.15.2:0/239824528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c1a0040 con 0x560866b16c80 2023-06-23T18:35:20.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.640+0000 7f8342561700 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b16c80 0x560866b27310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8338095920 tx=0x7f83380a1890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32bea448b92d8f74 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.640+0000 7f8325ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:35:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.640+0000 7f8325ffb700 1 -- 172.21.15.2:0/239824528 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8338003810 con 0x560866b16c80 2023-06-23T18:35:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.640+0000 7f8325ffb700 1 -- 172.21.15.2:0/239824528 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f833809d450 con 0x560866b16c80 2023-06-23T18:35:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.640+0000 7f835094ab80 1 -- 172.21.15.2:0/239824528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8338098d80 con 0x560866b16c80 2023-06-23T18:35:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.640+0000 7f8325ffb700 1 -- 172.21.15.2:0/239824528 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8338099420 con 0x560866b16c80 2023-06-23T18:35:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.640+0000 7f835094ab80 10 client.26282 fetch_fsmap learned FSMap version 39 2023-06-23T18:35:20.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.640+0000 7f835094ab80 1 -- 172.21.15.2:0/239824528 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x56086687ec60 con 0x560866b16c80 2023-06-23T18:35:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f8325ffb700 1 -- 172.21.15.2:0/239824528 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8338098d80 con 0x560866b16c80 2023-06-23T18:35:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f8325ffb700 1 -- 172.21.15.2:0/239824528 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 39) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8338098910 con 0x560866b16c80 2023-06-23T18:35:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f835094ab80 10 client.26282 fetch_fsmap finished waiting for FSMap version 39 2023-06-23T18:35:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f835094ab80 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:35:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f835094ab80 20 client.26282 populate_metadata read hostname 'smithi002' 2023-06-23T18:35:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f835094ab80 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560866b815e0 0x560866b839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:20.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f835094ab80 4 client.26282 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:35:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f835094ab80 1 -- 172.21.15.2:0/239824528 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560866a23050 con 0x560866b815e0 2023-06-23T18:35:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.641+0000 7f8341d60700 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560866b815e0 0x560866b839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.643+0000 7f8325ffb700 10 client.26282 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:35:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.643+0000 7f8341d60700 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560866b815e0 0x560866b839f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa3bfd2e67e8b6c0 server_cookie=5d49207262bf5ee2 in_seq=0 out_seq=0 2023-06-23T18:35:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f8325ffb700 1 -- 172.21.15.2:0/239824528 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560866a23050 con 0x560866b815e0 2023-06-23T18:35:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f8325ffb700 10 client.26282 handle_command_reply: tid=0 2023-06-23T18:35:20.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 client.26282 shutdown 2023-06-23T18:35:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 -- 172.21.15.2:0/239824528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b16c80 msgr2=0x560866b27310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b16c80 0x560866b27310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8338095920 tx=0x7f83380a1890 comp rx=0 tx=0).stop 2023-06-23T18:35:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 -- 172.21.15.2:0/239824528 shutdown_connections 2023-06-23T18:35:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560866b815e0 0x560866b839f0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560866b18000 0x560866b23db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f8325ffb700 0 client.26282 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:35:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560866b16c80 0x560866b27310 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 --2- 172.21.15.2:0/239824528 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560866b175d0 0x560866b27850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 -- 172.21.15.2:0/239824528 >> 172.21.15.2:0/239824528 conn(0x560866b0c3f0 msgr2=0x560866b1af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 -- 172.21.15.2:0/239824528 shutdown_connections 2023-06-23T18:35:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.644+0000 7f835094ab80 1 -- 172.21.15.2:0/239824528 wait complete. 2023-06-23T18:35:20.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.645+0000 7f835094ab80 20 client.26282 trim_cache size 0 max 16384 2023-06-23T18:35:20.654 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:35:20.655 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:35:20.655 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:35:20.655 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:35:20.655 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:35:20.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:35:20.655 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:35:20.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:35:20.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:35:20.656 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:35:20.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.645+0000 7f83447c5700 1 -- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f83280a1150 msgr2=0x7f83280a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.645+0000 7f83447c5700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f83280a1150 0x7f83280a3600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f832c00b2e0 tx=0x7f832c005fd0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.645+0000 7f83447c5700 1 -- 172.21.15.2:0/2584281338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0fea80 msgr2=0x7f833c1979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:20.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.645+0000 7f83447c5700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0fea80 0x7f833c1979b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f833800b3c0 tx=0x7f8338009ab0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.646+0000 7f83447c5700 1 -- 172.21.15.2:0/2584281338 shutdown_connections 2023-06-23T18:35:20.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.646+0000 7f83447c5700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f83280a1150 0x7f83280a3600 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.646+0000 7f83447c5700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f833c100600 0x7f833c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.646+0000 7f83447c5700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f833c0fea80 0x7f833c1979b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.646+0000 7f83447c5700 1 --2- 172.21.15.2:0/2584281338 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f833c0ffc70 0x7f833c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:20.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.646+0000 7f83447c5700 1 -- 172.21.15.2:0/2584281338 >> 172.21.15.2:0/2584281338 conn(0x7f833c0fa0e0 msgr2=0x7f833c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:20.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.646+0000 7f83447c5700 1 -- 172.21.15.2:0/2584281338 shutdown_connections 2023-06-23T18:35:20.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:20.646+0000 7f83447c5700 1 -- 172.21.15.2:0/2584281338 wait complete. 2023-06-23T18:35:20.669 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:35:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:20 smithi017 ceph-mon[97472]: pgmap v695: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 15 MiB/s wr, 1.12k op/s 2023-06-23T18:35:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:20 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:35:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 90.0 s, 392.6 tps, lat 99.138 ms stddev 16.441 2023-06-23T18:35:22.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:22 smithi002 ceph-mon[95947]: pgmap v696: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-06-23T18:35:22.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:22 smithi082 ceph-mon[97889]: pgmap v696: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-06-23T18:35:22.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:22 smithi017 ceph-mon[97472]: pgmap v696: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2023-06-23T18:35:24.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:24 smithi002 ceph-mon[95947]: pgmap v697: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-06-23T18:35:24.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:24 smithi082 ceph-mon[97889]: pgmap v697: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-06-23T18:35:24.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:24 smithi017 ceph-mon[97472]: pgmap v697: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 14 MiB/s wr, 1.07k op/s 2023-06-23T18:35:26.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:26 smithi002 ceph-mon[95947]: pgmap v698: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:35:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:26 smithi082 ceph-mon[97889]: pgmap v698: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:35:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:26 smithi017 ceph-mon[97472]: pgmap v698: 97 pgs: 97 active+clean; 8.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:35:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 95.0 s, 392.0 tps, lat 99.500 ms stddev 17.747 2023-06-23T18:35:28.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:28 smithi002 ceph-mon[95947]: pgmap v699: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:35:28.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:28 smithi082 ceph-mon[97889]: pgmap v699: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:35:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:28 smithi017 ceph-mon[97472]: pgmap v699: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:35:30.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:30 smithi002 ceph-mon[95947]: pgmap v700: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:35:30.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:30 smithi082 ceph-mon[97889]: pgmap v700: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:35:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:30 smithi017 ceph-mon[97472]: pgmap v700: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:35:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 100.0 s, 355.0 tps, lat 109.612 ms stddev 22.200 2023-06-23T18:35:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:32 smithi002 ceph-mon[95947]: pgmap v701: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2023-06-23T18:35:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:32 smithi082 ceph-mon[97889]: pgmap v701: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2023-06-23T18:35:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:32 smithi017 ceph-mon[97472]: pgmap v701: 97 pgs: 97 active+clean; 8.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 616 KiB/s rd, 14 MiB/s wr, 1.18k op/s 2023-06-23T18:35:34.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:34 smithi002 ceph-mon[95947]: pgmap v702: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-06-23T18:35:34.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:34 smithi082 ceph-mon[97889]: pgmap v702: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-06-23T18:35:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:34 smithi017 ceph-mon[97472]: pgmap v702: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-06-23T18:35:35.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:35 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:35:35.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:35 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:35:35.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:35 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:35:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:36 smithi002 ceph-mon[95947]: pgmap v703: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:35:36.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:36 smithi082 ceph-mon[97889]: pgmap v703: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:35:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:36 smithi017 ceph-mon[97472]: pgmap v703: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:35:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 105.0 s, 382.2 tps, lat 102.601 ms stddev 56.205 2023-06-23T18:35:38.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:38 smithi002 ceph-mon[95947]: pgmap v704: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:35:38.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:38 smithi082 ceph-mon[97889]: pgmap v704: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:35:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:38 smithi017 ceph-mon[97472]: pgmap v704: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:35:40.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:40 smithi002 ceph-mon[95947]: pgmap v705: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-06-23T18:35:40.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:40 smithi082 ceph-mon[97889]: pgmap v705: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-06-23T18:35:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:40 smithi017 ceph-mon[97472]: pgmap v705: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-06-23T18:35:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 110.0 s, 356.0 tps, lat 109.165 ms stddev 21.233 2023-06-23T18:35:42.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:42 smithi002 ceph-mon[95947]: pgmap v706: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:35:42.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:42 smithi082 ceph-mon[97889]: pgmap v706: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:35:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:42 smithi017 ceph-mon[97472]: pgmap v706: 97 pgs: 97 active+clean; 8.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:35:44.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:44 smithi002 ceph-mon[95947]: pgmap v707: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:44.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:44 smithi082 ceph-mon[97889]: pgmap v707: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:44 smithi017 ceph-mon[97472]: pgmap v707: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:46.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:46 smithi002 ceph-mon[95947]: pgmap v708: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:35:46.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:46 smithi082 ceph-mon[97889]: pgmap v708: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:35:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:46 smithi017 ceph-mon[97472]: pgmap v708: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:35:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 115.0 s, 338.8 tps, lat 114.610 ms stddev 27.053 2023-06-23T18:35:48.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:48 smithi002 ceph-mon[95947]: pgmap v709: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:48.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:48 smithi082 ceph-mon[97889]: pgmap v709: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:48 smithi017 ceph-mon[97472]: pgmap v709: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 13 MiB/s wr, 1.11k op/s 2023-06-23T18:35:50.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:50 smithi002 ceph-mon[95947]: pgmap v710: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:35:50.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:50 smithi082 ceph-mon[97889]: pgmap v710: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:35:50.670 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:35:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:50 smithi017 ceph-mon[97472]: pgmap v710: 97 pgs: 97 active+clean; 8.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2023-06-23T18:35:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.866+0000 7f5340dc0700 1 -- 172.21.15.2:0/600297054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0fea80 msgr2=0x7f533c0fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.866+0000 7f5340dc0700 1 --2- 172.21.15.2:0/600297054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0fea80 0x7f533c0fee90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f532c0098f0 tx=0x7f532c009c00 comp rx=0 tx=0).stop 2023-06-23T18:35:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.868+0000 7f5340dc0700 1 -- 172.21.15.2:0/600297054 shutdown_connections 2023-06-23T18:35:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.868+0000 7f5340dc0700 1 --2- 172.21.15.2:0/600297054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533c100600 0x7f533c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.868+0000 7f5340dc0700 1 --2- 172.21.15.2:0/600297054 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0fea80 0x7f533c0fee90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.868+0000 7f5340dc0700 1 --2- 172.21.15.2:0/600297054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f533c0ffc70 0x7f533c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.868+0000 7f5340dc0700 1 -- 172.21.15.2:0/600297054 >> 172.21.15.2:0/600297054 conn(0x7f533c0fa0e0 msgr2=0x7f533c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.869+0000 7f5340dc0700 1 -- 172.21.15.2:0/600297054 shutdown_connections 2023-06-23T18:35:50.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.869+0000 7f5340dc0700 1 -- 172.21.15.2:0/600297054 wait complete. 2023-06-23T18:35:50.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.870+0000 7f5340dc0700 1 Processor -- start 2023-06-23T18:35:50.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.870+0000 7f5340dc0700 1 -- start start 2023-06-23T18:35:50.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.870+0000 7f5340dc0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533c0fea80 0x7f533c068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.870+0000 7f5340dc0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0ffc70 0x7f533c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.870+0000 7f5340dc0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f533c100600 0x7f533c071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.870+0000 7f5340dc0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1a4ab0 con 0x7f533c0fea80 2023-06-23T18:35:50.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.870+0000 7f5340dc0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c071570 con 0x7f533c0ffc70 2023-06-23T18:35:50.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.870+0000 7f5340dc0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c071730 con 0x7f533c100600 2023-06-23T18:35:50.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0ffc70 0x7f533c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f533c100600 0x7f533c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0ffc70 0x7f533c0709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:47992/0 (socket says 172.21.15.2:47992) 2023-06-23T18:35:50.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 -- 172.21.15.2:0/262793263 learned_addr learned my addr 172.21.15.2:0/262793263 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f533a59c700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533c0fea80 0x7f533c068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 -- 172.21.15.2:0/262793263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f533c100600 msgr2=0x7f533c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f533c100600 0x7f533c071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 -- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533c0fea80 msgr2=0x7f533c068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533c0fea80 0x7f533c068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 -- 172.21.15.2:0/262793263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53300095f0 con 0x7f533c0ffc70 2023-06-23T18:35:50.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f5339d9b700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0ffc70 0x7f533c0709b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f533000ec10 tx=0x7f533000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.871+0000 7f532b7fe700 1 -- 172.21.15.2:0/262793263 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533000cc90 con 0x7f533c0ffc70 2023-06-23T18:35:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.872+0000 7f532b7fe700 1 -- 172.21.15.2:0/262793263 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f533000cdf0 con 0x7f533c0ffc70 2023-06-23T18:35:50.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.872+0000 7f532b7fe700 1 -- 172.21.15.2:0/262793263 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330003e80 con 0x7f533c0ffc70 2023-06-23T18:35:50.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.872+0000 7f5340dc0700 1 -- 172.21.15.2:0/262793263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c009520 con 0x7f533c0ffc70 2023-06-23T18:35:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.872+0000 7f5340dc0700 1 -- 172.21.15.2:0/262793263 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c071d40 con 0x7f533c0ffc70 2023-06-23T18:35:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.876+0000 7f532b7fe700 1 -- 172.21.15.2:0/262793263 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5330004000 con 0x7f533c0ffc70 2023-06-23T18:35:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.877+0000 7f532b7fe700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53240a11d0 0x7f53240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.877+0000 7f533a59c700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53240a11d0 0x7f53240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.877+0000 7f532b7fe700 1 -- 172.21.15.2:0/262793263 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5330014070 con 0x7f533c0ffc70 2023-06-23T18:35:50.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.878+0000 7f533a59c700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53240a11d0 0x7f53240a3680 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f532c0098c0 tx=0x7f532c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.878+0000 7f534cf45b80 1 Processor -- start 2023-06-23T18:35:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f534cf45b80 1 -- start start 2023-06-23T18:35:50.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ecc80 0x55a7f95ed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ed5d0 0x55a7f95eda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f534cf45b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f94f1cb0 con 0x55a7f95ee000 2023-06-23T18:35:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f534cf45b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f9544a40 con 0x55a7f95ed5d0 2023-06-23T18:35:50.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f534cf45b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f953ded0 con 0x55a7f95ecc80 2023-06-23T18:35:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f5339d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ed5d0 0x55a7f95eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f533a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ecc80 0x55a7f95ed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f5339d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ed5d0 0x55a7f95eda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48002/0 (socket says 172.21.15.2:48002) 2023-06-23T18:35:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f5339d9b700 1 -- 172.21.15.2:0/1842886845 learned_addr learned my addr 172.21.15.2:0/1842886845 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f533a59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ecc80 0x55a7f95ed090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57746/0 (socket says 172.21.15.2:57746) 2023-06-23T18:35:50.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.879+0000 7f533ad9d700 1 --2- 172.21.15.2:0/1842886845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f533a59c700 1 -- 172.21.15.2:0/1842886845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ed5d0 msgr2=0x55a7f95eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f533a59c700 1 --2- 172.21.15.2:0/1842886845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ed5d0 0x55a7f95eda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f533a59c700 1 -- 172.21.15.2:0/1842886845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 msgr2=0x55a7f95ee470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f533a59c700 1 --2- 172.21.15.2:0/1842886845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95ee470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f533a59c700 1 -- 172.21.15.2:0/1842886845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7f93bf630 con 0x55a7f95ecc80 2023-06-23T18:35:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f533a59c700 1 --2- 172.21.15.2:0/1842886845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ecc80 0x55a7f95ed090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f532c000c00 tx=0x7f532c01e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df3176be541343ff server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f5328ff9700 1 -- 172.21.15.2:0/1842886845 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00fde0 con 0x55a7f95ecc80 2023-06-23T18:35:50.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f5328ff9700 1 -- 172.21.15.2:0/1842886845 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f532c01d040 con 0x55a7f95ecc80 2023-06-23T18:35:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.880+0000 7f5328ff9700 1 -- 172.21.15.2:0/1842886845 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c005650 con 0x55a7f95ecc80 2023-06-23T18:35:50.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.881+0000 7f534cf45b80 1 -- 172.21.15.2:0/1842886845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ecc80 msgr2=0x55a7f95ed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.881+0000 7f534cf45b80 1 --2- 172.21.15.2:0/1842886845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ecc80 0x55a7f95ed090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f532c000c00 tx=0x7f532c01e000 comp rx=0 tx=0).stop 2023-06-23T18:35:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 -- 172.21.15.2:0/1842886845 shutdown_connections 2023-06-23T18:35:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 --2- 172.21.15.2:0/1842886845 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95ee470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 --2- 172.21.15.2:0/1842886845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ecc80 0x55a7f95ed090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 --2- 172.21.15.2:0/1842886845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ed5d0 0x55a7f95eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 -- 172.21.15.2:0/1842886845 >> 172.21.15.2:0/1842886845 conn(0x55a7f95e23f0 msgr2=0x55a7f95e7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 -- 172.21.15.2:0/1842886845 shutdown_connections 2023-06-23T18:35:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 -- 172.21.15.2:0/1842886845 wait complete. 2023-06-23T18:35:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 -- start start 2023-06-23T18:35:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.882+0000 7f534cf45b80 1 Processor -- start 2023-06-23T18:35:50.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.883+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.883+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.883+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.883+0000 7f534cf45b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f94ddac0 con 0x55a7f95ee000 2023-06-23T18:35:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.883+0000 7f534cf45b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f94dd7c0 con 0x55a7f95ecc80 2023-06-23T18:35:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.883+0000 7f534cf45b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f93c1920 con 0x55a7f95ed5d0 2023-06-23T18:35:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.883+0000 7f534cf45b80 10 client.0 _ll_register_callbacks cb 0x55a7f94e3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:35:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48008/0 (socket says 172.21.15.2:48008) 2023-06-23T18:35:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95fdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57414/0 (socket says 172.21.15.2:57414) 2023-06-23T18:35:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 -- 172.21.15.2:0/3035336904 learned_addr learned my addr 172.21.15.2:0/3035336904 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f5339d9b700 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 -- 172.21.15.2:0/3035336904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ed5d0 msgr2=0x55a7f95f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 -- 172.21.15.2:0/3035336904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 msgr2=0x55a7f95fdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95fdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 -- 172.21.15.2:0/3035336904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1077f0 con 0x55a7f95ecc80 2023-06-23T18:35:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f533a59c700 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f6560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f532c00a010 tx=0x7f532c0055c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593b098300f02afe server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f5321ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:35:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.884+0000 7f5321ffb700 1 -- 172.21.15.2:0/3035336904 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00b910 con 0x55a7f95ecc80 2023-06-23T18:35:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 1 -- 172.21.15.2:0/3035336904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7f95dfbb0 con 0x55a7f95ecc80 2023-06-23T18:35:50.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f5321ffb700 1 -- 172.21.15.2:0/3035336904 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f532c00f510 con 0x55a7f95ecc80 2023-06-23T18:35:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f5321ffb700 1 -- 172.21.15.2:0/3035336904 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f532c00ba70 con 0x55a7f95ecc80 2023-06-23T18:35:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 10 client.26294 fetch_fsmap learned FSMap version 40 2023-06-23T18:35:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 1 -- 172.21.15.2:0/3035336904 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a7f936bde0 con 0x55a7f95ecc80 2023-06-23T18:35:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f5321ffb700 1 -- 172.21.15.2:0/3035336904 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a7f95dfbb0 con 0x55a7f95ecc80 2023-06-23T18:35:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f5321ffb700 1 -- 172.21.15.2:0/3035336904 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f532c025720 con 0x55a7f95ecc80 2023-06-23T18:35:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 10 client.26294 fetch_fsmap finished waiting for FSMap version 40 2023-06-23T18:35:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 10 client.26294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:35:50.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 20 client.26294 populate_metadata read hostname 'smithi002' 2023-06-23T18:35:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a7f9601df0 0x55a7f96041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 4 client.26294 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:35:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.885+0000 7f534cf45b80 1 -- 172.21.15.2:0/3035336904 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7f9536eb0 con 0x55a7f9601df0 2023-06-23T18:35:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.886+0000 7f5339d9b700 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a7f9601df0 0x55a7f96041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.886+0000 7f5321ffb700 10 client.26294 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:35:50.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.886+0000 7f5339d9b700 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a7f9601df0 0x55a7f96041d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6d0812f8a9e287 server_cookie=5e8efd9a811cc5c0 in_seq=0 out_seq=0 2023-06-23T18:35:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f5321ffb700 1 -- 172.21.15.2:0/3035336904 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a7f9536eb0 con 0x55a7f9601df0 2023-06-23T18:35:50.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f5321ffb700 10 client.26294 handle_command_reply: tid=0 2023-06-23T18:35:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 client.26294 shutdown 2023-06-23T18:35:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 -- 172.21.15.2:0/3035336904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 msgr2=0x55a7f95f6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f6560 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f532c00a010 tx=0x7f532c0055c0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 -- 172.21.15.2:0/3035336904 shutdown_connections 2023-06-23T18:35:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a7f9601df0 0x55a7f96041d0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ee000 0x55a7f95fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f5321ffb700 0 client.26294 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:35:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f6560 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3035336904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.887+0000 7f534cf45b80 1 -- 172.21.15.2:0/3035336904 >> 172.21.15.2:0/3035336904 conn(0x55a7f95e23f0 msgr2=0x55a7f95e6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.888+0000 7f534cf45b80 1 -- 172.21.15.2:0/3035336904 shutdown_connections 2023-06-23T18:35:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.888+0000 7f534cf45b80 1 -- 172.21.15.2:0/3035336904 wait complete. 2023-06-23T18:35:50.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.890+0000 7f534cf45b80 20 client.26294 trim_cache size 0 max 16384 2023-06-23T18:35:50.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.921+0000 7f534cf45b80 1 Processor -- start 2023-06-23T18:35:50.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.921+0000 7f534cf45b80 1 -- start start 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.921+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.921+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.921+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f534cf45b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f95e7e20 con 0x55a7f95ed5d0 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f534cf45b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f95f5a10 con 0x55a7f95ecc80 2023-06-23T18:35:50.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f534cf45b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f95dfad0 con 0x55a7f95ee000 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f5339d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f5339d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57422/0 (socket says 172.21.15.2:57422) 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533ad9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95fdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57764/0 (socket says 172.21.15.2:57764) 2023-06-23T18:35:50.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f5339d9b700 1 -- 172.21.15.2:0/3830601258 learned_addr learned my addr 172.21.15.2:0/3830601258 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533a59c700 1 -- 172.21.15.2:0/3830601258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 msgr2=0x55a7f95fdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533a59c700 1 --2- 172.21.15.2:0/3830601258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95fdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533a59c700 1 -- 172.21.15.2:0/3830601258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 msgr2=0x55a7f95f1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533a59c700 1 --2- 172.21.15.2:0/3830601258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533a59c700 1 -- 172.21.15.2:0/3830601258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7f9354c60 con 0x55a7f95ecc80 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f5339d9b700 1 --2- 172.21.15.2:0/3830601258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:50.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.922+0000 7f533a59c700 1 --2- 172.21.15.2:0/3830601258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f0f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f532c0098f0 tx=0x7f532c005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e26340e23d580d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.923+0000 7f53297fa700 1 -- 172.21.15.2:0/3830601258 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c02a070 con 0x55a7f95ecc80 2023-06-23T18:35:50.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.923+0000 7f53297fa700 1 -- 172.21.15.2:0/3830601258 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f532c0258c0 con 0x55a7f95ecc80 2023-06-23T18:35:50.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.923+0000 7f53297fa700 1 -- 172.21.15.2:0/3830601258 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c0038c0 con 0x55a7f95ecc80 2023-06-23T18:35:50.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 -- 172.21.15.2:0/3830601258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 msgr2=0x55a7f95f0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3830601258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f0f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f532c0098f0 tx=0x7f532c005a40 comp rx=0 tx=0).stop 2023-06-23T18:35:50.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 -- 172.21.15.2:0/3830601258 shutdown_connections 2023-06-23T18:35:50.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3830601258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3830601258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95f0f00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 --2- 172.21.15.2:0/3830601258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95f1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 -- 172.21.15.2:0/3830601258 >> 172.21.15.2:0/3830601258 conn(0x55a7f95e23f0 msgr2=0x55a7f95f5200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 -- 172.21.15.2:0/3830601258 shutdown_connections 2023-06-23T18:35:50.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 -- 172.21.15.2:0/3830601258 wait complete. 2023-06-23T18:35:50.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.924+0000 7f534cf45b80 1 -- start start 2023-06-23T18:35:50.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f534cf45b80 1 Processor -- start 2023-06-23T18:35:50.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95fd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95fd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f534cf45b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f534cf45b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f95fa480 con 0x55a7f95ed5d0 2023-06-23T18:35:50.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f534cf45b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f95fa5f0 con 0x55a7f95ecc80 2023-06-23T18:35:50.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f534cf45b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7f95fa7b0 con 0x55a7f95ee000 2023-06-23T18:35:50.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f534cf45b80 10 client.0 _ll_register_callbacks cb 0x55a7f94e3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:35:50.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f5339d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95fd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f533a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95fd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f5339d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95fd950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57428/0 (socket says 172.21.15.2:57428) 2023-06-23T18:35:50.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f533a59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95fd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48022/0 (socket says 172.21.15.2:48022) 2023-06-23T18:35:50.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f5339d9b700 1 -- 172.21.15.2:0/897024910 learned_addr learned my addr 172.21.15.2:0/897024910 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:35:50.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.925+0000 7f533ad9d700 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5339d9b700 1 -- 172.21.15.2:0/897024910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 msgr2=0x55a7f95f9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5339d9b700 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95f9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5339d9b700 1 -- 172.21.15.2:0/897024910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 msgr2=0x55a7f95fd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:35:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5339d9b700 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95fd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5339d9b700 1 -- 172.21.15.2:0/897024910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c071d40 con 0x55a7f95ed5d0 2023-06-23T18:35:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f533ad9d700 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:35:50.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5339d9b700 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95fd950 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f53300965c0 tx=0x7f5330004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7120d2707f6816e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:35:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5321ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:35:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5321ffb700 1 -- 172.21.15.2:0/897024910 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300a2510 con 0x55a7f95ed5d0 2023-06-23T18:35:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f5321ffb700 1 -- 172.21.15.2:0/897024910 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5330098460 con 0x55a7f95ed5d0 2023-06-23T18:35:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.926+0000 7f534cf45b80 1 -- 172.21.15.2:0/897024910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f532c01d040 con 0x55a7f95ed5d0 2023-06-23T18:35:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f5321ffb700 1 -- 172.21.15.2:0/897024910 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53300042b0 con 0x55a7f95ed5d0 2023-06-23T18:35:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f534cf45b80 10 client.16791 fetch_fsmap learned FSMap version 40 2023-06-23T18:35:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f534cf45b80 1 -- 172.21.15.2:0/897024910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55a7f936bde0 con 0x55a7f95ed5d0 2023-06-23T18:35:50.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f5321ffb700 1 -- 172.21.15.2:0/897024910 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f532c01d040 con 0x55a7f95ed5d0 2023-06-23T18:35:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f5321ffb700 1 -- 172.21.15.2:0/897024910 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f533009a320 con 0x55a7f95ed5d0 2023-06-23T18:35:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f534cf45b80 10 client.16791 fetch_fsmap finished waiting for FSMap version 40 2023-06-23T18:35:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f534cf45b80 10 client.16791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:35:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f534cf45b80 20 client.16791 populate_metadata read hostname 'smithi002' 2023-06-23T18:35:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f534cf45b80 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a7f9668170 0x55a7f966a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:35:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f534cf45b80 4 client.16791 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:35:50.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f534cf45b80 1 -- 172.21.15.2:0/897024910 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7f94f9050 con 0x55a7f9668170 2023-06-23T18:35:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.927+0000 7f533a59c700 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a7f9668170 0x55a7f966a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:35:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.928+0000 7f5321ffb700 10 client.16791 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:35:50.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.928+0000 7f533a59c700 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a7f9668170 0x55a7f966a580 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb9a614b5a6d23a9 server_cookie=cb3cd9fb63261629 in_seq=0 out_seq=0 2023-06-23T18:35:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f5321ffb700 1 -- 172.21.15.2:0/897024910 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a7f94f9050 con 0x55a7f9668170 2023-06-23T18:35:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f5321ffb700 10 client.16791 handle_command_reply: tid=0 2023-06-23T18:35:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 client.16791 shutdown 2023-06-23T18:35:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 -- 172.21.15.2:0/897024910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 msgr2=0x55a7f95fd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95fd950 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f53300965c0 tx=0x7f5330004640 comp rx=0 tx=0).stop 2023-06-23T18:35:50.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 -- 172.21.15.2:0/897024910 shutdown_connections 2023-06-23T18:35:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a7f9668170 0x55a7f966a580 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a7f95ee000 0x55a7f95f9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a7f95ecc80 0x55a7f95fd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 --2- 172.21.15.2:0/897024910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a7f95ed5d0 0x55a7f95fd950 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.929+0000 7f534cf45b80 1 -- 172.21.15.2:0/897024910 >> 172.21.15.2:0/897024910 conn(0x55a7f95e23f0 msgr2=0x55a7f95f0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.930+0000 7f5321ffb700 0 client.16791 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:35:50.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.930+0000 7f534cf45b80 1 -- 172.21.15.2:0/897024910 shutdown_connections 2023-06-23T18:35:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.930+0000 7f534cf45b80 1 -- 172.21.15.2:0/897024910 wait complete. 2023-06-23T18:35:50.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.930+0000 7f534cf45b80 20 client.16791 trim_cache size 0 max 16384 2023-06-23T18:35:50.943 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:35:50.943 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:35:50.943 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:35:50.943 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:35:50.944 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:35:50.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:35:50.944 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:35:50.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:35:50.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:35:50.944 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:35:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 -- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53240a11d0 msgr2=0x7f53240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53240a11d0 0x7f53240a3680 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f532c0098c0 tx=0x7f532c019040 comp rx=0 tx=0).stop 2023-06-23T18:35:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 -- 172.21.15.2:0/262793263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0ffc70 msgr2=0x7f533c0709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:35:50.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0ffc70 0x7f533c0709b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f533000ec10 tx=0x7f533000ef20 comp rx=0 tx=0).stop 2023-06-23T18:35:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 -- 172.21.15.2:0/262793263 shutdown_connections 2023-06-23T18:35:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f533c100600 0x7f533c071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f533c0fea80 0x7f533c068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f53240a11d0 0x7f53240a3680 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 --2- 172.21.15.2:0/262793263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f533c0ffc70 0x7f533c0709b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:35:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 -- 172.21.15.2:0/262793263 >> 172.21.15.2:0/262793263 conn(0x7f533c0fa0e0 msgr2=0x7f533c1031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:35:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 -- 172.21.15.2:0/262793263 shutdown_connections 2023-06-23T18:35:50.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:35:50.931+0000 7f5340dc0700 1 -- 172.21.15.2:0/262793263 wait complete. 2023-06-23T18:35:50.953 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:35:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 120.0 s, 335.8 tps, lat 116.896 ms stddev 20.257 2023-06-23T18:35:52.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:52 smithi002 ceph-mon[95947]: pgmap v711: 97 pgs: 97 active+clean; 8.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2023-06-23T18:35:52.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:52 smithi082 ceph-mon[97889]: pgmap v711: 97 pgs: 97 active+clean; 8.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2023-06-23T18:35:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:52 smithi017 ceph-mon[97472]: pgmap v711: 97 pgs: 97 active+clean; 8.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 13 MiB/s wr, 1.14k op/s 2023-06-23T18:35:54.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:54 smithi002 ceph-mon[95947]: pgmap v712: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-06-23T18:35:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:54 smithi082 ceph-mon[97889]: pgmap v712: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-06-23T18:35:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:54 smithi017 ceph-mon[97472]: pgmap v712: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 13 MiB/s wr, 1.16k op/s 2023-06-23T18:35:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:56 smithi082 ceph-mon[97889]: pgmap v713: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-06-23T18:35:56.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:56 smithi017 ceph-mon[97472]: pgmap v713: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-06-23T18:35:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:56 smithi002 ceph-mon[95947]: pgmap v713: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2023-06-23T18:35:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 125.0 s, 382.6 tps, lat 102.132 ms stddev 18.518 2023-06-23T18:35:58.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:35:58 smithi082 ceph-mon[97889]: pgmap v714: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:35:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:35:58 smithi017 ceph-mon[97472]: pgmap v714: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:35:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:35:58 smithi002 ceph-mon[95947]: pgmap v714: 97 pgs: 97 active+clean; 8.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 14 MiB/s wr, 1.16k op/s 2023-06-23T18:36:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:00 smithi082 ceph-mon[97889]: pgmap v715: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-06-23T18:36:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:00 smithi017 ceph-mon[97472]: pgmap v715: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-06-23T18:36:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:00 smithi002 ceph-mon[95947]: pgmap v715: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 15 MiB/s wr, 1.19k op/s 2023-06-23T18:36:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 130.0 s, 372.4 tps, lat 1727.573 ms stddev 8499.940 2023-06-23T18:36:02.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:02 smithi082 ceph-mon[97889]: pgmap v716: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:36:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:02 smithi017 ceph-mon[97472]: pgmap v716: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:36:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:02 smithi002 ceph-mon[95947]: pgmap v716: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:36:04.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:04 smithi082 ceph-mon[97889]: pgmap v717: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:36:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:04 smithi017 ceph-mon[97472]: pgmap v717: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:36:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:04 smithi002 ceph-mon[95947]: pgmap v717: 97 pgs: 97 active+clean; 8.4 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:36:06.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:06 smithi082 ceph-mon[97889]: pgmap v718: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 13 MiB/s wr, 943 op/s 2023-06-23T18:36:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:06 smithi017 ceph-mon[97472]: pgmap v718: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 13 MiB/s wr, 943 op/s 2023-06-23T18:36:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:06 smithi002 ceph-mon[95947]: pgmap v718: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 13 MiB/s wr, 943 op/s 2023-06-23T18:36:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 135.0 s, 382.0 tps, lat 260.581 ms stddev 38.719 2023-06-23T18:36:08.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:08 smithi082 ceph-mon[97889]: pgmap v719: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 13 MiB/s wr, 932 op/s 2023-06-23T18:36:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:08 smithi017 ceph-mon[97472]: pgmap v719: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 13 MiB/s wr, 932 op/s 2023-06-23T18:36:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:08 smithi002 ceph-mon[95947]: pgmap v719: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 13 MiB/s wr, 932 op/s 2023-06-23T18:36:10.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:10 smithi082 ceph-mon[97889]: pgmap v720: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 12 MiB/s wr, 949 op/s 2023-06-23T18:36:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:10 smithi017 ceph-mon[97472]: pgmap v720: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 12 MiB/s wr, 949 op/s 2023-06-23T18:36:10.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:10 smithi002 ceph-mon[95947]: pgmap v720: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 12 MiB/s wr, 949 op/s 2023-06-23T18:36:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 140.0 s, 379.6 tps, lat 264.185 ms stddev 41.769 2023-06-23T18:36:12.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:12 smithi082 ceph-mon[97889]: pgmap v721: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 11 MiB/s wr, 901 op/s 2023-06-23T18:36:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:12 smithi017 ceph-mon[97472]: pgmap v721: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 11 MiB/s wr, 901 op/s 2023-06-23T18:36:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:12 smithi002 ceph-mon[95947]: pgmap v721: 97 pgs: 97 active+clean; 8.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 11 MiB/s wr, 901 op/s 2023-06-23T18:36:14.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:14 smithi082 ceph-mon[97889]: pgmap v722: 97 pgs: 97 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 12 MiB/s wr, 960 op/s 2023-06-23T18:36:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:14 smithi017 ceph-mon[97472]: pgmap v722: 97 pgs: 97 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 12 MiB/s wr, 960 op/s 2023-06-23T18:36:14.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:14 smithi002 ceph-mon[95947]: pgmap v722: 97 pgs: 97 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 12 MiB/s wr, 960 op/s 2023-06-23T18:36:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:36:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:15 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:36:15.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:15 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:36:15.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:15.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:15 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:16 smithi082 ceph-mon[97889]: pgmap v723: 97 pgs: 97 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 13 MiB/s wr, 994 op/s 2023-06-23T18:36:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:36:16.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:36:16.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:16 smithi017 ceph-mon[97472]: pgmap v723: 97 pgs: 97 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 13 MiB/s wr, 994 op/s 2023-06-23T18:36:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:36:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:36:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:16 smithi002 ceph-mon[95947]: pgmap v723: 97 pgs: 97 active+clean; 8.4 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 13 MiB/s wr, 994 op/s 2023-06-23T18:36:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:36:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:36:16.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:36:17.027 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 145.0 s, 389.8 tps, lat 256.928 ms stddev 36.116 2023-06-23T18:36:18.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:18 smithi082 ceph-mon[97889]: pgmap v724: 97 pgs: 97 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-06-23T18:36:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:18 smithi017 ceph-mon[97472]: pgmap v724: 97 pgs: 97 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-06-23T18:36:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:18 smithi002 ceph-mon[95947]: pgmap v724: 97 pgs: 97 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-06-23T18:36:20.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:20 smithi082 ceph-mon[97889]: pgmap v725: 97 pgs: 97 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:36:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:20 smithi017 ceph-mon[97472]: pgmap v725: 97 pgs: 97 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:36:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:20 smithi002 ceph-mon[95947]: pgmap v725: 97 pgs: 97 active+clean; 8.4 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2023-06-23T18:36:20.954 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:36:21.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 -- 172.21.15.2:0/1621084315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0ff320 msgr2=0x7f70fc101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1621084315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0ff320 0x7f70fc101700 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009990 tx=0x7f70ec009ca0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 -- 172.21.15.2:0/1621084315 shutdown_connections 2023-06-23T18:36:21.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1621084315 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70fc0fe050 0x7f70fc0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1621084315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0ff320 0x7f70fc101700 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1621084315 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70fc101c40 0x7f70fc104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 -- 172.21.15.2:0/1621084315 >> 172.21.15.2:0/1621084315 conn(0x7f70fc0f9370 msgr2=0x7f70fc0fb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 -- 172.21.15.2:0/1621084315 shutdown_connections 2023-06-23T18:36:21.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.148+0000 7f7100c0d700 1 -- 172.21.15.2:0/1621084315 wait complete. 2023-06-23T18:36:21.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.149+0000 7f7100c0d700 1 Processor -- start 2023-06-23T18:36:21.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.149+0000 7f7100c0d700 1 -- start start 2023-06-23T18:36:21.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f7100c0d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0fe050 0x7f70fc19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f7100c0d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70fc0ff320 0x7f70fc19c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f7100c0d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70fc101c40 0x7f70fc1a33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f7100c0d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc1a39b0 con 0x7f70fc101c40 2023-06-23T18:36:21.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f7100c0d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc069ef0 con 0x7f70fc0ff320 2023-06-23T18:36:21.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f7100c0d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70fc06a080 con 0x7f70fc0fe050 2023-06-23T18:36:21.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70fa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0fe050 0x7f70fc19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70f9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70fc0ff320 0x7f70fc19c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70fa59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0fe050 0x7f70fc19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57782/0 (socket says 172.21.15.2:57782) 2023-06-23T18:36:21.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70fa59c700 1 -- 172.21.15.2:0/1304076207 learned_addr learned my addr 172.21.15.2:0/1304076207 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:21.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70fad9d700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70fc101c40 0x7f70fc1a33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70f9d9b700 1 -- 172.21.15.2:0/1304076207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0fe050 msgr2=0x7f70fc19be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0fe050 0x7f70fc19be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70f9d9b700 1 -- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70fc101c40 msgr2=0x7f70fc1a33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70fc101c40 0x7f70fc1a33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70f9d9b700 1 -- 172.21.15.2:0/1304076207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec0095f0 con 0x7f70fc0ff320 2023-06-23T18:36:21.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70fa59c700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0fe050 0x7f70fc19be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:21.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.150+0000 7f70fad9d700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70fc101c40 0x7f70fc1a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:21.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.151+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70fc0ff320 0x7f70fc19c370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9f0 tx=0x7f70f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.151+0000 7f70eb7fe700 1 -- 172.21.15.2:0/1304076207 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f000c9c0 con 0x7f70fc0ff320 2023-06-23T18:36:21.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.151+0000 7f70eb7fe700 1 -- 172.21.15.2:0/1304076207 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70f0010430 con 0x7f70fc0ff320 2023-06-23T18:36:21.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.151+0000 7f70eb7fe700 1 -- 172.21.15.2:0/1304076207 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0018680 con 0x7f70fc0ff320 2023-06-23T18:36:21.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.151+0000 7f7100c0d700 1 -- 172.21.15.2:0/1304076207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70fc06a360 con 0x7f70fc0ff320 2023-06-23T18:36:21.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.151+0000 7f7100c0d700 1 -- 172.21.15.2:0/1304076207 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70fc06a820 con 0x7f70fc0ff320 2023-06-23T18:36:21.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.154+0000 7f70eb7fe700 1 -- 172.21.15.2:0/1304076207 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f70f00105a0 con 0x7f70fc0ff320 2023-06-23T18:36:21.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.155+0000 7f70eb7fe700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70e40a1150 0x7f70e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.155+0000 7f70eb7fe700 1 -- 172.21.15.2:0/1304076207 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f70f0014070 con 0x7f70fc0ff320 2023-06-23T18:36:21.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.155+0000 7f70fa59c700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70e40a1150 0x7f70e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f710cd92b80 1 Processor -- start 2023-06-23T18:36:21.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f70fa59c700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70e40a1150 0x7f70e40a3600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00b2e0 tx=0x7f70ec005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f710cd92b80 1 -- start start 2023-06-23T18:36:21.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 0x557ebaeda090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaeda5d0 0x557ebaedaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaedb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f710cd92b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebaddecb0 con 0x557ebaeda5d0 2023-06-23T18:36:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f710cd92b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebae31a40 con 0x557ebaedb000 2023-06-23T18:36:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.156+0000 7f710cd92b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebae2aed0 con 0x557ebaed9c80 2023-06-23T18:36:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaedb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaedb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48042/0 (socket says 172.21.15.2:48042) 2023-06-23T18:36:21.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 -- 172.21.15.2:0/1595363591 learned_addr learned my addr 172.21.15.2:0/1595363591 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:21.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fa59c700 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 0x557ebaeda090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaeda5d0 0x557ebaedaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 -- 172.21.15.2:0/1595363591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 msgr2=0x557ebaeda090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 0x557ebaeda090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 -- 172.21.15.2:0/1595363591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaeda5d0 msgr2=0x557ebaedaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaeda5d0 0x557ebaedaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 -- 172.21.15.2:0/1595363591 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebacac630 con 0x557ebaedb000 2023-06-23T18:36:21.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaeda5d0 0x557ebaedaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:21.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70fad9d700 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaedb470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f70f40078d0 tx=0x7f70f4007be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=741e6c442233d702 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.157+0000 7f70e8ff9700 1 -- 172.21.15.2:0/1595363591 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4004bb0 con 0x557ebaedb000 2023-06-23T18:36:21.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f70e8ff9700 1 -- 172.21.15.2:0/1595363591 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70f400f040 con 0x557ebaedb000 2023-06-23T18:36:21.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f70e8ff9700 1 -- 172.21.15.2:0/1595363591 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4005300 con 0x557ebaedb000 2023-06-23T18:36:21.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f710cd92b80 1 -- 172.21.15.2:0/1595363591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 msgr2=0x557ebaedb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f710cd92b80 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaedb470 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f70f40078d0 tx=0x7f70f4007be0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f710cd92b80 1 -- 172.21.15.2:0/1595363591 shutdown_connections 2023-06-23T18:36:21.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f710cd92b80 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaedb470 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f710cd92b80 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 0x557ebaeda090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f710cd92b80 1 --2- 172.21.15.2:0/1595363591 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaeda5d0 0x557ebaedaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f710cd92b80 1 -- 172.21.15.2:0/1595363591 >> 172.21.15.2:0/1595363591 conn(0x557ebaecf3f0 msgr2=0x557ebaed4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.158+0000 7f710cd92b80 1 -- 172.21.15.2:0/1595363591 shutdown_connections 2023-06-23T18:36:21.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 -- 172.21.15.2:0/1595363591 wait complete. 2023-06-23T18:36:21.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 -- start start 2023-06-23T18:36:21.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 Processor -- start 2023-06-23T18:36:21.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 0x557ebaee3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaeeaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebadcaac0 con 0x557ebaed9c80 2023-06-23T18:36:21.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebadca7c0 con 0x557ebaedb000 2023-06-23T18:36:21.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.159+0000 7f710cd92b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebacae920 con 0x557ebaeda5d0 2023-06-23T18:36:21.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f710cd92b80 10 client.0 _ll_register_callbacks cb 0x557ebadd0260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:36:21.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57794/0 (socket says 172.21.15.2:57794) 2023-06-23T18:36:21.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 -- 172.21.15.2:0/945268383 learned_addr learned my addr 172.21.15.2:0/945268383 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:21.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70fad9d700 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaeeaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70fa59c700 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 0x557ebaee3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 -- 172.21.15.2:0/945268383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 msgr2=0x557ebaeeaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaeeaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 -- 172.21.15.2:0/945268383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 msgr2=0x557ebaee3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 0x557ebaee3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 -- 172.21.15.2:0/945268383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f4007500 con 0x557ebaeda5d0 2023-06-23T18:36:21.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee3a70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70f0099e30 tx=0x7f70f009a3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=390b89b3e4cda7d8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:36:21.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70e1ffb700 1 -- 172.21.15.2:0/945268383 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f00a2060 con 0x557ebaeda5d0 2023-06-23T18:36:21.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f710cd92b80 1 -- 172.21.15.2:0/945268383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ebaeccbb0 con 0x557ebaeda5d0 2023-06-23T18:36:21.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.160+0000 7f70e1ffb700 1 -- 172.21.15.2:0/945268383 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f70f009ded0 con 0x557ebaeda5d0 2023-06-23T18:36:21.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f70e1ffb700 1 -- 172.21.15.2:0/945268383 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f00a22f0 con 0x557ebaeda5d0 2023-06-23T18:36:21.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f710cd92b80 10 client.26518 fetch_fsmap learned FSMap version 40 2023-06-23T18:36:21.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f710cd92b80 1 -- 172.21.15.2:0/945268383 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557ebac58de0 con 0x557ebaeda5d0 2023-06-23T18:36:21.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f70e1ffb700 1 -- 172.21.15.2:0/945268383 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70f00c8080 con 0x557ebaeda5d0 2023-06-23T18:36:21.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f70e1ffb700 1 -- 172.21.15.2:0/945268383 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f70f00c9f80 con 0x557ebaeda5d0 2023-06-23T18:36:21.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f710cd92b80 10 client.26518 fetch_fsmap finished waiting for FSMap version 40 2023-06-23T18:36:21.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f710cd92b80 10 client.26518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:36:21.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f710cd92b80 20 client.26518 populate_metadata read hostname 'smithi002' 2023-06-23T18:36:21.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f710cd92b80 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ebaeef390 0x557ebaef17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f710cd92b80 4 client.26518 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:36:21.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f710cd92b80 1 -- 172.21.15.2:0/945268383 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ebae23eb0 con 0x557ebaeef390 2023-06-23T18:36:21.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.161+0000 7f70fa59c700 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ebaeef390 0x557ebaef17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.162+0000 7f70e1ffb700 10 client.26518 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:36:21.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.162+0000 7f70fa59c700 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ebaeef390 0x557ebaef17a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf835f2b15af96b6 server_cookie=7d1cd4396af08db3 in_seq=0 out_seq=0 2023-06-23T18:36:21.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f70e1ffb700 1 -- 172.21.15.2:0/945268383 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557ebae23eb0 con 0x557ebaeef390 2023-06-23T18:36:21.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f70e1ffb700 10 client.26518 handle_command_reply: tid=0 2023-06-23T18:36:21.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 client.26518 shutdown 2023-06-23T18:36:21.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 -- 172.21.15.2:0/945268383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 msgr2=0x557ebaee3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee3a70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70f0099e30 tx=0x7f70f009a3e0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 -- 172.21.15.2:0/945268383 shutdown_connections 2023-06-23T18:36:21.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ebaeef390 0x557ebaef17a0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaeeaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 0x557ebaee3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 --2- 172.21.15.2:0/945268383 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee3a70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f70e1ffb700 0 client.26518 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:36:21.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 -- 172.21.15.2:0/945268383 >> 172.21.15.2:0/945268383 conn(0x557ebaecf3f0 msgr2=0x557ebaed3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 -- 172.21.15.2:0/945268383 shutdown_connections 2023-06-23T18:36:21.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 1 -- 172.21.15.2:0/945268383 wait complete. 2023-06-23T18:36:21.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.163+0000 7f710cd92b80 20 client.26518 trim_cache size 0 max 16384 2023-06-23T18:36:21.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.194+0000 7f710cd92b80 1 Processor -- start 2023-06-23T18:36:21.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f710cd92b80 1 -- start start 2023-06-23T18:36:21.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 0x557ebaef6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaeda5d0 0x557ebaeeaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaedb000 0x557ebaee3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f710cd92b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebaeec7a0 con 0x557ebaedb000 2023-06-23T18:36:21.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f710cd92b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebaee4430 con 0x557ebaeda5d0 2023-06-23T18:36:21.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f710cd92b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebaee30a0 con 0x557ebaed9c80 2023-06-23T18:36:21.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f70f9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaeda5d0 0x557ebaeeaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f70fad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaedb000 0x557ebaee3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f70f9d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaeda5d0 0x557ebaeeaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48056/0 (socket says 172.21.15.2:48056) 2023-06-23T18:36:21.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f70fad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaedb000 0x557ebaee3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57466/0 (socket says 172.21.15.2:57466) 2023-06-23T18:36:21.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f70f9d9b700 1 -- 172.21.15.2:0/1210882584 learned_addr learned my addr 172.21.15.2:0/1210882584 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:21.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.195+0000 7f70fa59c700 1 --2- 172.21.15.2:0/1210882584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 0x557ebaef6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70fad9d700 1 -- 172.21.15.2:0/1210882584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 msgr2=0x557ebaef6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70fad9d700 1 --2- 172.21.15.2:0/1210882584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 0x557ebaef6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70fad9d700 1 -- 172.21.15.2:0/1210882584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaeda5d0 msgr2=0x557ebaeeaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70fad9d700 1 --2- 172.21.15.2:0/1210882584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaeda5d0 0x557ebaeeaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70fad9d700 1 -- 172.21.15.2:0/1210882584 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebac41c60 con 0x557ebaedb000 2023-06-23T18:36:21.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70fad9d700 1 --2- 172.21.15.2:0/1210882584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaedb000 0x557ebaee3530 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000f80 tx=0x7f70f4006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea3d54f71862838 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70e97fa700 1 -- 172.21.15.2:0/1210882584 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f4003b00 con 0x557ebaedb000 2023-06-23T18:36:21.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70e97fa700 1 -- 172.21.15.2:0/1210882584 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70f4003c60 con 0x557ebaedb000 2023-06-23T18:36:21.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.196+0000 7f70e97fa700 1 -- 172.21.15.2:0/1210882584 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f40053d0 con 0x557ebaedb000 2023-06-23T18:36:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.197+0000 7f710cd92b80 1 -- 172.21.15.2:0/1210882584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaedb000 msgr2=0x557ebaee3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.197+0000 7f710cd92b80 1 --2- 172.21.15.2:0/1210882584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaedb000 0x557ebaee3530 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f70f4000f80 tx=0x7f70f4006000 comp rx=0 tx=0).stop 2023-06-23T18:36:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.197+0000 7f710cd92b80 1 -- 172.21.15.2:0/1210882584 shutdown_connections 2023-06-23T18:36:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.197+0000 7f710cd92b80 1 --2- 172.21.15.2:0/1210882584 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaedb000 0x557ebaee3530 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.197+0000 7f710cd92b80 1 --2- 172.21.15.2:0/1210882584 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaed9c80 0x557ebaef6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.197+0000 7f710cd92b80 1 --2- 172.21.15.2:0/1210882584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaeda5d0 0x557ebaeeaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.197+0000 7f710cd92b80 1 -- 172.21.15.2:0/1210882584 >> 172.21.15.2:0/1210882584 conn(0x557ebaecf3f0 msgr2=0x557ebaed08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.197+0000 7f710cd92b80 1 -- 172.21.15.2:0/1210882584 shutdown_connections 2023-06-23T18:36:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.199+0000 7f710cd92b80 1 -- 172.21.15.2:0/1210882584 wait complete. 2023-06-23T18:36:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.199+0000 7f710cd92b80 1 -- start start 2023-06-23T18:36:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.199+0000 7f710cd92b80 1 Processor -- start 2023-06-23T18:36:21.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 0x557ebaee6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f710cd92b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaeea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f710cd92b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebaeea780 con 0x557ebaed9c80 2023-06-23T18:36:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f710cd92b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebaeea8f0 con 0x557ebaedb000 2023-06-23T18:36:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f710cd92b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebaeeaab0 con 0x557ebaeda5d0 2023-06-23T18:36:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f710cd92b80 10 client.0 _ll_register_callbacks cb 0x557ebadd0260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:36:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70f9d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70fa59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 0x557ebaee6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70f9d9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57810/0 (socket says 172.21.15.2:57810) 2023-06-23T18:36:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70f9d9b700 1 -- 172.21.15.2:0/2398052852 learned_addr learned my addr 172.21.15.2:0/2398052852 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70fad9d700 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaeea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70f9d9b700 1 -- 172.21.15.2:0/2398052852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 msgr2=0x557ebaeea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaeea240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70f9d9b700 1 -- 172.21.15.2:0/2398052852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 msgr2=0x557ebaee6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:21.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 0x557ebaee6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.200+0000 7f70f9d9b700 1 -- 172.21.15.2:0/2398052852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ebacac630 con 0x557ebaeda5d0 2023-06-23T18:36:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f70f9d9b700 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee7220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f70f0099de0 tx=0x7f70f0010ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7dc66d6ee5888ef server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f70e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:36:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f70e1ffb700 1 -- 172.21.15.2:0/2398052852 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f009ded0 con 0x557ebaeda5d0 2023-06-23T18:36:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f70e1ffb700 1 -- 172.21.15.2:0/2398052852 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f70f00b8020 con 0x557ebaeda5d0 2023-06-23T18:36:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f710cd92b80 1 -- 172.21.15.2:0/2398052852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70f400f040 con 0x557ebaeda5d0 2023-06-23T18:36:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f70e1ffb700 1 -- 172.21.15.2:0/2398052852 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f00ca3d0 con 0x557ebaeda5d0 2023-06-23T18:36:21.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f710cd92b80 10 client.26524 fetch_fsmap learned FSMap version 40 2023-06-23T18:36:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f710cd92b80 1 -- 172.21.15.2:0/2398052852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x557ebac7f250 con 0x557ebaeda5d0 2023-06-23T18:36:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.201+0000 7f70e1ffb700 1 -- 172.21.15.2:0/2398052852 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70f400f040 con 0x557ebaeda5d0 2023-06-23T18:36:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.202+0000 7f70e1ffb700 1 -- 172.21.15.2:0/2398052852 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f70f0094ab0 con 0x557ebaeda5d0 2023-06-23T18:36:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.202+0000 7f710cd92b80 10 client.26524 fetch_fsmap finished waiting for FSMap version 40 2023-06-23T18:36:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.202+0000 7f710cd92b80 10 client.26524 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:36:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.202+0000 7f710cd92b80 20 client.26524 populate_metadata read hostname 'smithi002' 2023-06-23T18:36:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.202+0000 7f710cd92b80 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ebaf44380 0x557ebaf46790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:21.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.202+0000 7f710cd92b80 4 client.26524 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:36:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.202+0000 7f710cd92b80 1 -- 172.21.15.2:0/2398052852 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557ebade6050 con 0x557ebaf44380 2023-06-23T18:36:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.203+0000 7f70fa59c700 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ebaf44380 0x557ebaf46790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:21.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.203+0000 7f70e1ffb700 10 client.26524 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:36:21.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.203+0000 7f70fa59c700 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ebaf44380 0x557ebaf46790 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94bcce699c360b1b server_cookie=1755e337b21d889 in_seq=0 out_seq=0 2023-06-23T18:36:21.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.204+0000 7f70e1ffb700 1 -- 172.21.15.2:0/2398052852 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557ebade6050 con 0x557ebaf44380 2023-06-23T18:36:21.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.204+0000 7f70e1ffb700 10 client.26524 handle_command_reply: tid=0 2023-06-23T18:36:21.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.204+0000 7f710cd92b80 1 client.26524 shutdown 2023-06-23T18:36:21.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 -- 172.21.15.2:0/2398052852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 msgr2=0x557ebaee7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee7220 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f70f0099de0 tx=0x7f70f0010ae0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 -- 172.21.15.2:0/2398052852 shutdown_connections 2023-06-23T18:36:21.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x557ebaf44380 0x557ebaf46790 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x557ebaedb000 0x557ebaeea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x557ebaed9c80 0x557ebaee6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 --2- 172.21.15.2:0/2398052852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x557ebaeda5d0 0x557ebaee7220 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 -- 172.21.15.2:0/2398052852 >> 172.21.15.2:0/2398052852 conn(0x557ebaecf3f0 msgr2=0x557ebaef6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f70e1ffb700 0 client.26524 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:36:21.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 -- 172.21.15.2:0/2398052852 shutdown_connections 2023-06-23T18:36:21.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 1 -- 172.21.15.2:0/2398052852 wait complete. 2023-06-23T18:36:21.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.205+0000 7f710cd92b80 20 client.26524 trim_cache size 0 max 16384 2023-06-23T18:36:21.215 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:36:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:36:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:36:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:36:21.215 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:36:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:36:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:36:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:36:21.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:36:21.216 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:36:21.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 -- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70e40a1150 msgr2=0x7f70e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70e40a1150 0x7f70e40a3600 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00b2e0 tx=0x7f70ec005fd0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 -- 172.21.15.2:0/1304076207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70fc0ff320 msgr2=0x7f70fc19c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:21.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70fc0ff320 0x7f70fc19c370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9f0 tx=0x7f70f000ed00 comp rx=0 tx=0).stop 2023-06-23T18:36:21.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 -- 172.21.15.2:0/1304076207 shutdown_connections 2023-06-23T18:36:21.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70fc101c40 0x7f70fc1a33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70fc0fe050 0x7f70fc19be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70e40a1150 0x7f70e40a3600 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 --2- 172.21.15.2:0/1304076207 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70fc0ff320 0x7f70fc19c370 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:21.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 -- 172.21.15.2:0/1304076207 >> 172.21.15.2:0/1304076207 conn(0x7f70fc0f9370 msgr2=0x7f70fc0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:21.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 -- 172.21.15.2:0/1304076207 shutdown_connections 2023-06-23T18:36:21.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:21.208+0000 7f7100c0d700 1 -- 172.21.15.2:0/1304076207 wait complete. 2023-06-23T18:36:21.231 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:36:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 150.0 s, 394.6 tps, lat 254.398 ms stddev 47.002 2023-06-23T18:36:22.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:22 smithi082 ceph-mon[97889]: pgmap v726: 97 pgs: 97 active+clean; 8.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-06-23T18:36:22.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:22 smithi017 ceph-mon[97472]: pgmap v726: 97 pgs: 97 active+clean; 8.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-06-23T18:36:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:22 smithi002 ceph-mon[95947]: pgmap v726: 97 pgs: 97 active+clean; 8.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-06-23T18:36:24.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:24 smithi082 ceph-mon[97889]: pgmap v727: 97 pgs: 97 active+clean; 8.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-06-23T18:36:24.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:24 smithi017 ceph-mon[97472]: pgmap v727: 97 pgs: 97 active+clean; 8.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-06-23T18:36:24.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:24 smithi002 ceph-mon[95947]: pgmap v727: 97 pgs: 97 active+clean; 8.4 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2023-06-23T18:36:26.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:26 smithi082 ceph-mon[97889]: pgmap v728: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:36:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:26 smithi017 ceph-mon[97472]: pgmap v728: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:36:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:26 smithi002 ceph-mon[95947]: pgmap v728: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:36:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 155.0 s, 405.6 tps, lat 245.854 ms stddev 35.604 2023-06-23T18:36:28.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:28 smithi082 ceph-mon[97889]: pgmap v729: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:36:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:28 smithi017 ceph-mon[97472]: pgmap v729: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:36:28.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:28 smithi002 ceph-mon[95947]: pgmap v729: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:36:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:30 smithi082 ceph-mon[97889]: pgmap v730: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-06-23T18:36:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:30 smithi017 ceph-mon[97472]: pgmap v730: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-06-23T18:36:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:30 smithi002 ceph-mon[95947]: pgmap v730: 97 pgs: 97 active+clean; 8.4 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 14 MiB/s wr, 1.14k op/s 2023-06-23T18:36:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 160.0 s, 393.4 tps, lat 252.115 ms stddev 41.985 2023-06-23T18:36:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:32 smithi082 ceph-mon[97889]: pgmap v731: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:36:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:32 smithi017 ceph-mon[97472]: pgmap v731: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:36:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:32 smithi002 ceph-mon[95947]: pgmap v731: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:36:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:34 smithi017 ceph-mon[97472]: pgmap v732: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:36:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:34 smithi002 ceph-mon[95947]: pgmap v732: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:36:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:34 smithi082 ceph-mon[97889]: pgmap v732: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:36:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:36 smithi017 ceph-mon[97472]: pgmap v733: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:36:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:36 smithi002 ceph-mon[95947]: pgmap v733: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:36:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:36 smithi082 ceph-mon[97889]: pgmap v733: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:36:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 165.0 s, 390.8 tps, lat 257.657 ms stddev 45.966 2023-06-23T18:36:38.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:38 smithi017 ceph-mon[97472]: pgmap v734: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:36:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:38 smithi002 ceph-mon[95947]: pgmap v734: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:36:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:38 smithi082 ceph-mon[97889]: pgmap v734: 97 pgs: 97 active+clean; 8.4 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:36:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:40 smithi017 ceph-mon[97472]: pgmap v735: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:40 smithi002 ceph-mon[95947]: pgmap v735: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:40 smithi082 ceph-mon[97889]: pgmap v735: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 170.0 s, 320.4 tps, lat 217.678 ms stddev 115.147 2023-06-23T18:36:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:42 smithi017 ceph-mon[97472]: pgmap v736: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:42 smithi002 ceph-mon[95947]: pgmap v736: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:42 smithi082 ceph-mon[97889]: pgmap v736: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 587 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:44 smithi017 ceph-mon[97472]: pgmap v737: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:44 smithi002 ceph-mon[95947]: pgmap v737: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:44 smithi082 ceph-mon[97889]: pgmap v737: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:36:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:46 smithi017 ceph-mon[97472]: pgmap v738: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 11 MiB/s wr, 975 op/s 2023-06-23T18:36:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:46 smithi002 ceph-mon[95947]: pgmap v738: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 11 MiB/s wr, 975 op/s 2023-06-23T18:36:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:46 smithi082 ceph-mon[97889]: pgmap v738: 97 pgs: 97 active+clean; 8.4 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 11 MiB/s wr, 975 op/s 2023-06-23T18:36:47.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 175.0 s, 253.2 tps, lat 27.596 ms stddev 7.636 2023-06-23T18:36:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:48 smithi017 ceph-mon[97472]: pgmap v739: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 12 MiB/s wr, 969 op/s 2023-06-23T18:36:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:48 smithi002 ceph-mon[95947]: pgmap v739: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 12 MiB/s wr, 969 op/s 2023-06-23T18:36:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:48 smithi082 ceph-mon[97889]: pgmap v739: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 12 MiB/s wr, 969 op/s 2023-06-23T18:36:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:50 smithi017 ceph-mon[97472]: pgmap v740: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:36:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:50 smithi002 ceph-mon[95947]: pgmap v740: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:36:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:50 smithi082 ceph-mon[97889]: pgmap v740: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:36:51.233 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:36:51.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.399+0000 7fb34db88700 1 -- 172.21.15.2:0/595937499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb348100620 msgr2=0x7fb348102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.399+0000 7fb34db88700 1 --2- 172.21.15.2:0/595937499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb348100620 0x7fb348102a00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb338009990 tx=0x7fb338009ca0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 -- 172.21.15.2:0/595937499 shutdown_connections 2023-06-23T18:36:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 --2- 172.21.15.2:0/595937499 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb348105860 0x7fb348107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 --2- 172.21.15.2:0/595937499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb348100620 0x7fb348102a00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 --2- 172.21.15.2:0/595937499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb348102f40 0x7fb348105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 -- 172.21.15.2:0/595937499 >> 172.21.15.2:0/595937499 conn(0x7fb3480fa060 msgr2=0x7fb3480fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 -- 172.21.15.2:0/595937499 shutdown_connections 2023-06-23T18:36:51.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 -- 172.21.15.2:0/595937499 wait complete. 2023-06-23T18:36:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 Processor -- start 2023-06-23T18:36:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.400+0000 7fb34db88700 1 -- start start 2023-06-23T18:36:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb34db88700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb348100620 0x7fb348195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb34db88700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb348102f40 0x7fb348195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb34db88700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb348105860 0x7fb34819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb34db88700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34819d2c0 con 0x7fb348105860 2023-06-23T18:36:51.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb34db88700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34819d400 con 0x7fb348102f40 2023-06-23T18:36:51.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb34db88700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34819d590 con 0x7fb348100620 2023-06-23T18:36:51.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb348100620 0x7fb348195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb348100620 0x7fb348195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57822/0 (socket says 172.21.15.2:57822) 2023-06-23T18:36:51.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb3477fe700 1 -- 172.21.15.2:0/2399014340 learned_addr learned my addr 172.21.15.2:0/2399014340 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:51.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb346ffd700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb348102f40 0x7fb348195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb347fff700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb348105860 0x7fb34819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb3477fe700 1 -- 172.21.15.2:0/2399014340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb348102f40 msgr2=0x7fb348195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb3477fe700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb348102f40 0x7fb348195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb3477fe700 1 -- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb348105860 msgr2=0x7fb34819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb3477fe700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb348105860 0x7fb34819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb3477fe700 1 -- 172.21.15.2:0/2399014340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380095f0 con 0x7fb348100620 2023-06-23T18:36:51.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.401+0000 7fb346ffd700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb348102f40 0x7fb348195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:36:51.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.402+0000 7fb3477fe700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb348100620 0x7fb348195740 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb338003f60 tx=0x7fb3380057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.402+0000 7fb344ff9700 1 -- 172.21.15.2:0/2399014340 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33801d070 con 0x7fb348100620 2023-06-23T18:36:51.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.402+0000 7fb344ff9700 1 -- 172.21.15.2:0/2399014340 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb33800f460 con 0x7fb348100620 2023-06-23T18:36:51.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.402+0000 7fb34db88700 1 -- 172.21.15.2:0/2399014340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34819d810 con 0x7fb348100620 2023-06-23T18:36:51.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.402+0000 7fb344ff9700 1 -- 172.21.15.2:0/2399014340 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb338021670 con 0x7fb348100620 2023-06-23T18:36:51.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.402+0000 7fb34db88700 1 -- 172.21.15.2:0/2399014340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34819dcc0 con 0x7fb348100620 2023-06-23T18:36:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.404+0000 7fb344ff9700 1 -- 172.21.15.2:0/2399014340 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb33800f840 con 0x7fb348100620 2023-06-23T18:36:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.405+0000 7fb344ff9700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3300a11d0 0x7fb3300a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.405+0000 7fb344ff9700 1 -- 172.21.15.2:0/2399014340 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb3380cdc50 con 0x7fb348100620 2023-06-23T18:36:51.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.405+0000 7fb346ffd700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3300a11d0 0x7fb3300a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.406+0000 7fb346ffd700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3300a11d0 0x7fb3300a3680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00afd0 tx=0x7fb33c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb359d0db80 1 Processor -- start 2023-06-23T18:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb359d0db80 1 -- start start 2023-06-23T18:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37dc80 0x55eeea37e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37e5d0 0x55eeea37ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 0x55eeea37f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb359d0db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea282cb0 con 0x55eeea37dc80 2023-06-23T18:36:51.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb359d0db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea2d5a40 con 0x55eeea37f000 2023-06-23T18:36:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb359d0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea2ceed0 con 0x55eeea37e5d0 2023-06-23T18:36:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37dc80 0x55eeea37e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37dc80 0x55eeea37e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57492/0 (socket says 172.21.15.2:57492) 2023-06-23T18:36:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb3477fe700 1 -- 172.21.15.2:0/824610252 learned_addr learned my addr 172.21.15.2:0/824610252 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb347fff700 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 0x55eeea37f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb346ffd700 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37e5d0 0x55eeea37ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb3477fe700 1 -- 172.21.15.2:0/824610252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37e5d0 msgr2=0x55eeea37ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb3477fe700 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37e5d0 0x55eeea37ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb3477fe700 1 -- 172.21.15.2:0/824610252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 msgr2=0x55eeea37f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb3477fe700 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 0x55eeea37f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb3477fe700 1 -- 172.21.15.2:0/824610252 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeea150630 con 0x55eeea37dc80 2023-06-23T18:36:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.407+0000 7fb346ffd700 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37e5d0 0x55eeea37ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:36:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.408+0000 7fb3477fe700 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37dc80 0x55eeea37e090 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb3380975b0 tx=0x7fb338005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5e2c25cdd2f402 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.408+0000 7fb32dffb700 1 -- 172.21.15.2:0/824610252 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800fe40 con 0x55eeea37dc80 2023-06-23T18:36:51.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.408+0000 7fb32dffb700 1 -- 172.21.15.2:0/824610252 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb33809b0b0 con 0x55eeea37dc80 2023-06-23T18:36:51.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.408+0000 7fb32dffb700 1 -- 172.21.15.2:0/824610252 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33809a5a0 con 0x55eeea37dc80 2023-06-23T18:36:51.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 -- 172.21.15.2:0/824610252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37dc80 msgr2=0x55eeea37e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37dc80 0x55eeea37e090 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb3380975b0 tx=0x7fb338005fd0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 -- 172.21.15.2:0/824610252 shutdown_connections 2023-06-23T18:36:51.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 0x55eeea37f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37dc80 0x55eeea37e090 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 --2- 172.21.15.2:0/824610252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37e5d0 0x55eeea37ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 -- 172.21.15.2:0/824610252 >> 172.21.15.2:0/824610252 conn(0x55eeea3733f0 msgr2=0x55eeea378520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:51.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 -- 172.21.15.2:0/824610252 shutdown_connections 2023-06-23T18:36:51.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 -- 172.21.15.2:0/824610252 wait complete. 2023-06-23T18:36:51.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 -- start start 2023-06-23T18:36:51.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.409+0000 7fb359d0db80 1 Processor -- start 2023-06-23T18:36:51.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3871f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea387730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 0x55eeea38e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb359d0db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea26eac0 con 0x55eeea37e5d0 2023-06-23T18:36:51.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb359d0db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea26e7c0 con 0x55eeea37dc80 2023-06-23T18:36:51.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb359d0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea152920 con 0x55eeea37f000 2023-06-23T18:36:51.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb359d0db80 10 client.0 _ll_register_callbacks cb 0x55eeea274260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:36:51.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3871f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3871f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48090/0 (socket says 172.21.15.2:48090) 2023-06-23T18:36:51.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 -- 172.21.15.2:0/1222590440 learned_addr learned my addr 172.21.15.2:0/1222590440 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:51.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb346ffd700 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea387730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb347fff700 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 0x55eeea38e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 -- 172.21.15.2:0/1222590440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 msgr2=0x55eeea38e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:51.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 0x55eeea38e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 -- 172.21.15.2:0/1222590440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 msgr2=0x55eeea387730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:51.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea387730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 -- 172.21.15.2:0/1222590440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34819dcc0 con 0x55eeea37dc80 2023-06-23T18:36:51.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb3477fe700 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3871f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb338005cd0 tx=0x7fb33809ab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a202733a15c647d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.410+0000 7fb326ffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:36:51.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.411+0000 7fb326ffd700 1 -- 172.21.15.2:0/1222590440 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3380b93f0 con 0x55eeea37dc80 2023-06-23T18:36:51.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.411+0000 7fb359d0db80 1 -- 172.21.15.2:0/1222590440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eeea370bb0 con 0x55eeea37dc80 2023-06-23T18:36:51.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.411+0000 7fb326ffd700 1 -- 172.21.15.2:0/1222590440 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb33800fe40 con 0x55eeea37dc80 2023-06-23T18:36:51.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.411+0000 7fb326ffd700 1 -- 172.21.15.2:0/1222590440 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3380cbbd0 con 0x55eeea37dc80 2023-06-23T18:36:51.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.411+0000 7fb359d0db80 10 client.26327 fetch_fsmap learned FSMap version 40 2023-06-23T18:36:51.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.411+0000 7fb359d0db80 1 -- 172.21.15.2:0/1222590440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55eeea0fcde0 con 0x55eeea37dc80 2023-06-23T18:36:51.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb326ffd700 1 -- 172.21.15.2:0/1222590440 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb33807d080 con 0x55eeea37dc80 2023-06-23T18:36:51.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb326ffd700 1 -- 172.21.15.2:0/1222590440 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb338096ea0 con 0x55eeea37dc80 2023-06-23T18:36:51.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb359d0db80 10 client.26327 fetch_fsmap finished waiting for FSMap version 40 2023-06-23T18:36:51.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb359d0db80 10 client.26327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:36:51.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb359d0db80 20 client.26327 populate_metadata read hostname 'smithi002' 2023-06-23T18:36:51.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55eeea392a50 0x55eeea394e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb359d0db80 4 client.26327 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:36:51.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb359d0db80 1 -- 172.21.15.2:0/1222590440 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eeea2c7eb0 con 0x55eeea392a50 2023-06-23T18:36:51.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.412+0000 7fb346ffd700 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55eeea392a50 0x55eeea394e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.413+0000 7fb346ffd700 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55eeea392a50 0x55eeea394e60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d1ca75ed3eec072 server_cookie=bcabed7384378c9b in_seq=0 out_seq=0 2023-06-23T18:36:51.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.413+0000 7fb326ffd700 10 client.26327 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:36:51.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.414+0000 7fb326ffd700 1 -- 172.21.15.2:0/1222590440 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eeea2c7eb0 con 0x55eeea392a50 2023-06-23T18:36:51.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.414+0000 7fb326ffd700 10 client.26327 handle_command_reply: tid=0 2023-06-23T18:36:51.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.414+0000 7fb359d0db80 1 client.26327 shutdown 2023-06-23T18:36:51.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 -- 172.21.15.2:0/1222590440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 msgr2=0x55eeea3871f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3871f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb338005cd0 tx=0x7fb33809ab40 comp rx=0 tx=0).stop 2023-06-23T18:36:51.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 -- 172.21.15.2:0/1222590440 shutdown_connections 2023-06-23T18:36:51.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55eeea392a50 0x55eeea394e60 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 0x55eeea38e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3871f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb326ffd700 0 client.26327 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:36:51.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1222590440 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea387730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 -- 172.21.15.2:0/1222590440 >> 172.21.15.2:0/1222590440 conn(0x55eeea3733f0 msgr2=0x55eeea377c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:51.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 -- 172.21.15.2:0/1222590440 shutdown_connections 2023-06-23T18:36:51.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 1 -- 172.21.15.2:0/1222590440 wait complete. 2023-06-23T18:36:51.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.415+0000 7fb359d0db80 20 client.26327 trim_cache size 0 max 16384 2023-06-23T18:36:51.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.439+0000 7fb359d0db80 1 Processor -- start 2023-06-23T18:36:51.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb359d0db80 1 -- start start 2023-06-23T18:36:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37dc80 0x55eeea3997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea38e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 0x55eeea385f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb359d0db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea379820 con 0x55eeea37e5d0 2023-06-23T18:36:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb359d0db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea379630 con 0x55eeea37f000 2023-06-23T18:36:51.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb359d0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea399d00 con 0x55eeea37dc80 2023-06-23T18:36:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37dc80 0x55eeea3997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37dc80 0x55eeea3997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57844/0 (socket says 172.21.15.2:57844) 2023-06-23T18:36:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb3477fe700 1 -- 172.21.15.2:0/1611241809 learned_addr learned my addr 172.21.15.2:0/1611241809 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb347fff700 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 0x55eeea385f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.440+0000 7fb346ffd700 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea38e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb3477fe700 1 -- 172.21.15.2:0/1611241809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 msgr2=0x55eeea385f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb3477fe700 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 0x55eeea385f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb3477fe700 1 -- 172.21.15.2:0/1611241809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 msgr2=0x55eeea38e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:51.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb3477fe700 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea38e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb3477fe700 1 -- 172.21.15.2:0/1611241809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeea0e5c60 con 0x55eeea37dc80 2023-06-23T18:36:51.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb3477fe700 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37dc80 0x55eeea3997c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3380975b0 tx=0x7fb3380092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50355a307ccf793 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb32e7fc700 1 -- 172.21.15.2:0/1611241809 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33809abf0 con 0x55eeea37dc80 2023-06-23T18:36:51.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb32e7fc700 1 -- 172.21.15.2:0/1611241809 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eeea370bb0 con 0x55eeea37dc80 2023-06-23T18:36:51.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.441+0000 7fb32e7fc700 1 -- 172.21.15.2:0/1611241809 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3380b9450 con 0x55eeea37dc80 2023-06-23T18:36:51.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 -- 172.21.15.2:0/1611241809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37dc80 msgr2=0x55eeea3997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37dc80 0x55eeea3997c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb3380975b0 tx=0x7fb3380092d0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 -- 172.21.15.2:0/1611241809 shutdown_connections 2023-06-23T18:36:51.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37f000 0x55eeea385f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37dc80 0x55eeea3997c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 --2- 172.21.15.2:0/1611241809 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea38e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 -- 172.21.15.2:0/1611241809 >> 172.21.15.2:0/1611241809 conn(0x55eeea3733f0 msgr2=0x55eeea3748c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:51.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 -- 172.21.15.2:0/1611241809 shutdown_connections 2023-06-23T18:36:51.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.442+0000 7fb359d0db80 1 -- 172.21.15.2:0/1611241809 wait complete. 2023-06-23T18:36:51.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 1 -- start start 2023-06-23T18:36:51.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 1 Processor -- start 2023-06-23T18:36:51.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea388df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 0x55eeea38dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea38e180 con 0x55eeea37e5d0 2023-06-23T18:36:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea38e2f0 con 0x55eeea37dc80 2023-06-23T18:36:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeea38e4b0 con 0x55eeea37f000 2023-06-23T18:36:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb359d0db80 10 client.0 _ll_register_callbacks cb 0x55eeea274260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:36:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb3477fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3888b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48104/0 (socket says 172.21.15.2:48104) 2023-06-23T18:36:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb3477fe700 1 -- 172.21.15.2:0/2990172237 learned_addr learned my addr 172.21.15.2:0/2990172237 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:36:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb346ffd700 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea388df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb3477fe700 1 -- 172.21.15.2:0/2990172237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 msgr2=0x55eeea38dc40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:36:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb3477fe700 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 0x55eeea38dc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.443+0000 7fb3477fe700 1 -- 172.21.15.2:0/2990172237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 msgr2=0x55eeea388df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:36:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb3477fe700 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea388df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb3477fe700 1 -- 172.21.15.2:0/2990172237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeea0fcde0 con 0x55eeea37dc80 2023-06-23T18:36:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb3477fe700 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3888b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb338079fd0 tx=0x7fb33800fb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57cc73d1805f0fb7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:36:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb326ffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:36:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb326ffd700 1 -- 172.21.15.2:0/2990172237 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33809abf0 con 0x55eeea37dc80 2023-06-23T18:36:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb326ffd700 1 -- 172.21.15.2:0/2990172237 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb33809f0e0 con 0x55eeea37dc80 2023-06-23T18:36:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb359d0db80 1 -- 172.21.15.2:0/2990172237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb33809b0b0 con 0x55eeea37dc80 2023-06-23T18:36:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb326ffd700 1 -- 172.21.15.2:0/2990172237 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb33800fe40 con 0x55eeea37dc80 2023-06-23T18:36:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb359d0db80 10 client.26333 fetch_fsmap learned FSMap version 40 2023-06-23T18:36:51.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb359d0db80 1 -- 172.21.15.2:0/2990172237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55eeea123250 con 0x55eeea37dc80 2023-06-23T18:36:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.444+0000 7fb326ffd700 1 -- 172.21.15.2:0/2990172237 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb33809b0b0 con 0x55eeea37dc80 2023-06-23T18:36:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb326ffd700 1 -- 172.21.15.2:0/2990172237 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb3380aed20 con 0x55eeea37dc80 2023-06-23T18:36:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb359d0db80 10 client.26333 fetch_fsmap finished waiting for FSMap version 40 2023-06-23T18:36:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb359d0db80 10 client.26333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:36:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb359d0db80 20 client.26333 populate_metadata read hostname 'smithi002' 2023-06-23T18:36:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb359d0db80 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55eeea400bb0 0x55eeea402f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:36:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb359d0db80 4 client.26333 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:36:51.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb359d0db80 1 -- 172.21.15.2:0/2990172237 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eeea28a050 con 0x55eeea400bb0 2023-06-23T18:36:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb346ffd700 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55eeea400bb0 0x55eeea402f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:36:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb346ffd700 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55eeea400bb0 0x55eeea402f90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e21943d0491c9d8 server_cookie=9044c2f5e3a69ca3 in_seq=0 out_seq=0 2023-06-23T18:36:51.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.445+0000 7fb326ffd700 10 client.26333 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:36:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb326ffd700 1 -- 172.21.15.2:0/2990172237 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55eeea28a050 con 0x55eeea400bb0 2023-06-23T18:36:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb326ffd700 10 client.26333 handle_command_reply: tid=0 2023-06-23T18:36:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 client.26333 shutdown 2023-06-23T18:36:51.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 -- 172.21.15.2:0/2990172237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 msgr2=0x55eeea3888b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3888b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb338079fd0 tx=0x7fb33800fb50 comp rx=0 tx=0).stop 2023-06-23T18:36:51.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 -- 172.21.15.2:0/2990172237 shutdown_connections 2023-06-23T18:36:51.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55eeea400bb0 0x55eeea402f90 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55eeea37f000 0x55eeea38dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55eeea37dc80 0x55eeea3888b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 --2- 172.21.15.2:0/2990172237 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55eeea37e5d0 0x55eeea388df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb326ffd700 0 client.26333 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:36:51.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 -- 172.21.15.2:0/2990172237 >> 172.21.15.2:0/2990172237 conn(0x55eeea3733f0 msgr2=0x55eeea385f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 -- 172.21.15.2:0/2990172237 shutdown_connections 2023-06-23T18:36:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 1 -- 172.21.15.2:0/2990172237 wait complete. 2023-06-23T18:36:51.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.446+0000 7fb359d0db80 20 client.26333 trim_cache size 0 max 16384 2023-06-23T18:36:51.459 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:36:51.459 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:36:51.459 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:36:51.460 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:36:51.460 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:36:51.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:36:51.460 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:36:51.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:36:51.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:36:51.460 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:36:51.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 -- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3300a11d0 msgr2=0x7fb3300a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3300a11d0 0x7fb3300a3680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00afd0 tx=0x7fb33c00a380 comp rx=0 tx=0).stop 2023-06-23T18:36:51.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 -- 172.21.15.2:0/2399014340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb348100620 msgr2=0x7fb348195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:36:51.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb348100620 0x7fb348195740 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb338003f60 tx=0x7fb3380057a0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 -- 172.21.15.2:0/2399014340 shutdown_connections 2023-06-23T18:36:51.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb348105860 0x7fb34819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb348100620 0x7fb348195740 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fb3300a11d0 0x7fb3300a3680 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 --2- 172.21.15.2:0/2399014340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb348102f40 0x7fb348195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:36:51.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 -- 172.21.15.2:0/2399014340 >> 172.21.15.2:0/2399014340 conn(0x7fb3480fa060 msgr2=0x7fb348101280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:36:51.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 -- 172.21.15.2:0/2399014340 shutdown_connections 2023-06-23T18:36:51.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:36:51.447+0000 7fb34db88700 1 -- 172.21.15.2:0/2399014340 wait complete. 2023-06-23T18:36:51.466 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:36:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 180.0 s, 323.2 tps, lat 691.471 ms stddev 2152.792 2023-06-23T18:36:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:52 smithi017 ceph-mon[97472]: pgmap v741: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 12 MiB/s wr, 960 op/s 2023-06-23T18:36:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:52 smithi002 ceph-mon[95947]: pgmap v741: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 12 MiB/s wr, 960 op/s 2023-06-23T18:36:52.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:52 smithi082 ceph-mon[97889]: pgmap v741: 97 pgs: 97 active+clean; 8.4 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 12 MiB/s wr, 960 op/s 2023-06-23T18:36:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:54 smithi017 ceph-mon[97472]: pgmap v742: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 10 MiB/s wr, 916 op/s 2023-06-23T18:36:54.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:54 smithi002 ceph-mon[95947]: pgmap v742: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 10 MiB/s wr, 916 op/s 2023-06-23T18:36:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:54 smithi082 ceph-mon[97889]: pgmap v742: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 532 KiB/s rd, 10 MiB/s wr, 916 op/s 2023-06-23T18:36:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:56 smithi017 ceph-mon[97472]: pgmap v743: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 9.7 MiB/s wr, 877 op/s 2023-06-23T18:36:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:56 smithi002 ceph-mon[95947]: pgmap v743: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 9.7 MiB/s wr, 877 op/s 2023-06-23T18:36:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:56 smithi082 ceph-mon[97889]: pgmap v743: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 9.7 MiB/s wr, 877 op/s 2023-06-23T18:36:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 185.0 s, 381.0 tps, lat 260.378 ms stddev 44.728 2023-06-23T18:36:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:36:58 smithi017 ceph-mon[97472]: pgmap v744: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 10 MiB/s wr, 905 op/s 2023-06-23T18:36:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:36:58 smithi002 ceph-mon[95947]: pgmap v744: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 10 MiB/s wr, 905 op/s 2023-06-23T18:36:58.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:36:58 smithi082 ceph-mon[97889]: pgmap v744: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 10 MiB/s wr, 905 op/s 2023-06-23T18:37:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:00 smithi017 ceph-mon[97472]: pgmap v745: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:37:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:00 smithi002 ceph-mon[95947]: pgmap v745: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:37:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:00 smithi082 ceph-mon[97889]: pgmap v745: 97 pgs: 97 active+clean; 8.4 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:37:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 190.0 s, 294.6 tps, lat 336.688 ms stddev 74.959 2023-06-23T18:37:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:02 smithi017 ceph-mon[97472]: pgmap v746: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:37:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:02 smithi002 ceph-mon[95947]: pgmap v746: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:37:02.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:02 smithi082 ceph-mon[97889]: pgmap v746: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:37:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:04 smithi017 ceph-mon[97472]: pgmap v747: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 11 MiB/s wr, 976 op/s 2023-06-23T18:37:04.754 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:04 smithi082 ceph-mon[97889]: pgmap v747: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 11 MiB/s wr, 976 op/s 2023-06-23T18:37:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:04 smithi002 ceph-mon[95947]: pgmap v747: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 11 MiB/s wr, 976 op/s 2023-06-23T18:37:05.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:05 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:37:05.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:05 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:37:05.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:05 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:37:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:06 smithi017 ceph-mon[97472]: pgmap v748: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 11 MiB/s wr, 961 op/s 2023-06-23T18:37:06.716 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:06 smithi082 ceph-mon[97889]: pgmap v748: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 11 MiB/s wr, 961 op/s 2023-06-23T18:37:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:06 smithi002 ceph-mon[95947]: pgmap v748: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 544 KiB/s rd, 11 MiB/s wr, 961 op/s 2023-06-23T18:37:07.027 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 195.0 s, 338.6 tps, lat 296.624 ms stddev 65.853 2023-06-23T18:37:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:08 smithi017 ceph-mon[97472]: pgmap v749: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 12 MiB/s wr, 995 op/s 2023-06-23T18:37:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:08 smithi002 ceph-mon[95947]: pgmap v749: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 12 MiB/s wr, 995 op/s 2023-06-23T18:37:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:08 smithi082 ceph-mon[97889]: pgmap v749: 97 pgs: 97 active+clean; 8.4 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 12 MiB/s wr, 995 op/s 2023-06-23T18:37:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:10 smithi017 ceph-mon[97472]: pgmap v750: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:37:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:10 smithi002 ceph-mon[95947]: pgmap v750: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:37:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:10 smithi082 ceph-mon[97889]: pgmap v750: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 14 MiB/s wr, 1.12k op/s 2023-06-23T18:37:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 200.0 s, 373.2 tps, lat 271.043 ms stddev 48.810 2023-06-23T18:37:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:12 smithi017 ceph-mon[97472]: pgmap v751: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:37:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:12 smithi002 ceph-mon[95947]: pgmap v751: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:37:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:12 smithi082 ceph-mon[97889]: pgmap v751: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:37:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:14 smithi017 ceph-mon[97472]: pgmap v752: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:37:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:14 smithi002 ceph-mon[95947]: pgmap v752: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:37:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:14 smithi082 ceph-mon[97889]: pgmap v752: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 13 MiB/s wr, 1.09k op/s 2023-06-23T18:37:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:16 smithi017 ceph-mon[97472]: pgmap v753: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:37:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:16 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:37:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:16 smithi002 ceph-mon[95947]: pgmap v753: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:37:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:16 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:37:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:16 smithi082 ceph-mon[97889]: pgmap v753: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 14 MiB/s wr, 1.10k op/s 2023-06-23T18:37:16.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:16 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:37:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 205.0 s, 418.4 tps, lat 238.910 ms stddev 33.975 2023-06-23T18:37:17.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:37:17.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:37:17.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:37:17.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:37:17.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:37:17.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:37:17.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:37:17.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:37:17.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:37:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:18 smithi002 ceph-mon[95947]: pgmap v754: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:37:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:18 smithi082 ceph-mon[97889]: pgmap v754: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:37:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:18 smithi017 ceph-mon[97472]: pgmap v754: 97 pgs: 97 active+clean; 8.4 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:37:21.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:20 smithi002 ceph-mon[95947]: pgmap v755: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:37:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:20 smithi082 ceph-mon[97889]: pgmap v755: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:37:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:20 smithi017 ceph-mon[97472]: pgmap v755: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 14 MiB/s wr, 1.11k op/s 2023-06-23T18:37:21.468 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:37:21.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 -- 172.21.15.2:0/4225540067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40ffc80 msgr2=0x7f31b41000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/4225540067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40ffc80 0x7f31b41000d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009990 tx=0x7f31a8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 -- 172.21.15.2:0/4225540067 shutdown_connections 2023-06-23T18:37:21.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/4225540067 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31b4100610 0x7f31b4106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/4225540067 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31b40fea90 0x7f31b40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/4225540067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40ffc80 0x7f31b41000d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 -- 172.21.15.2:0/4225540067 >> 172.21.15.2:0/4225540067 conn(0x7f31b40fa090 msgr2=0x7f31b40fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 -- 172.21.15.2:0/4225540067 shutdown_connections 2023-06-23T18:37:21.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.663+0000 7f31b8bf6700 1 -- 172.21.15.2:0/4225540067 wait complete. 2023-06-23T18:37:21.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.664+0000 7f31b8bf6700 1 Processor -- start 2023-06-23T18:37:21.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.664+0000 7f31b8bf6700 1 -- start start 2023-06-23T18:37:21.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.664+0000 7f31b8bf6700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40fea90 0x7f31b419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.664+0000 7f31b8bf6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31b40ffc80 0x7f31b419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.664+0000 7f31b8bf6700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31b4100610 0x7f31b41a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.664+0000 7f31b8bf6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41a3990 con 0x7f31b40ffc80 2023-06-23T18:37:21.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b8bf6700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41a3ad0 con 0x7f31b4100610 2023-06-23T18:37:21.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b8bf6700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41a3c60 con 0x7f31b40fea90 2023-06-23T18:37:21.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40fea90 0x7f31b419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40fea90 0x7f31b419be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57864/0 (socket says 172.21.15.2:57864) 2023-06-23T18:37:21.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b259c700 1 -- 172.21.15.2:0/1088218711 learned_addr learned my addr 172.21.15.2:0/1088218711 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:21.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31b4100610 0x7f31b41a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b1d9b700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31b40ffc80 0x7f31b419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b2d9d700 1 -- 172.21.15.2:0/1088218711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40fea90 msgr2=0x7f31b419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40fea90 0x7f31b419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b2d9d700 1 -- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31b40ffc80 msgr2=0x7f31b419c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31b40ffc80 0x7f31b419c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.665+0000 7f31b2d9d700 1 -- 172.21.15.2:0/1088218711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a80095f0 con 0x7f31b4100610 2023-06-23T18:37:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.666+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31b4100610 0x7f31b41a33c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00be20 tx=0x7f31ac009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.666+0000 7f31a37fe700 1 -- 172.21.15.2:0/1088218711 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00cb90 con 0x7f31b4100610 2023-06-23T18:37:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.666+0000 7f31a37fe700 1 -- 172.21.15.2:0/1088218711 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31ac00ccf0 con 0x7f31b4100610 2023-06-23T18:37:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.666+0000 7f31b8bf6700 1 -- 172.21.15.2:0/1088218711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b41a3f40 con 0x7f31b4100610 2023-06-23T18:37:21.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.666+0000 7f31a37fe700 1 -- 172.21.15.2:0/1088218711 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0125a0 con 0x7f31b4100610 2023-06-23T18:37:21.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.666+0000 7f31b8bf6700 1 -- 172.21.15.2:0/1088218711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b41a4440 con 0x7f31b4100610 2023-06-23T18:37:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.669+0000 7f31a37fe700 1 -- 172.21.15.2:0/1088218711 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f31ac014440 con 0x7f31b4100610 2023-06-23T18:37:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.670+0000 7f31a37fe700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f319c0a1250 0x7f319c0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.670+0000 7f31a37fe700 1 -- 172.21.15.2:0/1088218711 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f31ac006e40 con 0x7f31b4100610 2023-06-23T18:37:21.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.671+0000 7f31b259c700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f319c0a1250 0x7f319c0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.672+0000 7f31b259c700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f319c0a1250 0x7f319c0a3700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f31a400b3c0 tx=0x7f31a400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.695+0000 7f31c4d7bb80 1 Processor -- start 2023-06-23T18:37:21.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.695+0000 7f31c4d7bb80 1 -- start start 2023-06-23T18:37:21.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.695+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da2c80 0x55c651da3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.695+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651da3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.695+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651da4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.695+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651ca7cb0 con 0x55c651da35d0 2023-06-23T18:37:21.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.695+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651cfaa40 con 0x55c651da2c80 2023-06-23T18:37:21.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.695+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651cf3ed0 con 0x55c651da4000 2023-06-23T18:37:21.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da2c80 0x55c651da3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b259c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da2c80 0x55c651da3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48126/0 (socket says 172.21.15.2:48126) 2023-06-23T18:37:21.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b259c700 1 -- 172.21.15.2:0/1974760010 learned_addr learned my addr 172.21.15.2:0/1974760010 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:21.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651da4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b2d9d700 1 -- 172.21.15.2:0/1974760010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da2c80 msgr2=0x55c651da3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da2c80 0x55c651da3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b2d9d700 1 -- 172.21.15.2:0/1974760010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 msgr2=0x55c651da3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651da3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b2d9d700 1 -- 172.21.15.2:0/1974760010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c651b75630 con 0x55c651da4000 2023-06-23T18:37:21.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.696+0000 7f31b259c700 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da2c80 0x55c651da3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:21.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.697+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651da4470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0994b0 tx=0x7f31ac099420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=338d0a7893216882 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.697+0000 7f31a0ff9700 1 -- 172.21.15.2:0/1974760010 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac095040 con 0x55c651da4000 2023-06-23T18:37:21.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.697+0000 7f31a0ff9700 1 -- 172.21.15.2:0/1974760010 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31ac09d550 con 0x55c651da4000 2023-06-23T18:37:21.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.697+0000 7f31a0ff9700 1 -- 172.21.15.2:0/1974760010 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac09d390 con 0x55c651da4000 2023-06-23T18:37:21.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1974760010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 msgr2=0x55c651da4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651da4470 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0994b0 tx=0x7f31ac099420 comp rx=0 tx=0).stop 2023-06-23T18:37:21.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1974760010 shutdown_connections 2023-06-23T18:37:21.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651da4470 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da2c80 0x55c651da3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1974760010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651da3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1974760010 >> 172.21.15.2:0/1974760010 conn(0x55c651d983f0 msgr2=0x55c651d9d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1974760010 shutdown_connections 2023-06-23T18:37:21.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.698+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1974760010 wait complete. 2023-06-23T18:37:21.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 1 -- start start 2023-06-23T18:37:21.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 1 Processor -- start 2023-06-23T18:37:21.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da2c80 0x55c651dbd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dbdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651dc4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651c93ac0 con 0x55c651da2c80 2023-06-23T18:37:21.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651c937c0 con 0x55c651da35d0 2023-06-23T18:37:21.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651b77920 con 0x55c651da4000 2023-06-23T18:37:21.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31c4d7bb80 10 client.0 _ll_register_callbacks cb 0x55c651c99260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:37:21.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31b1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dbdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31b1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dbdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48132/0 (socket says 172.21.15.2:48132) 2023-06-23T18:37:21.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.699+0000 7f31b1d9b700 1 -- 172.21.15.2:0/2223595900 learned_addr learned my addr 172.21.15.2:0/2223595900 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:21.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651dc4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f31b1d9b700 1 -- 172.21.15.2:0/2223595900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 msgr2=0x55c651dc4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f31b1d9b700 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651dc4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f31b1d9b700 1 -- 172.21.15.2:0/2223595900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da2c80 msgr2=0x55c651dbd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f31b1d9b700 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da2c80 0x55c651dbd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f31b1d9b700 1 -- 172.21.15.2:0/2223595900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b41a4440 con 0x55c651da35d0 2023-06-23T18:37:21.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f31b1d9b700 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dbdb80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009990 tx=0x7f31a80054f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f703da565fdb3004 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f3199ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:37:21.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f3199ffb700 1 -- 172.21.15.2:0/2223595900 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a801d070 con 0x55c651da35d0 2023-06-23T18:37:21.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f3199ffb700 1 -- 172.21.15.2:0/2223595900 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f31a8003be0 con 0x55c651da35d0 2023-06-23T18:37:21.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.700+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/2223595900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c651d95bb0 con 0x55c651da35d0 2023-06-23T18:37:21.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.706+0000 7f3199ffb700 1 -- 172.21.15.2:0/2223595900 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31a800b710 con 0x55c651da35d0 2023-06-23T18:37:21.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f31c4d7bb80 10 client.26342 fetch_fsmap learned FSMap version 41 2023-06-23T18:37:21.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/2223595900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c651b21de0 con 0x55c651da35d0 2023-06-23T18:37:21.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f3199ffb700 1 -- 172.21.15.2:0/2223595900 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31a8031020 con 0x55c651da35d0 2023-06-23T18:37:21.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f3199ffb700 1 -- 172.21.15.2:0/2223595900 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f31a8004860 con 0x55c651da35d0 2023-06-23T18:37:21.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f31c4d7bb80 10 client.26342 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:37:21.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f31c4d7bb80 10 client.26342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:37:21.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f31c4d7bb80 20 client.26342 populate_metadata read hostname 'smithi002' 2023-06-23T18:37:21.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c651dc9430 0x55c651dcb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f31c4d7bb80 4 client.26342 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:37:21.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.707+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/2223595900 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c651ceceb0 con 0x55c651dc9430 2023-06-23T18:37:21.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.708+0000 7f31b259c700 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c651dc9430 0x55c651dcb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.708+0000 7f3199ffb700 10 client.26342 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:37:21.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.709+0000 7f31b259c700 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c651dc9430 0x55c651dcb840 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c8ac7d06479538 server_cookie=1aa5fed82564d112 in_seq=0 out_seq=0 2023-06-23T18:37:21.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f3199ffb700 1 -- 172.21.15.2:0/2223595900 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c651ceceb0 con 0x55c651dc9430 2023-06-23T18:37:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f3199ffb700 10 client.26342 handle_command_reply: tid=0 2023-06-23T18:37:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 client.26342 shutdown 2023-06-23T18:37:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/2223595900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 msgr2=0x55c651dbdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dbdb80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009990 tx=0x7f31a80054f0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/2223595900 shutdown_connections 2023-06-23T18:37:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c651dc9430 0x55c651dcb840 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da4000 0x55c651dc4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da2c80 0x55c651dbd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/2223595900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dbdb80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/2223595900 >> 172.21.15.2:0/2223595900 conn(0x55c651d983f0 msgr2=0x55c651d9cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f3199ffb700 0 client.26342 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:37:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/2223595900 shutdown_connections 2023-06-23T18:37:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/2223595900 wait complete. 2023-06-23T18:37:21.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.710+0000 7f31c4d7bb80 20 client.26342 trim_cache size 0 max 16384 2023-06-23T18:37:21.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.737+0000 7f31c4d7bb80 1 Processor -- start 2023-06-23T18:37:21.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.737+0000 7f31c4d7bb80 1 -- start start 2023-06-23T18:37:21.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.737+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.737+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dc4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.737+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da4000 0x55c651da6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.737+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651dbe830 con 0x55c651da4000 2023-06-23T18:37:21.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.737+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651d956f0 con 0x55c651da35d0 2023-06-23T18:37:21.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.737+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651d9e810 con 0x55c651da2c80 2023-06-23T18:37:21.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da4000 0x55c651da6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dc4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da4000 0x55c651da6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57552/0 (socket says 172.21.15.2:57552) 2023-06-23T18:37:21.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57886/0 (socket says 172.21.15.2:57886) 2023-06-23T18:37:21.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b1d9b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dc4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48142/0 (socket says 172.21.15.2:48142) 2023-06-23T18:37:21.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 -- 172.21.15.2:0/4117634444 learned_addr learned my addr 172.21.15.2:0/4117634444 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:21.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 -- 172.21.15.2:0/4117634444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 msgr2=0x55c651dbc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/4117634444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 -- 172.21.15.2:0/4117634444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 msgr2=0x55c651dc4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/4117634444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dc4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 -- 172.21.15.2:0/4117634444 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c651b0ac60 con 0x55c651da4000 2023-06-23T18:37:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b259c700 1 --2- 172.21.15.2:0/4117634444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/4117634444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da4000 0x55c651da6f50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00bfd0 tx=0x7f31ac004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53e42ea40401e63 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31a17fa700 1 -- 172.21.15.2:0/4117634444 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac0097f0 con 0x55c651da4000 2023-06-23T18:37:21.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31a17fa700 1 -- 172.21.15.2:0/4117634444 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31ac095030 con 0x55c651da4000 2023-06-23T18:37:21.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.738+0000 7f31a17fa700 1 -- 172.21.15.2:0/4117634444 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac09d390 con 0x55c651da4000 2023-06-23T18:37:21.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/4117634444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da4000 msgr2=0x55c651da6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/4117634444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da4000 0x55c651da6f50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00bfd0 tx=0x7f31ac004ad0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/4117634444 shutdown_connections 2023-06-23T18:37:21.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/4117634444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da4000 0x55c651da6f50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/4117634444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/4117634444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da35d0 0x55c651dc4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/4117634444 >> 172.21.15.2:0/4117634444 conn(0x55c651d983f0 msgr2=0x55c651dcb440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/4117634444 shutdown_connections 2023-06-23T18:37:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.739+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/4117634444 wait complete. 2023-06-23T18:37:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 1 -- start start 2023-06-23T18:37:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 1 Processor -- start 2023-06-23T18:37:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651dbf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da4000 0x55c651dc41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651dc4710 con 0x55c651da35d0 2023-06-23T18:37:21.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651dc4880 con 0x55c651da4000 2023-06-23T18:37:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c651dc4a40 con 0x55c651da2c80 2023-06-23T18:37:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31c4d7bb80 10 client.0 _ll_register_callbacks cb 0x55c651c99260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:37:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31b1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651dbf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31b1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651dbf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57556/0 (socket says 172.21.15.2:57556) 2023-06-23T18:37:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31b259c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57892/0 (socket says 172.21.15.2:57892) 2023-06-23T18:37:21.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31b1d9b700 1 -- 172.21.15.2:0/1444660869 learned_addr learned my addr 172.21.15.2:0/1444660869 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.740+0000 7f31b2d9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da4000 0x55c651dc41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31b1d9b700 1 -- 172.21.15.2:0/1444660869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 msgr2=0x55c651dbeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31b1d9b700 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31b1d9b700 1 -- 172.21.15.2:0/1444660869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da4000 msgr2=0x55c651dc41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31b1d9b700 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da4000 0x55c651dc41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31b1d9b700 1 -- 172.21.15.2:0/1444660869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a401d040 con 0x55c651da35d0 2023-06-23T18:37:21.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31b2d9d700 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da4000 0x55c651dc41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:37:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31b259c700 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31b1d9b700 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651dbf2e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55c651d9df20 tx=0x7f31a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b038ed4b11e4baa1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f3199ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:37:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f3199ffb700 1 -- 172.21.15.2:0/1444660869 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a801d070 con 0x55c651da35d0 2023-06-23T18:37:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1444660869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f31ac09d550 con 0x55c651da35d0 2023-06-23T18:37:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.741+0000 7f3199ffb700 1 -- 172.21.15.2:0/1444660869 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f31a8000e40 con 0x55c651da35d0 2023-06-23T18:37:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f3199ffb700 1 -- 172.21.15.2:0/1444660869 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31a8003d60 con 0x55c651da35d0 2023-06-23T18:37:21.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31c4d7bb80 10 client.16839 fetch_fsmap learned FSMap version 41 2023-06-23T18:37:21.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1444660869 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55c651b75630 con 0x55c651da35d0 2023-06-23T18:37:21.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f3199ffb700 1 -- 172.21.15.2:0/1444660869 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f31ac09d550 con 0x55c651da35d0 2023-06-23T18:37:21.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f3199ffb700 1 -- 172.21.15.2:0/1444660869 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f31a8034070 con 0x55c651da35d0 2023-06-23T18:37:21.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31c4d7bb80 10 client.16839 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:37:21.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31c4d7bb80 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:37:21.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31c4d7bb80 20 client.16839 populate_metadata read hostname 'smithi002' 2023-06-23T18:37:21.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c651e2f190 0x55c651e315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:21.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31c4d7bb80 4 client.16839 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:37:21.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1444660869 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c651caf050 con 0x55c651e2f190 2023-06-23T18:37:21.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.742+0000 7f31b259c700 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c651e2f190 0x55c651e315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:21.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.743+0000 7f3199ffb700 10 client.16839 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:37:21.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.743+0000 7f31b259c700 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c651e2f190 0x55c651e315a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e78e9360e71151 server_cookie=7f308e5e9ce58f2c in_seq=0 out_seq=0 2023-06-23T18:37:21.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f3199ffb700 1 -- 172.21.15.2:0/1444660869 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c651caf050 con 0x55c651e2f190 2023-06-23T18:37:21.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f3199ffb700 10 client.16839 handle_command_reply: tid=0 2023-06-23T18:37:21.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 client.16839 shutdown 2023-06-23T18:37:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1444660869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 msgr2=0x55c651dbf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651dbf2e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x55c651d9df20 tx=0x7f31a8000bc0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1444660869 shutdown_connections 2023-06-23T18:37:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55c651e2f190 0x55c651e315a0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55c651da4000 0x55c651dc41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c651da2c80 0x55c651dbeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 --2- 172.21.15.2:0/1444660869 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55c651da35d0 0x55c651dbf2e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1444660869 >> 172.21.15.2:0/1444660869 conn(0x55c651d983f0 msgr2=0x55c651dbc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f3199ffb700 0 client.16839 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:37:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1444660869 shutdown_connections 2023-06-23T18:37:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 1 -- 172.21.15.2:0/1444660869 wait complete. 2023-06-23T18:37:21.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.744+0000 7f31c4d7bb80 20 client.16839 trim_cache size 0 max 16384 2023-06-23T18:37:21.760 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:37:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:37:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:37:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:37:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:37:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:37:21.761 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:37:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:37:21.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:37:21.762 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:37:21.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.745+0000 7f31b8bf6700 1 -- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f319c0a1250 msgr2=0x7f319c0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.745+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f319c0a1250 0x7f319c0a3700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f31a400b3c0 tx=0x7f31a400d040 comp rx=0 tx=0).stop 2023-06-23T18:37:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.745+0000 7f31b8bf6700 1 -- 172.21.15.2:0/1088218711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31b4100610 msgr2=0x7f31b41a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:21.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.745+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31b4100610 0x7f31b41a33c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00be20 tx=0x7f31ac009ed0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.746+0000 7f31b8bf6700 1 -- 172.21.15.2:0/1088218711 shutdown_connections 2023-06-23T18:37:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.746+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f319c0a1250 0x7f319c0a3700 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.746+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f31b4100610 0x7f31b41a33c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.746+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f31b40fea90 0x7f31b419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.746+0000 7f31b8bf6700 1 --2- 172.21.15.2:0/1088218711 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31b40ffc80 0x7f31b419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.746+0000 7f31b8bf6700 1 -- 172.21.15.2:0/1088218711 >> 172.21.15.2:0/1088218711 conn(0x7f31b40fa090 msgr2=0x7f31b4103190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:21.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.746+0000 7f31b8bf6700 1 -- 172.21.15.2:0/1088218711 shutdown_connections 2023-06-23T18:37:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:21.746+0000 7f31b8bf6700 1 -- 172.21.15.2:0/1088218711 wait complete. 2023-06-23T18:37:21.769 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:37:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 210.0 s, 399.4 tps, lat 250.021 ms stddev 45.004 2023-06-23T18:37:23.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:22 smithi002 ceph-mon[95947]: pgmap v756: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 13 MiB/s wr, 988 op/s 2023-06-23T18:37:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:22 smithi082 ceph-mon[97889]: pgmap v756: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 13 MiB/s wr, 988 op/s 2023-06-23T18:37:23.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:22 smithi017 ceph-mon[97472]: pgmap v756: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 592 KiB/s rd, 13 MiB/s wr, 988 op/s 2023-06-23T18:37:25.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:24 smithi002 ceph-mon[95947]: pgmap v757: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 12 MiB/s wr, 953 op/s 2023-06-23T18:37:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:24 smithi082 ceph-mon[97889]: pgmap v757: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 12 MiB/s wr, 953 op/s 2023-06-23T18:37:25.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:24 smithi017 ceph-mon[97472]: pgmap v757: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 12 MiB/s wr, 953 op/s 2023-06-23T18:37:27.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 215.0 s, 368.8 tps, lat 270.252 ms stddev 40.249 2023-06-23T18:37:27.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:26 smithi002 ceph-mon[95947]: pgmap v758: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 12 MiB/s wr, 920 op/s 2023-06-23T18:37:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:26 smithi082 ceph-mon[97889]: pgmap v758: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 12 MiB/s wr, 920 op/s 2023-06-23T18:37:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:26 smithi017 ceph-mon[97472]: pgmap v758: 97 pgs: 97 active+clean; 8.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 12 MiB/s wr, 920 op/s 2023-06-23T18:37:29.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:28 smithi002 ceph-mon[95947]: pgmap v759: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 12 MiB/s wr, 904 op/s 2023-06-23T18:37:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:28 smithi082 ceph-mon[97889]: pgmap v759: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 12 MiB/s wr, 904 op/s 2023-06-23T18:37:29.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:28 smithi017 ceph-mon[97472]: pgmap v759: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 12 MiB/s wr, 904 op/s 2023-06-23T18:37:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:30 smithi002 ceph-mon[95947]: pgmap v760: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 12 MiB/s wr, 977 op/s 2023-06-23T18:37:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:30 smithi082 ceph-mon[97889]: pgmap v760: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 12 MiB/s wr, 977 op/s 2023-06-23T18:37:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:30 smithi017 ceph-mon[97472]: pgmap v760: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 12 MiB/s wr, 977 op/s 2023-06-23T18:37:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 220.0 s, 383.0 tps, lat 260.469 ms stddev 39.926 2023-06-23T18:37:33.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:32 smithi002 ceph-mon[95947]: pgmap v761: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 13 MiB/s wr, 998 op/s 2023-06-23T18:37:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:32 smithi082 ceph-mon[97889]: pgmap v761: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 13 MiB/s wr, 998 op/s 2023-06-23T18:37:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:32 smithi017 ceph-mon[97472]: pgmap v761: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 13 MiB/s wr, 998 op/s 2023-06-23T18:37:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:34 smithi002 ceph-mon[95947]: pgmap v762: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-06-23T18:37:35.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:34 smithi082 ceph-mon[97889]: pgmap v762: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-06-23T18:37:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:34 smithi017 ceph-mon[97472]: pgmap v762: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2023-06-23T18:37:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 225.0 s, 394.6 tps, lat 255.603 ms stddev 48.650 2023-06-23T18:37:37.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:36 smithi002 ceph-mon[95947]: pgmap v763: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 12 MiB/s wr, 988 op/s 2023-06-23T18:37:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:36 smithi082 ceph-mon[97889]: pgmap v763: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 12 MiB/s wr, 988 op/s 2023-06-23T18:37:37.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:36 smithi017 ceph-mon[97472]: pgmap v763: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 567 KiB/s rd, 12 MiB/s wr, 988 op/s 2023-06-23T18:37:39.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:38 smithi002 ceph-mon[95947]: pgmap v764: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 12 MiB/s wr, 991 op/s 2023-06-23T18:37:39.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:38 smithi082 ceph-mon[97889]: pgmap v764: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 12 MiB/s wr, 991 op/s 2023-06-23T18:37:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:38 smithi017 ceph-mon[97472]: pgmap v764: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 12 MiB/s wr, 991 op/s 2023-06-23T18:37:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:40 smithi002 ceph-mon[95947]: pgmap v765: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:37:41.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:40 smithi082 ceph-mon[97889]: pgmap v765: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:37:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:40 smithi017 ceph-mon[97472]: pgmap v765: 97 pgs: 97 active+clean; 8.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 13 MiB/s wr, 1.08k op/s 2023-06-23T18:37:42.027 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 230.0 s, 213.1 tps, lat 288.523 ms stddev 66.356 2023-06-23T18:37:43.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:42 smithi002 ceph-mon[95947]: pgmap v766: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-06-23T18:37:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:42 smithi082 ceph-mon[97889]: pgmap v766: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-06-23T18:37:43.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:42 smithi017 ceph-mon[97472]: pgmap v766: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2023-06-23T18:37:45.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:44 smithi002 ceph-mon[95947]: pgmap v767: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 12 MiB/s wr, 996 op/s 2023-06-23T18:37:45.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:44 smithi082 ceph-mon[97889]: pgmap v767: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 12 MiB/s wr, 996 op/s 2023-06-23T18:37:45.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:44 smithi017 ceph-mon[97472]: pgmap v767: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 12 MiB/s wr, 996 op/s 2023-06-23T18:37:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 235.0 s, 322.1 tps, lat 431.372 ms stddev 788.263 2023-06-23T18:37:47.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:46 smithi002 ceph-mon[95947]: pgmap v768: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 12 MiB/s wr, 905 op/s 2023-06-23T18:37:47.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:46 smithi082 ceph-mon[97889]: pgmap v768: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 12 MiB/s wr, 905 op/s 2023-06-23T18:37:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:46 smithi017 ceph-mon[97472]: pgmap v768: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 12 MiB/s wr, 905 op/s 2023-06-23T18:37:49.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:48 smithi002 ceph-mon[95947]: pgmap v769: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 11 MiB/s wr, 848 op/s 2023-06-23T18:37:49.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:48 smithi082 ceph-mon[97889]: pgmap v769: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 11 MiB/s wr, 848 op/s 2023-06-23T18:37:49.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:48 smithi017 ceph-mon[97472]: pgmap v769: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 11 MiB/s wr, 848 op/s 2023-06-23T18:37:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:50 smithi002 ceph-mon[95947]: pgmap v770: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 11 MiB/s wr, 849 op/s 2023-06-23T18:37:51.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:50 smithi082 ceph-mon[97889]: pgmap v770: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 11 MiB/s wr, 849 op/s 2023-06-23T18:37:51.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:50 smithi017 ceph-mon[97472]: pgmap v770: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 11 MiB/s wr, 849 op/s 2023-06-23T18:37:51.771 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:37:51.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 -- 172.21.15.2:0/3958356890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb80fde80 msgr2=0x7f2eb80fe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:51.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/3958356890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb80fde80 0x7f2eb80fe290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009990 tx=0x7f2ea8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 -- 172.21.15.2:0/3958356890 shutdown_connections 2023-06-23T18:37:51.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/3958356890 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eb8101850 0x7f2eb81080f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/3958356890 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb80fde80 0x7f2eb80fe290 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/3958356890 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb8100ea0 0x7f2eb8101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 -- 172.21.15.2:0/3958356890 >> 172.21.15.2:0/3958356890 conn(0x7f2eb80f9850 msgr2=0x7f2eb80fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:51.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 -- 172.21.15.2:0/3958356890 shutdown_connections 2023-06-23T18:37:51.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.956+0000 7f2ebd43a700 1 -- 172.21.15.2:0/3958356890 wait complete. 2023-06-23T18:37:51.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2ebd43a700 1 Processor -- start 2023-06-23T18:37:51.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2ebd43a700 1 -- start start 2023-06-23T18:37:51.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2ebd43a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb80fde80 0x7f2eb8100810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2ebd43a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb8100ea0 0x7f2eb80fee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2ebd43a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eb8101850 0x7f2eb80ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2ebd43a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80ffb30 con 0x7f2eb80fde80 2023-06-23T18:37:51.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2ebd43a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80ffca0 con 0x7f2eb8100ea0 2023-06-23T18:37:51.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2ebd43a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb819f870 con 0x7f2eb8101850 2023-06-23T18:37:51.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2eb67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb8100ea0 0x7f2eb80fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2eb6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb80fde80 0x7f2eb8100810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2eb67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb8100ea0 0x7f2eb80fee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48162/0 (socket says 172.21.15.2:48162) 2023-06-23T18:37:51.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2eb67fc700 1 -- 172.21.15.2:0/1975086149 learned_addr learned my addr 172.21.15.2:0/1975086149 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:51.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.957+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eb8101850 0x7f2eb80ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2eb67fc700 1 -- 172.21.15.2:0/1975086149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eb8101850 msgr2=0x7f2eb80ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:51.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eb8101850 0x7f2eb80ff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2eb67fc700 1 -- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb80fde80 msgr2=0x7f2eb8100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:51.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb80fde80 0x7f2eb8100810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2eb67fc700 1 -- 172.21.15.2:0/1975086149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80095f0 con 0x7f2eb8100ea0 2023-06-23T18:37:51.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb8100ea0 0x7f2eb80fee50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00e9a0 tx=0x7f2eac00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:51.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2e9ffff700 1 -- 172.21.15.2:0/1975086149 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac009790 con 0x7f2eb8100ea0 2023-06-23T18:37:51.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2e9ffff700 1 -- 172.21.15.2:0/1975086149 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2eac010430 con 0x7f2eb8100ea0 2023-06-23T18:37:51.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2ebd43a700 1 -- 172.21.15.2:0/1975086149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb819fb50 con 0x7f2eb8100ea0 2023-06-23T18:37:51.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2e9ffff700 1 -- 172.21.15.2:0/1975086149 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00cea0 con 0x7f2eb8100ea0 2023-06-23T18:37:51.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.958+0000 7f2ebd43a700 1 -- 172.21.15.2:0/1975086149 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb81a00b0 con 0x7f2eb8100ea0 2023-06-23T18:37:51.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.961+0000 7f2e9ffff700 1 -- 172.21.15.2:0/1975086149 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2eac0105a0 con 0x7f2eb8100ea0 2023-06-23T18:37:51.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.962+0000 7f2e9ffff700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2ea00a11d0 0x7f2ea00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.962+0000 7f2e9ffff700 1 -- 172.21.15.2:0/1975086149 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2eac014070 con 0x7f2eb8100ea0 2023-06-23T18:37:51.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.962+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2ea00a11d0 0x7f2ea00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.962+0000 7f2ec95bfb80 1 Processor -- start 2023-06-23T18:37:51.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.962+0000 7f2ec95bfb80 1 -- start start 2023-06-23T18:37:51.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2ea00a11d0 0x7f2ea00a3680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8006010 tx=0x7f2ea800b2a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:51.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd11458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1146280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd1146c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1041cb0 con 0x560fd1145490 2023-06-23T18:37:51.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1094a40 con 0x560fd1145de0 2023-06-23T18:37:51.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd108ded0 con 0x560fd1146810 2023-06-23T18:37:51.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd11458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd11458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57576/0 (socket says 172.21.15.2:57576) 2023-06-23T18:37:51.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/3291593805 learned_addr learned my addr 172.21.15.2:0/3291593805 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:51.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd1146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb67fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1146280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/3291593805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 msgr2=0x560fd1146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:51.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd1146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/3291593805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 msgr2=0x560fd1146280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:51.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1146280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/3291593805 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fd0f0f630 con 0x560fd1145490 2023-06-23T18:37:51.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.963+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:51.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.964+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd1146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:51.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.965+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd11458a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8067520 tx=0x7f2ea8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb16d72944beed1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:51.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.965+0000 7f2e9d7fa700 1 -- 172.21.15.2:0/3291593805 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8003ce0 con 0x560fd1145490 2023-06-23T18:37:51.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.965+0000 7f2e9d7fa700 1 -- 172.21.15.2:0/3291593805 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ea801e040 con 0x560fd1145490 2023-06-23T18:37:51.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.965+0000 7f2e9d7fa700 1 -- 172.21.15.2:0/3291593805 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8004860 con 0x560fd1145490 2023-06-23T18:37:51.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.966+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3291593805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 msgr2=0x560fd11458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:51.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.966+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd11458a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8067520 tx=0x7f2ea8020000 comp rx=0 tx=0).stop 2023-06-23T18:37:51.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.966+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3291593805 shutdown_connections 2023-06-23T18:37:51.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.966+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd1146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.966+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd11458a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.966+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3291593805 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1146280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.966+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3291593805 >> 172.21.15.2:0/3291593805 conn(0x560fd11323f0 msgr2=0x560fd1137520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.967+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3291593805 shutdown_connections 2023-06-23T18:37:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.967+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3291593805 wait complete. 2023-06-23T18:37:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.967+0000 7f2ec95bfb80 1 -- start start 2023-06-23T18:37:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.967+0000 7f2ec95bfb80 1 Processor -- start 2023-06-23T18:37:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145490 0x560fd114ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145de0 0x560fd114f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 0x560fd11561f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd102dac0 con 0x560fd1145de0 2023-06-23T18:37:51.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd102d7c0 con 0x560fd1146810 2023-06-23T18:37:51.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd0f11920 con 0x560fd1145490 2023-06-23T18:37:51.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2ec95bfb80 10 client.0 _ll_register_callbacks cb 0x560fd1033260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:37:51.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145de0 0x560fd114f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145de0 0x560fd114f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57584/0 (socket says 172.21.15.2:57584) 2023-06-23T18:37:51.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 -- 172.21.15.2:0/4210985169 learned_addr learned my addr 172.21.15.2:0/4210985169 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:51.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145490 0x560fd114ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 0x560fd11561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 -- 172.21.15.2:0/4210985169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145490 msgr2=0x560fd114ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:51.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145490 0x560fd114ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 -- 172.21.15.2:0/4210985169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 msgr2=0x560fd11561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:51.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 0x560fd11561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 -- 172.21.15.2:0/4210985169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb0011230 con 0x560fd1145de0 2023-06-23T18:37:51.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145490 0x560fd114ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:51.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.968+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145de0 0x560fd114f1b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0ca570 tx=0x7f2eac099e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9926eed4317a2f91 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:51.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.969+0000 7f2e9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:37:51.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.969+0000 7f2e9aff5700 1 -- 172.21.15.2:0/4210985169 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac099570 con 0x560fd1145de0 2023-06-23T18:37:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.969+0000 7f2e9aff5700 1 -- 172.21.15.2:0/4210985169 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2eac0a22d0 con 0x560fd1145de0 2023-06-23T18:37:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.969+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/4210985169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fd112fbb0 con 0x560fd1145de0 2023-06-23T18:37:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.969+0000 7f2e9aff5700 1 -- 172.21.15.2:0/4210985169 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eac0a1e40 con 0x560fd1145de0 2023-06-23T18:37:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.969+0000 7f2ec95bfb80 10 client.16851 fetch_fsmap learned FSMap version 41 2023-06-23T18:37:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.969+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/4210985169 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f2eb81088e0 con 0x560fd1145de0 2023-06-23T18:37:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2e9aff5700 1 -- 172.21.15.2:0/4210985169 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560fd112fbb0 con 0x560fd1145de0 2023-06-23T18:37:51.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2e9aff5700 1 -- 172.21.15.2:0/4210985169 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2eac0add10 con 0x560fd1145de0 2023-06-23T18:37:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2ec95bfb80 10 client.16851 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:37:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2ec95bfb80 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:37:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2ec95bfb80 20 client.16851 populate_metadata read hostname 'smithi002' 2023-06-23T18:37:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560fd115a970 0x560fd115cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2ec95bfb80 4 client.16851 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:37:51.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/4210985169 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fd1086eb0 con 0x560fd115a970 2023-06-23T18:37:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.970+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560fd115a970 0x560fd115cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.971+0000 7f2e9aff5700 10 client.16851 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:37:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.971+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560fd115a970 0x560fd115cd80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16f83b0229f3af70 server_cookie=e79fa094a3542826 in_seq=0 out_seq=0 2023-06-23T18:37:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.972+0000 7f2e9aff5700 1 -- 172.21.15.2:0/4210985169 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560fd1086eb0 con 0x560fd115a970 2023-06-23T18:37:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.972+0000 7f2e9aff5700 10 client.16851 handle_command_reply: tid=0 2023-06-23T18:37:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.972+0000 7f2ec95bfb80 1 client.16851 shutdown 2023-06-23T18:37:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/4210985169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145de0 msgr2=0x560fd114f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:51.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145de0 0x560fd114f1b0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0ca570 tx=0x7f2eac099e80 comp rx=0 tx=0).stop 2023-06-23T18:37:51.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/4210985169 shutdown_connections 2023-06-23T18:37:51.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560fd115a970 0x560fd115cd80 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 0x560fd11561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145490 0x560fd114ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/4210985169 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145de0 0x560fd114f1b0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/4210985169 >> 172.21.15.2:0/4210985169 conn(0x560fd11323f0 msgr2=0x560fd1136c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:51.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2e9aff5700 0 client.16851 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:37:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/4210985169 shutdown_connections 2023-06-23T18:37:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.973+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/4210985169 wait complete. 2023-06-23T18:37:51.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:51.974+0000 7f2ec95bfb80 20 client.16851 trim_cache size 0 max 16384 2023-06-23T18:37:52.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.001+0000 7f2ec95bfb80 1 Processor -- start 2023-06-23T18:37:52.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.001+0000 7f2ec95bfb80 1 -- start start 2023-06-23T18:37:52.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1149630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:52.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145de0 0x560fd11561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:52.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 0x560fd1149b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:52.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd114d800 con 0x560fd1145490 2023-06-23T18:37:52.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1086eb0 con 0x560fd1146810 2023-06-23T18:37:52.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd114fe60 con 0x560fd1145de0 2023-06-23T18:37:52.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1149630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:52.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb67fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145de0 0x560fd11561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:52.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1149630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57590/0 (socket says 172.21.15.2:57590) 2023-06-23T18:37:52.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145de0 0x560fd11561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57930/0 (socket says 172.21.15.2:57930) 2023-06-23T18:37:52.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/673209451 learned_addr learned my addr 172.21.15.2:0/673209451 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:52.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 0x560fd1149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/673209451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145de0 msgr2=0x560fd11561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145de0 0x560fd11561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/673209451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 msgr2=0x560fd1149b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 0x560fd1149b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/673209451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fd1146eb0 con 0x560fd1145490 2023-06-23T18:37:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.002+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145de0 0x560fd11561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.003+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1149630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8000c00 tx=0x7f2ea8005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4830255b816cab server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.003+0000 7f2e9dffb700 1 -- 172.21.15.2:0/673209451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8004500 con 0x560fd1145490 2023-06-23T18:37:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.003+0000 7f2e9dffb700 1 -- 172.21.15.2:0/673209451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ea803c050 con 0x560fd1145490 2023-06-23T18:37:52.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.003+0000 7f2e9dffb700 1 -- 172.21.15.2:0/673209451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8004b00 con 0x560fd1145490 2023-06-23T18:37:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.004+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/673209451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 msgr2=0x560fd1149630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.004+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1149630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8000c00 tx=0x7f2ea8005a40 comp rx=0 tx=0).stop 2023-06-23T18:37:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.004+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/673209451 shutdown_connections 2023-06-23T18:37:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.004+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1146810 0x560fd1149b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.004+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1149630 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.004+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/673209451 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1145de0 0x560fd11561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.004+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/673209451 >> 172.21.15.2:0/673209451 conn(0x560fd11323f0 msgr2=0x560fd114cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.004+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/673209451 shutdown_connections 2023-06-23T18:37:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.005+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/673209451 wait complete. 2023-06-23T18:37:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.005+0000 7f2ec95bfb80 1 -- start start 2023-06-23T18:37:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.005+0000 7f2ec95bfb80 1 Processor -- start 2023-06-23T18:37:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.005+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1152250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.005+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1152790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.005+0000 7f2ec95bfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd11557b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:52.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1155cf0 con 0x560fd1145490 2023-06-23T18:37:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1155e60 con 0x560fd1145de0 2023-06-23T18:37:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2ec95bfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1156020 con 0x560fd1146810 2023-06-23T18:37:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2ec95bfb80 10 client.0 _ll_register_callbacks cb 0x560fd1033260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:37:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57596/0 (socket says 172.21.15.2:57596) 2023-06-23T18:37:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb6ffd700 1 -- 172.21.15.2:0/3471162064 learned_addr learned my addr 172.21.15.2:0/3471162064 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:37:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd11557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:52.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb77fe700 1 -- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 msgr2=0x560fd1152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb77fe700 1 -- 172.21.15.2:0/3471162064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 msgr2=0x560fd1152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:37:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb77fe700 1 -- 172.21.15.2:0/3471162064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb81088e0 con 0x560fd1146810 2023-06-23T18:37:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb67fc700 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:37:52.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2eb77fe700 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd11557b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x560fd1157f50 tx=0x7f2eb00045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=519a0b856a6dff64 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:37:52.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2e9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:37:52.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2e9aff5700 1 -- 172.21.15.2:0/3471162064 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb001c070 con 0x560fd1146810 2023-06-23T18:37:52.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2e9aff5700 1 -- 172.21.15.2:0/3471162064 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2eb0004bb0 con 0x560fd1146810 2023-06-23T18:37:52.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.006+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3471162064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2ea801e040 con 0x560fd1146810 2023-06-23T18:37:52.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2e9aff5700 1 -- 172.21.15.2:0/3471162064 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eb000e040 con 0x560fd1146810 2023-06-23T18:37:52.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2ec95bfb80 10 client.26575 fetch_fsmap learned FSMap version 41 2023-06-23T18:37:52.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3471162064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560fd0f0f630 con 0x560fd1146810 2023-06-23T18:37:52.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2e9aff5700 1 -- 172.21.15.2:0/3471162064 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2ea801e040 con 0x560fd1146810 2023-06-23T18:37:52.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2e9aff5700 1 -- 172.21.15.2:0/3471162064 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2eb0021630 con 0x560fd1146810 2023-06-23T18:37:52.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2ec95bfb80 10 client.26575 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:37:52.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2ec95bfb80 10 client.26575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:37:52.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2ec95bfb80 20 client.26575 populate_metadata read hostname 'smithi002' 2023-06-23T18:37:52.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560fd11afb20 0x560fd11b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:37:52.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2ec95bfb80 4 client.26575 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:37:52.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3471162064 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560fd1049050 con 0x560fd11afb20 2023-06-23T18:37:52.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.007+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560fd11afb20 0x560fd11b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:37:52.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.008+0000 7f2eb6ffd700 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560fd11afb20 0x560fd11b1f30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13aef46437a26865 server_cookie=5bbe68b0f49e3ade in_seq=0 out_seq=0 2023-06-23T18:37:52.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.008+0000 7f2e9aff5700 10 client.26575 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:37:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2e9aff5700 1 -- 172.21.15.2:0/3471162064 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560fd1049050 con 0x560fd11afb20 2023-06-23T18:37:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2e9aff5700 10 client.26575 handle_command_reply: tid=0 2023-06-23T18:37:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 client.26575 shutdown 2023-06-23T18:37:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3471162064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 msgr2=0x560fd11557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd11557b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x560fd1157f50 tx=0x7f2eb00045a0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3471162064 shutdown_connections 2023-06-23T18:37:52.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560fd11afb20 0x560fd11b1f30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560fd1146810 0x560fd11557b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560fd1145490 0x560fd1152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 --2- 172.21.15.2:0/3471162064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560fd1145de0 0x560fd1152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2e9aff5700 0 client.26575 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:37:52.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3471162064 >> 172.21.15.2:0/3471162064 conn(0x560fd11323f0 msgr2=0x560fd114cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:52.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3471162064 shutdown_connections 2023-06-23T18:37:52.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 1 -- 172.21.15.2:0/3471162064 wait complete. 2023-06-23T18:37:52.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.009+0000 7f2ec95bfb80 20 client.26575 trim_cache size 0 max 16384 2023-06-23T18:37:52.029 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:37:52.029 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:37:52.029 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:37:52.029 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:37:52.029 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:37:52.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:37:52.030 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:37:52.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:37:52.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:37:52.030 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:37:52.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 -- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2ea00a11d0 msgr2=0x7f2ea00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:52.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2ea00a11d0 0x7f2ea00a3680 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8006010 tx=0x7f2ea800b2a0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 -- 172.21.15.2:0/1975086149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb8100ea0 msgr2=0x7f2eb80fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:37:52.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb8100ea0 0x7f2eb80fee50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00e9a0 tx=0x7f2eac00ecb0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 -- 172.21.15.2:0/1975086149 shutdown_connections 2023-06-23T18:37:52.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2eb8101850 0x7f2eb80ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2eb80fde80 0x7f2eb8100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2ea00a11d0 0x7f2ea00a3680 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 --2- 172.21.15.2:0/1975086149 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eb8100ea0 0x7f2eb80fee50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:37:52.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 -- 172.21.15.2:0/1975086149 >> 172.21.15.2:0/1975086149 conn(0x7f2eb80f9850 msgr2=0x7f2eb80fbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:37:52.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 -- 172.21.15.2:0/1975086149 shutdown_connections 2023-06-23T18:37:52.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:37:52.010+0000 7f2ebd43a700 1 -- 172.21.15.2:0/1975086149 wait complete. 2023-06-23T18:37:52.033 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 240.0 s, 405.0 tps, lat 244.005 ms stddev 38.707 2023-06-23T18:37:52.034 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:37:53.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:52 smithi002 ceph-mon[95947]: pgmap v771: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 9.7 MiB/s wr, 789 op/s 2023-06-23T18:37:53.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:52 smithi082 ceph-mon[97889]: pgmap v771: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 9.7 MiB/s wr, 789 op/s 2023-06-23T18:37:53.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:52 smithi017 ceph-mon[97472]: pgmap v771: 97 pgs: 97 active+clean; 8.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 9.7 MiB/s wr, 789 op/s 2023-06-23T18:37:55.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:54 smithi002 ceph-mon[95947]: pgmap v772: 97 pgs: 97 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 9.7 MiB/s wr, 793 op/s 2023-06-23T18:37:55.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:54 smithi082 ceph-mon[97889]: pgmap v772: 97 pgs: 97 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 9.7 MiB/s wr, 793 op/s 2023-06-23T18:37:55.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:54 smithi017 ceph-mon[97472]: pgmap v772: 97 pgs: 97 active+clean; 8.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 9.7 MiB/s wr, 793 op/s 2023-06-23T18:37:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 245.0 s, 390.4 tps, lat 256.593 ms stddev 39.321 2023-06-23T18:37:57.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:56 smithi002 ceph-mon[95947]: pgmap v773: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 10 MiB/s wr, 804 op/s 2023-06-23T18:37:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:56 smithi082 ceph-mon[97889]: pgmap v773: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 10 MiB/s wr, 804 op/s 2023-06-23T18:37:57.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:56 smithi017 ceph-mon[97472]: pgmap v773: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 10 MiB/s wr, 804 op/s 2023-06-23T18:37:59.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:37:58 smithi002 ceph-mon[95947]: pgmap v774: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 10 MiB/s wr, 866 op/s 2023-06-23T18:37:59.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:37:58 smithi082 ceph-mon[97889]: pgmap v774: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 10 MiB/s wr, 866 op/s 2023-06-23T18:37:59.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:37:58 smithi017 ceph-mon[97472]: pgmap v774: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 520 KiB/s rd, 10 MiB/s wr, 866 op/s 2023-06-23T18:38:01.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:00 smithi002 ceph-mon[95947]: pgmap v775: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:00 smithi082 ceph-mon[97889]: pgmap v775: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:00 smithi017 ceph-mon[97472]: pgmap v775: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 250.0 s, 380.0 tps, lat 263.850 ms stddev 41.845 2023-06-23T18:38:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:02 smithi002 ceph-mon[95947]: pgmap v776: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:38:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:02 smithi082 ceph-mon[97889]: pgmap v776: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:38:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:02 smithi017 ceph-mon[97472]: pgmap v776: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 14 MiB/s wr, 1.06k op/s 2023-06-23T18:38:05.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:04 smithi002 ceph-mon[95947]: pgmap v777: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:38:05.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:04 smithi082 ceph-mon[97889]: pgmap v777: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:38:05.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:04 smithi017 ceph-mon[97472]: pgmap v777: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:38:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 255.0 s, 381.6 tps, lat 263.198 ms stddev 45.849 2023-06-23T18:38:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:06 smithi002 ceph-mon[95947]: pgmap v778: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:06 smithi082 ceph-mon[97889]: pgmap v778: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:06 smithi017 ceph-mon[97472]: pgmap v778: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:08 smithi002 ceph-mon[95947]: pgmap v779: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 12 MiB/s wr, 994 op/s 2023-06-23T18:38:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:08 smithi082 ceph-mon[97889]: pgmap v779: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 12 MiB/s wr, 994 op/s 2023-06-23T18:38:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:08 smithi017 ceph-mon[97472]: pgmap v779: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 12 MiB/s wr, 994 op/s 2023-06-23T18:38:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:10 smithi002 ceph-mon[95947]: pgmap v780: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:38:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:10 smithi082 ceph-mon[97889]: pgmap v780: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:38:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:10 smithi017 ceph-mon[97472]: pgmap v780: 97 pgs: 97 active+clean; 8.4 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2023-06-23T18:38:12.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 260.0 s, 382.4 tps, lat 260.725 ms stddev 39.445 2023-06-23T18:38:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:12 smithi082 ceph-mon[97889]: pgmap v781: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:38:13.184 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:12 smithi002 ceph-mon[95947]: pgmap v781: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:38:13.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:12 smithi017 ceph-mon[97472]: pgmap v781: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:38:15.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:14 smithi002 ceph-mon[95947]: pgmap v782: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:15.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:14 smithi082 ceph-mon[97889]: pgmap v782: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:15.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:14 smithi017 ceph-mon[97472]: pgmap v782: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 780 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 265.0 s, 359.2 tps, lat 275.980 ms stddev 54.611 2023-06-23T18:38:17.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:16 smithi002 ceph-mon[95947]: pgmap v783: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-06-23T18:38:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:16 smithi082 ceph-mon[97889]: pgmap v783: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-06-23T18:38:17.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:16 smithi017 ceph-mon[97472]: pgmap v783: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 13 MiB/s wr, 1.01k op/s 2023-06-23T18:38:18.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:17 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:38:18.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:17 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:38:18.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:17 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:38:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:18 smithi002 ceph-mon[95947]: pgmap v784: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 12 MiB/s wr, 983 op/s 2023-06-23T18:38:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:18 smithi082 ceph-mon[97889]: pgmap v784: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 12 MiB/s wr, 983 op/s 2023-06-23T18:38:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:18 smithi017 ceph-mon[97472]: pgmap v784: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 12 MiB/s wr, 983 op/s 2023-06-23T18:38:20.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:38:20.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:38:20.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:38:20.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:38:20.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:38:20.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:38:20.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:38:20.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:38:20.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:38:21.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:20 smithi002 ceph-mon[95947]: pgmap v785: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2023-06-23T18:38:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:20 smithi082 ceph-mon[97889]: pgmap v785: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2023-06-23T18:38:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:20 smithi017 ceph-mon[97472]: pgmap v785: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 12 MiB/s wr, 1.03k op/s 2023-06-23T18:38:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 270.0 s, 247.6 tps, lat 147.967 ms stddev 135.241 2023-06-23T18:38:22.035 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:38:22.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.202+0000 7feffc1ae700 1 -- 172.21.15.2:0/1623826976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff40fea90 msgr2=0x7feff40feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:22.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.202+0000 7feffc1ae700 1 --2- 172.21.15.2:0/1623826976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff40fea90 0x7feff40feea0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feff0009990 tx=0x7feff0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.203+0000 7feffc1ae700 1 -- 172.21.15.2:0/1623826976 shutdown_connections 2023-06-23T18:38:22.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.203+0000 7feffc1ae700 1 --2- 172.21.15.2:0/1623826976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feff4100610 0x7feff4106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.203+0000 7feffc1ae700 1 --2- 172.21.15.2:0/1623826976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff40fea90 0x7feff40feea0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.203+0000 7feffc1ae700 1 --2- 172.21.15.2:0/1623826976 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feff40ffc80 0x7feff41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.203+0000 7feffc1ae700 1 -- 172.21.15.2:0/1623826976 >> 172.21.15.2:0/1623826976 conn(0x7feff40fa090 msgr2=0x7feff40fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:22.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.203+0000 7feffc1ae700 1 -- 172.21.15.2:0/1623826976 shutdown_connections 2023-06-23T18:38:22.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.203+0000 7feffc1ae700 1 -- 172.21.15.2:0/1623826976 wait complete. 2023-06-23T18:38:22.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.203+0000 7feffc1ae700 1 Processor -- start 2023-06-23T18:38:22.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feffc1ae700 1 -- start start 2023-06-23T18:38:22.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feffc1ae700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feff40fea90 0x7feff419be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feffc1ae700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff40ffc80 0x7feff419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feffc1ae700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feff4100610 0x7feff41a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feffc1ae700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff41a3980 con 0x7feff40fea90 2023-06-23T18:38:22.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feffc1ae700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff41a3ac0 con 0x7feff4100610 2023-06-23T18:38:22.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feffc1ae700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff41a3c50 con 0x7feff40ffc80 2023-06-23T18:38:22.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feff9f4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feff40fea90 0x7feff419be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feff9f4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feff40fea90 0x7feff419be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57610/0 (socket says 172.21.15.2:57610) 2023-06-23T18:38:22.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feff9f4a700 1 -- 172.21.15.2:0/3937225009 learned_addr learned my addr 172.21.15.2:0/3937225009 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:22.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.204+0000 7feff9749700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff40ffc80 0x7feff419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7feff9f4a700 1 -- 172.21.15.2:0/3937225009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff40ffc80 msgr2=0x7feff419c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:22.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7feff9f4a700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff40ffc80 0x7feff419c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7feff9f4a700 1 -- 172.21.15.2:0/3937225009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feff4100610 msgr2=0x7feff41a33b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:38:22.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7feff9f4a700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feff4100610 0x7feff41a33b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7feff9f4a700 1 -- 172.21.15.2:0/3937225009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefec00bc60 con 0x7feff40fea90 2023-06-23T18:38:22.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7feff9f4a700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feff40fea90 0x7feff419be00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feff0005910 tx=0x7feff0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:22.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7fefeaffd700 1 -- 172.21.15.2:0/3937225009 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff001d070 con 0x7feff40fea90 2023-06-23T18:38:22.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7fefeaffd700 1 -- 172.21.15.2:0/3937225009 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feff000be30 con 0x7feff40fea90 2023-06-23T18:38:22.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7feffc1ae700 1 -- 172.21.15.2:0/3937225009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff00095f0 con 0x7feff40fea90 2023-06-23T18:38:22.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7fefeaffd700 1 -- 172.21.15.2:0/3937225009 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0022dd0 con 0x7feff40fea90 2023-06-23T18:38:22.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.205+0000 7feffc1ae700 1 -- 172.21.15.2:0/3937225009 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff41a41c0 con 0x7feff40fea90 2023-06-23T18:38:22.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.208+0000 7fefeaffd700 1 -- 172.21.15.2:0/3937225009 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7feff0022570 con 0x7feff40fea90 2023-06-23T18:38:22.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.209+0000 7fefeaffd700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fefe00a11d0 0x7fefe00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.209+0000 7fefeaffd700 1 -- 172.21.15.2:0/3937225009 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7feff00cdea0 con 0x7feff40fea90 2023-06-23T18:38:22.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.209+0000 7feff9749700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fefe00a11d0 0x7fefe00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.210+0000 7feff9749700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fefe00a11d0 0x7fefe00a3680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fefe400ddd0 tx=0x7fefe400f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:22.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.210+0000 7ff008333b80 1 Processor -- start 2023-06-23T18:38:22.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.210+0000 7ff008333b80 1 -- start start 2023-06-23T18:38:22.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7ff008333b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd284c80 0x55ddcd285090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7ff008333b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd285a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7ff008333b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd286470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7ff008333b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd189cb0 con 0x55ddcd284c80 2023-06-23T18:38:22.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7ff008333b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd1dca40 con 0x55ddcd286000 2023-06-23T18:38:22.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7ff008333b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd1d5ed0 con 0x55ddcd2855d0 2023-06-23T18:38:22.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feffa74b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd286470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9f4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd284c80 0x55ddcd285090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feffa74b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd286470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48208/0 (socket says 172.21.15.2:48208) 2023-06-23T18:38:22.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9f4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd284c80 0x55ddcd285090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57620/0 (socket says 172.21.15.2:57620) 2023-06-23T18:38:22.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feffa74b700 1 -- 172.21.15.2:0/654572217 learned_addr learned my addr 172.21.15.2:0/654572217 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:22.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9749700 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd285a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9749700 1 -- 172.21.15.2:0/654572217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 msgr2=0x55ddcd286470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:22.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9749700 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd286470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9749700 1 -- 172.21.15.2:0/654572217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd284c80 msgr2=0x55ddcd285090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:22.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9749700 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd284c80 0x55ddcd285090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9749700 1 -- 172.21.15.2:0/654572217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddcd057630 con 0x55ddcd2855d0 2023-06-23T18:38:22.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feffa74b700 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd286470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:38:22.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.211+0000 7feff9749700 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd285a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fefe400d5c0 tx=0x7fefe4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35d2ccc86dba85f7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:22.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.212+0000 7fefdffff700 1 -- 172.21.15.2:0/654572217 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4020070 con 0x55ddcd2855d0 2023-06-23T18:38:22.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.212+0000 7fefdffff700 1 -- 172.21.15.2:0/654572217 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefe4013040 con 0x55ddcd2855d0 2023-06-23T18:38:22.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.212+0000 7fefdffff700 1 -- 172.21.15.2:0/654572217 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe401b990 con 0x55ddcd2855d0 2023-06-23T18:38:22.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 -- 172.21.15.2:0/654572217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 msgr2=0x55ddcd285a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:22.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd285a70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fefe400d5c0 tx=0x7fefe4014000 comp rx=0 tx=0).stop 2023-06-23T18:38:22.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 -- 172.21.15.2:0/654572217 shutdown_connections 2023-06-23T18:38:22.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd286470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd284c80 0x55ddcd285090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 --2- 172.21.15.2:0/654572217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd285a70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 -- 172.21.15.2:0/654572217 >> 172.21.15.2:0/654572217 conn(0x55ddcd27a3f0 msgr2=0x55ddcd27f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:22.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 -- 172.21.15.2:0/654572217 shutdown_connections 2023-06-23T18:38:22.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.213+0000 7ff008333b80 1 -- 172.21.15.2:0/654572217 wait complete. 2023-06-23T18:38:22.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 1 -- start start 2023-06-23T18:38:22.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 1 Processor -- start 2023-06-23T18:38:22.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd28e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd28ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd286000 0x55ddcd295ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd175ac0 con 0x55ddcd2855d0 2023-06-23T18:38:22.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd1757c0 con 0x55ddcd284c80 2023-06-23T18:38:22.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd059920 con 0x55ddcd286000 2023-06-23T18:38:22.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7ff008333b80 10 client.0 _ll_register_callbacks cb 0x55ddcd17b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:38:22.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.214+0000 7feff9f4a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd28e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9f4a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd28e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48214/0 (socket says 172.21.15.2:48214) 2023-06-23T18:38:22.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9f4a700 1 -- 172.21.15.2:0/794888900 learned_addr learned my addr 172.21.15.2:0/794888900 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:22.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9749700 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd28ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feffa74b700 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd286000 0x55ddcd295ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9f4a700 1 -- 172.21.15.2:0/794888900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd286000 msgr2=0x55ddcd295ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:22.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9f4a700 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd286000 0x55ddcd295ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9f4a700 1 -- 172.21.15.2:0/794888900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 msgr2=0x55ddcd28ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:22.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9f4a700 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd28ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9f4a700 1 -- 172.21.15.2:0/794888900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec0074e0 con 0x55ddcd284c80 2023-06-23T18:38:22.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7feff9f4a700 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd28e540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feff00cb980 tx=0x7feff0005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=762bcc7212493069 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:22.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7fefdd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:38:22.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7fefdd7fa700 1 -- 172.21.15.2:0/794888900 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff009aa10 con 0x55ddcd284c80 2023-06-23T18:38:22.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7ff008333b80 1 -- 172.21.15.2:0/794888900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddcd277bb0 con 0x55ddcd284c80 2023-06-23T18:38:22.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.215+0000 7fefdd7fa700 1 -- 172.21.15.2:0/794888900 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7feff00cbe40 con 0x55ddcd284c80 2023-06-23T18:38:22.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.216+0000 7fefdd7fa700 1 -- 172.21.15.2:0/794888900 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff009f350 con 0x55ddcd284c80 2023-06-23T18:38:22.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.216+0000 7ff008333b80 10 client.26375 fetch_fsmap learned FSMap version 41 2023-06-23T18:38:22.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.216+0000 7ff008333b80 1 -- 172.21.15.2:0/794888900 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fefe400caf0 con 0x55ddcd284c80 2023-06-23T18:38:22.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.216+0000 7fefdd7fa700 1 -- 172.21.15.2:0/794888900 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ddcd277bb0 con 0x55ddcd284c80 2023-06-23T18:38:22.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.218+0000 7fefdd7fa700 1 -- 172.21.15.2:0/794888900 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7feff000b9a0 con 0x55ddcd284c80 2023-06-23T18:38:22.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.218+0000 7ff008333b80 10 client.26375 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:38:22.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.218+0000 7ff008333b80 10 client.26375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:38:22.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.218+0000 7ff008333b80 20 client.26375 populate_metadata read hostname 'smithi002' 2023-06-23T18:38:22.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.218+0000 7ff008333b80 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ddcd299df0 0x55ddcd29c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.218+0000 7ff008333b80 4 client.26375 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:38:22.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.218+0000 7ff008333b80 1 -- 172.21.15.2:0/794888900 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddcd1ceeb0 con 0x55ddcd299df0 2023-06-23T18:38:22.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.218+0000 7feff9749700 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ddcd299df0 0x55ddcd29c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.219+0000 7feff9749700 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ddcd299df0 0x55ddcd29c200 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5623b657aca31c65 server_cookie=3006785db1775816 in_seq=0 out_seq=0 2023-06-23T18:38:22.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.219+0000 7fefdd7fa700 10 client.26375 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:38:22.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7fefdd7fa700 1 -- 172.21.15.2:0/794888900 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ddcd1ceeb0 con 0x55ddcd299df0 2023-06-23T18:38:22.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7fefdd7fa700 10 client.26375 handle_command_reply: tid=0 2023-06-23T18:38:22.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 client.26375 shutdown 2023-06-23T18:38:22.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 -- 172.21.15.2:0/794888900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 msgr2=0x55ddcd28e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:22.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd28e540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feff00cb980 tx=0x7feff0005ad0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 -- 172.21.15.2:0/794888900 shutdown_connections 2023-06-23T18:38:22.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ddcd299df0 0x55ddcd29c200 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd286000 0x55ddcd295ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd28e540 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7fefdd7fa700 0 client.26375 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:38:22.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 --2- 172.21.15.2:0/794888900 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd28ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 -- 172.21.15.2:0/794888900 >> 172.21.15.2:0/794888900 conn(0x55ddcd27a3f0 msgr2=0x55ddcd27ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:22.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 -- 172.21.15.2:0/794888900 shutdown_connections 2023-06-23T18:38:22.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.220+0000 7ff008333b80 1 -- 172.21.15.2:0/794888900 wait complete. 2023-06-23T18:38:22.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.221+0000 7ff008333b80 20 client.26375 trim_cache size 0 max 16384 2023-06-23T18:38:22.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.245+0000 7ff008333b80 1 Processor -- start 2023-06-23T18:38:22.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7ff008333b80 1 -- start start 2023-06-23T18:38:22.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7ff008333b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd284c80 0x55ddcd2963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7ff008333b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd295a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7ff008333b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd28e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7ff008333b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd1ceeb0 con 0x55ddcd2855d0 2023-06-23T18:38:22.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7ff008333b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd27c470 con 0x55ddcd286000 2023-06-23T18:38:22.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7ff008333b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd28f730 con 0x55ddcd284c80 2023-06-23T18:38:22.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feff9749700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd295a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feffa74b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd28e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feff9749700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd295a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57630/0 (socket says 172.21.15.2:57630) 2023-06-23T18:38:22.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feff9749700 1 -- 172.21.15.2:0/3946435700 learned_addr learned my addr 172.21.15.2:0/3946435700 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:22.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feff9749700 1 -- 172.21.15.2:0/3946435700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd284c80 msgr2=0x55ddcd2963b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:38:22.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feff9749700 1 --2- 172.21.15.2:0/3946435700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd284c80 0x55ddcd2963b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feff9749700 1 -- 172.21.15.2:0/3946435700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 msgr2=0x55ddcd28e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:22.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feff9749700 1 --2- 172.21.15.2:0/3946435700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd28e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feff9749700 1 -- 172.21.15.2:0/3946435700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddcd003de0 con 0x55ddcd2855d0 2023-06-23T18:38:22.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.246+0000 7feffa74b700 1 --2- 172.21.15.2:0/3946435700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd28e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:38:22.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.247+0000 7feff9749700 1 --2- 172.21.15.2:0/3946435700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd295a80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fefe400d5c0 tx=0x7fefe400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2627871ac1904798 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:22.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.247+0000 7fefe8ff9700 1 -- 172.21.15.2:0/3946435700 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4020040 con 0x55ddcd2855d0 2023-06-23T18:38:22.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.247+0000 7fefe8ff9700 1 -- 172.21.15.2:0/3946435700 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefe4021040 con 0x55ddcd2855d0 2023-06-23T18:38:22.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.247+0000 7fefe8ff9700 1 -- 172.21.15.2:0/3946435700 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4030370 con 0x55ddcd2855d0 2023-06-23T18:38:22.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 -- 172.21.15.2:0/3946435700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 msgr2=0x55ddcd295a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:22.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 --2- 172.21.15.2:0/3946435700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd295a80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fefe400d5c0 tx=0x7fefe400cf10 comp rx=0 tx=0).stop 2023-06-23T18:38:22.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 -- 172.21.15.2:0/3946435700 shutdown_connections 2023-06-23T18:38:22.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 --2- 172.21.15.2:0/3946435700 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd286000 0x55ddcd28e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 --2- 172.21.15.2:0/3946435700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd284c80 0x55ddcd2963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 --2- 172.21.15.2:0/3946435700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd2855d0 0x55ddcd295a80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 -- 172.21.15.2:0/3946435700 >> 172.21.15.2:0/3946435700 conn(0x55ddcd27a3f0 msgr2=0x55ddcd28d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:22.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 -- 172.21.15.2:0/3946435700 shutdown_connections 2023-06-23T18:38:22.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 -- 172.21.15.2:0/3946435700 wait complete. 2023-06-23T18:38:22.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 -- start start 2023-06-23T18:38:22.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.248+0000 7ff008333b80 1 Processor -- start 2023-06-23T18:38:22.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7ff008333b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd2958a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7ff008333b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd292340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7ff008333b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd286000 0x55ddcd292880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7ff008333b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd292ed0 con 0x55ddcd286000 2023-06-23T18:38:22.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7ff008333b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd2edc60 con 0x55ddcd284c80 2023-06-23T18:38:22.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7ff008333b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddcd2edda0 con 0x55ddcd2855d0 2023-06-23T18:38:22.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7ff008333b80 10 client.0 _ll_register_callbacks cb 0x55ddcd17b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:38:22.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feff9f4a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd2958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feff9f4a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd2958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48228/0 (socket says 172.21.15.2:48228) 2023-06-23T18:38:22.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feff9f4a700 1 -- 172.21.15.2:0/1236175158 learned_addr learned my addr 172.21.15.2:0/1236175158 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:22.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feff9749700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd292340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feffa74b700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd286000 0x55ddcd292880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feffa74b700 1 -- 172.21.15.2:0/1236175158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 msgr2=0x55ddcd292340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feffa74b700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd292340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feffa74b700 1 -- 172.21.15.2:0/1236175158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 msgr2=0x55ddcd2958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.249+0000 7feffa74b700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd2958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7feffa74b700 1 -- 172.21.15.2:0/1236175158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefec0076b0 con 0x55ddcd286000 2023-06-23T18:38:22.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7feff9f4a700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd2958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:38:22.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7feff9749700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd292340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:38:22.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7feffa74b700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd286000 0x55ddcd292880 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x55ddcd27a300 tx=0x7fefec018aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5eb4cb9de4b5b7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:22.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7fefdd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:38:22.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7fefdd7fa700 1 -- 172.21.15.2:0/1236175158 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec010040 con 0x55ddcd286000 2023-06-23T18:38:22.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7fefdd7fa700 1 -- 172.21.15.2:0/1236175158 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fefec004bb0 con 0x55ddcd286000 2023-06-23T18:38:22.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7ff008333b80 1 -- 172.21.15.2:0/1236175158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefe4013040 con 0x55ddcd286000 2023-06-23T18:38:22.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7fefdd7fa700 1 -- 172.21.15.2:0/1236175158 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefec004d70 con 0x55ddcd286000 2023-06-23T18:38:22.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7ff008333b80 10 client.16875 fetch_fsmap learned FSMap version 41 2023-06-23T18:38:22.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7ff008333b80 1 -- 172.21.15.2:0/1236175158 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ddcd057630 con 0x55ddcd286000 2023-06-23T18:38:22.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.250+0000 7fefdd7fa700 1 -- 172.21.15.2:0/1236175158 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fefe4013040 con 0x55ddcd286000 2023-06-23T18:38:22.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7fefdd7fa700 1 -- 172.21.15.2:0/1236175158 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fefec009a20 con 0x55ddcd286000 2023-06-23T18:38:22.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7ff008333b80 10 client.16875 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:38:22.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7ff008333b80 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:38:22.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7ff008333b80 20 client.16875 populate_metadata read hostname 'smithi002' 2023-06-23T18:38:22.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7ff008333b80 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ddcd2ef880 0x55ddcd2f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:22.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7ff008333b80 4 client.16875 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:38:22.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7ff008333b80 1 -- 172.21.15.2:0/1236175158 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ddcd191050 con 0x55ddcd2ef880 2023-06-23T18:38:22.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7feff9f4a700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ddcd2ef880 0x55ddcd2f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:22.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7fefdd7fa700 10 client.16875 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:38:22.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.251+0000 7feff9f4a700 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ddcd2ef880 0x55ddcd2f1c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80edd8c27aae3dde server_cookie=10144c3708dec0c in_seq=0 out_seq=0 2023-06-23T18:38:22.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.252+0000 7fefdd7fa700 1 -- 172.21.15.2:0/1236175158 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ddcd191050 con 0x55ddcd2ef880 2023-06-23T18:38:22.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.252+0000 7fefdd7fa700 10 client.16875 handle_command_reply: tid=0 2023-06-23T18:38:22.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.252+0000 7ff008333b80 1 client.16875 shutdown 2023-06-23T18:38:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.252+0000 7ff008333b80 1 -- 172.21.15.2:0/1236175158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd286000 msgr2=0x55ddcd292880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.252+0000 7ff008333b80 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd286000 0x55ddcd292880 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x55ddcd27a300 tx=0x7fefec018aa0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 1 -- 172.21.15.2:0/1236175158 shutdown_connections 2023-06-23T18:38:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55ddcd2ef880 0x55ddcd2f1c90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55ddcd286000 0x55ddcd292880 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7fefdd7fa700 0 client.16875 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:38:22.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55ddcd284c80 0x55ddcd2958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 1 --2- 172.21.15.2:0/1236175158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ddcd2855d0 0x55ddcd292340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 1 -- 172.21.15.2:0/1236175158 >> 172.21.15.2:0/1236175158 conn(0x55ddcd27a3f0 msgr2=0x55ddcd2963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:22.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 1 -- 172.21.15.2:0/1236175158 shutdown_connections 2023-06-23T18:38:22.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 1 -- 172.21.15.2:0/1236175158 wait complete. 2023-06-23T18:38:22.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7ff008333b80 20 client.16875 trim_cache size 0 max 16384 2023-06-23T18:38:22.265 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:38:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:38:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:38:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:38:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:38:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:38:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:38:22.266 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:38:22.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:38:22.267 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:38:22.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 -- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fefe00a11d0 msgr2=0x7fefe00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:22.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fefe00a11d0 0x7fefe00a3680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fefe400ddd0 tx=0x7fefe400f040 comp rx=0 tx=0).stop 2023-06-23T18:38:22.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 -- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feff40fea90 msgr2=0x7feff419be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:22.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feff40fea90 0x7feff419be00 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feff0005910 tx=0x7feff0005740 comp rx=0 tx=0).stop 2023-06-23T18:38:22.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 -- 172.21.15.2:0/3937225009 shutdown_connections 2023-06-23T18:38:22.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fefe00a11d0 0x7fefe00a3680 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feff4100610 0x7feff41a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feff40fea90 0x7feff419be00 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 --2- 172.21.15.2:0/3937225009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feff40ffc80 0x7feff419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:22.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 -- 172.21.15.2:0/3937225009 >> 172.21.15.2:0/3937225009 conn(0x7feff40fa090 msgr2=0x7feff40fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:22.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.253+0000 7feffc1ae700 1 -- 172.21.15.2:0/3937225009 shutdown_connections 2023-06-23T18:38:22.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:22.254+0000 7feffc1ae700 1 -- 172.21.15.2:0/3937225009 wait complete. 2023-06-23T18:38:22.271 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:38:23.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:22 smithi002 ceph-mon[95947]: pgmap v786: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 12 MiB/s wr, 989 op/s 2023-06-23T18:38:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:22 smithi082 ceph-mon[97889]: pgmap v786: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 12 MiB/s wr, 989 op/s 2023-06-23T18:38:23.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:22 smithi017 ceph-mon[97472]: pgmap v786: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 794 KiB/s rd, 12 MiB/s wr, 989 op/s 2023-06-23T18:38:25.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:24 smithi002 ceph-mon[95947]: pgmap v787: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 12 MiB/s wr, 993 op/s 2023-06-23T18:38:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:24 smithi082 ceph-mon[97889]: pgmap v787: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 12 MiB/s wr, 993 op/s 2023-06-23T18:38:25.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:24 smithi017 ceph-mon[97472]: pgmap v787: 97 pgs: 97 active+clean; 8.4 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 12 MiB/s wr, 993 op/s 2023-06-23T18:38:27.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 275.0 s, 232.4 tps, lat 707.873 ms stddev 2102.654 2023-06-23T18:38:27.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:26 smithi002 ceph-mon[95947]: pgmap v788: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 11 MiB/s wr, 934 op/s 2023-06-23T18:38:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:26 smithi082 ceph-mon[97889]: pgmap v788: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 11 MiB/s wr, 934 op/s 2023-06-23T18:38:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:26 smithi017 ceph-mon[97472]: pgmap v788: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 11 MiB/s wr, 934 op/s 2023-06-23T18:38:29.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:28 smithi002 ceph-mon[95947]: pgmap v789: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 10 MiB/s wr, 894 op/s 2023-06-23T18:38:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:28 smithi082 ceph-mon[97889]: pgmap v789: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 10 MiB/s wr, 894 op/s 2023-06-23T18:38:29.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:28 smithi017 ceph-mon[97472]: pgmap v789: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 10 MiB/s wr, 894 op/s 2023-06-23T18:38:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:30 smithi002 ceph-mon[95947]: pgmap v790: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 9.9 MiB/s wr, 924 op/s 2023-06-23T18:38:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:30 smithi082 ceph-mon[97889]: pgmap v790: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 9.9 MiB/s wr, 924 op/s 2023-06-23T18:38:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:30 smithi017 ceph-mon[97472]: pgmap v790: 97 pgs: 97 active+clean; 8.4 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 9.9 MiB/s wr, 924 op/s 2023-06-23T18:38:32.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 280.0 s, 358.8 tps, lat 276.445 ms stddev 45.900 2023-06-23T18:38:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:32 smithi082 ceph-mon[97889]: pgmap v791: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 10 MiB/s wr, 946 op/s 2023-06-23T18:38:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:32 smithi017 ceph-mon[97472]: pgmap v791: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 10 MiB/s wr, 946 op/s 2023-06-23T18:38:33.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:32 smithi002 ceph-mon[95947]: pgmap v791: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 10 MiB/s wr, 946 op/s 2023-06-23T18:38:35.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:34 smithi082 ceph-mon[97889]: pgmap v792: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:34 smithi017 ceph-mon[97472]: pgmap v792: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:35.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:34 smithi002 ceph-mon[95947]: pgmap v792: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-06-23T18:38:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 285.0 s, 333.6 tps, lat 294.717 ms stddev 83.834 2023-06-23T18:38:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:36 smithi082 ceph-mon[97889]: pgmap v793: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 11 MiB/s wr, 973 op/s 2023-06-23T18:38:37.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:36 smithi017 ceph-mon[97472]: pgmap v793: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 11 MiB/s wr, 973 op/s 2023-06-23T18:38:37.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:36 smithi002 ceph-mon[95947]: pgmap v793: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 11 MiB/s wr, 973 op/s 2023-06-23T18:38:39.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:38 smithi082 ceph-mon[97889]: pgmap v794: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 11 MiB/s wr, 965 op/s 2023-06-23T18:38:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:38 smithi017 ceph-mon[97472]: pgmap v794: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 11 MiB/s wr, 965 op/s 2023-06-23T18:38:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:38 smithi002 ceph-mon[95947]: pgmap v794: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 11 MiB/s wr, 965 op/s 2023-06-23T18:38:41.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:40 smithi082 ceph-mon[97889]: pgmap v795: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:38:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:40 smithi017 ceph-mon[97472]: pgmap v795: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:38:41.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:40 smithi002 ceph-mon[95947]: pgmap v795: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:38:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 290.0 s, 277.6 tps, lat 365.358 ms stddev 101.558 2023-06-23T18:38:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:42 smithi082 ceph-mon[97889]: pgmap v796: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 12 MiB/s wr, 993 op/s 2023-06-23T18:38:43.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:42 smithi017 ceph-mon[97472]: pgmap v796: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 12 MiB/s wr, 993 op/s 2023-06-23T18:38:43.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:42 smithi002 ceph-mon[95947]: pgmap v796: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 955 KiB/s rd, 12 MiB/s wr, 993 op/s 2023-06-23T18:38:45.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:44 smithi082 ceph-mon[97889]: pgmap v797: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 11 MiB/s wr, 932 op/s 2023-06-23T18:38:45.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:44 smithi017 ceph-mon[97472]: pgmap v797: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 11 MiB/s wr, 932 op/s 2023-06-23T18:38:45.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:44 smithi002 ceph-mon[95947]: pgmap v797: 97 pgs: 97 active+clean; 8.4 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 11 MiB/s wr, 932 op/s 2023-06-23T18:38:47.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 295.0 s, 365.8 tps, lat 274.334 ms stddev 44.989 2023-06-23T18:38:47.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:46 smithi082 ceph-mon[97889]: pgmap v798: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 9.1 MiB/s wr, 836 op/s 2023-06-23T18:38:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:46 smithi017 ceph-mon[97472]: pgmap v798: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 9.1 MiB/s wr, 836 op/s 2023-06-23T18:38:47.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:46 smithi002 ceph-mon[95947]: pgmap v798: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 9.1 MiB/s wr, 836 op/s 2023-06-23T18:38:49.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:48 smithi082 ceph-mon[97889]: pgmap v799: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2023-06-23T18:38:49.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:48 smithi017 ceph-mon[97472]: pgmap v799: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2023-06-23T18:38:49.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:48 smithi002 ceph-mon[95947]: pgmap v799: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 9.7 MiB/s wr, 820 op/s 2023-06-23T18:38:51.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:50 smithi082 ceph-mon[97889]: pgmap v800: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 878 op/s 2023-06-23T18:38:51.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:50 smithi017 ceph-mon[97472]: pgmap v800: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 878 op/s 2023-06-23T18:38:51.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:50 smithi002 ceph-mon[95947]: pgmap v800: 97 pgs: 97 active+clean; 8.4 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 878 op/s 2023-06-23T18:38:52.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 300.0 s, 353.0 tps, lat 283.573 ms stddev 38.947 2023-06-23T18:38:52.272 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:38:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 -- 172.21.15.2:0/2001728229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0108850 msgr2=0x7f1ae0108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/2001728229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0108850 0x7f1ae0108c60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8009990 tx=0x7f1ac8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 -- 172.21.15.2:0/2001728229 shutdown_connections 2023-06-23T18:38:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/2001728229 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ae0071c10 0x7f1ae010f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/2001728229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0108850 0x7f1ae0108c60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/2001728229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ae0071260 0x7f1ae00716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 -- 172.21.15.2:0/2001728229 >> 172.21.15.2:0/2001728229 conn(0x7f1ae006cde0 msgr2=0x7f1ae006f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 -- 172.21.15.2:0/2001728229 shutdown_connections 2023-06-23T18:38:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.497+0000 7f1ae51bd700 1 -- 172.21.15.2:0/2001728229 wait complete. 2023-06-23T18:38:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.498+0000 7f1ae51bd700 1 Processor -- start 2023-06-23T18:38:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.498+0000 7f1ae51bd700 1 -- start start 2023-06-23T18:38:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1ae51bd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0071260 0x7f1ae01a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1ae51bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ae0071c10 0x7f1ae01a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1ae51bd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ae0108850 0x7f1ae01abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1ae51bd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae01ac1c0 con 0x7f1ae0071c10 2023-06-23T18:38:52.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1ae51bd700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae01ac300 con 0x7f1ae0108850 2023-06-23T18:38:52.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1ae51bd700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae01ac490 con 0x7f1ae0071260 2023-06-23T18:38:52.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0071260 0x7f1ae01a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0071260 0x7f1ae01a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:57988/0 (socket says 172.21.15.2:57988) 2023-06-23T18:38:52.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1aded9d700 1 -- 172.21.15.2:0/1856252725 learned_addr learned my addr 172.21.15.2:0/1856252725 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:52.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1adf59e700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ae0108850 0x7f1ae01abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1ade59c700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ae0071c10 0x7f1ae01a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1adf59e700 1 -- 172.21.15.2:0/1856252725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0071260 msgr2=0x7f1ae01a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1adf59e700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0071260 0x7f1ae01a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1adf59e700 1 -- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ae0071c10 msgr2=0x7f1ae01a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1adf59e700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ae0071c10 0x7f1ae01a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1adf59e700 1 -- 172.21.15.2:0/1856252725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac80095f0 con 0x7f1ae0108850 2023-06-23T18:38:52.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.499+0000 7f1ade59c700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ae0071c10 0x7f1ae01a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:38:52.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.500+0000 7f1adf59e700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ae0108850 0x7f1ae01abbf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000be20 tx=0x7f1ad0009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:52.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.500+0000 7f1ad7fff700 1 -- 172.21.15.2:0/1856252725 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad000cb90 con 0x7f1ae0108850 2023-06-23T18:38:52.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.500+0000 7f1ad7fff700 1 -- 172.21.15.2:0/1856252725 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ad000ccf0 con 0x7f1ae0108850 2023-06-23T18:38:52.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.500+0000 7f1ad7fff700 1 -- 172.21.15.2:0/1856252725 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00125a0 con 0x7f1ae0108850 2023-06-23T18:38:52.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.500+0000 7f1ae51bd700 1 -- 172.21.15.2:0/1856252725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae01ac770 con 0x7f1ae0108850 2023-06-23T18:38:52.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.500+0000 7f1ae51bd700 1 -- 172.21.15.2:0/1856252725 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae01accb0 con 0x7f1ae0108850 2023-06-23T18:38:52.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.503+0000 7f1ad7fff700 1 -- 172.21.15.2:0/1856252725 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1ad0014440 con 0x7f1ae0108850 2023-06-23T18:38:52.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.504+0000 7f1ad7fff700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1acc0a11d0 0x7f1acc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.504+0000 7f1ad7fff700 1 -- 172.21.15.2:0/1856252725 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1ad0006e40 con 0x7f1ae0108850 2023-06-23T18:38:52.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.504+0000 7f1aded9d700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1acc0a11d0 0x7f1acc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1aded9d700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1acc0a11d0 0x7f1acc0a3680 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800b5c0 tx=0x7f1ac8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:52.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1af1342b80 1 Processor -- start 2023-06-23T18:38:52.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1af1342b80 1 -- start start 2023-06-23T18:38:52.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa05d0 0x562464aa0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa1000 0x562464aa1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1af1342b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624649a4cb0 con 0x562464aa1000 2023-06-23T18:38:52.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1af1342b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624649f7a40 con 0x562464a9fc80 2023-06-23T18:38:52.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1af1342b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624649f0ed0 con 0x562464aa05d0 2023-06-23T18:38:52.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48252/0 (socket says 172.21.15.2:48252) 2023-06-23T18:38:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.505+0000 7f1aded9d700 1 -- 172.21.15.2:0/1457586854 learned_addr learned my addr 172.21.15.2:0/1457586854 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1ade59c700 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa05d0 0x562464aa0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1aded9d700 1 -- 172.21.15.2:0/1457586854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa05d0 msgr2=0x562464aa0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1aded9d700 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa05d0 0x562464aa0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1aded9d700 1 -- 172.21.15.2:0/1457586854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa1000 msgr2=0x562464aa1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:52.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1aded9d700 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa1000 0x562464aa1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1aded9d700 1 -- 172.21.15.2:0/1457586854 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562464872630 con 0x562464a9fc80 2023-06-23T18:38:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1ade59c700 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa05d0 0x562464aa0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:38:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1aded9d700 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa0090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800bcf0 tx=0x7f1ac8020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39428315164e611a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1ad57fa700 1 -- 172.21.15.2:0/1457586854 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac802b070 con 0x562464a9fc80 2023-06-23T18:38:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1ad57fa700 1 -- 172.21.15.2:0/1457586854 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ac801e040 con 0x562464a9fc80 2023-06-23T18:38:52.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.506+0000 7f1ad57fa700 1 -- 172.21.15.2:0/1457586854 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8003880 con 0x562464a9fc80 2023-06-23T18:38:52.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.507+0000 7f1af1342b80 1 -- 172.21.15.2:0/1457586854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 msgr2=0x562464aa0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.507+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa0090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800bcf0 tx=0x7f1ac8020000 comp rx=0 tx=0).stop 2023-06-23T18:38:52.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.508+0000 7f1af1342b80 1 -- 172.21.15.2:0/1457586854 shutdown_connections 2023-06-23T18:38:52.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.508+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa1000 0x562464aa1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.508+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa0090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.508+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1457586854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa05d0 0x562464aa0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.508+0000 7f1af1342b80 1 -- 172.21.15.2:0/1457586854 >> 172.21.15.2:0/1457586854 conn(0x562464a953f0 msgr2=0x562464a9a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:52.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.508+0000 7f1af1342b80 1 -- 172.21.15.2:0/1457586854 shutdown_connections 2023-06-23T18:38:52.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.508+0000 7f1af1342b80 1 -- 172.21.15.2:0/1457586854 wait complete. 2023-06-23T18:38:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 1 -- start start 2023-06-23T18:38:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 1 Processor -- start 2023-06-23T18:38:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aa9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464ab0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562464990ac0 con 0x562464aa05d0 2023-06-23T18:38:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624649907c0 con 0x562464a9fc80 2023-06-23T18:38:52.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562464874920 con 0x562464aa1000 2023-06-23T18:38:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1af1342b80 10 client.0 _ll_register_callbacks cb 0x562464996260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:38:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1ade59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aa9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1ade59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aa9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57666/0 (socket says 172.21.15.2:57666) 2023-06-23T18:38:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1ade59c700 1 -- 172.21.15.2:0/1906668350 learned_addr learned my addr 172.21.15.2:0/1906668350 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.509+0000 7f1aded9d700 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1adf59e700 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464ab0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ade59c700 1 -- 172.21.15.2:0/1906668350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 msgr2=0x562464ab0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ade59c700 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464ab0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ade59c700 1 -- 172.21.15.2:0/1906668350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 msgr2=0x562464aa9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ade59c700 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ade59c700 1 -- 172.21.15.2:0/1906668350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad800ad10 con 0x562464aa05d0 2023-06-23T18:38:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ade59c700 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aa9a60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8010e90 tx=0x7f1ad800f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38719aad72fe3bd9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ac67fc700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:38:52.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ac67fc700 1 -- 172.21.15.2:0/1906668350 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8015070 con 0x562464aa05d0 2023-06-23T18:38:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1ac67fc700 1 -- 172.21.15.2:0/1906668350 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1ad8004830 con 0x562464aa05d0 2023-06-23T18:38:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.510+0000 7f1af1342b80 1 -- 172.21.15.2:0/1906668350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562464a92bb0 con 0x562464aa05d0 2023-06-23T18:38:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1ac67fc700 1 -- 172.21.15.2:0/1906668350 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ad800feb0 con 0x562464aa05d0 2023-06-23T18:38:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1af1342b80 10 client.16881 fetch_fsmap learned FSMap version 41 2023-06-23T18:38:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1af1342b80 1 -- 172.21.15.2:0/1906668350 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f1ae01074d0 con 0x562464aa05d0 2023-06-23T18:38:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1ac67fc700 1 -- 172.21.15.2:0/1906668350 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562464a92bb0 con 0x562464aa05d0 2023-06-23T18:38:52.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1ac67fc700 1 -- 172.21.15.2:0/1906668350 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ad8019410 con 0x562464aa05d0 2023-06-23T18:38:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1af1342b80 10 client.16881 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:38:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1af1342b80 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:38:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1af1342b80 20 client.16881 populate_metadata read hostname 'smithi002' 2023-06-23T18:38:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562464ab5360 0x562464ab7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1af1342b80 4 client.16881 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:38:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1af1342b80 1 -- 172.21.15.2:0/1906668350 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624649e9eb0 con 0x562464ab5360 2023-06-23T18:38:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.511+0000 7f1aded9d700 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562464ab5360 0x562464ab7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.512+0000 7f1ac67fc700 10 client.16881 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:38:52.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.512+0000 7f1aded9d700 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562464ab5360 0x562464ab7770 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3426fb5c85e5b708 server_cookie=ecc9886e9315c076 in_seq=0 out_seq=0 2023-06-23T18:38:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1ac67fc700 1 -- 172.21.15.2:0/1906668350 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5624649e9eb0 con 0x562464ab5360 2023-06-23T18:38:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1ac67fc700 10 client.16881 handle_command_reply: tid=0 2023-06-23T18:38:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 client.16881 shutdown 2023-06-23T18:38:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 -- 172.21.15.2:0/1906668350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 msgr2=0x562464aa9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aa9a60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8010e90 tx=0x7f1ad800f700 comp rx=0 tx=0).stop 2023-06-23T18:38:52.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 -- 172.21.15.2:0/1906668350 shutdown_connections 2023-06-23T18:38:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562464ab5360 0x562464ab7770 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464ab0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 --2- 172.21.15.2:0/1906668350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aa9a60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1ac67fc700 0 client.16881 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:38:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 -- 172.21.15.2:0/1906668350 >> 172.21.15.2:0/1906668350 conn(0x562464a953f0 msgr2=0x562464a99c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 -- 172.21.15.2:0/1906668350 shutdown_connections 2023-06-23T18:38:52.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 1 -- 172.21.15.2:0/1906668350 wait complete. 2023-06-23T18:38:52.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.513+0000 7f1af1342b80 20 client.16881 trim_cache size 0 max 16384 2023-06-23T18:38:52.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1af1342b80 1 Processor -- start 2023-06-23T18:38:52.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1af1342b80 1 -- start start 2023-06-23T18:38:52.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464ab0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464aa81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1af1342b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562464aa89d0 con 0x562464aa05d0 2023-06-23T18:38:52.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1af1342b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562464aa9090 con 0x562464a9fc80 2023-06-23T18:38:52.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1af1342b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562464aa7160 con 0x562464aa1000 2023-06-23T18:38:52.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48264/0 (socket says 172.21.15.2:48264) 2023-06-23T18:38:52.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1aded9d700 1 -- 172.21.15.2:0/2930819467 learned_addr learned my addr 172.21.15.2:0/2930819467 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:52.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.537+0000 7f1adf59e700 1 --2- 172.21.15.2:0/2930819467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464aa81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1aded9d700 1 -- 172.21.15.2:0/2930819467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 msgr2=0x562464aa81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:52.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1aded9d700 1 --2- 172.21.15.2:0/2930819467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464aa81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1aded9d700 1 -- 172.21.15.2:0/2930819467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 msgr2=0x562464ab0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:52.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1aded9d700 1 --2- 172.21.15.2:0/2930819467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464ab0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1aded9d700 1 -- 172.21.15.2:0/2930819467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56246481ede0 con 0x562464a9fc80 2023-06-23T18:38:52.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1aded9d700 1 --2- 172.21.15.2:0/2930819467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa1450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800b940 tx=0x7f1ac8004fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da1e51f990b9ced8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:52.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1ad5ffb700 1 -- 172.21.15.2:0/2930819467 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac802b040 con 0x562464a9fc80 2023-06-23T18:38:52.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1ad5ffb700 1 -- 172.21.15.2:0/2930819467 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ac803f070 con 0x562464a9fc80 2023-06-23T18:38:52.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.538+0000 7f1ad5ffb700 1 -- 172.21.15.2:0/2930819467 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8004930 con 0x562464a9fc80 2023-06-23T18:38:52.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 -- 172.21.15.2:0/2930819467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 msgr2=0x562464aa1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2930819467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa1450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800b940 tx=0x7f1ac8004fd0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 -- 172.21.15.2:0/2930819467 shutdown_connections 2023-06-23T18:38:52.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2930819467 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464aa81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2930819467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aa1450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2930819467 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464ab0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 -- 172.21.15.2:0/2930819467 >> 172.21.15.2:0/2930819467 conn(0x562464a953f0 msgr2=0x562464abc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:52.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 -- 172.21.15.2:0/2930819467 shutdown_connections 2023-06-23T18:38:52.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.539+0000 7f1af1342b80 1 -- 172.21.15.2:0/2930819467 wait complete. 2023-06-23T18:38:52.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 1 -- start start 2023-06-23T18:38:52.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 1 Processor -- start 2023-06-23T18:38:52.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aacbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464aafbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562464ab0120 con 0x562464aa05d0 2023-06-23T18:38:52.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562464ab0290 con 0x562464a9fc80 2023-06-23T18:38:52.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562464ab0450 con 0x562464aa1000 2023-06-23T18:38:52.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1af1342b80 10 client.0 _ll_register_callbacks cb 0x562464996260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:38:52.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1ade59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aacbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1adf59e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464aafbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1ade59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aacbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57680/0 (socket says 172.21.15.2:57680) 2023-06-23T18:38:52.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1ade59c700 1 -- 172.21.15.2:0/2843119586 learned_addr learned my addr 172.21.15.2:0/2843119586 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:38:52.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.540+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48270/0 (socket says 172.21.15.2:48270) 2023-06-23T18:38:52.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ade59c700 1 -- 172.21.15.2:0/2843119586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 msgr2=0x562464aafbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:52.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ade59c700 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464aafbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ade59c700 1 -- 172.21.15.2:0/2843119586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 msgr2=0x562464aac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:38:52.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ade59c700 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ade59c700 1 -- 172.21.15.2:0/2843119586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562464872630 con 0x562464aa05d0 2023-06-23T18:38:52.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ade59c700 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aacbc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80197c0 tx=0x7f1ad800fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ec50f021bde605 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:38:52.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ac67fc700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:38:52.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ac67fc700 1 -- 172.21.15.2:0/2843119586 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad8015070 con 0x562464aa05d0 2023-06-23T18:38:52.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1ac67fc700 1 -- 172.21.15.2:0/2843119586 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1ad8000e40 con 0x562464aa05d0 2023-06-23T18:38:52.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.541+0000 7f1af1342b80 1 -- 172.21.15.2:0/2843119586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ac801e040 con 0x562464aa05d0 2023-06-23T18:38:52.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1ac67fc700 1 -- 172.21.15.2:0/2843119586 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ad8019be0 con 0x562464aa05d0 2023-06-23T18:38:52.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1af1342b80 10 client.16884 fetch_fsmap learned FSMap version 41 2023-06-23T18:38:52.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1af1342b80 1 -- 172.21.15.2:0/2843119586 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562464807c60 con 0x562464aa05d0 2023-06-23T18:38:52.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1ac67fc700 1 -- 172.21.15.2:0/2843119586 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ac801e040 con 0x562464aa05d0 2023-06-23T18:38:52.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1ac67fc700 1 -- 172.21.15.2:0/2843119586 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1ad800e700 con 0x562464aa05d0 2023-06-23T18:38:52.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1af1342b80 10 client.16884 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:38:52.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1af1342b80 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:38:52.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1af1342b80 20 client.16884 populate_metadata read hostname 'smithi002' 2023-06-23T18:38:52.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562464b0a010 0x562464b0c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:38:52.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1af1342b80 4 client.16884 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:38:52.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.542+0000 7f1af1342b80 1 -- 172.21.15.2:0/2843119586 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5624649ac050 con 0x562464b0a010 2023-06-23T18:38:52.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1aded9d700 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562464b0a010 0x562464b0c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:38:52.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1ac67fc700 10 client.16884 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:38:52.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1aded9d700 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562464b0a010 0x562464b0c420 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3c17f93fa71c67 server_cookie=1c59888d02c1fe7d in_seq=0 out_seq=0 2023-06-23T18:38:52.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1ac67fc700 1 -- 172.21.15.2:0/2843119586 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5624649ac050 con 0x562464b0a010 2023-06-23T18:38:52.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1ac67fc700 10 client.16884 handle_command_reply: tid=0 2023-06-23T18:38:52.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 client.16884 shutdown 2023-06-23T18:38:52.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 -- 172.21.15.2:0/2843119586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 msgr2=0x562464aacbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aacbc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1ad80197c0 tx=0x7f1ad800fac0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 -- 172.21.15.2:0/2843119586 shutdown_connections 2023-06-23T18:38:52.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562464b0a010 0x562464b0c420 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562464aa1000 0x562464aafbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562464a9fc80 0x562464aac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1ac67fc700 0 client.16884 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:38:52.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 --2- 172.21.15.2:0/2843119586 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562464aa05d0 0x562464aacbc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 -- 172.21.15.2:0/2843119586 >> 172.21.15.2:0/2843119586 conn(0x562464a953f0 msgr2=0x562464aa81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:52.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.543+0000 7f1af1342b80 1 -- 172.21.15.2:0/2843119586 shutdown_connections 2023-06-23T18:38:52.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1af1342b80 1 -- 172.21.15.2:0/2843119586 wait complete. 2023-06-23T18:38:52.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1af1342b80 20 client.16884 trim_cache size 0 max 16384 2023-06-23T18:38:52.559 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:38:52.559 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:38:52.560 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:38:52.560 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:38:52.560 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:38:52.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:38:52.560 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:38:52.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:38:52.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:38:52.561 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:38:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 -- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1acc0a11d0 msgr2=0x7f1acc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1acc0a11d0 0x7f1acc0a3680 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800b5c0 tx=0x7f1ac8005fb0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 -- 172.21.15.2:0/1856252725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ae0108850 msgr2=0x7f1ae01abbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:38:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ae0108850 0x7f1ae01abbf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ad000be20 tx=0x7f1ad0009ed0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 -- 172.21.15.2:0/1856252725 shutdown_connections 2023-06-23T18:38:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f1acc0a11d0 0x7f1acc0a3680 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ae0108850 0x7f1ae01abbf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ae0071260 0x7f1ae01a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 --2- 172.21.15.2:0/1856252725 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1ae0071c10 0x7f1ae01a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:38:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 -- 172.21.15.2:0/1856252725 >> 172.21.15.2:0/1856252725 conn(0x7f1ae006cde0 msgr2=0x7f1ae006f010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:38:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 -- 172.21.15.2:0/1856252725 shutdown_connections 2023-06-23T18:38:52.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:38:52.544+0000 7f1ae51bd700 1 -- 172.21.15.2:0/1856252725 wait complete. 2023-06-23T18:38:52.565 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:38:53.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:52 smithi082 ceph-mon[97889]: pgmap v801: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 915 op/s 2023-06-23T18:38:53.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:52 smithi017 ceph-mon[97472]: pgmap v801: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 915 op/s 2023-06-23T18:38:53.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:52 smithi002 ceph-mon[95947]: pgmap v801: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 915 op/s 2023-06-23T18:38:55.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:54 smithi082 ceph-mon[97889]: pgmap v802: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 974 op/s 2023-06-23T18:38:55.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:54 smithi017 ceph-mon[97472]: pgmap v802: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 974 op/s 2023-06-23T18:38:55.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:54 smithi002 ceph-mon[95947]: pgmap v802: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 974 op/s 2023-06-23T18:38:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 305.0 s, 358.6 tps, lat 278.905 ms stddev 39.370 2023-06-23T18:38:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:56 smithi082 ceph-mon[97889]: pgmap v803: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 965 op/s 2023-06-23T18:38:57.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:56 smithi017 ceph-mon[97472]: pgmap v803: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 965 op/s 2023-06-23T18:38:57.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:56 smithi002 ceph-mon[95947]: pgmap v803: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 965 op/s 2023-06-23T18:38:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:38:58 smithi082 ceph-mon[97889]: pgmap v804: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 964 op/s 2023-06-23T18:38:59.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:38:58 smithi017 ceph-mon[97472]: pgmap v804: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 964 op/s 2023-06-23T18:38:59.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:38:58 smithi002 ceph-mon[95947]: pgmap v804: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 964 op/s 2023-06-23T18:39:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:00 smithi082 ceph-mon[97889]: pgmap v805: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-06-23T18:39:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:00 smithi017 ceph-mon[97472]: pgmap v805: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-06-23T18:39:01.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:00 smithi002 ceph-mon[95947]: pgmap v805: 97 pgs: 97 active+clean; 8.4 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-06-23T18:39:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 310.0 s, 356.4 tps, lat 281.012 ms stddev 48.408 2023-06-23T18:39:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:02 smithi082 ceph-mon[97889]: pgmap v806: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:02 smithi017 ceph-mon[97472]: pgmap v806: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:03.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:02 smithi002 ceph-mon[95947]: pgmap v806: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:05.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:04 smithi082 ceph-mon[97889]: pgmap v807: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:05.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:04 smithi017 ceph-mon[97472]: pgmap v807: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:05.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:04 smithi002 ceph-mon[95947]: pgmap v807: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 315.0 s, 365.0 tps, lat 273.271 ms stddev 39.977 2023-06-23T18:39:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:06 smithi017 ceph-mon[97472]: pgmap v808: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 980 op/s 2023-06-23T18:39:07.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:06 smithi002 ceph-mon[95947]: pgmap v808: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 980 op/s 2023-06-23T18:39:07.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:06 smithi082 ceph-mon[97889]: pgmap v808: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 980 op/s 2023-06-23T18:39:09.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:08 smithi017 ceph-mon[97472]: pgmap v809: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 986 op/s 2023-06-23T18:39:09.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:08 smithi002 ceph-mon[95947]: pgmap v809: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 986 op/s 2023-06-23T18:39:09.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:08 smithi082 ceph-mon[97889]: pgmap v809: 97 pgs: 97 active+clean; 8.4 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 12 MiB/s wr, 986 op/s 2023-06-23T18:39:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:10 smithi017 ceph-mon[97472]: pgmap v810: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-06-23T18:39:11.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:10 smithi002 ceph-mon[95947]: pgmap v810: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-06-23T18:39:11.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:10 smithi082 ceph-mon[97889]: pgmap v810: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-06-23T18:39:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 320.0 s, 330.4 tps, lat 300.930 ms stddev 194.030 2023-06-23T18:39:13.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:12 smithi017 ceph-mon[97472]: pgmap v811: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-06-23T18:39:13.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:12 smithi002 ceph-mon[95947]: pgmap v811: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-06-23T18:39:13.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:12 smithi082 ceph-mon[97889]: pgmap v811: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 13 MiB/s wr, 1.06k op/s 2023-06-23T18:39:15.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:14 smithi017 ceph-mon[97472]: pgmap v812: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:39:15.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:14 smithi002 ceph-mon[95947]: pgmap v812: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:39:15.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:14 smithi082 ceph-mon[97889]: pgmap v812: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:39:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 325.0 s, 368.4 tps, lat 273.110 ms stddev 46.747 2023-06-23T18:39:17.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:16 smithi017 ceph-mon[97472]: pgmap v813: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 974 op/s 2023-06-23T18:39:17.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:16 smithi002 ceph-mon[95947]: pgmap v813: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 974 op/s 2023-06-23T18:39:17.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:16 smithi082 ceph-mon[97889]: pgmap v813: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 974 op/s 2023-06-23T18:39:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:18 smithi017 ceph-mon[97472]: pgmap v814: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 986 op/s 2023-06-23T18:39:19.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:18 smithi002 ceph-mon[95947]: pgmap v814: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 986 op/s 2023-06-23T18:39:19.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:18 smithi082 ceph-mon[97889]: pgmap v814: 97 pgs: 97 active+clean; 8.4 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 986 op/s 2023-06-23T18:39:20.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:19 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:39:20.303 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:19 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:39:20.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:19 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:39:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:20 smithi017 ceph-mon[97472]: pgmap v815: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:39:21.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:20 smithi002 ceph-mon[95947]: pgmap v815: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:39:21.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:20 smithi082 ceph-mon[97889]: pgmap v815: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 1.05k op/s 2023-06-23T18:39:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 330.0 s, 371.6 tps, lat 268.842 ms stddev 41.385 2023-06-23T18:39:22.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:21 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:39:22.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:21 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:39:22.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:21 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:39:22.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:21 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:39:22.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:21 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:39:22.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:21 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:39:22.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:21 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:39:22.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:21 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:39:22.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:21 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:39:22.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:39:22.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.818+0000 7fac3a7fc700 1 -- 172.21.15.2:0/3683924609 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30003aa0 con 0x7fac3c102f20 2023-06-23T18:39:22.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.819+0000 7fac41ab4700 1 -- 172.21.15.2:0/3683924609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c102f20 msgr2=0x7fac3c105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.819+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3683924609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c102f20 0x7fac3c105300 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fac30009990 tx=0x7fac30009ca0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.820+0000 7fac41ab4700 1 -- 172.21.15.2:0/3683924609 shutdown_connections 2023-06-23T18:39:22.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.820+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3683924609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac3c105840 0x7fac3c107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.820+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3683924609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac3c100600 0x7fac3c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.820+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3683924609 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c102f20 0x7fac3c105300 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.820+0000 7fac41ab4700 1 -- 172.21.15.2:0/3683924609 >> 172.21.15.2:0/3683924609 conn(0x7fac3c0fa0e0 msgr2=0x7fac3c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:22.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.820+0000 7fac41ab4700 1 -- 172.21.15.2:0/3683924609 shutdown_connections 2023-06-23T18:39:22.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.820+0000 7fac41ab4700 1 -- 172.21.15.2:0/3683924609 wait complete. 2023-06-23T18:39:22.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac41ab4700 1 Processor -- start 2023-06-23T18:39:22.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac41ab4700 1 -- start start 2023-06-23T18:39:22.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac41ab4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c100600 0x7fac3c0fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac41ab4700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac3c102f20 0x7fac3c0fe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac41ab4700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac3c105840 0x7fac3c0feaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac41ab4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c0ff290 con 0x7fac3c100600 2023-06-23T18:39:22.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac41ab4700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c0ff400 con 0x7fac3c102f20 2023-06-23T18:39:22.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac41ab4700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3c19f840 con 0x7fac3c105840 2023-06-23T18:39:22.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c100600 0x7fac3c0fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.821+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c100600 0x7fac3c0fff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57692/0 (socket says 172.21.15.2:57692) 2023-06-23T18:39:22.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3b7fe700 1 -- 172.21.15.2:0/3322548687 learned_addr learned my addr 172.21.15.2:0/3322548687 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:22.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3bfff700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac3c105840 0x7fac3c0feaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3affd700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac3c102f20 0x7fac3c0fe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3bfff700 1 -- 172.21.15.2:0/3322548687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac3c102f20 msgr2=0x7fac3c0fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3bfff700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac3c102f20 0x7fac3c0fe5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3bfff700 1 -- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c100600 msgr2=0x7fac3c0fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3bfff700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c100600 0x7fac3c0fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3bfff700 1 -- 172.21.15.2:0/3322548687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac300095f0 con 0x7fac3c105840 2023-06-23T18:39:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c100600 0x7fac3c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:39:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.822+0000 7fac3bfff700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac3c105840 0x7fac3c0feaf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac3400be20 tx=0x7fac34009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:22.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.823+0000 7fac38ff9700 1 -- 172.21.15.2:0/3322548687 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3400cb90 con 0x7fac3c105840 2023-06-23T18:39:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.823+0000 7fac38ff9700 1 -- 172.21.15.2:0/3322548687 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac3400ccf0 con 0x7fac3c105840 2023-06-23T18:39:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.823+0000 7fac38ff9700 1 -- 172.21.15.2:0/3322548687 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340125a0 con 0x7fac3c105840 2023-06-23T18:39:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.823+0000 7fac41ab4700 1 -- 172.21.15.2:0/3322548687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac3c19fb20 con 0x7fac3c105840 2023-06-23T18:39:22.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.823+0000 7fac41ab4700 1 -- 172.21.15.2:0/3322548687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c1956a0 con 0x7fac3c105840 2023-06-23T18:39:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.826+0000 7fac38ff9700 1 -- 172.21.15.2:0/3322548687 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fac34014440 con 0x7fac3c105840 2023-06-23T18:39:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.826+0000 7fac38ff9700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fac240a12a0 0x7fac240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.826+0000 7fac38ff9700 1 -- 172.21.15.2:0/3322548687 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fac34006e40 con 0x7fac3c105840 2023-06-23T18:39:22.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.827+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fac240a12a0 0x7fac240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.827+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fac240a12a0 0x7fac240a3750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac2c005950 tx=0x7fac2c00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.827+0000 7fac4dc39b80 1 Processor -- start 2023-06-23T18:39:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.827+0000 7fac4dc39b80 1 -- start start 2023-06-23T18:39:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d22090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d22a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d23470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6c26cb0 con 0x562af6d21c80 2023-06-23T18:39:22.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6c79a40 con 0x562af6d23000 2023-06-23T18:39:22.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6c72ed0 con 0x562af6d225d0 2023-06-23T18:39:22.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d22090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d22090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57700/0 (socket says 172.21.15.2:57700) 2023-06-23T18:39:22.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3b7fe700 1 -- 172.21.15.2:0/3862481301 learned_addr learned my addr 172.21.15.2:0/3862481301 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:22.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3bfff700 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d23470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3affd700 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d22a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3b7fe700 1 -- 172.21.15.2:0/3862481301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 msgr2=0x562af6d22a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:22.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d22a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3b7fe700 1 -- 172.21.15.2:0/3862481301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 msgr2=0x562af6d23470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:22.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d23470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3b7fe700 1 -- 172.21.15.2:0/3862481301 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af6af4630 con 0x562af6d21c80 2023-06-23T18:39:22.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.828+0000 7fac3bfff700 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d23470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:39:22.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.829+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d22090 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0117d0 tx=0x7fac2c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5712a575b778a1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:22.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.829+0000 7fac21ffb700 1 -- 172.21.15.2:0/3862481301 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c004220 con 0x562af6d21c80 2023-06-23T18:39:22.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.829+0000 7fac21ffb700 1 -- 172.21.15.2:0/3862481301 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac2c015070 con 0x562af6d21c80 2023-06-23T18:39:22.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 -- 172.21.15.2:0/3862481301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 msgr2=0x562af6d22090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d22090 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fac2c0117d0 tx=0x7fac2c013000 comp rx=0 tx=0).stop 2023-06-23T18:39:22.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 -- 172.21.15.2:0/3862481301 shutdown_connections 2023-06-23T18:39:22.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d23470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d22090 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/3862481301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d22a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 -- 172.21.15.2:0/3862481301 >> 172.21.15.2:0/3862481301 conn(0x562af6d173f0 msgr2=0x562af6d1c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:22.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 -- 172.21.15.2:0/3862481301 shutdown_connections 2023-06-23T18:39:22.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.830+0000 7fac4dc39b80 1 -- 172.21.15.2:0/3862481301 wait complete. 2023-06-23T18:39:22.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 1 -- start start 2023-06-23T18:39:22.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 1 Processor -- start 2023-06-23T18:39:22.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d2bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d32b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6c12ac0 con 0x562af6d21c80 2023-06-23T18:39:22.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6c127c0 con 0x562af6d23000 2023-06-23T18:39:22.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6af6920 con 0x562af6d225d0 2023-06-23T18:39:22.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac4dc39b80 10 client.0 _ll_register_callbacks cb 0x562af6c18260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:39:22.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57706/0 (socket says 172.21.15.2:57706) 2023-06-23T18:39:22.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac3b7fe700 1 -- 172.21.15.2:0/721386154 learned_addr learned my addr 172.21.15.2:0/721386154 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:22.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.831+0000 7fac3bfff700 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d32b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.832+0000 7fac3b7fe700 1 -- 172.21.15.2:0/721386154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 msgr2=0x562af6d2bb50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:39:22.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.832+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d2bb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.832+0000 7fac3b7fe700 1 -- 172.21.15.2:0/721386154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 msgr2=0x562af6d32b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:22.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.832+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d32b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.832+0000 7fac3b7fe700 1 -- 172.21.15.2:0/721386154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c1956a0 con 0x562af6d21c80 2023-06-23T18:39:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.832+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2b610 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fac2c004470 tx=0x7fac2c0044a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d6261ff96bcc8a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.832+0000 7fac1affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:39:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.833+0000 7fac1affd700 1 -- 172.21.15.2:0/721386154 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c003c50 con 0x562af6d21c80 2023-06-23T18:39:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.833+0000 7fac1affd700 1 -- 172.21.15.2:0/721386154 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fac2c017070 con 0x562af6d21c80 2023-06-23T18:39:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.833+0000 7fac4dc39b80 1 -- 172.21.15.2:0/721386154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562af6d14bb0 con 0x562af6d21c80 2023-06-23T18:39:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.833+0000 7fac1affd700 1 -- 172.21.15.2:0/721386154 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac2c026440 con 0x562af6d21c80 2023-06-23T18:39:22.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.833+0000 7fac4dc39b80 10 client.16896 fetch_fsmap learned FSMap version 41 2023-06-23T18:39:22.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.833+0000 7fac4dc39b80 1 -- 172.21.15.2:0/721386154 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fac2c011480 con 0x562af6d21c80 2023-06-23T18:39:22.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.833+0000 7fac1affd700 1 -- 172.21.15.2:0/721386154 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562af6d14bb0 con 0x562af6d21c80 2023-06-23T18:39:22.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac1affd700 1 -- 172.21.15.2:0/721386154 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac2c024070 con 0x562af6d21c80 2023-06-23T18:39:22.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac4dc39b80 10 client.16896 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:39:22.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac4dc39b80 10 client.16896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:39:22.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac4dc39b80 20 client.16896 populate_metadata read hostname 'smithi002' 2023-06-23T18:39:22.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562af6d37310 0x562af6d39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac4dc39b80 4 client.16896 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:39:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac4dc39b80 1 -- 172.21.15.2:0/721386154 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562af6c6beb0 con 0x562af6d37310 2023-06-23T18:39:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac3affd700 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562af6d37310 0x562af6d39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac1affd700 10 client.16896 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:39:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.834+0000 7fac3affd700 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562af6d37310 0x562af6d39720 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894f88b8e3f688de server_cookie=2e4491cff67048e2 in_seq=0 out_seq=0 2023-06-23T18:39:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.835+0000 7fac1affd700 1 -- 172.21.15.2:0/721386154 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562af6c6beb0 con 0x562af6d37310 2023-06-23T18:39:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.835+0000 7fac1affd700 10 client.16896 handle_command_reply: tid=0 2023-06-23T18:39:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.835+0000 7fac4dc39b80 1 client.16896 shutdown 2023-06-23T18:39:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.835+0000 7fac4dc39b80 1 -- 172.21.15.2:0/721386154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 msgr2=0x562af6d2b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.835+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2b610 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fac2c004470 tx=0x7fac2c0044a0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 1 -- 172.21.15.2:0/721386154 shutdown_connections 2023-06-23T18:39:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562af6d37310 0x562af6d39720 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d32b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2b610 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/721386154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d2bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 1 -- 172.21.15.2:0/721386154 >> 172.21.15.2:0/721386154 conn(0x562af6d173f0 msgr2=0x562af6d1bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac1affd700 0 client.16896 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:39:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 1 -- 172.21.15.2:0/721386154 shutdown_connections 2023-06-23T18:39:22.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 1 -- 172.21.15.2:0/721386154 wait complete. 2023-06-23T18:39:22.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.836+0000 7fac4dc39b80 20 client.16896 trim_cache size 0 max 16384 2023-06-23T18:39:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.864+0000 7fac4dc39b80 1 Processor -- start 2023-06-23T18:39:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.864+0000 7fac4dc39b80 1 -- start start 2023-06-23T18:39:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.864+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d21c80 0x562af6d29390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.864+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d32b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.864+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d23000 0x562af6d25ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.864+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6d2b1e0 con 0x562af6d23000 2023-06-23T18:39:22.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.864+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6d2aac0 con 0x562af6d21c80 2023-06-23T18:39:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.864+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6d1d420 con 0x562af6d225d0 2023-06-23T18:39:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d32b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d32b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58050/0 (socket says 172.21.15.2:58050) 2023-06-23T18:39:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d23000 0x562af6d25ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 -- 172.21.15.2:0/1737304002 learned_addr learned my addr 172.21.15.2:0/1737304002 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 -- 172.21.15.2:0/1737304002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d21c80 msgr2=0x562af6d29390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:39:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 --2- 172.21.15.2:0/1737304002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d21c80 0x562af6d29390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 -- 172.21.15.2:0/1737304002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d23000 msgr2=0x562af6d25ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:22.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 --2- 172.21.15.2:0/1737304002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d23000 0x562af6d25ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 -- 172.21.15.2:0/1737304002 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562af6aa0de0 con 0x562af6d225d0 2023-06-23T18:39:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac3affd700 1 --2- 172.21.15.2:0/1737304002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d32b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac30009990 tx=0x7fac3000b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b10f189467f3db3c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac227fc700 1 -- 172.21.15.2:0/1737304002 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3001d040 con 0x562af6d225d0 2023-06-23T18:39:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac227fc700 1 -- 172.21.15.2:0/1737304002 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac3001e040 con 0x562af6d225d0 2023-06-23T18:39:22.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.865+0000 7fac227fc700 1 -- 172.21.15.2:0/1737304002 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30005040 con 0x562af6d225d0 2023-06-23T18:39:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.866+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1737304002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 msgr2=0x562af6d32b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.866+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1737304002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d32b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fac30009990 tx=0x7fac3000b670 comp rx=0 tx=0).stop 2023-06-23T18:39:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.866+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1737304002 shutdown_connections 2023-06-23T18:39:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.866+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1737304002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d23000 0x562af6d25ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.866+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1737304002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d21c80 0x562af6d29390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.866+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1737304002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d32b50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.866+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1737304002 >> 172.21.15.2:0/1737304002 conn(0x562af6d173f0 msgr2=0x562af6d2a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.867+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1737304002 shutdown_connections 2023-06-23T18:39:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.867+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1737304002 wait complete. 2023-06-23T18:39:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.867+0000 7fac4dc39b80 1 -- start start 2023-06-23T18:39:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.867+0000 7fac4dc39b80 1 Processor -- start 2023-06-23T18:39:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d2ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac4dc39b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6d323f0 con 0x562af6d21c80 2023-06-23T18:39:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6d32560 con 0x562af6d23000 2023-06-23T18:39:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac4dc39b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562af6d32720 con 0x562af6d225d0 2023-06-23T18:39:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac4dc39b80 10 client.0 _ll_register_callbacks cb 0x562af6c18260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:39:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac3b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac3affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d2ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac3affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d2ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58054/0 (socket says 172.21.15.2:58054) 2023-06-23T18:39:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac3affd700 1 -- 172.21.15.2:0/1868895119 learned_addr learned my addr 172.21.15.2:0/1868895119 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.868+0000 7fac3bfff700 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac3b7fe700 1 -- 172.21.15.2:0/1868895119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 msgr2=0x562af6d2ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d2ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac3b7fe700 1 -- 172.21.15.2:0/1868895119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 msgr2=0x562af6d31eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d31eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac3b7fe700 1 -- 172.21.15.2:0/1868895119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac3c108890 con 0x562af6d21c80 2023-06-23T18:39:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac3b7fe700 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2e950 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fac2c011dc0 tx=0x7fac2c01e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa629532d2e0ec1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac1affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:39:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac1affd700 1 -- 172.21.15.2:0/1868895119 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2c01ec80 con 0x562af6d21c80 2023-06-23T18:39:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1868895119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac2c015070 con 0x562af6d21c80 2023-06-23T18:39:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac1affd700 1 -- 172.21.15.2:0/1868895119 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fac2c017070 con 0x562af6d21c80 2023-06-23T18:39:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac1affd700 1 -- 172.21.15.2:0/1868895119 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac2c003c50 con 0x562af6d21c80 2023-06-23T18:39:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac4dc39b80 10 client.16902 fetch_fsmap learned FSMap version 41 2023-06-23T18:39:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.869+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1868895119 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x562af6a89c60 con 0x562af6d21c80 2023-06-23T18:39:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac1affd700 1 -- 172.21.15.2:0/1868895119 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fac2c015070 con 0x562af6d21c80 2023-06-23T18:39:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac1affd700 1 -- 172.21.15.2:0/1868895119 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fac2c011ae0 con 0x562af6d21c80 2023-06-23T18:39:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac4dc39b80 10 client.16902 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:39:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac4dc39b80 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:39:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac4dc39b80 20 client.16902 populate_metadata read hostname 'smithi002' 2023-06-23T18:39:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562af6d8bd10 0x562af6d8e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac4dc39b80 4 client.16902 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:39:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1868895119 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562af6c2e050 con 0x562af6d8bd10 2023-06-23T18:39:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac3affd700 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562af6d8bd10 0x562af6d8e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.870+0000 7fac1affd700 10 client.16902 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:39:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac3affd700 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562af6d8bd10 0x562af6d8e0f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1714dcdc590d0691 server_cookie=4974d80686531753 in_seq=0 out_seq=0 2023-06-23T18:39:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac1affd700 1 -- 172.21.15.2:0/1868895119 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562af6c2e050 con 0x562af6d8bd10 2023-06-23T18:39:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac1affd700 10 client.16902 handle_command_reply: tid=0 2023-06-23T18:39:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 client.16902 shutdown 2023-06-23T18:39:22.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1868895119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 msgr2=0x562af6d2e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2e950 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fac2c011dc0 tx=0x7fac2c01e760 comp rx=0 tx=0).stop 2023-06-23T18:39:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1868895119 shutdown_connections 2023-06-23T18:39:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x562af6d8bd10 0x562af6d8e0f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x562af6d23000 0x562af6d31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac1affd700 0 client.16902 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:39:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x562af6d21c80 0x562af6d2e950 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 --2- 172.21.15.2:0/1868895119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562af6d225d0 0x562af6d2ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1868895119 >> 172.21.15.2:0/1868895119 conn(0x562af6d173f0 msgr2=0x562af6d29390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:22.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.871+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1868895119 shutdown_connections 2023-06-23T18:39:22.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac4dc39b80 1 -- 172.21.15.2:0/1868895119 wait complete. 2023-06-23T18:39:22.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac4dc39b80 20 client.16902 trim_cache size 0 max 16384 2023-06-23T18:39:22.886 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:39:22.887 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:39:22.887 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:39:22.887 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:39:22.887 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:39:22.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:39:22.887 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:39:22.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:39:22.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:39:22.888 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:39:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 -- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fac240a12a0 msgr2=0x7fac240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fac240a12a0 0x7fac240a3750 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fac2c005950 tx=0x7fac2c00b500 comp rx=0 tx=0).stop 2023-06-23T18:39:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 -- 172.21.15.2:0/3322548687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac3c105840 msgr2=0x7fac3c0feaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac3c105840 0x7fac3c0feaf0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fac3400be20 tx=0x7fac34009ed0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 -- 172.21.15.2:0/3322548687 shutdown_connections 2023-06-23T18:39:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac3c105840 0x7fac3c0feaf0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fac3c100600 0x7fac3c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fac240a12a0 0x7fac240a3750 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 --2- 172.21.15.2:0/3322548687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac3c102f20 0x7fac3c0fe5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 -- 172.21.15.2:0/3322548687 >> 172.21.15.2:0/3322548687 conn(0x7fac3c0fa0e0 msgr2=0x7fac3c101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 -- 172.21.15.2:0/3322548687 shutdown_connections 2023-06-23T18:39:22.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:22.872+0000 7fac41ab4700 1 -- 172.21.15.2:0/3322548687 wait complete. 2023-06-23T18:39:22.898 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:39:23.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:22 smithi017 ceph-mon[97472]: pgmap v816: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-06-23T18:39:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:22 smithi002 ceph-mon[95947]: pgmap v816: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-06-23T18:39:23.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:22 smithi082 ceph-mon[97889]: pgmap v816: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 14 MiB/s wr, 1.05k op/s 2023-06-23T18:39:25.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:24 smithi017 ceph-mon[97472]: pgmap v817: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:39:25.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:24 smithi002 ceph-mon[95947]: pgmap v817: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:39:25.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:24 smithi082 ceph-mon[97889]: pgmap v817: 97 pgs: 97 active+clean; 8.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 13 MiB/s wr, 1.04k op/s 2023-06-23T18:39:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 335.0 s, 369.6 tps, lat 270.461 ms stddev 46.891 2023-06-23T18:39:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:26 smithi017 ceph-mon[97472]: pgmap v818: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 984 op/s 2023-06-23T18:39:27.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:26 smithi002 ceph-mon[95947]: pgmap v818: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 984 op/s 2023-06-23T18:39:27.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:26 smithi082 ceph-mon[97889]: pgmap v818: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 984 op/s 2023-06-23T18:39:29.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:28 smithi017 ceph-mon[97472]: pgmap v819: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:29.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:28 smithi002 ceph-mon[95947]: pgmap v819: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:29.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:28 smithi082 ceph-mon[97889]: pgmap v819: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:39:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:30 smithi017 ceph-mon[97472]: pgmap v820: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:31.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:30 smithi002 ceph-mon[95947]: pgmap v820: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:31.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:30 smithi082 ceph-mon[97889]: pgmap v820: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 340.0 s, 368.8 tps, lat 272.031 ms stddev 45.569 2023-06-23T18:39:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:32 smithi017 ceph-mon[97472]: pgmap v821: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:33.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:32 smithi002 ceph-mon[95947]: pgmap v821: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:33.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:32 smithi082 ceph-mon[97889]: pgmap v821: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:34.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:33 smithi002 ceph-mon[95947]: pgmap v822: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:34.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:33 smithi082 ceph-mon[97889]: pgmap v822: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:34.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:33 smithi017 ceph-mon[97472]: pgmap v822: 97 pgs: 97 active+clean; 8.4 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:36.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:36 smithi082 ceph-mon[97889]: pgmap v823: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-06-23T18:39:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:36 smithi017 ceph-mon[97472]: pgmap v823: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-06-23T18:39:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:36 smithi002 ceph-mon[95947]: pgmap v823: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-06-23T18:39:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 345.0 s, 368.2 tps, lat 272.852 ms stddev 43.439 2023-06-23T18:39:38.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:38 smithi082 ceph-mon[97889]: pgmap v824: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:38 smithi017 ceph-mon[97472]: pgmap v824: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:38 smithi002 ceph-mon[95947]: pgmap v824: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.07k op/s 2023-06-23T18:39:40.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:40 smithi082 ceph-mon[97889]: pgmap v825: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-06-23T18:39:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:40 smithi017 ceph-mon[97472]: pgmap v825: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-06-23T18:39:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:40 smithi002 ceph-mon[95947]: pgmap v825: 97 pgs: 97 active+clean; 8.4 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-06-23T18:39:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 350.0 s, 348.0 tps, lat 285.661 ms stddev 46.271 2023-06-23T18:39:42.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:42 smithi082 ceph-mon[97889]: pgmap v826: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-06-23T18:39:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:42 smithi017 ceph-mon[97472]: pgmap v826: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-06-23T18:39:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:42 smithi002 ceph-mon[95947]: pgmap v826: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 13 MiB/s wr, 1.12k op/s 2023-06-23T18:39:44.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:44 smithi082 ceph-mon[97889]: pgmap v827: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.11k op/s 2023-06-23T18:39:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:44 smithi017 ceph-mon[97472]: pgmap v827: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.11k op/s 2023-06-23T18:39:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:44 smithi002 ceph-mon[95947]: pgmap v827: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.11k op/s 2023-06-23T18:39:46.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:46 smithi082 ceph-mon[97889]: pgmap v828: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:39:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:46 smithi017 ceph-mon[97472]: pgmap v828: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:39:46.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:46 smithi002 ceph-mon[95947]: pgmap v828: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:39:47.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 355.0 s, 334.8 tps, lat 298.958 ms stddev 59.825 2023-06-23T18:39:48.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:48 smithi082 ceph-mon[97889]: pgmap v829: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:39:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:48 smithi017 ceph-mon[97472]: pgmap v829: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:39:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:48 smithi002 ceph-mon[95947]: pgmap v829: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:39:50.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:50 smithi082 ceph-mon[97889]: pgmap v830: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:39:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:50 smithi017 ceph-mon[97472]: pgmap v830: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:39:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:50 smithi002 ceph-mon[95947]: pgmap v830: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:39:52.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 360.0 s, 356.4 tps, lat 280.271 ms stddev 42.810 2023-06-23T18:39:52.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:52 smithi082 ceph-mon[97889]: pgmap v831: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:39:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:52 smithi017 ceph-mon[97472]: pgmap v831: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:39:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:52 smithi002 ceph-mon[95947]: pgmap v831: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:39:52.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:39:54.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 -- 172.21.15.2:0/896856025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878102f90 msgr2=0x7f28781033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 --2- 172.21.15.2:0/896856025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878102f90 0x7f28781033e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f28600098f0 tx=0x7f2860009c00 comp rx=0 tx=0).stop 2023-06-23T18:39:54.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 -- 172.21.15.2:0/896856025 shutdown_connections 2023-06-23T18:39:54.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 --2- 172.21.15.2:0/896856025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2878103920 0x7f287810a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 --2- 172.21.15.2:0/896856025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2878101da0 0x7f28781021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 --2- 172.21.15.2:0/896856025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878102f90 0x7f28781033e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 -- 172.21.15.2:0/896856025 >> 172.21.15.2:0/896856025 conn(0x7f28780fd3c0 msgr2=0x7f28780ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 -- 172.21.15.2:0/896856025 shutdown_connections 2023-06-23T18:39:54.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.055+0000 7f287db7f700 1 -- 172.21.15.2:0/896856025 wait complete. 2023-06-23T18:39:54.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f287db7f700 1 Processor -- start 2023-06-23T18:39:54.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f287db7f700 1 -- start start 2023-06-23T18:39:54.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f287db7f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878101da0 0x7f28781979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f287db7f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2878102f90 0x7f2878197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f287db7f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2878103920 0x7f287819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f287db7f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287819f570 con 0x7f2878102f90 2023-06-23T18:39:54.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f287db7f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287819f6b0 con 0x7f2878103920 2023-06-23T18:39:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f287db7f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287819f840 con 0x7f2878101da0 2023-06-23T18:39:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f28777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878101da0 0x7f28781979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f2877fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2878103920 0x7f287819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f28777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878101da0 0x7f28781979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58068/0 (socket says 172.21.15.2:58068) 2023-06-23T18:39:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f28777fe700 1 -- 172.21.15.2:0/4149440656 learned_addr learned my addr 172.21.15.2:0/4149440656 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.056+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2878102f90 0x7f2878197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f28777fe700 1 -- 172.21.15.2:0/4149440656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2878103920 msgr2=0x7f287819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f28777fe700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2878103920 0x7f287819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f28777fe700 1 -- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2878102f90 msgr2=0x7f2878197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f28777fe700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2878102f90 0x7f2878197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f28777fe700 1 -- 172.21.15.2:0/4149440656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2860009520 con 0x7f2878101da0 2023-06-23T18:39:54.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f2877fff700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2878103920 0x7f287819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:39:54.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f28777fe700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878101da0 0x7f28781979f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f286800e970 tx=0x7f286800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:54.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f2874ff9700 1 -- 172.21.15.2:0/4149440656 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868009790 con 0x7f2878101da0 2023-06-23T18:39:54.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f2874ff9700 1 -- 172.21.15.2:0/4149440656 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f286800c7e0 con 0x7f2878101da0 2023-06-23T18:39:54.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f2874ff9700 1 -- 172.21.15.2:0/4149440656 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868003bd0 con 0x7f2878101da0 2023-06-23T18:39:54.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f287db7f700 1 -- 172.21.15.2:0/4149440656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287819fb20 con 0x7f2878101da0 2023-06-23T18:39:54.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.057+0000 7f287db7f700 1 -- 172.21.15.2:0/4149440656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287819ffe0 con 0x7f2878101da0 2023-06-23T18:39:54.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.061+0000 7f2874ff9700 1 -- 172.21.15.2:0/4149440656 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f286800c950 con 0x7f2878101da0 2023-06-23T18:39:54.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.061+0000 7f2874ff9700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28640a1250 0x7f28640a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2874ff9700 1 -- 172.21.15.2:0/4149440656 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2868014070 con 0x7f2878101da0 2023-06-23T18:39:54.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2876ffd700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28640a1250 0x7f28640a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2889d04b80 1 Processor -- start 2023-06-23T18:39:54.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2889d04b80 1 -- start start 2023-06-23T18:39:54.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2876ffd700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28640a1250 0x7f28640a3700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2860005e90 tx=0x7f2860019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:54.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09d78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09d8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 0x5614f09d8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2889d04b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f08d3cb0 con 0x5614f09d7de0 2023-06-23T18:39:54.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2889d04b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f0926a40 con 0x5614f09d8810 2023-06-23T18:39:54.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f2889d04b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f091fed0 con 0x5614f09d7490 2023-06-23T18:39:54.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.062+0000 7f28777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09d78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f28777fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09d78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58076/0 (socket says 172.21.15.2:58076) 2023-06-23T18:39:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f28777fe700 1 -- 172.21.15.2:0/611392750 learned_addr learned my addr 172.21.15.2:0/611392750 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f2877fff700 1 --2- 172.21.15.2:0/611392750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 0x5614f09d8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f28777fe700 1 -- 172.21.15.2:0/611392750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 msgr2=0x5614f09d8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f28777fe700 1 --2- 172.21.15.2:0/611392750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 0x5614f09d8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f28777fe700 1 -- 172.21.15.2:0/611392750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 msgr2=0x5614f09d8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f28777fe700 1 --2- 172.21.15.2:0/611392750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09d8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f28777fe700 1 -- 172.21.15.2:0/611392750 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614f07a1630 con 0x5614f09d7490 2023-06-23T18:39:54.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f28777fe700 1 --2- 172.21.15.2:0/611392750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09d78a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2868099e10 tx=0x7f2868099ac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e805100b28b0044f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:54.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f286dffb700 1 -- 172.21.15.2:0/611392750 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868099e80 con 0x5614f09d7490 2023-06-23T18:39:54.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f286dffb700 1 -- 172.21.15.2:0/611392750 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f286809deb0 con 0x5614f09d7490 2023-06-23T18:39:54.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.063+0000 7f286dffb700 1 -- 172.21.15.2:0/611392750 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286809a340 con 0x5614f09d7490 2023-06-23T18:39:54.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 -- 172.21.15.2:0/611392750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 msgr2=0x5614f09d78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 --2- 172.21.15.2:0/611392750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09d78a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2868099e10 tx=0x7f2868099ac0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 -- 172.21.15.2:0/611392750 shutdown_connections 2023-06-23T18:39:54.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 --2- 172.21.15.2:0/611392750 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 0x5614f09d8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 --2- 172.21.15.2:0/611392750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09d78a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 --2- 172.21.15.2:0/611392750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09d8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 -- 172.21.15.2:0/611392750 >> 172.21.15.2:0/611392750 conn(0x5614f09c43f0 msgr2=0x5614f09c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 -- 172.21.15.2:0/611392750 shutdown_connections 2023-06-23T18:39:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.064+0000 7f2889d04b80 1 -- 172.21.15.2:0/611392750 wait complete. 2023-06-23T18:39:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 1 -- start start 2023-06-23T18:39:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 1 Processor -- start 2023-06-23T18:39:54.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 0x5614f09e8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f08bfac0 con 0x5614f09d7490 2023-06-23T18:39:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f08bf7c0 con 0x5614f09d7de0 2023-06-23T18:39:54.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f07a3920 con 0x5614f09d8810 2023-06-23T18:39:54.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.065+0000 7f2889d04b80 10 client.0 _ll_register_callbacks cb 0x5614f08c5260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:39:54.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48336/0 (socket says 172.21.15.2:48336) 2023-06-23T18:39:54.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 -- 172.21.15.2:0/3354842224 learned_addr learned my addr 172.21.15.2:0/3354842224 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:54.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f28777fe700 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2877fff700 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 0x5614f09e8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 -- 172.21.15.2:0/3354842224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 msgr2=0x5614f09e8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:54.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 0x5614f09e8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 -- 172.21.15.2:0/3354842224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 msgr2=0x5614f09e0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:54.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 -- 172.21.15.2:0/3354842224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287819ffe0 con 0x5614f09d7de0 2023-06-23T18:39:54.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2876ffd700 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e1300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f286000f9d0 tx=0x7f2860005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab4cb09dc2a8497 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:54.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2856ffd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:39:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.066+0000 7f2856ffd700 1 -- 172.21.15.2:0/3354842224 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860005240 con 0x5614f09d7de0 2023-06-23T18:39:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2856ffd700 1 -- 172.21.15.2:0/3354842224 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28600053a0 con 0x5614f09d7de0 2023-06-23T18:39:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 1 -- 172.21.15.2:0/3354842224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614f09c1bb0 con 0x5614f09d7de0 2023-06-23T18:39:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2856ffd700 1 -- 172.21.15.2:0/3354842224 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f286000bbd0 con 0x5614f09d7de0 2023-06-23T18:39:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 10 client.26420 fetch_fsmap learned FSMap version 41 2023-06-23T18:39:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 1 -- 172.21.15.2:0/3354842224 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5614f074dde0 con 0x5614f09d7de0 2023-06-23T18:39:54.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2856ffd700 1 -- 172.21.15.2:0/3354842224 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5614f09c1bb0 con 0x5614f09d7de0 2023-06-23T18:39:54.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2856ffd700 1 -- 172.21.15.2:0/3354842224 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f286002f440 con 0x5614f09d7de0 2023-06-23T18:39:54.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 10 client.26420 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:39:54.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 10 client.26420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:39:54.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 20 client.26420 populate_metadata read hostname 'smithi002' 2023-06-23T18:39:54.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5614f09ecb90 0x5614f09eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 4 client.26420 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:39:54.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.067+0000 7f2889d04b80 1 -- 172.21.15.2:0/3354842224 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614f0918eb0 con 0x5614f09ecb90 2023-06-23T18:39:54.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.068+0000 7f28777fe700 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5614f09ecb90 0x5614f09eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.068+0000 7f2856ffd700 10 client.26420 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:39:54.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.068+0000 7f28777fe700 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5614f09ecb90 0x5614f09eefa0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d322324f8a80c5 server_cookie=1ac55f78bce8e117 in_seq=0 out_seq=0 2023-06-23T18:39:54.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2856ffd700 1 -- 172.21.15.2:0/3354842224 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5614f0918eb0 con 0x5614f09ecb90 2023-06-23T18:39:54.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2856ffd700 10 client.26420 handle_command_reply: tid=0 2023-06-23T18:39:54.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 client.26420 shutdown 2023-06-23T18:39:54.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 -- 172.21.15.2:0/3354842224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 msgr2=0x5614f09e1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e1300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f286000f9d0 tx=0x7f2860005dc0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 -- 172.21.15.2:0/3354842224 shutdown_connections 2023-06-23T18:39:54.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5614f09ecb90 0x5614f09eefa0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 0x5614f09e8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 --2- 172.21.15.2:0/3354842224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e1300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 -- 172.21.15.2:0/3354842224 >> 172.21.15.2:0/3354842224 conn(0x5614f09c43f0 msgr2=0x5614f09c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:54.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2856ffd700 0 client.26420 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:39:54.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 -- 172.21.15.2:0/3354842224 shutdown_connections 2023-06-23T18:39:54.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.069+0000 7f2889d04b80 1 -- 172.21.15.2:0/3354842224 wait complete. 2023-06-23T18:39:54.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.070+0000 7f2889d04b80 20 client.26420 trim_cache size 0 max 16384 2023-06-23T18:39:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2889d04b80 1 Processor -- start 2023-06-23T18:39:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2889d04b80 1 -- start start 2023-06-23T18:39:54.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 0x5614f09e1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2889d04b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f0918eb0 con 0x5614f09d7490 2023-06-23T18:39:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2889d04b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f09c6470 con 0x5614f09d7de0 2023-06-23T18:39:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2889d04b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f09e1cc0 con 0x5614f09d8810 2023-06-23T18:39:54.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f28777fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f28777fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57752/0 (socket says 172.21.15.2:57752) 2023-06-23T18:39:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f28777fe700 1 -- 172.21.15.2:0/798949588 learned_addr learned my addr 172.21.15.2:0/798949588 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.098+0000 7f2877fff700 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 0x5614f09e1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f2876ffd700 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f28777fe700 1 -- 172.21.15.2:0/798949588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 msgr2=0x5614f09e1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f28777fe700 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 0x5614f09e1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f28777fe700 1 -- 172.21.15.2:0/798949588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 msgr2=0x5614f09e8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f28777fe700 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f28777fe700 1 -- 172.21.15.2:0/798949588 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614f0736c60 con 0x5614f09d7490 2023-06-23T18:39:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f2876ffd700 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:39:54.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f28777fe700 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f28680968f0 tx=0x7f2868000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf402006792a932 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:54.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f286e7fc700 1 -- 172.21.15.2:0/798949588 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680b82b0 con 0x5614f09d7490 2023-06-23T18:39:54.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f286e7fc700 1 -- 172.21.15.2:0/798949588 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28680ca780 con 0x5614f09d7490 2023-06-23T18:39:54.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.099+0000 7f286e7fc700 1 -- 172.21.15.2:0/798949588 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868096600 con 0x5614f09d7490 2023-06-23T18:39:54.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 -- 172.21.15.2:0/798949588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 msgr2=0x5614f09e0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f28680968f0 tx=0x7f2868000bc0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 -- 172.21.15.2:0/798949588 shutdown_connections 2023-06-23T18:39:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d8810 0x5614f09e1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7490 0x5614f09e0dc0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 --2- 172.21.15.2:0/798949588 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d7de0 0x5614f09e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 -- 172.21.15.2:0/798949588 >> 172.21.15.2:0/798949588 conn(0x5614f09c43f0 msgr2=0x5614f09dfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 -- 172.21.15.2:0/798949588 shutdown_connections 2023-06-23T18:39:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.100+0000 7f2889d04b80 1 -- 172.21.15.2:0/798949588 wait complete. 2023-06-23T18:39:54.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 1 -- start start 2023-06-23T18:39:54.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 1 Processor -- start 2023-06-23T18:39:54.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 0x5614f09e45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f09e4b20 con 0x5614f09d7de0 2023-06-23T18:39:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f09e4c90 con 0x5614f09d8810 2023-06-23T18:39:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614f09e4e50 con 0x5614f09d7490 2023-06-23T18:39:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2889d04b80 10 client.0 _ll_register_callbacks cb 0x5614f08c5260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:39:54.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2876ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09e8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57760/0 (socket says 172.21.15.2:57760) 2023-06-23T18:39:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f2876ffd700 1 -- 172.21.15.2:0/220597536 learned_addr learned my addr 172.21.15.2:0/220597536 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:39:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.101+0000 7f28777fe700 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2877fff700 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 0x5614f09e45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2876ffd700 1 -- 172.21.15.2:0/220597536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 msgr2=0x5614f09e7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2876ffd700 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09e7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2876ffd700 1 -- 172.21.15.2:0/220597536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 msgr2=0x5614f09e45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:39:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2876ffd700 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 0x5614f09e45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2876ffd700 1 -- 172.21.15.2:0/220597536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28680b6040 con 0x5614f09d7de0 2023-06-23T18:39:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f28777fe700 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09e7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:39:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2876ffd700 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09e8040 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f28600094b0 tx=0x7f286000bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5823cf18cd55093c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:39:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2856ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:39:54.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.102+0000 7f2856ffd700 1 -- 172.21.15.2:0/220597536 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286003e070 con 0x5614f09d7de0 2023-06-23T18:39:54.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2856ffd700 1 -- 172.21.15.2:0/220597536 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2860035030 con 0x5614f09d7de0 2023-06-23T18:39:54.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2889d04b80 1 -- 172.21.15.2:0/220597536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f286809deb0 con 0x5614f09d7de0 2023-06-23T18:39:54.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2856ffd700 1 -- 172.21.15.2:0/220597536 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f286002fba0 con 0x5614f09d7de0 2023-06-23T18:39:54.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2889d04b80 10 client.16917 fetch_fsmap learned FSMap version 41 2023-06-23T18:39:54.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2889d04b80 1 -- 172.21.15.2:0/220597536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5614f0774250 con 0x5614f09d7de0 2023-06-23T18:39:54.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2856ffd700 1 -- 172.21.15.2:0/220597536 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2860049020 con 0x5614f09d7de0 2023-06-23T18:39:54.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2856ffd700 1 -- 172.21.15.2:0/220597536 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2860041020 con 0x5614f09d7de0 2023-06-23T18:39:54.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2889d04b80 10 client.16917 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:39:54.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2889d04b80 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:39:54.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.103+0000 7f2889d04b80 20 client.16917 populate_metadata read hostname 'smithi002' 2023-06-23T18:39:54.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.104+0000 7f2889d04b80 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5614f0a41ac0 0x5614f0a43ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:39:54.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.104+0000 7f2889d04b80 4 client.16917 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:39:54.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.104+0000 7f2889d04b80 1 -- 172.21.15.2:0/220597536 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614f08db050 con 0x5614f0a41ac0 2023-06-23T18:39:54.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.104+0000 7f28777fe700 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5614f0a41ac0 0x5614f0a43ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:39:54.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.104+0000 7f2856ffd700 10 client.16917 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:39:54.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.104+0000 7f28777fe700 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5614f0a41ac0 0x5614f0a43ed0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe73571692fdfde server_cookie=718881eaabf249fc in_seq=0 out_seq=0 2023-06-23T18:39:54.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2856ffd700 1 -- 172.21.15.2:0/220597536 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5614f08db050 con 0x5614f0a41ac0 2023-06-23T18:39:54.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2856ffd700 10 client.16917 handle_command_reply: tid=0 2023-06-23T18:39:54.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 client.16917 shutdown 2023-06-23T18:39:54.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 -- 172.21.15.2:0/220597536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 msgr2=0x5614f09e8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09e8040 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f28600094b0 tx=0x7f286000bcc0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 -- 172.21.15.2:0/220597536 shutdown_connections 2023-06-23T18:39:54.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5614f0a41ac0 0x5614f0a43ed0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5614f09d8810 0x5614f09e45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2856ffd700 0 client.16917 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:39:54.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5614f09d7490 0x5614f09e7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 --2- 172.21.15.2:0/220597536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5614f09d7de0 0x5614f09e8040 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 -- 172.21.15.2:0/220597536 >> 172.21.15.2:0/220597536 conn(0x5614f09c43f0 msgr2=0x5614f09e0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:54.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 -- 172.21.15.2:0/220597536 shutdown_connections 2023-06-23T18:39:54.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 1 -- 172.21.15.2:0/220597536 wait complete. 2023-06-23T18:39:54.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.105+0000 7f2889d04b80 20 client.16917 trim_cache size 0 max 16384 2023-06-23T18:39:54.121 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:39:54.121 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:39:54.122 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:39:54.122 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:39:54.122 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:39:54.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:39:54.122 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:39:54.122 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:39:54.122 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:39:54.123 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:39:54.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 -- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28640a1250 msgr2=0x7f28640a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28640a1250 0x7f28640a3700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2860005e90 tx=0x7f2860019040 comp rx=0 tx=0).stop 2023-06-23T18:39:54.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 -- 172.21.15.2:0/4149440656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878101da0 msgr2=0x7f28781979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:39:54.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878101da0 0x7f28781979f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f286800e970 tx=0x7f286800ec80 comp rx=0 tx=0).stop 2023-06-23T18:39:54.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 -- 172.21.15.2:0/4149440656 shutdown_connections 2023-06-23T18:39:54.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f28640a1250 0x7f28640a3700 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2878103920 0x7f287819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2878101da0 0x7f28781979f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 --2- 172.21.15.2:0/4149440656 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2878102f90 0x7f2878197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:39:54.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 -- 172.21.15.2:0/4149440656 >> 172.21.15.2:0/4149440656 conn(0x7f28780fd3c0 msgr2=0x7f28780ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:39:54.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 -- 172.21.15.2:0/4149440656 shutdown_connections 2023-06-23T18:39:54.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:39:54.106+0000 7f287db7f700 1 -- 172.21.15.2:0/4149440656 wait complete. 2023-06-23T18:39:54.131 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:39:54.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:54 smithi002 ceph-mon[95947]: pgmap v832: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:39:54.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:54 smithi082 ceph-mon[97889]: pgmap v832: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:39:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:54 smithi017 ceph-mon[97472]: pgmap v832: 97 pgs: 97 active+clean; 8.4 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:39:56.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:56 smithi082 ceph-mon[97889]: pgmap v833: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:39:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:56 smithi017 ceph-mon[97472]: pgmap v833: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:39:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:56 smithi002 ceph-mon[95947]: pgmap v833: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:39:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 365.0 s, 382.6 tps, lat 262.273 ms stddev 42.722 2023-06-23T18:39:58.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:39:58 smithi082 ceph-mon[97889]: pgmap v834: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-06-23T18:39:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:39:58 smithi017 ceph-mon[97472]: pgmap v834: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-06-23T18:39:58.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:39:58 smithi002 ceph-mon[95947]: pgmap v834: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.12k op/s 2023-06-23T18:40:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:00 smithi082 ceph-mon[97889]: pgmap v835: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 1.23k op/s 2023-06-23T18:40:00.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:00 smithi082 ceph-mon[97889]: overall HEALTH_OK 2023-06-23T18:40:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:00 smithi017 ceph-mon[97472]: pgmap v835: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 1.23k op/s 2023-06-23T18:40:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:00 smithi017 ceph-mon[97472]: overall HEALTH_OK 2023-06-23T18:40:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:00 smithi002 ceph-mon[95947]: pgmap v835: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 1.23k op/s 2023-06-23T18:40:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:00 smithi002 ceph-mon[95947]: overall HEALTH_OK 2023-06-23T18:40:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 370.0 s, 393.2 tps, lat 253.153 ms stddev 38.984 2023-06-23T18:40:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:02 smithi017 ceph-mon[97472]: pgmap v836: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:40:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:02 smithi002 ceph-mon[95947]: pgmap v836: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:40:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:02 smithi082 ceph-mon[97889]: pgmap v836: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 1.13k op/s 2023-06-23T18:40:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:04 smithi017 ceph-mon[97472]: pgmap v837: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-06-23T18:40:04.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:04 smithi002 ceph-mon[95947]: pgmap v837: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-06-23T18:40:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:04 smithi082 ceph-mon[97889]: pgmap v837: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-06-23T18:40:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:06 smithi002 ceph-mon[95947]: pgmap v838: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:40:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:06 smithi082 ceph-mon[97889]: pgmap v838: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:40:07.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 375.0 s, 257.4 tps, lat 151.896 ms stddev 111.631 2023-06-23T18:40:07.133 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:06 smithi017 ceph-mon[97472]: pgmap v838: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-06-23T18:40:08.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:08 smithi017 ceph-mon[97472]: pgmap v839: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 956 op/s 2023-06-23T18:40:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:08 smithi002 ceph-mon[95947]: pgmap v839: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 956 op/s 2023-06-23T18:40:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:08 smithi082 ceph-mon[97889]: pgmap v839: 97 pgs: 97 active+clean; 8.4 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 956 op/s 2023-06-23T18:40:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:10 smithi017 ceph-mon[97472]: pgmap v840: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:40:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:10 smithi002 ceph-mon[95947]: pgmap v840: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:40:10.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:10 smithi082 ceph-mon[97889]: pgmap v840: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:40:12.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 380.0 s, 278.0 tps, lat 35.944 ms stddev 10.120 2023-06-23T18:40:12.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:12 smithi017 ceph-mon[97472]: pgmap v841: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 906 op/s 2023-06-23T18:40:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:12 smithi002 ceph-mon[95947]: pgmap v841: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 906 op/s 2023-06-23T18:40:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:12 smithi082 ceph-mon[97889]: pgmap v841: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 906 op/s 2023-06-23T18:40:14.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:14 smithi017 ceph-mon[97472]: pgmap v842: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 910 op/s 2023-06-23T18:40:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:14 smithi002 ceph-mon[95947]: pgmap v842: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 910 op/s 2023-06-23T18:40:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:14 smithi082 ceph-mon[97889]: pgmap v842: 97 pgs: 97 active+clean; 8.4 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 910 op/s 2023-06-23T18:40:16.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:16 smithi017 ceph-mon[97472]: pgmap v843: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 925 op/s 2023-06-23T18:40:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:16 smithi002 ceph-mon[95947]: pgmap v843: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 925 op/s 2023-06-23T18:40:16.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:16 smithi082 ceph-mon[97889]: pgmap v843: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 925 op/s 2023-06-23T18:40:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 385.0 s, 250.4 tps, lat 39.949 ms stddev 10.147 2023-06-23T18:40:18.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:18 smithi017 ceph-mon[97472]: pgmap v844: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 891 op/s 2023-06-23T18:40:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:18 smithi002 ceph-mon[95947]: pgmap v844: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 891 op/s 2023-06-23T18:40:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:18 smithi082 ceph-mon[97889]: pgmap v844: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 891 op/s 2023-06-23T18:40:20.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:20 smithi017 ceph-mon[97472]: pgmap v845: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 1k op/s 2023-06-23T18:40:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:20 smithi002 ceph-mon[95947]: pgmap v845: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 1k op/s 2023-06-23T18:40:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:20 smithi082 ceph-mon[97889]: pgmap v845: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 1k op/s 2023-06-23T18:40:21.658 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:21 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:40:21.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:21 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:40:21.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:21 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:40:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 390.0 s, 354.8 tps, lat 960.854 ms stddev 3085.339 2023-06-23T18:40:22.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: pgmap v846: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.9 MiB/s wr, 949 op/s 2023-06-23T18:40:22.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-06-23T18:40:22.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-06-23T18:40:22.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:40:22.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:40:22.694 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:40:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: pgmap v846: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.9 MiB/s wr, 949 op/s 2023-06-23T18:40:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-06-23T18:40:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-06-23T18:40:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:40:22.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:40:22.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:40:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: pgmap v846: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.9 MiB/s wr, 949 op/s 2023-06-23T18:40:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi082 to 196.2M 2023-06-23T18:40:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: Unable to set osd_memory_target on smithi082 to 205745254: error parsing value: Value '205745254' is below minimum 939524096 2023-06-23T18:40:22.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-06-23T18:40:22.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:40:22.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:40:22.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:40:23.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:23 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi017 to 197.2M 2023-06-23T18:40:23.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:23 smithi017 ceph-mon[97472]: Unable to set osd_memory_target on smithi017 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:40:23.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:23 smithi017 ceph-mon[97472]: Adjusting osd_memory_target on smithi002 to 197.2M 2023-06-23T18:40:23.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:23 smithi017 ceph-mon[97472]: Unable to set osd_memory_target on smithi002 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:40:23.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:23 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi017 to 197.2M 2023-06-23T18:40:23.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:23 smithi002 ceph-mon[95947]: Unable to set osd_memory_target on smithi017 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:40:23.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:23 smithi002 ceph-mon[95947]: Adjusting osd_memory_target on smithi002 to 197.2M 2023-06-23T18:40:23.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:23 smithi002 ceph-mon[95947]: Unable to set osd_memory_target on smithi002 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:40:23.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:23 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi017 to 197.2M 2023-06-23T18:40:23.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:23 smithi082 ceph-mon[97889]: Unable to set osd_memory_target on smithi017 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:40:23.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:23 smithi082 ceph-mon[97889]: Adjusting osd_memory_target on smithi002 to 197.2M 2023-06-23T18:40:23.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:23 smithi082 ceph-mon[97889]: Unable to set osd_memory_target on smithi002 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-06-23T18:40:24.133 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:40:24.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.348+0000 7f35e19aa700 1 -- 172.21.15.2:0/922599092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fe5b0 msgr2=0x7f35dc108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.348+0000 7f35e19aa700 1 --2- 172.21.15.2:0/922599092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fe5b0 0x7f35dc108190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35d400b0a0 tx=0x7f35d400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.349+0000 7f35e19aa700 1 -- 172.21.15.2:0/922599092 shutdown_connections 2023-06-23T18:40:24.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.349+0000 7f35e19aa700 1 --2- 172.21.15.2:0/922599092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fe5b0 0x7f35dc108190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.349+0000 7f35e19aa700 1 --2- 172.21.15.2:0/922599092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35dc1011b0 0x7f35dc1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.349+0000 7f35e19aa700 1 --2- 172.21.15.2:0/922599092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0fdc20 0x7f35dc0fe070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.349+0000 7f35e19aa700 1 -- 172.21.15.2:0/922599092 >> 172.21.15.2:0/922599092 conn(0x7f35dc0f9850 msgr2=0x7f35dc0fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:24.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.349+0000 7f35e19aa700 1 -- 172.21.15.2:0/922599092 shutdown_connections 2023-06-23T18:40:24.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.349+0000 7f35e19aa700 1 -- 172.21.15.2:0/922599092 wait complete. 2023-06-23T18:40:24.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35e19aa700 1 Processor -- start 2023-06-23T18:40:24.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35e19aa700 1 -- start start 2023-06-23T18:40:24.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35e19aa700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fdc20 0x7f35dc1008b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35e19aa700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0fe5b0 0x7f35dc0feef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35e19aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35dc1011b0 0x7f35dc0ff430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35e19aa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0ffbd0 con 0x7f35dc1011b0 2023-06-23T18:40:24.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35e19aa700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc0ffd40 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35e19aa700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35dc19f900 con 0x7f35dc0fdc20 2023-06-23T18:40:24.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.350+0000 7f35daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fdc20 0x7f35dc1008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fdc20 0x7f35dc1008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58114/0 (socket says 172.21.15.2:58114) 2023-06-23T18:40:24.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35daffd700 1 -- 172.21.15.2:0/3508715928 learned_addr learned my addr 172.21.15.2:0/3508715928 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:24.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35da7fc700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0fe5b0 0x7f35dc0feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35db7fe700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35dc1011b0 0x7f35dc0ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35da7fc700 1 -- 172.21.15.2:0/3508715928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fdc20 msgr2=0x7f35dc1008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35da7fc700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fdc20 0x7f35dc1008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35da7fc700 1 -- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35dc1011b0 msgr2=0x7f35dc0ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35da7fc700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35dc1011b0 0x7f35dc0ff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35da7fc700 1 -- 172.21.15.2:0/3508715928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4009d00 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35daffd700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fdc20 0x7f35dc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:40:24.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35da7fc700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0fe5b0 0x7f35dc0feef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00e9f0 tx=0x7f35cc00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:24.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35e09a8700 1 -- 172.21.15.2:0/3508715928 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00c9c0 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35e19aa700 1 -- 172.21.15.2:0/3508715928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35dc19fbe0 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35e09a8700 1 -- 172.21.15.2:0/3508715928 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35cc010430 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35e09a8700 1 -- 172.21.15.2:0/3508715928 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc018680 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.351+0000 7f35e19aa700 1 -- 172.21.15.2:0/3508715928 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc1a0110 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.355+0000 7f35e09a8700 1 -- 172.21.15.2:0/3508715928 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f35cc0105a0 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.356+0000 7f35e09a8700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f35c40a1150 0x7f35c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.356+0000 7f35e09a8700 1 -- 172.21.15.2:0/3508715928 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35cc014070 con 0x7f35dc0fe5b0 2023-06-23T18:40:24.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.357+0000 7f35edb2fb80 1 Processor -- start 2023-06-23T18:40:24.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.357+0000 7f35daffd700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f35c40a1150 0x7f35c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35edb2fb80 1 -- start start 2023-06-23T18:40:24.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35daffd700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f35c40a1150 0x7f35c40a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f35d0005950 tx=0x7f35d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:24.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2e9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b1e4cb0 con 0x55e65b2e9810 2023-06-23T18:40:24.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b237a40 con 0x55e65b2e8de0 2023-06-23T18:40:24.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b230ed0 con 0x55e65b2e8490 2023-06-23T18:40:24.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35da7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2e88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58118/0 (socket says 172.21.15.2:58118) 2023-06-23T18:40:24.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.358+0000 7f35daffd700 1 -- 172.21.15.2:0/2622653280 learned_addr learned my addr 172.21.15.2:0/2622653280 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:24.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35db7fe700 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2e9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35da7fc700 1 -- 172.21.15.2:0/2622653280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 msgr2=0x55e65b2e88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2e88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35da7fc700 1 -- 172.21.15.2:0/2622653280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 msgr2=0x55e65b2e9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2e9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35da7fc700 1 -- 172.21.15.2:0/2622653280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e65b0b2630 con 0x55e65b2e8de0 2023-06-23T18:40:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35db7fe700 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2e9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:40:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2e9280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0ca540 tx=0x7f35cc099cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae1a2849362fbfe0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:24.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35c1ffb700 1 -- 172.21.15.2:0/2622653280 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc010880 con 0x55e65b2e8de0 2023-06-23T18:40:24.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35c1ffb700 1 -- 172.21.15.2:0/2622653280 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35cc099e50 con 0x55e65b2e8de0 2023-06-23T18:40:24.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.359+0000 7f35c1ffb700 1 -- 172.21.15.2:0/2622653280 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0a1e10 con 0x55e65b2e8de0 2023-06-23T18:40:24.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2622653280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 msgr2=0x55e65b2e9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2e9280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0ca540 tx=0x7f35cc099cf0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2622653280 shutdown_connections 2023-06-23T18:40:24.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2e9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2e88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2622653280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2e9280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2622653280 >> 172.21.15.2:0/2622653280 conn(0x55e65b2d53f0 msgr2=0x55e65b2da520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2622653280 shutdown_connections 2023-06-23T18:40:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.360+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2622653280 wait complete. 2023-06-23T18:40:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 1 -- start start 2023-06-23T18:40:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 1 Processor -- start 2023-06-23T18:40:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2f2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2f9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b1d0ac0 con 0x55e65b2e9810 2023-06-23T18:40:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b1d07c0 con 0x55e65b2e8de0 2023-06-23T18:40:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b0b4920 con 0x55e65b2e8490 2023-06-23T18:40:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35edb2fb80 10 client.0 _ll_register_callbacks cb 0x55e65b1d6260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:40:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35daffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2f1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58124/0 (socket says 172.21.15.2:58124) 2023-06-23T18:40:24.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.361+0000 7f35daffd700 1 -- 172.21.15.2:0/2376467176 learned_addr learned my addr 172.21.15.2:0/2376467176 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:24.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2f2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35daffd700 1 -- 172.21.15.2:0/2376467176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 msgr2=0x55e65b2f2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:24.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35db7fe700 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35daffd700 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2f2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35daffd700 1 -- 172.21.15.2:0/2376467176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 msgr2=0x55e65b2f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:24.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35daffd700 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2f9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35daffd700 1 -- 172.21.15.2:0/2376467176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc1a0110 con 0x55e65b2e8490 2023-06-23T18:40:24.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35db7fe700 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:40:24.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35daffd700 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2f1de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009cf0 tx=0x7f35d00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f11a32d61a6973 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:24.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35baffd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:40:24.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35baffd700 1 -- 172.21.15.2:0/2376467176 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0022070 con 0x55e65b2e8490 2023-06-23T18:40:24.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2376467176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e65b2d2bb0 con 0x55e65b2e8490 2023-06-23T18:40:24.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.362+0000 7f35baffd700 1 -- 172.21.15.2:0/2376467176 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35d001ddc0 con 0x55e65b2e8490 2023-06-23T18:40:24.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35baffd700 1 -- 172.21.15.2:0/2376467176 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35d000eea0 con 0x55e65b2e8490 2023-06-23T18:40:24.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35edb2fb80 10 client.26653 fetch_fsmap learned FSMap version 41 2023-06-23T18:40:24.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2376467176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e65b05ede0 con 0x55e65b2e8490 2023-06-23T18:40:24.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35baffd700 1 -- 172.21.15.2:0/2376467176 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e65b2d2bb0 con 0x55e65b2e8490 2023-06-23T18:40:24.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35baffd700 1 -- 172.21.15.2:0/2376467176 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f35d001c520 con 0x55e65b2e8490 2023-06-23T18:40:24.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35edb2fb80 10 client.26653 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:40:24.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35edb2fb80 10 client.26653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:40:24.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35edb2fb80 20 client.26653 populate_metadata read hostname 'smithi002' 2023-06-23T18:40:24.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e65b2fdb80 0x55e65b2fff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35edb2fb80 4 client.26653 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:40:24.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2376467176 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e65b229eb0 con 0x55e65b2fdb80 2023-06-23T18:40:24.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.363+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e65b2fdb80 0x55e65b2fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.364+0000 7f35baffd700 10 client.26653 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:40:24.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.364+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e65b2fdb80 0x55e65b2fff90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eae19c2d5849944 server_cookie=a35a9d0696913e5 in_seq=0 out_seq=0 2023-06-23T18:40:24.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.364+0000 7f35baffd700 1 -- 172.21.15.2:0/2376467176 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e65b229eb0 con 0x55e65b2fdb80 2023-06-23T18:40:24.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.364+0000 7f35baffd700 10 client.26653 handle_command_reply: tid=0 2023-06-23T18:40:24.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.364+0000 7f35edb2fb80 1 client.26653 shutdown 2023-06-23T18:40:24.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2376467176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 msgr2=0x55e65b2f1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2f1de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009cf0 tx=0x7f35d00047c0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2376467176 shutdown_connections 2023-06-23T18:40:24.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e65b2fdb80 0x55e65b2fff90 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35baffd700 0 client.26653 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:40:24.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e9810 0x55e65b2f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e8490 0x55e65b2f1de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2376467176 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2f2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2376467176 >> 172.21.15.2:0/2376467176 conn(0x55e65b2d53f0 msgr2=0x55e65b2d9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:24.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2376467176 shutdown_connections 2023-06-23T18:40:24.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2376467176 wait complete. 2023-06-23T18:40:24.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.365+0000 7f35edb2fb80 20 client.26653 trim_cache size 0 max 16384 2023-06-23T18:40:24.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35edb2fb80 1 Processor -- start 2023-06-23T18:40:24.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35edb2fb80 1 -- start start 2023-06-23T18:40:24.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8490 0x55e65b2ec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b2f19b0 con 0x55e65b2e8490 2023-06-23T18:40:24.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b2f08c0 con 0x55e65b2e8de0 2023-06-23T18:40:24.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b2dc010 con 0x55e65b2e9810 2023-06-23T18:40:24.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35daffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8490 0x55e65b2ec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35db7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35daffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8490 0x55e65b2ec6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57794/0 (socket says 172.21.15.2:57794) 2023-06-23T18:40:24.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35daffd700 1 -- 172.21.15.2:0/1554247466 learned_addr learned my addr 172.21.15.2:0/1554247466 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:24.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35da7fc700 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.392+0000 7f35daffd700 1 -- 172.21.15.2:0/1554247466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 msgr2=0x55e65b2f9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:24.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35daffd700 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35daffd700 1 -- 172.21.15.2:0/1554247466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 msgr2=0x55e65b2ecc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:24.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35daffd700 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2ecc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35daffd700 1 -- 172.21.15.2:0/1554247466 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e65b047c60 con 0x55e65b2e8490 2023-06-23T18:40:24.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35da7fc700 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2ecc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:40:24.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35db7fe700 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:40:24.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35daffd700 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8490 0x55e65b2ec6d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009e60 tx=0x7f35d001db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fb4fdb0a853f4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:24.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35c27fc700 1 -- 172.21.15.2:0/1554247466 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0022070 con 0x55e65b2e8490 2023-06-23T18:40:24.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35c27fc700 1 -- 172.21.15.2:0/1554247466 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35d000e460 con 0x55e65b2e8490 2023-06-23T18:40:24.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.393+0000 7f35c27fc700 1 -- 172.21.15.2:0/1554247466 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d0004e30 con 0x55e65b2e8490 2023-06-23T18:40:24.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 -- 172.21.15.2:0/1554247466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8490 msgr2=0x55e65b2ec6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8490 0x55e65b2ec6d0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009e60 tx=0x7f35d001db40 comp rx=0 tx=0).stop 2023-06-23T18:40:24.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 -- 172.21.15.2:0/1554247466 shutdown_connections 2023-06-23T18:40:24.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8490 0x55e65b2ec6d0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/1554247466 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8de0 0x55e65b2ecc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 -- 172.21.15.2:0/1554247466 >> 172.21.15.2:0/1554247466 conn(0x55e65b2d53f0 msgr2=0x55e65b2f0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 -- 172.21.15.2:0/1554247466 shutdown_connections 2023-06-23T18:40:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.394+0000 7f35edb2fb80 1 -- 172.21.15.2:0/1554247466 wait complete. 2023-06-23T18:40:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 1 -- start start 2023-06-23T18:40:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 1 Processor -- start 2023-06-23T18:40:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8490 0x55e65b2f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8de0 0x55e65b2f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b2f5c90 con 0x55e65b2e8de0 2023-06-23T18:40:24.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b2f5e00 con 0x55e65b2e8490 2023-06-23T18:40:24.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e65b2f5fc0 con 0x55e65b2e9810 2023-06-23T18:40:24.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35edb2fb80 10 client.0 _ll_register_callbacks cb 0x55e65b1d6260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:40:24.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35da7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8de0 0x55e65b2f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35daffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8490 0x55e65b2f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35da7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8de0 0x55e65b2f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57800/0 (socket says 172.21.15.2:57800) 2023-06-23T18:40:24.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35da7fc700 1 -- 172.21.15.2:0/2540295285 learned_addr learned my addr 172.21.15.2:0/2540295285 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:24.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35db7fe700 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35da7fc700 1 -- 172.21.15.2:0/2540295285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 msgr2=0x55e65b2f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:24.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35da7fc700 1 -- 172.21.15.2:0/2540295285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8490 msgr2=0x55e65b2f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:24.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8490 0x55e65b2f8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35da7fc700 1 -- 172.21.15.2:0/2540295285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35dc1088e0 con 0x55e65b2e8de0 2023-06-23T18:40:24.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.395+0000 7f35daffd700 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8490 0x55e65b2f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:40:24.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35da7fc700 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8de0 0x55e65b2f91a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0968c0 tx=0x7f35cc009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dbe5bb2021d47a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:24.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35baffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:40:24.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35baffd700 1 -- 172.21.15.2:0/2540295285 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0b8230 con 0x55e65b2e8de0 2023-06-23T18:40:24.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35baffd700 1 -- 172.21.15.2:0/2540295285 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f35cc09de80 con 0x55e65b2e8de0 2023-06-23T18:40:24.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2540295285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e65b1ec050 con 0x55e65b2e8de0 2023-06-23T18:40:24.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35baffd700 1 -- 172.21.15.2:0/2540295285 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35cc0a22a0 con 0x55e65b2e8de0 2023-06-23T18:40:24.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35edb2fb80 10 client.16929 fetch_fsmap learned FSMap version 41 2023-06-23T18:40:24.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2540295285 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e65b05ede0 con 0x55e65b2e8de0 2023-06-23T18:40:24.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.396+0000 7f35baffd700 1 -- 172.21.15.2:0/2540295285 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e65b1ec050 con 0x55e65b2e8de0 2023-06-23T18:40:24.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35baffd700 1 -- 172.21.15.2:0/2540295285 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f35cc09c030 con 0x55e65b2e8de0 2023-06-23T18:40:24.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35edb2fb80 10 client.16929 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:40:24.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35edb2fb80 10 client.16929 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:40:24.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35edb2fb80 20 client.16929 populate_metadata read hostname 'smithi002' 2023-06-23T18:40:24.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e65b2fdb80 0x55e65b354d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:24.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35edb2fb80 4 client.16929 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:40:24.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2540295285 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e65b07ff70 con 0x55e65b2fdb80 2023-06-23T18:40:24.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35daffd700 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e65b2fdb80 0x55e65b354d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:24.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35baffd700 10 client.16929 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:40:24.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.397+0000 7f35daffd700 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e65b2fdb80 0x55e65b354d50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662b93f5d5511682 server_cookie=1b34407b9a1c410 in_seq=0 out_seq=0 2023-06-23T18:40:24.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35baffd700 1 -- 172.21.15.2:0/2540295285 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e65b07ff70 con 0x55e65b2fdb80 2023-06-23T18:40:24.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35baffd700 10 client.16929 handle_command_reply: tid=0 2023-06-23T18:40:24.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 client.16929 shutdown 2023-06-23T18:40:24.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2540295285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8de0 msgr2=0x55e65b2f91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8de0 0x55e65b2f91a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0968c0 tx=0x7f35cc009790 comp rx=0 tx=0).stop 2023-06-23T18:40:24.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2540295285 shutdown_connections 2023-06-23T18:40:24.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e65b2fdb80 0x55e65b354d50 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e65b2e9810 0x55e65b2f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e65b2e8490 0x55e65b2f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 --2- 172.21.15.2:0/2540295285 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e65b2e8de0 0x55e65b2f91a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2540295285 >> 172.21.15.2:0/2540295285 conn(0x55e65b2d53f0 msgr2=0x55e65b2ec6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:24.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35baffd700 0 client.16929 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:40:24.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2540295285 shutdown_connections 2023-06-23T18:40:24.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 1 -- 172.21.15.2:0/2540295285 wait complete. 2023-06-23T18:40:24.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.398+0000 7f35edb2fb80 20 client.16929 trim_cache size 0 max 16384 2023-06-23T18:40:24.412 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:40:24.412 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:40:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:40:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:40:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:40:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:40:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:40:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:40:24.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:40:24.413 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:40:24.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 -- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f35c40a1150 msgr2=0x7f35c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f35c40a1150 0x7f35c40a3600 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f35d0005950 tx=0x7f35d0009260 comp rx=0 tx=0).stop 2023-06-23T18:40:24.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 -- 172.21.15.2:0/3508715928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0fe5b0 msgr2=0x7f35dc0feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:24.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0fe5b0 0x7f35dc0feef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00e9f0 tx=0x7f35cc00ed00 comp rx=0 tx=0).stop 2023-06-23T18:40:24.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 -- 172.21.15.2:0/3508715928 shutdown_connections 2023-06-23T18:40:24.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f35dc1011b0 0x7f35dc0ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35dc0fdc20 0x7f35dc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f35c40a1150 0x7f35c40a3600 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 --2- 172.21.15.2:0/3508715928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f35dc0fe5b0 0x7f35dc0feef0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:24.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 -- 172.21.15.2:0/3508715928 >> 172.21.15.2:0/3508715928 conn(0x7f35dc0f9850 msgr2=0x7f35dc104400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:24.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 -- 172.21.15.2:0/3508715928 shutdown_connections 2023-06-23T18:40:24.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:24.399+0000 7f35e19aa700 1 -- 172.21.15.2:0/3508715928 wait complete. 2023-06-23T18:40:24.421 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:40:24.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:24 smithi002 ceph-mon[95947]: pgmap v847: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:40:24.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:24 smithi017 ceph-mon[97472]: pgmap v847: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:40:24.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:24 smithi082 ceph-mon[97889]: pgmap v847: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:40:26.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:26 smithi017 ceph-mon[97472]: pgmap v848: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:26 smithi002 ceph-mon[95947]: pgmap v848: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:26 smithi082 ceph-mon[97889]: pgmap v848: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:27.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 395.0 s, 312.6 tps, lat 317.414 ms stddev 65.061 2023-06-23T18:40:28.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:28 smithi017 ceph-mon[97472]: pgmap v849: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 980 op/s 2023-06-23T18:40:28.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:28 smithi002 ceph-mon[95947]: pgmap v849: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 980 op/s 2023-06-23T18:40:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:28 smithi082 ceph-mon[97889]: pgmap v849: 97 pgs: 97 active+clean; 8.4 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 980 op/s 2023-06-23T18:40:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:30 smithi017 ceph-mon[97472]: pgmap v850: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.9 MiB/s wr, 1.04k op/s 2023-06-23T18:40:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:30 smithi002 ceph-mon[95947]: pgmap v850: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.9 MiB/s wr, 1.04k op/s 2023-06-23T18:40:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:30 smithi082 ceph-mon[97889]: pgmap v850: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.9 MiB/s wr, 1.04k op/s 2023-06-23T18:40:32.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 400.0 s, 279.0 tps, lat 358.781 ms stddev 87.859 2023-06-23T18:40:32.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:32 smithi017 ceph-mon[97472]: pgmap v851: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:32 smithi002 ceph-mon[95947]: pgmap v851: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:32 smithi082 ceph-mon[97889]: pgmap v851: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:34 smithi017 ceph-mon[97472]: pgmap v852: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:40:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:34 smithi002 ceph-mon[95947]: pgmap v852: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:40:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:34 smithi082 ceph-mon[97889]: pgmap v852: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:40:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:36 smithi017 ceph-mon[97472]: pgmap v853: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.8 MiB/s wr, 952 op/s 2023-06-23T18:40:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:36 smithi002 ceph-mon[95947]: pgmap v853: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.8 MiB/s wr, 952 op/s 2023-06-23T18:40:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:36 smithi082 ceph-mon[97889]: pgmap v853: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.8 MiB/s wr, 952 op/s 2023-06-23T18:40:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 405.0 s, 322.2 tps, lat 312.372 ms stddev 243.741 2023-06-23T18:40:38.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:38 smithi017 ceph-mon[97472]: pgmap v854: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 936 op/s 2023-06-23T18:40:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:38 smithi002 ceph-mon[95947]: pgmap v854: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 936 op/s 2023-06-23T18:40:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:38 smithi082 ceph-mon[97889]: pgmap v854: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 936 op/s 2023-06-23T18:40:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:40 smithi017 ceph-mon[97472]: pgmap v855: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 997 op/s 2023-06-23T18:40:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:40 smithi002 ceph-mon[95947]: pgmap v855: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 997 op/s 2023-06-23T18:40:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:40 smithi082 ceph-mon[97889]: pgmap v855: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 997 op/s 2023-06-23T18:40:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 410.0 s, 318.6 tps, lat 314.194 ms stddev 85.017 2023-06-23T18:40:42.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:42 smithi017 ceph-mon[97472]: pgmap v856: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 1k op/s 2023-06-23T18:40:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:42 smithi002 ceph-mon[95947]: pgmap v856: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 1k op/s 2023-06-23T18:40:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:42 smithi082 ceph-mon[97889]: pgmap v856: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 1k op/s 2023-06-23T18:40:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:44 smithi017 ceph-mon[97472]: pgmap v857: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:40:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:44 smithi002 ceph-mon[95947]: pgmap v857: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:40:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:44 smithi082 ceph-mon[97889]: pgmap v857: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:40:46.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:46 smithi017 ceph-mon[97472]: pgmap v858: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 996 op/s 2023-06-23T18:40:46.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:46 smithi002 ceph-mon[95947]: pgmap v858: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 996 op/s 2023-06-23T18:40:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:46 smithi082 ceph-mon[97889]: pgmap v858: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 996 op/s 2023-06-23T18:40:47.028 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 415.0 s, 340.2 tps, lat 294.003 ms stddev 50.519 2023-06-23T18:40:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:48 smithi017 ceph-mon[97472]: pgmap v859: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:48 smithi002 ceph-mon[95947]: pgmap v859: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:48 smithi082 ceph-mon[97889]: pgmap v859: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:40:50.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:50 smithi017 ceph-mon[97472]: pgmap v860: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:50.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:50 smithi002 ceph-mon[95947]: pgmap v860: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:50 smithi082 ceph-mon[97889]: pgmap v860: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:52.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 420.0 s, 350.8 tps, lat 283.466 ms stddev 50.791 2023-06-23T18:40:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:52 smithi017 ceph-mon[97472]: pgmap v861: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:52 smithi002 ceph-mon[95947]: pgmap v861: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:52.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:52 smithi082 ceph-mon[97889]: pgmap v861: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:54.422 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:40:54.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 -- 172.21.15.2:0/3657361421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70e8100600 msgr2=0x7f70e8106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 --2- 172.21.15.2:0/3657361421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70e8100600 0x7f70e8106ec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b0a0 tx=0x7f70e000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 -- 172.21.15.2:0/3657361421 shutdown_connections 2023-06-23T18:40:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 --2- 172.21.15.2:0/3657361421 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70e8100600 0x7f70e8106ec0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 --2- 172.21.15.2:0/3657361421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e80fea80 0x7f70e80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 --2- 172.21.15.2:0/3657361421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e80ffc70 0x7f70e81000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 -- 172.21.15.2:0/3657361421 >> 172.21.15.2:0/3657361421 conn(0x7f70e80fa0e0 msgr2=0x7f70e80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:54.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 -- 172.21.15.2:0/3657361421 shutdown_connections 2023-06-23T18:40:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.638+0000 7f70f0301700 1 -- 172.21.15.2:0/3657361421 wait complete. 2023-06-23T18:40:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70f0301700 1 Processor -- start 2023-06-23T18:40:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70f0301700 1 -- start start 2023-06-23T18:40:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70f0301700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70e80fea80 0x7f70e8068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70f0301700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e80ffc70 0x7f70e819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70f0301700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e8100600 0x7f70e819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70f0301700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e819d3d0 con 0x7f70e80fea80 2023-06-23T18:40:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70f0301700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e819d540 con 0x7f70e8100600 2023-06-23T18:40:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70f0301700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e819d700 con 0x7f70e80ffc70 2023-06-23T18:40:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70ee89e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e8100600 0x7f70e819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70ed89c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e80ffc70 0x7f70e819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70ee89e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e8100600 0x7f70e819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48408/0 (socket says 172.21.15.2:48408) 2023-06-23T18:40:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70ee09d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70e80fea80 0x7f70e8068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70ee89e700 1 -- 172.21.15.2:0/1736679506 learned_addr learned my addr 172.21.15.2:0/1736679506 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:54.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.639+0000 7f70ed89c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e80ffc70 0x7f70e819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58152/0 (socket says 172.21.15.2:58152) 2023-06-23T18:40:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70ee89e700 1 -- 172.21.15.2:0/1736679506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e80ffc70 msgr2=0x7f70e819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70ee89e700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e80ffc70 0x7f70e819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70ee89e700 1 -- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70e80fea80 msgr2=0x7f70e8068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70ee89e700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70e80fea80 0x7f70e8068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70ee89e700 1 -- 172.21.15.2:0/1736679506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4009520 con 0x7f70e8100600 2023-06-23T18:40:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70ee89e700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e8100600 0x7f70e819cc30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70e00095e0 tx=0x7f70e000ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:54.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70df7fe700 1 -- 172.21.15.2:0/1736679506 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e000e040 con 0x7f70e8100600 2023-06-23T18:40:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70f0301700 1 -- 172.21.15.2:0/1736679506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e0009d00 con 0x7f70e8100600 2023-06-23T18:40:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70df7fe700 1 -- 172.21.15.2:0/1736679506 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70e000be90 con 0x7f70e8100600 2023-06-23T18:40:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70df7fe700 1 -- 172.21.15.2:0/1736679506 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0005040 con 0x7f70e8100600 2023-06-23T18:40:54.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.640+0000 7f70f0301700 1 -- 172.21.15.2:0/1736679506 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e819dcc0 con 0x7f70e8100600 2023-06-23T18:40:54.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.643+0000 7f70df7fe700 1 -- 172.21.15.2:0/1736679506 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f70e0003c00 con 0x7f70e8100600 2023-06-23T18:40:54.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.643+0000 7f70df7fe700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70d40a1150 0x7f70d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.644+0000 7f70ee09d700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70d40a1150 0x7f70d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.644+0000 7f70df7fe700 1 -- 172.21.15.2:0/1736679506 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f70e00cdbe0 con 0x7f70e8100600 2023-06-23T18:40:54.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.644+0000 7f70ee09d700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70d40a1150 0x7f70d40a3600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f70e80ffad0 tx=0x7f70e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:54.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.644+0000 7f70fc486b80 1 Processor -- start 2023-06-23T18:40:54.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.644+0000 7f70fc486b80 1 -- start start 2023-06-23T18:40:54.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 0x5568076f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x5568076f2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70fc486b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568075f1cb0 con 0x5568076f1090 2023-06-23T18:40:54.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70fc486b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556807644a40 con 0x5568076f2410 2023-06-23T18:40:54.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70fc486b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55680763ded0 con 0x5568076f19e0 2023-06-23T18:40:54.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee09d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee09d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076f14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57820/0 (socket says 172.21.15.2:57820) 2023-06-23T18:40:54.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee09d700 1 -- 172.21.15.2:0/3803787983 learned_addr learned my addr 172.21.15.2:0/3803787983 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:54.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ed89c700 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 0x5568076f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee89e700 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x5568076f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee89e700 1 -- 172.21.15.2:0/3803787983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 msgr2=0x5568076f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:54.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee89e700 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 0x5568076f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee89e700 1 -- 172.21.15.2:0/3803787983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 msgr2=0x5568076f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:54.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee89e700 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee89e700 1 -- 172.21.15.2:0/3803787983 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5568074bf630 con 0x5568076f2410 2023-06-23T18:40:54.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee09d700 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:40:54.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.645+0000 7f70ee89e700 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x5568076f2880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b780 tx=0x7f70e009ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67c0394b5b11072 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:54.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.646+0000 7f70dcdac700 1 -- 172.21.15.2:0/3803787983 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e009f070 con 0x5568076f2410 2023-06-23T18:40:54.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.646+0000 7f70dcdac700 1 -- 172.21.15.2:0/3803787983 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70e00a3400 con 0x5568076f2410 2023-06-23T18:40:54.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.646+0000 7f70dcdac700 1 -- 172.21.15.2:0/3803787983 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e009a530 con 0x5568076f2410 2023-06-23T18:40:54.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.646+0000 7f70fc486b80 1 -- 172.21.15.2:0/3803787983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 msgr2=0x5568076f2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.646+0000 7f70fc486b80 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x5568076f2880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b780 tx=0x7f70e009ab80 comp rx=0 tx=0).stop 2023-06-23T18:40:54.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 -- 172.21.15.2:0/3803787983 shutdown_connections 2023-06-23T18:40:54.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x5568076f2880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 --2- 172.21.15.2:0/3803787983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 0x5568076f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 -- 172.21.15.2:0/3803787983 >> 172.21.15.2:0/3803787983 conn(0x5568076e23f0 msgr2=0x5568076e7520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:54.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 -- 172.21.15.2:0/3803787983 shutdown_connections 2023-06-23T18:40:54.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 -- 172.21.15.2:0/3803787983 wait complete. 2023-06-23T18:40:54.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 -- start start 2023-06-23T18:40:54.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.647+0000 7f70fc486b80 1 Processor -- start 2023-06-23T18:40:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f19e0 0x5568076fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x556807701b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70fc486b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568075ddac0 con 0x5568076f1090 2023-06-23T18:40:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70fc486b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568075dd7c0 con 0x5568076f19e0 2023-06-23T18:40:54.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70fc486b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568074c1920 con 0x5568076f2410 2023-06-23T18:40:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70fc486b80 10 client.0 _ll_register_callbacks cb 0x5568075e3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:40:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee09d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee09d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57826/0 (socket says 172.21.15.2:57826) 2023-06-23T18:40:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee09d700 1 -- 172.21.15.2:0/835471452 learned_addr learned my addr 172.21.15.2:0/835471452 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee89e700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x556807701b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ed89c700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f19e0 0x5568076fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee89e700 1 -- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f19e0 msgr2=0x5568076fab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee89e700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f19e0 0x5568076fab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee89e700 1 -- 172.21.15.2:0/835471452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 msgr2=0x5568076fa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee89e700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee89e700 1 -- 172.21.15.2:0/835471452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e4017070 con 0x5568076f2410 2023-06-23T18:40:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ed89c700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f19e0 0x5568076fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:40:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.648+0000 7f70ee09d700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:40:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70ee89e700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x556807701b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5568076f9190 tx=0x7f70e00074e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=257e00095071475b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:54.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:40:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70d1ffb700 1 -- 172.21.15.2:0/835471452 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0099240 con 0x5568076f2410 2023-06-23T18:40:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70d1ffb700 1 -- 172.21.15.2:0/835471452 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f70e00a2f70 con 0x5568076f2410 2023-06-23T18:40:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70fc486b80 1 -- 172.21.15.2:0/835471452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5568076dfbb0 con 0x5568076f2410 2023-06-23T18:40:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70d1ffb700 1 -- 172.21.15.2:0/835471452 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70e00a3130 con 0x5568076f2410 2023-06-23T18:40:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70fc486b80 10 client.26671 fetch_fsmap learned FSMap version 41 2023-06-23T18:40:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70fc486b80 1 -- 172.21.15.2:0/835471452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f70e819dcc0 con 0x5568076f2410 2023-06-23T18:40:54.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70d1ffb700 1 -- 172.21.15.2:0/835471452 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5568076dfbb0 con 0x5568076f2410 2023-06-23T18:40:54.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.649+0000 7f70d1ffb700 1 -- 172.21.15.2:0/835471452 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f70e00b9270 con 0x5568076f2410 2023-06-23T18:40:54.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70fc486b80 10 client.26671 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:40:54.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70fc486b80 10 client.26671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:40:54.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70fc486b80 20 client.26671 populate_metadata read hostname 'smithi002' 2023-06-23T18:40:54.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70fc486b80 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556807705ed0 0x5568077082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70fc486b80 4 client.26671 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:40:54.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70fc486b80 1 -- 172.21.15.2:0/835471452 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556807636eb0 con 0x556807705ed0 2023-06-23T18:40:54.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70ee09d700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556807705ed0 0x5568077082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70d1ffb700 10 client.26671 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:40:54.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.650+0000 7f70ee09d700 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556807705ed0 0x5568077082e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2999c8d8b83b860d server_cookie=45aaee629b669e08 in_seq=0 out_seq=0 2023-06-23T18:40:54.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.651+0000 7f70d1ffb700 1 -- 172.21.15.2:0/835471452 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556807636eb0 con 0x556807705ed0 2023-06-23T18:40:54.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.651+0000 7f70d1ffb700 10 client.26671 handle_command_reply: tid=0 2023-06-23T18:40:54.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.651+0000 7f70fc486b80 1 client.26671 shutdown 2023-06-23T18:40:54.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.651+0000 7f70fc486b80 1 -- 172.21.15.2:0/835471452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 msgr2=0x556807701b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.651+0000 7f70fc486b80 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x556807701b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5568076f9190 tx=0x7f70e00074e0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 1 -- 172.21.15.2:0/835471452 shutdown_connections 2023-06-23T18:40:54.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556807705ed0 0x5568077082e0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x556807701b80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70d1ffb700 0 client.26671 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:40:54.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 1 --2- 172.21.15.2:0/835471452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f19e0 0x5568076fab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 1 -- 172.21.15.2:0/835471452 >> 172.21.15.2:0/835471452 conn(0x5568076e23f0 msgr2=0x5568076e6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:54.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 1 -- 172.21.15.2:0/835471452 shutdown_connections 2023-06-23T18:40:54.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 1 -- 172.21.15.2:0/835471452 wait complete. 2023-06-23T18:40:54.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.652+0000 7f70fc486b80 20 client.26671 trim_cache size 0 max 16384 2023-06-23T18:40:54.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70fc486b80 1 Processor -- start 2023-06-23T18:40:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70fc486b80 1 -- start start 2023-06-23T18:40:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f1090 0x556807702470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f19e0 0x556807701b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x5568076fa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70fc486b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568076f9ab0 con 0x5568076f19e0 2023-06-23T18:40:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70fc486b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568076e81f0 con 0x5568076f1090 2023-06-23T18:40:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70fc486b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568076fa170 con 0x5568076f2410 2023-06-23T18:40:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.674+0000 7f70ed89c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f19e0 0x556807701b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ed89c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f19e0 0x556807701b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57834/0 (socket says 172.21.15.2:57834) 2023-06-23T18:40:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ed89c700 1 -- 172.21.15.2:0/203447730 learned_addr learned my addr 172.21.15.2:0/203447730 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ee09d700 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f1090 0x556807702470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ee89e700 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x5568076fa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ee09d700 1 -- 172.21.15.2:0/203447730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 msgr2=0x5568076fa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ee09d700 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x5568076fa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ee09d700 1 -- 172.21.15.2:0/203447730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f19e0 msgr2=0x556807701b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ee09d700 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f19e0 0x556807701b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ee09d700 1 -- 172.21.15.2:0/203447730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55680746bde0 con 0x5568076f1090 2023-06-23T18:40:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ed89c700 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f19e0 0x556807701b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:40:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70ee09d700 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f1090 0x556807702470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x5568077062a0 tx=0x7f70e4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd62ff2667314415 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70dd5ad700 1 -- 172.21.15.2:0/203447730 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4024040 con 0x5568076f1090 2023-06-23T18:40:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70dd5ad700 1 -- 172.21.15.2:0/203447730 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70e40295b0 con 0x5568076f1090 2023-06-23T18:40:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.675+0000 7f70dd5ad700 1 -- 172.21.15.2:0/203447730 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e401ebb0 con 0x5568076f1090 2023-06-23T18:40:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 -- 172.21.15.2:0/203447730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f1090 msgr2=0x556807702470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f1090 0x556807702470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x5568077062a0 tx=0x7f70e4014000 comp rx=0 tx=0).stop 2023-06-23T18:40:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 -- 172.21.15.2:0/203447730 shutdown_connections 2023-06-23T18:40:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f2410 0x5568076fa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f1090 0x556807702470 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 --2- 172.21.15.2:0/203447730 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f19e0 0x556807701b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 -- 172.21.15.2:0/203447730 >> 172.21.15.2:0/203447730 conn(0x5568076e23f0 msgr2=0x55680770cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 -- 172.21.15.2:0/203447730 shutdown_connections 2023-06-23T18:40:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.676+0000 7f70fc486b80 1 -- 172.21.15.2:0/203447730 wait complete. 2023-06-23T18:40:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 1 -- start start 2023-06-23T18:40:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 1 Processor -- start 2023-06-23T18:40:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 0x5568076fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x556807701280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5568077017c0 con 0x5568076f1090 2023-06-23T18:40:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556807701930 con 0x5568076f2410 2023-06-23T18:40:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556807701af0 con 0x5568076f19e0 2023-06-23T18:40:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70fc486b80 10 client.0 _ll_register_callbacks cb 0x5568075e3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:40:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70ee89e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x556807701280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70ee89e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x556807701280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48432/0 (socket says 172.21.15.2:48432) 2023-06-23T18:40:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70ee89e700 1 -- 172.21.15.2:0/568738684 learned_addr learned my addr 172.21.15.2:0/568738684 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:40:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.677+0000 7f70ed89c700 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 0x5568076fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70ee89e700 1 -- 172.21.15.2:0/568738684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 msgr2=0x5568076fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:40:54.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70ee89e700 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 0x5568076fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70ee89e700 1 -- 172.21.15.2:0/568738684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 msgr2=0x5568076fdd70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:40:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70ee89e700 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fdd70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70ee89e700 1 -- 172.21.15.2:0/568738684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d8012450 con 0x5568076f2410 2023-06-23T18:40:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70ee89e700 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x556807701280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b780 tx=0x7f70e000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0f1bbab58ebc4c4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:40:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:40:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70d1ffb700 1 -- 172.21.15.2:0/568738684 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e00b9320 con 0x5568076f2410 2023-06-23T18:40:54.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70d1ffb700 1 -- 172.21.15.2:0/568738684 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f70e00cb020 con 0x5568076f2410 2023-06-23T18:40:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70fc486b80 1 -- 172.21.15.2:0/568738684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70e00a3400 con 0x5568076f2410 2023-06-23T18:40:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70d1ffb700 1 -- 172.21.15.2:0/568738684 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70e0099240 con 0x5568076f2410 2023-06-23T18:40:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70fc486b80 10 client.26459 fetch_fsmap learned FSMap version 41 2023-06-23T18:40:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.678+0000 7f70fc486b80 1 -- 172.21.15.2:0/568738684 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f70e4003680 con 0x5568076f2410 2023-06-23T18:40:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70d1ffb700 1 -- 172.21.15.2:0/568738684 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f70e00a3400 con 0x5568076f2410 2023-06-23T18:40:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70d1ffb700 1 -- 172.21.15.2:0/568738684 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f70e00cb1e0 con 0x5568076f2410 2023-06-23T18:40:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70fc486b80 10 client.26459 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:40:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70fc486b80 10 client.26459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:40:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70fc486b80 20 client.26459 populate_metadata read hostname 'smithi002' 2023-06-23T18:40:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70fc486b80 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55680775ad10 0x55680775d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:40:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70fc486b80 4 client.26459 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:40:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70fc486b80 1 -- 172.21.15.2:0/568738684 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5568075f9050 con 0x55680775ad10 2023-06-23T18:40:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.679+0000 7f70ee09d700 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55680775ad10 0x55680775d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:40:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70d1ffb700 10 client.26459 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:40:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70ee09d700 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55680775ad10 0x55680775d120 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d65bace2e0be74 server_cookie=f48b446974478ff9 in_seq=0 out_seq=0 2023-06-23T18:40:54.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70d1ffb700 1 -- 172.21.15.2:0/568738684 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5568075f9050 con 0x55680775ad10 2023-06-23T18:40:54.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70d1ffb700 10 client.26459 handle_command_reply: tid=0 2023-06-23T18:40:54.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 client.26459 shutdown 2023-06-23T18:40:54.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 -- 172.21.15.2:0/568738684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 msgr2=0x556807701280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x556807701280 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f70e000b780 tx=0x7f70e000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 -- 172.21.15.2:0/568738684 shutdown_connections 2023-06-23T18:40:54.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55680775ad10 0x55680775d120 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5568076f2410 0x556807701280 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5568076f1090 0x5568076fdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 --2- 172.21.15.2:0/568738684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5568076f19e0 0x5568076fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70d1ffb700 0 client.26459 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:40:54.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 -- 172.21.15.2:0/568738684 >> 172.21.15.2:0/568738684 conn(0x5568076e23f0 msgr2=0x5568076f5330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:54.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 -- 172.21.15.2:0/568738684 shutdown_connections 2023-06-23T18:40:54.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 1 -- 172.21.15.2:0/568738684 wait complete. 2023-06-23T18:40:54.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.680+0000 7f70fc486b80 20 client.26459 trim_cache size 0 max 16384 2023-06-23T18:40:54.694 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:40:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:40:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:40:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:40:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:40:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:40:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:40:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:40:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:40:54.695 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:40:54.696 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:54 smithi017 ceph-mon[97472]: pgmap v862: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:54.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 -- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70d40a1150 msgr2=0x7f70d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70d40a1150 0x7f70d40a3600 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f70e80ffad0 tx=0x7f70e4009260 comp rx=0 tx=0).stop 2023-06-23T18:40:54.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 -- 172.21.15.2:0/1736679506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e8100600 msgr2=0x7f70e819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:40:54.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e8100600 0x7f70e819cc30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f70e00095e0 tx=0x7f70e000ba50 comp rx=0 tx=0).stop 2023-06-23T18:40:54.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 -- 172.21.15.2:0/1736679506 shutdown_connections 2023-06-23T18:40:54.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f70d40a1150 0x7f70d40a3600 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70e8100600 0x7f70e819cc30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70e80fea80 0x7f70e8068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 --2- 172.21.15.2:0/1736679506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e80ffc70 0x7f70e819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:40:54.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 -- 172.21.15.2:0/1736679506 >> 172.21.15.2:0/1736679506 conn(0x7f70e80fa0e0 msgr2=0x7f70e81030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:40:54.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 -- 172.21.15.2:0/1736679506 shutdown_connections 2023-06-23T18:40:54.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:40:54.681+0000 7f70f0301700 1 -- 172.21.15.2:0/1736679506 wait complete. 2023-06-23T18:40:54.700 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:54 smithi002 ceph-mon[95947]: pgmap v862: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:54.701 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:40:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:54 smithi082 ceph-mon[97889]: pgmap v862: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:40:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:56 smithi017 ceph-mon[97472]: pgmap v863: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-06-23T18:40:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:56 smithi002 ceph-mon[95947]: pgmap v863: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-06-23T18:40:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:56 smithi082 ceph-mon[97889]: pgmap v863: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-06-23T18:40:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 425.0 s, 370.2 tps, lat 271.085 ms stddev 49.018 2023-06-23T18:40:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:40:58 smithi017 ceph-mon[97472]: pgmap v864: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.7 MiB/s wr, 1.04k op/s 2023-06-23T18:40:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:40:58 smithi002 ceph-mon[95947]: pgmap v864: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.7 MiB/s wr, 1.04k op/s 2023-06-23T18:40:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:40:58 smithi082 ceph-mon[97889]: pgmap v864: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.7 MiB/s wr, 1.04k op/s 2023-06-23T18:41:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:00 smithi017 ceph-mon[97472]: pgmap v865: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:00 smithi002 ceph-mon[95947]: pgmap v865: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:00 smithi082 ceph-mon[97889]: pgmap v865: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:02.027 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 430.0 s, 369.1 tps, lat 271.035 ms stddev 43.417 2023-06-23T18:41:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:02 smithi002 ceph-mon[95947]: pgmap v866: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-06-23T18:41:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:02 smithi082 ceph-mon[97889]: pgmap v866: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-06-23T18:41:02.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:02 smithi017 ceph-mon[97472]: pgmap v866: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-06-23T18:41:04.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:04 smithi002 ceph-mon[95947]: pgmap v867: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:04 smithi082 ceph-mon[97889]: pgmap v867: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:04.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:04 smithi017 ceph-mon[97472]: pgmap v867: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:06.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:06 smithi002 ceph-mon[95947]: pgmap v868: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:06.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:06 smithi082 ceph-mon[97889]: pgmap v868: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:06.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:06 smithi017 ceph-mon[97472]: pgmap v868: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:07.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 435.0 s, 352.5 tps, lat 284.052 ms stddev 44.526 2023-06-23T18:41:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:08 smithi002 ceph-mon[95947]: pgmap v869: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:41:08.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:08 smithi082 ceph-mon[97889]: pgmap v869: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:41:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:08 smithi017 ceph-mon[97472]: pgmap v869: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:41:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:10 smithi002 ceph-mon[95947]: pgmap v870: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 1.17k op/s 2023-06-23T18:41:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:10 smithi082 ceph-mon[97889]: pgmap v870: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 1.17k op/s 2023-06-23T18:41:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:10 smithi017 ceph-mon[97472]: pgmap v870: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 12 MiB/s wr, 1.17k op/s 2023-06-23T18:41:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 440.0 s, 337.4 tps, lat 296.355 ms stddev 49.341 2023-06-23T18:41:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:12 smithi002 ceph-mon[95947]: pgmap v871: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:41:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:12 smithi082 ceph-mon[97889]: pgmap v871: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:41:12.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:12 smithi017 ceph-mon[97472]: pgmap v871: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:41:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:14 smithi002 ceph-mon[95947]: pgmap v872: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-06-23T18:41:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:14 smithi082 ceph-mon[97889]: pgmap v872: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-06-23T18:41:14.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:14 smithi017 ceph-mon[97472]: pgmap v872: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-06-23T18:41:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:16 smithi002 ceph-mon[95947]: pgmap v873: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.11k op/s 2023-06-23T18:41:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:16 smithi082 ceph-mon[97889]: pgmap v873: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.11k op/s 2023-06-23T18:41:16.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:16 smithi017 ceph-mon[97472]: pgmap v873: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.11k op/s 2023-06-23T18:41:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 445.0 s, 303.2 tps, lat 327.544 ms stddev 79.384 2023-06-23T18:41:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:18 smithi002 ceph-mon[95947]: pgmap v874: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:18 smithi082 ceph-mon[97889]: pgmap v874: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:18.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:18 smithi017 ceph-mon[97472]: pgmap v874: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:20 smithi002 ceph-mon[95947]: pgmap v875: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 1.22k op/s 2023-06-23T18:41:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:20 smithi082 ceph-mon[97889]: pgmap v875: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 1.22k op/s 2023-06-23T18:41:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:20 smithi017 ceph-mon[97472]: pgmap v875: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 1.22k op/s 2023-06-23T18:41:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 450.0 s, 342.0 tps, lat 294.946 ms stddev 58.285 2023-06-23T18:41:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:22 smithi002 ceph-mon[95947]: pgmap v876: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.17k op/s 2023-06-23T18:41:22.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:22 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:41:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:22 smithi082 ceph-mon[97889]: pgmap v876: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.17k op/s 2023-06-23T18:41:22.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:22 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:41:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:22 smithi017 ceph-mon[97472]: pgmap v876: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.17k op/s 2023-06-23T18:41:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:22 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:41:23.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:23 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:41:23.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:23 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:41:23.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:23 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:41:23.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:23 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:41:23.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:23 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:41:23.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:23 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:41:23.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:23 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:41:23.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:23 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:41:23.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:23 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:41:24.702 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:41:24.743 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:24 smithi002 ceph-mon[95947]: pgmap v877: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.18k op/s 2023-06-23T18:41:24.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:24 smithi082 ceph-mon[97889]: pgmap v877: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.18k op/s 2023-06-23T18:41:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:24 smithi017 ceph-mon[97472]: pgmap v877: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.18k op/s 2023-06-23T18:41:24.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.960+0000 7f3881bb7700 1 -- 172.21.15.2:0/868441110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c1017a0 msgr2=0x7f387c108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:24.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.960+0000 7f3881bb7700 1 --2- 172.21.15.2:0/868441110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c1017a0 0x7f387c108040 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f387400b0a0 tx=0x7f387400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.960+0000 7f3881bb7700 1 -- 172.21.15.2:0/868441110 shutdown_connections 2023-06-23T18:41:24.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.960+0000 7f3881bb7700 1 --2- 172.21.15.2:0/868441110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c1017a0 0x7f387c108040 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.960+0000 7f3881bb7700 1 --2- 172.21.15.2:0/868441110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f387c0ffb60 0x7f387c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.960+0000 7f3881bb7700 1 --2- 172.21.15.2:0/868441110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c100df0 0x7f387c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.960+0000 7f3881bb7700 1 -- 172.21.15.2:0/868441110 >> 172.21.15.2:0/868441110 conn(0x7f387c0fb180 msgr2=0x7f387c0fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:24.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.961+0000 7f3881bb7700 1 -- 172.21.15.2:0/868441110 shutdown_connections 2023-06-23T18:41:24.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.962+0000 7f3881bb7700 1 -- 172.21.15.2:0/868441110 wait complete. 2023-06-23T18:41:24.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.962+0000 7f3881bb7700 1 Processor -- start 2023-06-23T18:41:24.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.962+0000 7f3881bb7700 1 -- start start 2023-06-23T18:41:24.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f3881bb7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c0ffb60 0x7f387c19bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f3881bb7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c100df0 0x7f387c19c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f3881bb7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f387c1017a0 0x7f387c1a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f3881bb7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1a3a10 con 0x7f387c0ffb60 2023-06-23T18:41:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f3881bb7700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1a3b50 con 0x7f387c100df0 2023-06-23T18:41:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f3881bb7700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c1a3ce0 con 0x7f387c1017a0 2023-06-23T18:41:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c0ffb60 0x7f387c19bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c0ffb60 0x7f387c19bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57850/0 (socket says 172.21.15.2:57850) 2023-06-23T18:41:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f387c1017a0 0x7f387c1a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387b7fe700 1 -- 172.21.15.2:0/2615101758 learned_addr learned my addr 172.21.15.2:0/2615101758 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387affd700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c100df0 0x7f387c19c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387b7fe700 1 -- 172.21.15.2:0/2615101758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f387c1017a0 msgr2=0x7f387c1a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f387c1017a0 0x7f387c1a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387b7fe700 1 -- 172.21.15.2:0/2615101758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c100df0 msgr2=0x7f387c19c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c100df0 0x7f387c19c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387b7fe700 1 -- 172.21.15.2:0/2615101758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874009d00 con 0x7f387c0ffb60 2023-06-23T18:41:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.963+0000 7f387affd700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c100df0 0x7f387c19c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:41:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.964+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c0ffb60 0x7f387c19bec0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e9a0 tx=0x7f386c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.964+0000 7f3878ff9700 1 -- 172.21.15.2:0/2615101758 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c009790 con 0x7f387c0ffb60 2023-06-23T18:41:24.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.964+0000 7f3881bb7700 1 -- 172.21.15.2:0/2615101758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387c1a3fc0 con 0x7f387c0ffb60 2023-06-23T18:41:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.964+0000 7f3878ff9700 1 -- 172.21.15.2:0/2615101758 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f386c010430 con 0x7f387c0ffb60 2023-06-23T18:41:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.964+0000 7f3878ff9700 1 -- 172.21.15.2:0/2615101758 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c00cea0 con 0x7f387c0ffb60 2023-06-23T18:41:24.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.964+0000 7f3881bb7700 1 -- 172.21.15.2:0/2615101758 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c1a43e0 con 0x7f387c0ffb60 2023-06-23T18:41:24.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.968+0000 7f3878ff9700 1 -- 172.21.15.2:0/2615101758 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f386c0105a0 con 0x7f387c0ffb60 2023-06-23T18:41:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f388dd3cb80 1 Processor -- start 2023-06-23T18:41:24.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f3878ff9700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38640a1150 0x7f38640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f3878ff9700 1 -- 172.21.15.2:0/2615101758 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f386c014070 con 0x7f387c0ffb60 2023-06-23T18:41:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f388dd3cb80 1 -- start start 2023-06-23T18:41:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68baddc80 0x55e68bade090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68badea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68badf000 0x55e68badf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b9e2cb0 con 0x55e68badf000 2023-06-23T18:41:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68ba35a40 con 0x55e68baddc80 2023-06-23T18:41:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68ba2eed0 con 0x55e68bade5d0 2023-06-23T18:41:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f387bfff700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38640a1150 0x7f38640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68baddc80 0x55e68bade090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68baddc80 0x55e68bade090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48450/0 (socket says 172.21.15.2:48450) 2023-06-23T18:41:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f387b7fe700 1 -- 172.21.15.2:0/2724260670 learned_addr learned my addr 172.21.15.2:0/2724260670 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.969+0000 7f387affd700 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68badea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387b7fe700 1 -- 172.21.15.2:0/2724260670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 msgr2=0x55e68badea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68badea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387b7fe700 1 -- 172.21.15.2:0/2724260670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68badf000 msgr2=0x55e68badf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387bfff700 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68badf000 0x55e68badf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68badf000 0x55e68badf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387b7fe700 1 -- 172.21.15.2:0/2724260670 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e68b8b0630 con 0x55e68baddc80 2023-06-23T18:41:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387bfff700 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68badf000 0x55e68badf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:41:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68baddc80 0x55e68bade090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f386c096400 tx=0x7f386c099930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d7bbe2f5818aeb8 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:24.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f387bfff700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38640a1150 0x7f38640a3600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f387400b780 tx=0x7f3874009320 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f3861ffb700 1 -- 172.21.15.2:0/2724260670 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c09df30 con 0x55e68baddc80 2023-06-23T18:41:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f3861ffb700 1 -- 172.21.15.2:0/2724260670 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f386c0a2350 con 0x55e68baddc80 2023-06-23T18:41:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.970+0000 7f3861ffb700 1 -- 172.21.15.2:0/2724260670 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0a2120 con 0x55e68baddc80 2023-06-23T18:41:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2724260670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68baddc80 msgr2=0x55e68bade090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68baddc80 0x55e68bade090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f386c096400 tx=0x7f386c099930 comp rx=0 tx=0).stop 2023-06-23T18:41:24.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2724260670 shutdown_connections 2023-06-23T18:41:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68badf000 0x55e68badf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68baddc80 0x55e68bade090 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2724260670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68badea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2724260670 >> 172.21.15.2:0/2724260670 conn(0x55e68bad33f0 msgr2=0x55e68bad8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2724260670 shutdown_connections 2023-06-23T18:41:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.971+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2724260670 wait complete. 2023-06-23T18:41:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.972+0000 7f388dd3cb80 1 -- start start 2023-06-23T18:41:24.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.972+0000 7f388dd3cb80 1 Processor -- start 2023-06-23T18:41:24.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.972+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68baddc80 0x55e68bae75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.972+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68bade5d0 0x55e68bae7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.972+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 0x55e68baeeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.972+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b9ceac0 con 0x55e68bade5d0 2023-06-23T18:41:24.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.972+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b9ce7c0 con 0x55e68badf000 2023-06-23T18:41:24.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.972+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68b8b2920 con 0x55e68baddc80 2023-06-23T18:41:24.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f388dd3cb80 10 client.0 _ll_register_callbacks cb 0x55e68b9d4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:41:24.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68baddc80 0x55e68bae75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68baddc80 0x55e68bae75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58202/0 (socket says 172.21.15.2:58202) 2023-06-23T18:41:24.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 -- 172.21.15.2:0/3838404642 learned_addr learned my addr 172.21.15.2:0/3838404642 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:24.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387bfff700 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 0x55e68baeeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387affd700 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68bade5d0 0x55e68bae7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 -- 172.21.15.2:0/3838404642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 msgr2=0x55e68baeeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:24.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 0x55e68baeeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 -- 172.21.15.2:0/3838404642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68bade5d0 msgr2=0x55e68bae7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:24.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68bade5d0 0x55e68bae7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 -- 172.21.15.2:0/3838404642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387c1a43e0 con 0x55e68baddc80 2023-06-23T18:41:24.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387affd700 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68bade5d0 0x55e68bae7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:41:24.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f387b7fe700 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68baddc80 0x55e68bae75f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f386c09a560 tx=0x7f386c010a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc43d0d5df7b5249 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:24.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.973+0000 7f385affd700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:41:24.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f385affd700 1 -- 172.21.15.2:0/3838404642 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0ca200 con 0x55e68baddc80 2023-06-23T18:41:24.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f385affd700 1 -- 172.21.15.2:0/3838404642 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f386c0ca360 con 0x55e68baddc80 2023-06-23T18:41:24.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 1 -- 172.21.15.2:0/3838404642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e68bad0bb0 con 0x55e68baddc80 2023-06-23T18:41:24.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f385affd700 1 -- 172.21.15.2:0/3838404642 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f386c00ca10 con 0x55e68baddc80 2023-06-23T18:41:24.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 10 client.26689 fetch_fsmap learned FSMap version 41 2023-06-23T18:41:24.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 1 -- 172.21.15.2:0/3838404642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e68b85cde0 con 0x55e68baddc80 2023-06-23T18:41:24.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f385affd700 1 -- 172.21.15.2:0/3838404642 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f386c07c080 con 0x55e68baddc80 2023-06-23T18:41:24.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f385affd700 1 -- 172.21.15.2:0/3838404642 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f386c095dd0 con 0x55e68baddc80 2023-06-23T18:41:24.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 10 client.26689 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:41:24.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 10 client.26689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:41:24.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 20 client.26689 populate_metadata read hostname 'smithi002' 2023-06-23T18:41:24.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e68bb03ef0 0x55e68bb06300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:24.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 4 client.26689 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:41:24.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.974+0000 7f388dd3cb80 1 -- 172.21.15.2:0/3838404642 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e68ba27eb0 con 0x55e68bb03ef0 2023-06-23T18:41:24.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.975+0000 7f387affd700 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e68bb03ef0 0x55e68bb06300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:24.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.975+0000 7f385affd700 10 client.26689 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:41:24.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.975+0000 7f387affd700 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e68bb03ef0 0x55e68bb06300 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c53e88fc0445f57 server_cookie=2027f2c03cc50fc8 in_seq=0 out_seq=0 2023-06-23T18:41:24.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.976+0000 7f385affd700 1 -- 172.21.15.2:0/3838404642 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e68ba27eb0 con 0x55e68bb03ef0 2023-06-23T18:41:24.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.976+0000 7f385affd700 10 client.26689 handle_command_reply: tid=0 2023-06-23T18:41:24.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.976+0000 7f388dd3cb80 1 client.26689 shutdown 2023-06-23T18:41:24.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 -- 172.21.15.2:0/3838404642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68baddc80 msgr2=0x55e68bae75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:24.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68baddc80 0x55e68bae75f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f386c09a560 tx=0x7f386c010a70 comp rx=0 tx=0).stop 2023-06-23T18:41:24.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 -- 172.21.15.2:0/3838404642 shutdown_connections 2023-06-23T18:41:24.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e68bb03ef0 0x55e68bb06300 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 0x55e68baeeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68baddc80 0x55e68bae75f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f385affd700 0 client.26689 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:41:24.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/3838404642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68bade5d0 0x55e68bae7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:24.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 -- 172.21.15.2:0/3838404642 >> 172.21.15.2:0/3838404642 conn(0x55e68bad33f0 msgr2=0x55e68bad7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:24.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 -- 172.21.15.2:0/3838404642 shutdown_connections 2023-06-23T18:41:24.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 1 -- 172.21.15.2:0/3838404642 wait complete. 2023-06-23T18:41:24.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:24.977+0000 7f388dd3cb80 20 client.26689 trim_cache size 0 max 16384 2023-06-23T18:41:25.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.001+0000 7f388dd3cb80 1 Processor -- start 2023-06-23T18:41:25.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.001+0000 7f388dd3cb80 1 -- start start 2023-06-23T18:41:25.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68bb00470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:25.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68bade5d0 0x55e68baeeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:25.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68badf000 0x55e68bae73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:25.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68bae71c0 con 0x55e68baddc80 2023-06-23T18:41:25.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68bad2f70 con 0x55e68bade5d0 2023-06-23T18:41:25.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68bad9ff0 con 0x55e68badf000 2023-06-23T18:41:25.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68bb00470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:25.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68bb00470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57872/0 (socket says 172.21.15.2:57872) 2023-06-23T18:41:25.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 -- 172.21.15.2:0/452904052 learned_addr learned my addr 172.21.15.2:0/452904052 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:25.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387affd700 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68bade5d0 0x55e68baeeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:25.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387bfff700 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68badf000 0x55e68bae73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:25.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 -- 172.21.15.2:0/452904052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68badf000 msgr2=0x55e68bae73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:25.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68badf000 0x55e68bae73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 -- 172.21.15.2:0/452904052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68bade5d0 msgr2=0x55e68baeeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:25.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68bade5d0 0x55e68baeeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 -- 172.21.15.2:0/452904052 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e68b845c60 con 0x55e68baddc80 2023-06-23T18:41:25.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387affd700 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68bade5d0 0x55e68baeeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:41:25.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f387b7fe700 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68bb00470 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f386c096400 tx=0x7f386c0984b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de324a6af6a1b856 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:25.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.002+0000 7f38627fc700 1 -- 172.21.15.2:0/452904052 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0cab50 con 0x55e68baddc80 2023-06-23T18:41:25.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.003+0000 7f38627fc700 1 -- 172.21.15.2:0/452904052 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e68bad0bb0 con 0x55e68baddc80 2023-06-23T18:41:25.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.003+0000 7f38627fc700 1 -- 172.21.15.2:0/452904052 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0ca200 con 0x55e68baddc80 2023-06-23T18:41:25.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.003+0000 7f388dd3cb80 1 -- 172.21.15.2:0/452904052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 msgr2=0x55e68bb00470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:25.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.003+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68bb00470 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f386c096400 tx=0x7f386c0984b0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.003+0000 7f388dd3cb80 1 -- 172.21.15.2:0/452904052 shutdown_connections 2023-06-23T18:41:25.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68badf000 0x55e68bae73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68bb00470 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/452904052 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68bade5d0 0x55e68baeeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 -- 172.21.15.2:0/452904052 >> 172.21.15.2:0/452904052 conn(0x55e68bad33f0 msgr2=0x55e68bae62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:25.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 -- 172.21.15.2:0/452904052 shutdown_connections 2023-06-23T18:41:25.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 -- 172.21.15.2:0/452904052 wait complete. 2023-06-23T18:41:25.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 -- start start 2023-06-23T18:41:25.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 Processor -- start 2023-06-23T18:41:25.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68baee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:25.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68baee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:25.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 0x55e68baeadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:25.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68baeb340 con 0x55e68baddc80 2023-06-23T18:41:25.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68baeb480 con 0x55e68badf000 2023-06-23T18:41:25.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e68baeb640 con 0x55e68bade5d0 2023-06-23T18:41:25.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f388dd3cb80 10 client.0 _ll_register_callbacks cb 0x55e68b9d4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:41:25.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68baee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:25.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f387b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68baee320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57878/0 (socket says 172.21.15.2:57878) 2023-06-23T18:41:25.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f387b7fe700 1 -- 172.21.15.2:0/2865705654 learned_addr learned my addr 172.21.15.2:0/2865705654 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:25.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.004+0000 7f387affd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68baee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:25.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f387b7fe700 1 -- 172.21.15.2:0/2865705654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 msgr2=0x55e68baee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:25.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68baee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f387b7fe700 1 -- 172.21.15.2:0/2865705654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 msgr2=0x55e68baeadc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:41:25.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 0x55e68baeadc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f387b7fe700 1 -- 172.21.15.2:0/2865705654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f387001fa40 con 0x55e68baddc80 2023-06-23T18:41:25.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f387affd700 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68baee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:41:25.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f387b7fe700 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68baee320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f386c096160 tx=0x7f386c010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db99ccb32e3e5e1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:25.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f385affd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:41:25.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f385affd700 1 -- 172.21.15.2:0/2865705654 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0cacc0 con 0x55e68baddc80 2023-06-23T18:41:25.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f385affd700 1 -- 172.21.15.2:0/2865705654 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f386c0ca400 con 0x55e68baddc80 2023-06-23T18:41:25.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2865705654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f386c0a2350 con 0x55e68baddc80 2023-06-23T18:41:25.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f385affd700 1 -- 172.21.15.2:0/2865705654 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f386c095cf0 con 0x55e68baddc80 2023-06-23T18:41:25.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f388dd3cb80 10 client.16959 fetch_fsmap learned FSMap version 41 2023-06-23T18:41:25.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.005+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2865705654 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e68b85cde0 con 0x55e68baddc80 2023-06-23T18:41:25.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f385affd700 1 -- 172.21.15.2:0/2865705654 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f386c07c030 con 0x55e68baddc80 2023-06-23T18:41:25.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f385affd700 1 -- 172.21.15.2:0/2865705654 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f386c098e60 con 0x55e68baddc80 2023-06-23T18:41:25.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f388dd3cb80 10 client.16959 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:41:25.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f388dd3cb80 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:41:25.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f388dd3cb80 20 client.16959 populate_metadata read hostname 'smithi002' 2023-06-23T18:41:25.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e68bb58d40 0x55e68bb5b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:25.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f388dd3cb80 4 client.16959 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:41:25.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2865705654 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e68b9ea050 con 0x55e68bb58d40 2023-06-23T18:41:25.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.006+0000 7f387affd700 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e68bb58d40 0x55e68bb5b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:25.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.007+0000 7f387affd700 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e68bb58d40 0x55e68bb5b120 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d4dbc0e3fcab88 server_cookie=59398c303d60b9a4 in_seq=0 out_seq=0 2023-06-23T18:41:25.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.007+0000 7f385affd700 10 client.16959 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:41:25.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.007+0000 7f385affd700 1 -- 172.21.15.2:0/2865705654 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55e68b9ea050 con 0x55e68bb58d40 2023-06-23T18:41:25.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.007+0000 7f385affd700 10 client.16959 handle_command_reply: tid=0 2023-06-23T18:41:25.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.007+0000 7f388dd3cb80 1 client.16959 shutdown 2023-06-23T18:41:25.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.007+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2865705654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 msgr2=0x55e68baee320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:25.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.007+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68baee320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f386c096160 tx=0x7f386c010ae0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2865705654 shutdown_connections 2023-06-23T18:41:25.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55e68bb58d40 0x55e68bb5b120 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55e68badf000 0x55e68baeadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f385affd700 0 client.16959 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:41:25.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55e68baddc80 0x55e68baee320 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 1 --2- 172.21.15.2:0/2865705654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e68bade5d0 0x55e68baee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2865705654 >> 172.21.15.2:0/2865705654 conn(0x55e68bad33f0 msgr2=0x55e68bae1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:25.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2865705654 shutdown_connections 2023-06-23T18:41:25.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 1 -- 172.21.15.2:0/2865705654 wait complete. 2023-06-23T18:41:25.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f388dd3cb80 20 client.16959 trim_cache size 0 max 16384 2023-06-23T18:41:25.022 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:41:25.022 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:41:25.022 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:41:25.022 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:41:25.023 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:41:25.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:41:25.023 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:41:25.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:41:25.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:41:25.023 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:41:25.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f3881bb7700 1 -- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38640a1150 msgr2=0x7f38640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:25.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f3881bb7700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38640a1150 0x7f38640a3600 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f387400b780 tx=0x7f3874009320 comp rx=0 tx=0).stop 2023-06-23T18:41:25.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f3881bb7700 1 -- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c0ffb60 msgr2=0x7f387c19bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:25.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f3881bb7700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c0ffb60 0x7f387c19bec0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f386c00e9a0 tx=0x7f386c00ecb0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.008+0000 7f3881bb7700 1 -- 172.21.15.2:0/2615101758 shutdown_connections 2023-06-23T18:41:25.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.009+0000 7f3881bb7700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f387c1017a0 0x7f387c1a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.009+0000 7f3881bb7700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f387c0ffb60 0x7f387c19bec0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.009+0000 7f3881bb7700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f38640a1150 0x7f38640a3600 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.009+0000 7f3881bb7700 1 --2- 172.21.15.2:0/2615101758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f387c100df0 0x7f387c19c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:25.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.009+0000 7f3881bb7700 1 -- 172.21.15.2:0/2615101758 >> 172.21.15.2:0/2615101758 conn(0x7f387c0fb180 msgr2=0x7f387c0fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:25.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.009+0000 7f3881bb7700 1 -- 172.21.15.2:0/2615101758 shutdown_connections 2023-06-23T18:41:25.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:25.009+0000 7f3881bb7700 1 -- 172.21.15.2:0/2615101758 wait complete. 2023-06-23T18:41:25.030 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:41:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:26 smithi002 ceph-mon[95947]: pgmap v878: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:26 smithi082 ceph-mon[97889]: pgmap v878: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:26 smithi017 ceph-mon[97472]: pgmap v878: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:41:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 455.0 s, 350.2 tps, lat 285.427 ms stddev 49.694 2023-06-23T18:41:28.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:28 smithi002 ceph-mon[95947]: pgmap v879: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-06-23T18:41:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:28 smithi082 ceph-mon[97889]: pgmap v879: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-06-23T18:41:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:28 smithi017 ceph-mon[97472]: pgmap v879: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.12k op/s 2023-06-23T18:41:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:30 smithi002 ceph-mon[95947]: pgmap v880: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.19k op/s 2023-06-23T18:41:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:30 smithi082 ceph-mon[97889]: pgmap v880: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.19k op/s 2023-06-23T18:41:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:30 smithi017 ceph-mon[97472]: pgmap v880: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.19k op/s 2023-06-23T18:41:32.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 460.0 s, 402.6 tps, lat 248.059 ms stddev 39.871 2023-06-23T18:41:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:32 smithi002 ceph-mon[95947]: pgmap v881: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:32.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:32 smithi082 ceph-mon[97889]: pgmap v881: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:32 smithi017 ceph-mon[97472]: pgmap v881: 97 pgs: 97 active+clean; 8.4 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:34 smithi002 ceph-mon[95947]: pgmap v882: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:34 smithi082 ceph-mon[97889]: pgmap v882: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:34 smithi017 ceph-mon[97472]: pgmap v882: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:36 smithi002 ceph-mon[95947]: pgmap v883: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:41:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:36 smithi082 ceph-mon[97889]: pgmap v883: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:41:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:36 smithi017 ceph-mon[97472]: pgmap v883: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:41:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 465.0 s, 343.4 tps, lat 293.662 ms stddev 42.166 2023-06-23T18:41:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:38 smithi002 ceph-mon[95947]: pgmap v884: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:41:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:38 smithi082 ceph-mon[97889]: pgmap v884: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:41:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:38 smithi017 ceph-mon[97472]: pgmap v884: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:41:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:40 smithi002 ceph-mon[95947]: pgmap v885: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:40 smithi082 ceph-mon[97889]: pgmap v885: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:40.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:40 smithi017 ceph-mon[97472]: pgmap v885: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:41:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 470.0 s, 318.4 tps, lat 312.505 ms stddev 54.622 2023-06-23T18:41:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:42 smithi002 ceph-mon[95947]: pgmap v886: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:41:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:42 smithi082 ceph-mon[97889]: pgmap v886: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:41:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:42 smithi017 ceph-mon[97472]: pgmap v886: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:41:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:44 smithi002 ceph-mon[95947]: pgmap v887: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:41:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:44 smithi082 ceph-mon[97889]: pgmap v887: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:41:44.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:44 smithi017 ceph-mon[97472]: pgmap v887: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-06-23T18:41:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:46 smithi002 ceph-mon[95947]: pgmap v888: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:41:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:46 smithi082 ceph-mon[97889]: pgmap v888: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:41:46.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:46 smithi017 ceph-mon[97472]: pgmap v888: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:41:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 475.0 s, 309.4 tps, lat 322.546 ms stddev 122.512 2023-06-23T18:41:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:48 smithi002 ceph-mon[95947]: pgmap v889: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:41:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:48 smithi082 ceph-mon[97889]: pgmap v889: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:41:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:48 smithi017 ceph-mon[97472]: pgmap v889: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:41:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:50 smithi002 ceph-mon[95947]: pgmap v890: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2023-06-23T18:41:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:50 smithi082 ceph-mon[97889]: pgmap v890: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2023-06-23T18:41:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:50 smithi017 ceph-mon[97472]: pgmap v890: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.2 MiB/s wr, 1.03k op/s 2023-06-23T18:41:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 480.0 s, 323.4 tps, lat 306.795 ms stddev 53.007 2023-06-23T18:41:52.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:52 smithi002 ceph-mon[95947]: pgmap v891: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:41:52.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:52 smithi082 ceph-mon[97889]: pgmap v891: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:41:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:52 smithi017 ceph-mon[97472]: pgmap v891: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:41:54.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:54 smithi002 ceph-mon[95947]: pgmap v892: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-06-23T18:41:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:54 smithi082 ceph-mon[97889]: pgmap v892: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-06-23T18:41:54.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:54 smithi017 ceph-mon[97472]: pgmap v892: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-06-23T18:41:55.032 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:41:55.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.246+0000 7f6c55c5b700 1 -- 172.21.15.2:0/1655474330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50101670 msgr2=0x7f6c50107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.246+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/1655474330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50101670 0x7f6c50107f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009f90 tx=0x7f6c4800b330 comp rx=0 tx=0).stop 2023-06-23T18:41:55.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.246+0000 7f6c55c5b700 1 -- 172.21.15.2:0/1655474330 shutdown_connections 2023-06-23T18:41:55.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.246+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/1655474330 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50101670 0x7f6c50107f30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.246+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/1655474330 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c500ffaf0 0x7f6c500fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.246+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/1655474330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c50100ce0 0x7f6c50101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.246+0000 7f6c55c5b700 1 -- 172.21.15.2:0/1655474330 >> 172.21.15.2:0/1655474330 conn(0x7f6c500fb0f0 msgr2=0x7f6c500fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:55.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.246+0000 7f6c55c5b700 1 -- 172.21.15.2:0/1655474330 shutdown_connections 2023-06-23T18:41:55.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.247+0000 7f6c55c5b700 1 -- 172.21.15.2:0/1655474330 wait complete. 2023-06-23T18:41:55.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.247+0000 7f6c55c5b700 1 Processor -- start 2023-06-23T18:41:55.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.247+0000 7f6c55c5b700 1 -- start start 2023-06-23T18:41:55.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c55c5b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c500ffaf0 0x7f6c50193630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c55c5b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50100ce0 0x7f6c50193b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c55c5b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c50101670 0x7f6c5019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c55c5b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c501ac240 con 0x7f6c500ffaf0 2023-06-23T18:41:55.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c55c5b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c501ac380 con 0x7f6c50100ce0 2023-06-23T18:41:55.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c55c5b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c501ac4c0 con 0x7f6c50101670 2023-06-23T18:41:55.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50100ce0 0x7f6c50193b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50100ce0 0x7f6c50193b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48486/0 (socket says 172.21.15.2:48486) 2023-06-23T18:41:55.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4effd700 1 -- 172.21.15.2:0/3971038570 learned_addr learned my addr 172.21.15.2:0/3971038570 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:55.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c50101670 0x7f6c5019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c500ffaf0 0x7f6c50193630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4ffff700 1 -- 172.21.15.2:0/3971038570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50100ce0 msgr2=0x7f6c50193b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50100ce0 0x7f6c50193b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4ffff700 1 -- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c500ffaf0 msgr2=0x7f6c50193630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c500ffaf0 0x7f6c50193630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.248+0000 7f6c4ffff700 1 -- 172.21.15.2:0/3971038570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48009c40 con 0x7f6c50101670 2023-06-23T18:41:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.249+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c50101670 0x7f6c5019abe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800b6c0 tx=0x7f6c48008c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.249+0000 7f6c4cff9700 1 -- 172.21.15.2:0/3971038570 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4800e040 con 0x7f6c50101670 2023-06-23T18:41:55.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.249+0000 7f6c4cff9700 1 -- 172.21.15.2:0/3971038570 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c4800bc00 con 0x7f6c50101670 2023-06-23T18:41:55.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.249+0000 7f6c4cff9700 1 -- 172.21.15.2:0/3971038570 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48005040 con 0x7f6c50101670 2023-06-23T18:41:55.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.249+0000 7f6c55c5b700 1 -- 172.21.15.2:0/3971038570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c501ac740 con 0x7f6c50101670 2023-06-23T18:41:55.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.249+0000 7f6c55c5b700 1 -- 172.21.15.2:0/3971038570 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c501acb70 con 0x7f6c50101670 2023-06-23T18:41:55.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.252+0000 7f6c4cff9700 1 -- 172.21.15.2:0/3971038570 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6c4801d470 con 0x7f6c50101670 2023-06-23T18:41:55.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.253+0000 7f6c4cff9700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c380a1150 0x7f6c380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.253+0000 7f6c4cff9700 1 -- 172.21.15.2:0/3971038570 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c480cdc60 con 0x7f6c50101670 2023-06-23T18:41:55.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.253+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c380a1150 0x7f6c380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.253+0000 7f6c61de0b80 1 Processor -- start 2023-06-23T18:41:55.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c61de0b80 1 -- start start 2023-06-23T18:41:55.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 0x559ad626ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626c000 0x559ad626c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad616fcb0 con 0x559ad626ac80 2023-06-23T18:41:55.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad61c2a40 con 0x559ad626c000 2023-06-23T18:41:55.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad61bbed0 con 0x559ad626b5d0 2023-06-23T18:41:55.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57900/0 (socket says 172.21.15.2:57900) 2023-06-23T18:41:55.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/2230458611 learned_addr learned my addr 172.21.15.2:0/2230458611 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:55.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626c000 0x559ad626c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4effd700 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 0x559ad626ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4ffff700 1 -- 172.21.15.2:0/2230458611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 msgr2=0x559ad626ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:55.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 0x559ad626ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4ffff700 1 -- 172.21.15.2:0/2230458611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 msgr2=0x559ad626b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:55.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4ffff700 1 -- 172.21.15.2:0/2230458611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ad603d630 con 0x559ad626c000 2023-06-23T18:41:55.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:41:55.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.254+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c380a1150 0x7f6c380a3600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6c50100b40 tx=0x7f6c40009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:55.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.255+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626c000 0x559ad626c470 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007890 tx=0x7f6c4800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8b3a517318edd1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:55.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.255+0000 7f6c35ffb700 1 -- 172.21.15.2:0/2230458611 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4809f130 con 0x559ad626c000 2023-06-23T18:41:55.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.255+0000 7f6c35ffb700 1 -- 172.21.15.2:0/2230458611 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c480a34c0 con 0x559ad626c000 2023-06-23T18:41:55.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.255+0000 7f6c35ffb700 1 -- 172.21.15.2:0/2230458611 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4809b360 con 0x559ad626c000 2023-06-23T18:41:55.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2230458611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626c000 msgr2=0x559ad626c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626c000 0x559ad626c470 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6c48007890 tx=0x7f6c4800b330 comp rx=0 tx=0).stop 2023-06-23T18:41:55.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2230458611 shutdown_connections 2023-06-23T18:41:55.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626c000 0x559ad626c470 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2230458611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 0x559ad626ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2230458611 >> 172.21.15.2:0/2230458611 conn(0x559ad62603f0 msgr2=0x559ad6265520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:55.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2230458611 shutdown_connections 2023-06-23T18:41:55.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2230458611 wait complete. 2023-06-23T18:41:55.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.256+0000 7f6c61de0b80 1 -- start start 2023-06-23T18:41:55.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c61de0b80 1 Processor -- start 2023-06-23T18:41:55.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad6274390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad62748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 0x559ad627b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad615bac0 con 0x559ad626ac80 2023-06-23T18:41:55.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad615b7c0 con 0x559ad626b5d0 2023-06-23T18:41:55.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad603f920 con 0x559ad626c000 2023-06-23T18:41:55.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c61de0b80 10 client.0 _ll_register_callbacks cb 0x559ad6161260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:41:55.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad6274390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad6274390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57906/0 (socket says 172.21.15.2:57906) 2023-06-23T18:41:55.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4effd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad62748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/2401110118 learned_addr learned my addr 172.21.15.2:0/2401110118 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:55.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 0x559ad627b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/2401110118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 msgr2=0x559ad627b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:55.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 0x559ad627b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/2401110118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 msgr2=0x559ad62748d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:55.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad62748d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.257+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/2401110118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c40017070 con 0x559ad626ac80 2023-06-23T18:41:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.258+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad6274390 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6c4001db80 tx=0x7f6c400047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707ba178475f11f5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:55.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.258+0000 7f6c2effd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:41:55.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.258+0000 7f6c2effd700 1 -- 172.21.15.2:0/2401110118 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000ec50 con 0x559ad626ac80 2023-06-23T18:41:55.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.258+0000 7f6c2effd700 1 -- 172.21.15.2:0/2401110118 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c4000edb0 con 0x559ad626ac80 2023-06-23T18:41:55.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.258+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2401110118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ad625dbb0 con 0x559ad626ac80 2023-06-23T18:41:55.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.258+0000 7f6c2effd700 1 -- 172.21.15.2:0/2401110118 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c40026400 con 0x559ad626ac80 2023-06-23T18:41:55.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.258+0000 7f6c61de0b80 10 client.16971 fetch_fsmap learned FSMap version 41 2023-06-23T18:41:55.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.258+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2401110118 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f6c501acb70 con 0x559ad626ac80 2023-06-23T18:41:55.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c2effd700 1 -- 172.21.15.2:0/2401110118 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559ad625dbb0 con 0x559ad626ac80 2023-06-23T18:41:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c2effd700 1 -- 172.21.15.2:0/2401110118 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6c400253a0 con 0x559ad626ac80 2023-06-23T18:41:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c61de0b80 10 client.16971 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:41:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c61de0b80 10 client.16971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:41:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c61de0b80 20 client.16971 populate_metadata read hostname 'smithi002' 2023-06-23T18:41:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559ad6280130 0x559ad6282540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c61de0b80 4 client.16971 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:41:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2401110118 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ad61b4eb0 con 0x559ad6280130 2023-06-23T18:41:55.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.259+0000 7f6c4effd700 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559ad6280130 0x559ad6282540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.260+0000 7f6c4effd700 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559ad6280130 0x559ad6282540 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c4fbf4f1df4712e server_cookie=f77e76b0414e7086 in_seq=0 out_seq=0 2023-06-23T18:41:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.260+0000 7f6c2effd700 10 client.16971 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:41:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c2effd700 1 -- 172.21.15.2:0/2401110118 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559ad61b4eb0 con 0x559ad6280130 2023-06-23T18:41:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c2effd700 10 client.16971 handle_command_reply: tid=0 2023-06-23T18:41:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 client.16971 shutdown 2023-06-23T18:41:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2401110118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 msgr2=0x559ad6274390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad6274390 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6c4001db80 tx=0x7f6c400047c0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2401110118 shutdown_connections 2023-06-23T18:41:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559ad6280130 0x559ad6282540 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 0x559ad627b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c2effd700 0 client.16971 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:41:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad6274390 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/2401110118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad62748d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2401110118 >> 172.21.15.2:0/2401110118 conn(0x559ad62603f0 msgr2=0x559ad6264c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:55.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2401110118 shutdown_connections 2023-06-23T18:41:55.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 1 -- 172.21.15.2:0/2401110118 wait complete. 2023-06-23T18:41:55.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.261+0000 7f6c61de0b80 20 client.16971 trim_cache size 0 max 16384 2023-06-23T18:41:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c61de0b80 1 Processor -- start 2023-06-23T18:41:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c61de0b80 1 -- start start 2023-06-23T18:41:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad627b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 0x559ad626f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad627d630 con 0x559ad626ac80 2023-06-23T18:41:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad6266420 con 0x559ad626b5d0 2023-06-23T18:41:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad6272220 con 0x559ad626c000 2023-06-23T18:41:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57914/0 (socket says 172.21.15.2:57914) 2023-06-23T18:41:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/3631321710 learned_addr learned my addr 172.21.15.2:0/3631321710 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:55.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 0x559ad626f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4effd700 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad627b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/3631321710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 msgr2=0x559ad626f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 0x559ad626f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/3631321710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 msgr2=0x559ad627b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad627b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/3631321710 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ad5fe9de0 con 0x559ad626ac80 2023-06-23T18:41:55.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.289+0000 7f6c4effd700 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad627b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:41:55.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.290+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626ef60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6c4001d440 tx=0x7f6c40005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9bf7751eb8718e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:55.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.290+0000 7f6c367fc700 1 -- 172.21.15.2:0/3631321710 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40025a20 con 0x559ad626ac80 2023-06-23T18:41:55.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.290+0000 7f6c367fc700 1 -- 172.21.15.2:0/3631321710 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c40025c60 con 0x559ad626ac80 2023-06-23T18:41:55.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.290+0000 7f6c367fc700 1 -- 172.21.15.2:0/3631321710 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40032810 con 0x559ad626ac80 2023-06-23T18:41:55.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 -- 172.21.15.2:0/3631321710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 msgr2=0x559ad626ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626ef60 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6c4001d440 tx=0x7f6c40005d20 comp rx=0 tx=0).stop 2023-06-23T18:41:55.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 -- 172.21.15.2:0/3631321710 shutdown_connections 2023-06-23T18:41:55.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626c000 0x559ad626f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626ac80 0x559ad626ef60 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/3631321710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626b5d0 0x559ad627b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 -- 172.21.15.2:0/3631321710 >> 172.21.15.2:0/3631321710 conn(0x559ad62603f0 msgr2=0x559ad6273060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 -- 172.21.15.2:0/3631321710 shutdown_connections 2023-06-23T18:41:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.291+0000 7f6c61de0b80 1 -- 172.21.15.2:0/3631321710 wait complete. 2023-06-23T18:41:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 1 -- start start 2023-06-23T18:41:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 1 Processor -- start 2023-06-23T18:41:55.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626ac80 0x559ad62783b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 0x559ad62788f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626c000 0x559ad62d3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad627b910 con 0x559ad626c000 2023-06-23T18:41:55.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad627ba80 con 0x559ad626ac80 2023-06-23T18:41:55.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ad627bc40 con 0x559ad626b5d0 2023-06-23T18:41:55.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c61de0b80 10 client.0 _ll_register_callbacks cb 0x559ad6161260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:41:55.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626ac80 0x559ad62783b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c4f7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626ac80 0x559ad62783b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48514/0 (socket says 172.21.15.2:48514) 2023-06-23T18:41:55.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/835654090 learned_addr learned my addr 172.21.15.2:0/835654090 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:41:55.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c4ffff700 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626c000 0x559ad62d3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.292+0000 7f6c4effd700 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 0x559ad62788f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/835654090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 msgr2=0x559ad62788f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:55.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 0x559ad62788f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/835654090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626c000 msgr2=0x559ad62d3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:41:55.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626c000 0x559ad62d3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c4f7fe700 1 -- 172.21.15.2:0/835654090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c501acb70 con 0x559ad626ac80 2023-06-23T18:41:55.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c4f7fe700 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626ac80 0x559ad62783b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c40025730 tx=0x7f6c4000eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ae65de268a2cab0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:41:55.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c2effd700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:41:55.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c2effd700 1 -- 172.21.15.2:0/835654090 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000e690 con 0x559ad626ac80 2023-06-23T18:41:55.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c2effd700 1 -- 172.21.15.2:0/835654090 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6c40025990 con 0x559ad626ac80 2023-06-23T18:41:55.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c61de0b80 1 -- 172.21.15.2:0/835654090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c480a34c0 con 0x559ad626ac80 2023-06-23T18:41:55.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c2effd700 1 -- 172.21.15.2:0/835654090 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c40004020 con 0x559ad626ac80 2023-06-23T18:41:55.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c61de0b80 10 client.26489 fetch_fsmap learned FSMap version 41 2023-06-23T18:41:55.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.293+0000 7f6c61de0b80 1 -- 172.21.15.2:0/835654090 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x559ad5fd2c60 con 0x559ad626ac80 2023-06-23T18:41:55.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.294+0000 7f6c2effd700 1 -- 172.21.15.2:0/835654090 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c480a34c0 con 0x559ad626ac80 2023-06-23T18:41:55.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.295+0000 7f6c2effd700 1 -- 172.21.15.2:0/835654090 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6c40043070 con 0x559ad626ac80 2023-06-23T18:41:55.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.295+0000 7f6c61de0b80 10 client.26489 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:41:55.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.295+0000 7f6c61de0b80 10 client.26489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:41:55.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.295+0000 7f6c61de0b80 20 client.26489 populate_metadata read hostname 'smithi002' 2023-06-23T18:41:55.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.295+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559ad62d5a20 0x559ad62d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:41:55.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.295+0000 7f6c61de0b80 4 client.26489 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:41:55.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.295+0000 7f6c61de0b80 1 -- 172.21.15.2:0/835654090 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ad6177050 con 0x559ad62d5a20 2023-06-23T18:41:55.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.295+0000 7f6c4effd700 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559ad62d5a20 0x559ad62d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:41:55.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.296+0000 7f6c4effd700 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559ad62d5a20 0x559ad62d7e30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d91d6380ccafe3 server_cookie=9910e64e64bf26be in_seq=0 out_seq=0 2023-06-23T18:41:55.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.296+0000 7f6c2effd700 10 client.26489 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:41:55.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.298+0000 7f6c2effd700 1 -- 172.21.15.2:0/835654090 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559ad6177050 con 0x559ad62d5a20 2023-06-23T18:41:55.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.298+0000 7f6c2effd700 10 client.26489 handle_command_reply: tid=0 2023-06-23T18:41:55.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.298+0000 7f6c61de0b80 1 client.26489 shutdown 2023-06-23T18:41:55.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.298+0000 7f6c61de0b80 1 -- 172.21.15.2:0/835654090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626ac80 msgr2=0x559ad62783b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.298+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626ac80 0x559ad62783b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6c40025730 tx=0x7f6c4000eea0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.298+0000 7f6c61de0b80 1 -- 172.21.15.2:0/835654090 shutdown_connections 2023-06-23T18:41:55.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.298+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559ad62d5a20 0x559ad62d7e30 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c2effd700 0 client.26489 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:41:55.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559ad626c000 0x559ad62d3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559ad626ac80 0x559ad62783b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c61de0b80 1 --2- 172.21.15.2:0/835654090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559ad626b5d0 0x559ad62788f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c61de0b80 1 -- 172.21.15.2:0/835654090 >> 172.21.15.2:0/835654090 conn(0x559ad62603f0 msgr2=0x559ad626ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:55.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c61de0b80 1 -- 172.21.15.2:0/835654090 shutdown_connections 2023-06-23T18:41:55.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c61de0b80 1 -- 172.21.15.2:0/835654090 wait complete. 2023-06-23T18:41:55.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c61de0b80 20 client.26489 trim_cache size 0 max 16384 2023-06-23T18:41:55.310 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:41:55.310 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:41:55.310 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:41:55.310 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:41:55.310 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:41:55.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:41:55.310 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:41:55.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:41:55.311 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:41:55.311 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:41:55.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c55c5b700 1 -- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c380a1150 msgr2=0x7f6c380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c380a1150 0x7f6c380a3600 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6c50100b40 tx=0x7f6c40009310 comp rx=0 tx=0).stop 2023-06-23T18:41:55.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c55c5b700 1 -- 172.21.15.2:0/3971038570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c50101670 msgr2=0x7f6c5019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:41:55.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.299+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c50101670 0x7f6c5019abe0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800b6c0 tx=0x7f6c48008c80 comp rx=0 tx=0).stop 2023-06-23T18:41:55.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.300+0000 7f6c55c5b700 1 -- 172.21.15.2:0/3971038570 shutdown_connections 2023-06-23T18:41:55.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.300+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c50101670 0x7f6c5019abe0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.300+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c500ffaf0 0x7f6c50193630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.300+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6c380a1150 0x7f6c380a3600 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.300+0000 7f6c55c5b700 1 --2- 172.21.15.2:0/3971038570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c50100ce0 0x7f6c50193b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:41:55.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.300+0000 7f6c55c5b700 1 -- 172.21.15.2:0/3971038570 >> 172.21.15.2:0/3971038570 conn(0x7f6c500fb0f0 msgr2=0x7f6c5010a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:41:55.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.300+0000 7f6c55c5b700 1 -- 172.21.15.2:0/3971038570 shutdown_connections 2023-06-23T18:41:55.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:41:55.300+0000 7f6c55c5b700 1 -- 172.21.15.2:0/3971038570 wait complete. 2023-06-23T18:41:55.321 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:41:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:56 smithi002 ceph-mon[95947]: pgmap v893: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 971 op/s 2023-06-23T18:41:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:56 smithi082 ceph-mon[97889]: pgmap v893: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 971 op/s 2023-06-23T18:41:56.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:56 smithi017 ceph-mon[97472]: pgmap v893: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 971 op/s 2023-06-23T18:41:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 485.0 s, 339.2 tps, lat 295.518 ms stddev 51.065 2023-06-23T18:41:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:41:58 smithi002 ceph-mon[95947]: pgmap v894: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.4 MiB/s wr, 993 op/s 2023-06-23T18:41:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:41:58 smithi082 ceph-mon[97889]: pgmap v894: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.4 MiB/s wr, 993 op/s 2023-06-23T18:41:58.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:41:58 smithi017 ceph-mon[97472]: pgmap v894: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.4 MiB/s wr, 993 op/s 2023-06-23T18:42:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:00 smithi002 ceph-mon[95947]: pgmap v895: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.07k op/s 2023-06-23T18:42:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:00 smithi082 ceph-mon[97889]: pgmap v895: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.07k op/s 2023-06-23T18:42:00.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:00 smithi017 ceph-mon[97472]: pgmap v895: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.07k op/s 2023-06-23T18:42:02.028 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 490.0 s, 187.1 tps, lat 247.798 ms stddev 91.660 2023-06-23T18:42:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:02 smithi082 ceph-mon[97889]: pgmap v896: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.08k op/s 2023-06-23T18:42:02.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:02 smithi017 ceph-mon[97472]: pgmap v896: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.08k op/s 2023-06-23T18:42:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:02 smithi002 ceph-mon[95947]: pgmap v896: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.08k op/s 2023-06-23T18:42:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:04 smithi082 ceph-mon[97889]: pgmap v897: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.07k op/s 2023-06-23T18:42:04.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:04 smithi017 ceph-mon[97472]: pgmap v897: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.07k op/s 2023-06-23T18:42:05.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:04 smithi002 ceph-mon[95947]: pgmap v897: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.07k op/s 2023-06-23T18:42:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:06 smithi082 ceph-mon[97889]: pgmap v898: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 967 op/s 2023-06-23T18:42:06.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:06 smithi017 ceph-mon[97472]: pgmap v898: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 967 op/s 2023-06-23T18:42:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 495.0 s, 225.9 tps, lat 682.888 ms stddev 1519.765 2023-06-23T18:42:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:06 smithi002 ceph-mon[95947]: pgmap v898: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 967 op/s 2023-06-23T18:42:08.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:08 smithi082 ceph-mon[97889]: pgmap v899: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 868 op/s 2023-06-23T18:42:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:08 smithi017 ceph-mon[97472]: pgmap v899: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 868 op/s 2023-06-23T18:42:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:08 smithi002 ceph-mon[95947]: pgmap v899: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 868 op/s 2023-06-23T18:42:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:10 smithi082 ceph-mon[97889]: pgmap v900: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 904 op/s 2023-06-23T18:42:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:10 smithi017 ceph-mon[97472]: pgmap v900: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 904 op/s 2023-06-23T18:42:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:10 smithi002 ceph-mon[95947]: pgmap v900: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 904 op/s 2023-06-23T18:42:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 500.0 s, 350.4 tps, lat 286.053 ms stddev 41.662 2023-06-23T18:42:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:12 smithi082 ceph-mon[97889]: pgmap v901: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 866 op/s 2023-06-23T18:42:12.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:12 smithi017 ceph-mon[97472]: pgmap v901: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 866 op/s 2023-06-23T18:42:13.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:12 smithi002 ceph-mon[95947]: pgmap v901: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.1 MiB/s wr, 866 op/s 2023-06-23T18:42:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:14 smithi082 ceph-mon[97889]: pgmap v902: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.1 MiB/s wr, 876 op/s 2023-06-23T18:42:14.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:14 smithi017 ceph-mon[97472]: pgmap v902: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.1 MiB/s wr, 876 op/s 2023-06-23T18:42:15.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:14 smithi002 ceph-mon[95947]: pgmap v902: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.1 MiB/s wr, 876 op/s 2023-06-23T18:42:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:16 smithi082 ceph-mon[97889]: pgmap v903: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 825 op/s 2023-06-23T18:42:16.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:16 smithi017 ceph-mon[97472]: pgmap v903: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 825 op/s 2023-06-23T18:42:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 505.0 s, 334.0 tps, lat 291.504 ms stddev 79.412 2023-06-23T18:42:17.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:16 smithi002 ceph-mon[95947]: pgmap v903: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 825 op/s 2023-06-23T18:42:18.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:18 smithi082 ceph-mon[97889]: pgmap v904: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.9 MiB/s wr, 898 op/s 2023-06-23T18:42:18.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:18 smithi017 ceph-mon[97472]: pgmap v904: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.9 MiB/s wr, 898 op/s 2023-06-23T18:42:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:18 smithi002 ceph-mon[95947]: pgmap v904: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.9 MiB/s wr, 898 op/s 2023-06-23T18:42:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:20 smithi082 ceph-mon[97889]: pgmap v905: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.06k op/s 2023-06-23T18:42:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:20 smithi017 ceph-mon[97472]: pgmap v905: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.06k op/s 2023-06-23T18:42:21.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:20 smithi002 ceph-mon[95947]: pgmap v905: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.06k op/s 2023-06-23T18:42:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 510.0 s, 230.0 tps, lat 445.011 ms stddev 205.955 2023-06-23T18:42:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:22 smithi082 ceph-mon[97889]: pgmap v906: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:42:22.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:22 smithi017 ceph-mon[97472]: pgmap v906: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:42:23.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:22 smithi002 ceph-mon[95947]: pgmap v906: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:42:23.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:23 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:42:23.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:23 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:42:24.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:23 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:42:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:24 smithi082 ceph-mon[97889]: pgmap v907: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.7 MiB/s wr, 981 op/s 2023-06-23T18:42:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:42:24.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:42:24.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:42:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:24 smithi017 ceph-mon[97472]: pgmap v907: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.7 MiB/s wr, 981 op/s 2023-06-23T18:42:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:42:24.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:42:24.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:42:25.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:24 smithi002 ceph-mon[95947]: pgmap v907: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.7 MiB/s wr, 981 op/s 2023-06-23T18:42:25.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:42:25.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:42:25.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:42:25.322 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:42:25.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 -- 172.21.15.2:0/1793936259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00fea50 msgr2=0x7fcea00fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 --2- 172.21.15.2:0/1793936259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00fea50 0x7fcea00fee60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce90009990 tx=0x7fce90009ca0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 -- 172.21.15.2:0/1793936259 shutdown_connections 2023-06-23T18:42:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 --2- 172.21.15.2:0/1793936259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea0100690 0x7fcea0106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 --2- 172.21.15.2:0/1793936259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00fea50 0x7fcea00fee60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 --2- 172.21.15.2:0/1793936259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcea00ffce0 0x7fcea0100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 -- 172.21.15.2:0/1793936259 >> 172.21.15.2:0/1793936259 conn(0x7fcea00fa050 msgr2=0x7fcea00fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:25.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 -- 172.21.15.2:0/1793936259 shutdown_connections 2023-06-23T18:42:25.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.519+0000 7fcea554f700 1 -- 172.21.15.2:0/1793936259 wait complete. 2023-06-23T18:42:25.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fcea554f700 1 Processor -- start 2023-06-23T18:42:25.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fcea554f700 1 -- start start 2023-06-23T18:42:25.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fcea554f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00fea50 0x7fcea006ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fcea554f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcea00ffce0 0x7fcea006f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fcea554f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea0100690 0x7fcea01a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fcea554f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea00723a0 con 0x7fcea00ffce0 2023-06-23T18:42:25.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fcea554f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea0072510 con 0x7fcea00fea50 2023-06-23T18:42:25.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fcea554f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea01075e0 con 0x7fcea0100690 2023-06-23T18:42:25.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcea00ffce0 0x7fcea006f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcea00ffce0 0x7fcea006f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57936/0 (socket says 172.21.15.2:57936) 2023-06-23T18:42:25.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.520+0000 7fce9e7fc700 1 -- 172.21.15.2:0/956522812 learned_addr learned my addr 172.21.15.2:0/956522812 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:25.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea0100690 0x7fcea01a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9effd700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00fea50 0x7fcea006ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9f7fe700 1 -- 172.21.15.2:0/956522812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00fea50 msgr2=0x7fcea006ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00fea50 0x7fcea006ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9f7fe700 1 -- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcea00ffce0 msgr2=0x7fcea006f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcea00ffce0 0x7fcea006f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9f7fe700 1 -- 172.21.15.2:0/956522812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce900095f0 con 0x7fcea0100690 2023-06-23T18:42:25.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9e7fc700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcea00ffce0 0x7fcea006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:42:25.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea0100690 0x7fcea01a5920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fce9800efc0 tx=0x7fce9800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:25.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce97fff700 1 -- 172.21.15.2:0/956522812 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98018070 con 0x7fcea0100690 2023-06-23T18:42:25.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fcea554f700 1 -- 172.21.15.2:0/956522812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea0107860 con 0x7fcea0100690 2023-06-23T18:42:25.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce97fff700 1 -- 172.21.15.2:0/956522812 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce9800f040 con 0x7fcea0100690 2023-06-23T18:42:25.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fce97fff700 1 -- 172.21.15.2:0/956522812 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9800cb40 con 0x7fcea0100690 2023-06-23T18:42:25.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.521+0000 7fcea554f700 1 -- 172.21.15.2:0/956522812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea0107d70 con 0x7fcea0100690 2023-06-23T18:42:25.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.525+0000 7fce97fff700 1 -- 172.21.15.2:0/956522812 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fce98003a00 con 0x7fcea0100690 2023-06-23T18:42:25.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.525+0000 7fce97fff700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce8c0a11d0 0x7fce8c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.526+0000 7fce97fff700 1 -- 172.21.15.2:0/956522812 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fce980cc340 con 0x7fcea0100690 2023-06-23T18:42:25.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.526+0000 7fceb16d4b80 1 Processor -- start 2023-06-23T18:42:25.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.527+0000 7fceb16d4b80 1 -- start start 2023-06-23T18:42:25.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.527+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 0x556d3fa538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.527+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53de0 0x556d3fa54280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa54c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3f94fcb0 con 0x556d3fa53de0 2023-06-23T18:42:25.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3f9a2a40 con 0x556d3fa53490 2023-06-23T18:42:25.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3f99bed0 con 0x556d3fa54810 2023-06-23T18:42:25.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53de0 0x556d3fa54280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa54c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53de0 0x556d3fa54280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57942/0 (socket says 172.21.15.2:57942) 2023-06-23T18:42:25.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9e7fc700 1 -- 172.21.15.2:0/1053745259 learned_addr learned my addr 172.21.15.2:0/1053745259 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:25.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9f7fe700 1 -- 172.21.15.2:0/1053745259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 msgr2=0x556d3fa538a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:42:25.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/1053745259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 0x556d3fa538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9f7fe700 1 -- 172.21.15.2:0/1053745259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53de0 msgr2=0x556d3fa54280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:25.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/1053745259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53de0 0x556d3fa54280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9f7fe700 1 -- 172.21.15.2:0/1053745259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d3f81d630 con 0x556d3fa54810 2023-06-23T18:42:25.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.528+0000 7fce9effd700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce8c0a11d0 0x7fce8c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.529+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/1053745259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa54c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fce980994e0 tx=0x7fce980993a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ed57f1b89c42e81 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:25.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.529+0000 7fce9effd700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce8c0a11d0 0x7fce8c0a3680 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fce90006010 tx=0x7fce90005c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:25.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.529+0000 7fce957fa700 1 -- 172.21.15.2:0/1053745259 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980997e0 con 0x556d3fa54810 2023-06-23T18:42:25.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.529+0000 7fce957fa700 1 -- 172.21.15.2:0/1053745259 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce9809d810 con 0x556d3fa54810 2023-06-23T18:42:25.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.529+0000 7fce957fa700 1 -- 172.21.15.2:0/1053745259 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98099a40 con 0x556d3fa54810 2023-06-23T18:42:25.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.530+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1053745259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 msgr2=0x556d3fa54c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.530+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1053745259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa54c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fce980994e0 tx=0x7fce980993a0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.530+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1053745259 shutdown_connections 2023-06-23T18:42:25.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.530+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1053745259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa54c80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.530+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1053745259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 0x556d3fa538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.530+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1053745259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53de0 0x556d3fa54280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.530+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1053745259 >> 172.21.15.2:0/1053745259 conn(0x556d3fa403f0 msgr2=0x556d3fa45520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:25.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1053745259 shutdown_connections 2023-06-23T18:42:25.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1053745259 wait complete. 2023-06-23T18:42:25.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 -- start start 2023-06-23T18:42:25.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 Processor -- start 2023-06-23T18:42:25.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53490 0x556d3fa4c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3f93bac0 con 0x556d3fa53490 2023-06-23T18:42:25.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3f93b7c0 con 0x556d3fa53de0 2023-06-23T18:42:25.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3f81f920 con 0x556d3fa54810 2023-06-23T18:42:25.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.531+0000 7fceb16d4b80 10 client.0 _ll_register_callbacks cb 0x556d3f941260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:42:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53490 0x556d3fa4c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48540/0 (socket says 172.21.15.2:48540) 2023-06-23T18:42:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9f7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58290/0 (socket says 172.21.15.2:58290) 2023-06-23T18:42:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53490 0x556d3fa4c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57950/0 (socket says 172.21.15.2:57950) 2023-06-23T18:42:25.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9e7fc700 1 -- 172.21.15.2:0/1297303255 learned_addr learned my addr 172.21.15.2:0/1297303255 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9f7fe700 1 -- 172.21.15.2:0/1297303255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 msgr2=0x556d3fa4c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9f7fe700 1 -- 172.21.15.2:0/1297303255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53490 msgr2=0x556d3fa4c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53490 0x556d3fa4c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9f7fe700 1 -- 172.21.15.2:0/1297303255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea0107d70 con 0x556d3fa54810 2023-06-23T18:42:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa518a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce98099770 tx=0x7fce9800cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=230c102acdc31308 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.532+0000 7fce867fc700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:42:25.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fce867fc700 1 -- 172.21.15.2:0/1297303255 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980b3070 con 0x556d3fa54810 2023-06-23T18:42:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fce867fc700 1 -- 172.21.15.2:0/1297303255 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fce98097b80 con 0x556d3fa54810 2023-06-23T18:42:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1297303255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d3fa3dbb0 con 0x556d3fa54810 2023-06-23T18:42:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fce867fc700 1 -- 172.21.15.2:0/1297303255 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce980a1c20 con 0x556d3fa54810 2023-06-23T18:42:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 10 client.26725 fetch_fsmap learned FSMap version 41 2023-06-23T18:42:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1297303255 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x556d3f7c9de0 con 0x556d3fa54810 2023-06-23T18:42:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fce867fc700 1 -- 172.21.15.2:0/1297303255 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556d3fa3dbb0 con 0x556d3fa54810 2023-06-23T18:42:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fce867fc700 1 -- 172.21.15.2:0/1297303255 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fce980a1710 con 0x556d3fa54810 2023-06-23T18:42:25.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 10 client.26725 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:42:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:42:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 20 client.26725 populate_metadata read hostname 'smithi002' 2023-06-23T18:42:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556d3fa70ef0 0x556d3fa73300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 4 client.26725 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:42:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1297303255 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d3f994eb0 con 0x556d3fa70ef0 2023-06-23T18:42:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.533+0000 7fce9effd700 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556d3fa70ef0 0x556d3fa73300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.534+0000 7fce867fc700 10 client.26725 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:42:25.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.534+0000 7fce9effd700 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556d3fa70ef0 0x556d3fa73300 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3430caac0dd1528d server_cookie=f258a5146a1d0026 in_seq=0 out_seq=0 2023-06-23T18:42:25.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.535+0000 7fce867fc700 1 -- 172.21.15.2:0/1297303255 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556d3f994eb0 con 0x556d3fa70ef0 2023-06-23T18:42:25.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.535+0000 7fce867fc700 10 client.26725 handle_command_reply: tid=0 2023-06-23T18:42:25.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 client.26725 shutdown 2023-06-23T18:42:25.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1297303255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 msgr2=0x556d3fa518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa518a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce98099770 tx=0x7fce9800cf60 comp rx=0 tx=0).stop 2023-06-23T18:42:25.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1297303255 shutdown_connections 2023-06-23T18:42:25.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556d3fa70ef0 0x556d3fa73300 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa54810 0x556d3fa518a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa53490 0x556d3fa4c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1297303255 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1297303255 >> 172.21.15.2:0/1297303255 conn(0x556d3fa403f0 msgr2=0x556d3fa44c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:25.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.538+0000 7fce867fc700 0 client.26725 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:42:25.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.539+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1297303255 shutdown_connections 2023-06-23T18:42:25.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.539+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1297303255 wait complete. 2023-06-23T18:42:25.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.539+0000 7fceb16d4b80 20 client.26725 trim_cache size 0 max 16384 2023-06-23T18:42:25.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fceb16d4b80 1 Processor -- start 2023-06-23T18:42:25.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fceb16d4b80 1 -- start start 2023-06-23T18:42:25.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53490 0x556d3fa80b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fa4c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3fa4b830 con 0x556d3fa54810 2023-06-23T18:42:25.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3fa46fa0 con 0x556d3fa53de0 2023-06-23T18:42:25.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3fa46810 con 0x556d3fa53490 2023-06-23T18:42:25.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fce9effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53490 0x556d3fa80b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fce9f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fa4c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48548/0 (socket says 172.21.15.2:48548) 2023-06-23T18:42:25.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fce9effd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53490 0x556d3fa80b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58296/0 (socket says 172.21.15.2:58296) 2023-06-23T18:42:25.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.567+0000 7fce9e7fc700 1 -- 172.21.15.2:0/2627091979 learned_addr learned my addr 172.21.15.2:0/2627091979 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:25.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce9f7fe700 1 -- 172.21.15.2:0/2627091979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53490 msgr2=0x556d3fa80b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:25.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53490 0x556d3fa80b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce9f7fe700 1 -- 172.21.15.2:0/2627091979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 msgr2=0x556d3fa4c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:25.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce9f7fe700 1 -- 172.21.15.2:0/2627091979 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d3f7b2c60 con 0x556d3fa54810 2023-06-23T18:42:25.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce9e7fc700 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:42:25.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce9effd700 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53490 0x556d3fa80b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:42:25.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fa4c870 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fce98009470 tx=0x7fce98009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3de3d2e97b41da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:25.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce95ffb700 1 -- 172.21.15.2:0/2627091979 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980a1c20 con 0x556d3fa54810 2023-06-23T18:42:25.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce95ffb700 1 -- 172.21.15.2:0/2627091979 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce980b5070 con 0x556d3fa54810 2023-06-23T18:42:25.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.568+0000 7fce95ffb700 1 -- 172.21.15.2:0/2627091979 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce980a1710 con 0x556d3fa54810 2023-06-23T18:42:25.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 -- 172.21.15.2:0/2627091979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 msgr2=0x556d3fa4c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fa4c870 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fce98009470 tx=0x7fce98009f90 comp rx=0 tx=0).stop 2023-06-23T18:42:25.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 -- 172.21.15.2:0/2627091979 shutdown_connections 2023-06-23T18:42:25.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fa4c870 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53490 0x556d3fa80b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/2627091979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53de0 0x556d3fa4c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 -- 172.21.15.2:0/2627091979 >> 172.21.15.2:0/2627091979 conn(0x556d3fa403f0 msgr2=0x556d3fa77c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:25.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 -- 172.21.15.2:0/2627091979 shutdown_connections 2023-06-23T18:42:25.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.569+0000 7fceb16d4b80 1 -- 172.21.15.2:0/2627091979 wait complete. 2023-06-23T18:42:25.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 1 -- start start 2023-06-23T18:42:25.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 1 Processor -- start 2023-06-23T18:42:25.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 0x556d3fabf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53de0 0x556d3fac1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fac3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3fac46d0 con 0x556d3fa54810 2023-06-23T18:42:25.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3fac4840 con 0x556d3fa53490 2023-06-23T18:42:25.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d3fac4a00 con 0x556d3fa53de0 2023-06-23T18:42:25.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fceb16d4b80 10 client.0 _ll_register_callbacks cb 0x556d3f941260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:42:25.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.570+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53de0 0x556d3fac1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9e7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53de0 0x556d3fac1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58300/0 (socket says 172.21.15.2:58300) 2023-06-23T18:42:25.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9e7fc700 1 -- 172.21.15.2:0/1968434891 learned_addr learned my addr 172.21.15.2:0/1968434891 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:25.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9f7fe700 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fac3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9effd700 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 0x556d3fabf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9e7fc700 1 -- 172.21.15.2:0/1968434891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 msgr2=0x556d3fabf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:25.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9e7fc700 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 0x556d3fabf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9e7fc700 1 -- 172.21.15.2:0/1968434891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 msgr2=0x556d3fac3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:25.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9e7fc700 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fac3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9e7fc700 1 -- 172.21.15.2:0/1968434891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea0068b30 con 0x556d3fa53de0 2023-06-23T18:42:25.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce9e7fc700 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53de0 0x556d3fac1a10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce88009fd0 tx=0x7fce8800f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35f853b72c9fd434 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:25.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.571+0000 7fce867fc700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:42:25.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fce867fc700 1 -- 172.21.15.2:0/1968434891 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88017070 con 0x556d3fa53de0 2023-06-23T18:42:25.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fce867fc700 1 -- 172.21.15.2:0/1968434891 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fce88004050 con 0x556d3fa53de0 2023-06-23T18:42:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1968434891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce9809d810 con 0x556d3fa53de0 2023-06-23T18:42:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fce867fc700 1 -- 172.21.15.2:0/1968434891 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce880122d0 con 0x556d3fa53de0 2023-06-23T18:42:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 10 client.26731 fetch_fsmap learned FSMap version 41 2023-06-23T18:42:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1968434891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fce88012470 con 0x556d3fa53de0 2023-06-23T18:42:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fce867fc700 1 -- 172.21.15.2:0/1968434891 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce8802c020 con 0x556d3fa53de0 2023-06-23T18:42:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fce867fc700 1 -- 172.21.15.2:0/1968434891 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fce8801d440 con 0x556d3fa53de0 2023-06-23T18:42:25.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 10 client.26731 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:42:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:42:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 20 client.26731 populate_metadata read hostname 'smithi002' 2023-06-23T18:42:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556d3fac6450 0x556d3fac8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 4 client.26731 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:42:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.572+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1968434891 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556d3f957050 con 0x556d3fac6450 2023-06-23T18:42:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.573+0000 7fce9effd700 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556d3fac6450 0x556d3fac8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.573+0000 7fce867fc700 10 client.26731 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:42:25.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.573+0000 7fce9effd700 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556d3fac6450 0x556d3fac8860 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db8582b4c6a1bfd server_cookie=8bd0f87cd9922618 in_seq=0 out_seq=0 2023-06-23T18:42:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.573+0000 7fce867fc700 1 -- 172.21.15.2:0/1968434891 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556d3f957050 con 0x556d3fac6450 2023-06-23T18:42:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.573+0000 7fce867fc700 10 client.26731 handle_command_reply: tid=0 2023-06-23T18:42:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.573+0000 7fceb16d4b80 1 client.26731 shutdown 2023-06-23T18:42:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1968434891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53de0 msgr2=0x556d3fac1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53de0 0x556d3fac1a10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce88009fd0 tx=0x7fce8800f6a0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1968434891 shutdown_connections 2023-06-23T18:42:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x556d3fac6450 0x556d3fac8860 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556d3fa54810 0x556d3fac3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x556d3fa53490 0x556d3fabf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 --2- 172.21.15.2:0/1968434891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556d3fa53de0 0x556d3fac1a10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1968434891 >> 172.21.15.2:0/1968434891 conn(0x556d3fa403f0 msgr2=0x556d3fa57820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fce867fc700 0 client.26731 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:42:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1968434891 shutdown_connections 2023-06-23T18:42:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 1 -- 172.21.15.2:0/1968434891 wait complete. 2023-06-23T18:42:25.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.574+0000 7fceb16d4b80 20 client.26731 trim_cache size 0 max 16384 2023-06-23T18:42:25.589 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:42:25.589 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:42:25.589 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:42:25.589 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:42:25.589 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:42:25.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:42:25.590 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:42:25.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:42:25.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:42:25.590 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:42:25.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 -- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce8c0a11d0 msgr2=0x7fce8c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce8c0a11d0 0x7fce8c0a3680 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fce90006010 tx=0x7fce90005c20 comp rx=0 tx=0).stop 2023-06-23T18:42:25.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 -- 172.21.15.2:0/956522812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea0100690 msgr2=0x7fcea01a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:25.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea0100690 0x7fcea01a5920 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fce9800efc0 tx=0x7fce9800c3a0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 -- 172.21.15.2:0/956522812 shutdown_connections 2023-06-23T18:42:25.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcea0100690 0x7fcea01a5920 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fce8c0a11d0 0x7fce8c0a3680 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00fea50 0x7fcea006ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 --2- 172.21.15.2:0/956522812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcea00ffce0 0x7fcea006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:25.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 -- 172.21.15.2:0/956522812 >> 172.21.15.2:0/956522812 conn(0x7fcea00fa050 msgr2=0x7fcea00fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:25.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 -- 172.21.15.2:0/956522812 shutdown_connections 2023-06-23T18:42:25.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:25.575+0000 7fcea554f700 1 -- 172.21.15.2:0/956522812 wait complete. 2023-06-23T18:42:25.596 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:42:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:26 smithi082 ceph-mon[97889]: pgmap v908: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.1 MiB/s wr, 938 op/s 2023-06-23T18:42:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:26 smithi017 ceph-mon[97472]: pgmap v908: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.1 MiB/s wr, 938 op/s 2023-06-23T18:42:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 515.0 s, 355.4 tps, lat 280.487 ms stddev 47.248 2023-06-23T18:42:27.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:26 smithi002 ceph-mon[95947]: pgmap v908: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.1 MiB/s wr, 938 op/s 2023-06-23T18:42:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:28 smithi082 ceph-mon[97889]: pgmap v909: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 973 op/s 2023-06-23T18:42:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:28 smithi017 ceph-mon[97472]: pgmap v909: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 973 op/s 2023-06-23T18:42:29.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:28 smithi002 ceph-mon[95947]: pgmap v909: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 973 op/s 2023-06-23T18:42:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:30 smithi082 ceph-mon[97889]: pgmap v910: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:42:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:30 smithi017 ceph-mon[97472]: pgmap v910: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:42:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:30 smithi002 ceph-mon[95947]: pgmap v910: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:42:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 520.0 s, 346.6 tps, lat 288.248 ms stddev 50.849 2023-06-23T18:42:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:32 smithi017 ceph-mon[97472]: pgmap v911: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 1.00k op/s 2023-06-23T18:42:33.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:32 smithi002 ceph-mon[95947]: pgmap v911: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 1.00k op/s 2023-06-23T18:42:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:32 smithi082 ceph-mon[97889]: pgmap v911: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 1.00k op/s 2023-06-23T18:42:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:34 smithi017 ceph-mon[97472]: pgmap v912: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-06-23T18:42:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:34 smithi002 ceph-mon[95947]: pgmap v912: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-06-23T18:42:35.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:34 smithi082 ceph-mon[97889]: pgmap v912: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.05k op/s 2023-06-23T18:42:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:36 smithi017 ceph-mon[97472]: pgmap v913: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-06-23T18:42:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 525.0 s, 309.6 tps, lat 322.813 ms stddev 57.142 2023-06-23T18:42:37.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:36 smithi002 ceph-mon[95947]: pgmap v913: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-06-23T18:42:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:36 smithi082 ceph-mon[97889]: pgmap v913: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-06-23T18:42:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:38 smithi017 ceph-mon[97472]: pgmap v914: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-06-23T18:42:39.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:38 smithi002 ceph-mon[95947]: pgmap v914: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-06-23T18:42:39.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:38 smithi082 ceph-mon[97889]: pgmap v914: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-06-23T18:42:40.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:40 smithi017 ceph-mon[97472]: pgmap v915: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 1.05k op/s 2023-06-23T18:42:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:40 smithi002 ceph-mon[95947]: pgmap v915: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 1.05k op/s 2023-06-23T18:42:41.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:40 smithi082 ceph-mon[97889]: pgmap v915: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 1.05k op/s 2023-06-23T18:42:42.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 530.0 s, 335.2 tps, lat 299.207 ms stddev 66.126 2023-06-23T18:42:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:42 smithi017 ceph-mon[97472]: pgmap v916: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:42:43.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:42 smithi002 ceph-mon[95947]: pgmap v916: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:42:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:42 smithi082 ceph-mon[97889]: pgmap v916: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-06-23T18:42:44.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:44 smithi017 ceph-mon[97472]: pgmap v917: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:42:45.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:44 smithi002 ceph-mon[95947]: pgmap v917: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:42:45.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:44 smithi082 ceph-mon[97889]: pgmap v917: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:42:46.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:46 smithi017 ceph-mon[97472]: pgmap v918: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:42:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 535.0 s, 346.4 tps, lat 289.126 ms stddev 55.778 2023-06-23T18:42:47.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:46 smithi002 ceph-mon[95947]: pgmap v918: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:42:47.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:46 smithi082 ceph-mon[97889]: pgmap v918: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:42:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:48 smithi017 ceph-mon[97472]: pgmap v919: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-06-23T18:42:49.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:48 smithi002 ceph-mon[95947]: pgmap v919: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-06-23T18:42:49.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:48 smithi082 ceph-mon[97889]: pgmap v919: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-06-23T18:42:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:50 smithi017 ceph-mon[97472]: pgmap v920: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:42:51.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:50 smithi002 ceph-mon[95947]: pgmap v920: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:42:51.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:50 smithi082 ceph-mon[97889]: pgmap v920: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:42:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 540.0 s, 335.4 tps, lat 296.754 ms stddev 50.957 2023-06-23T18:42:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:52 smithi017 ceph-mon[97472]: pgmap v921: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:42:53.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:52 smithi002 ceph-mon[95947]: pgmap v921: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:42:53.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:52 smithi082 ceph-mon[97889]: pgmap v921: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 1.14k op/s 2023-06-23T18:42:54.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:54 smithi017 ceph-mon[97472]: pgmap v922: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:42:55.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:54 smithi002 ceph-mon[95947]: pgmap v922: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:42:55.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:54 smithi082 ceph-mon[97889]: pgmap v922: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.14k op/s 2023-06-23T18:42:55.597 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:42:55.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.774+0000 7fe952a4a700 1 -- 172.21.15.2:0/4214316253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0ffc80 msgr2=0x7fe94c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:55.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.774+0000 7fe952a4a700 1 --2- 172.21.15.2:0/4214316253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0ffc80 0x7fe94c1000d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe940009990 tx=0x7fe940009ca0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.774+0000 7fe952a4a700 1 -- 172.21.15.2:0/4214316253 shutdown_connections 2023-06-23T18:42:55.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.774+0000 7fe952a4a700 1 --2- 172.21.15.2:0/4214316253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c100610 0x7fe94c106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.774+0000 7fe952a4a700 1 --2- 172.21.15.2:0/4214316253 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe94c0fea90 0x7fe94c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.774+0000 7fe952a4a700 1 --2- 172.21.15.2:0/4214316253 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0ffc80 0x7fe94c1000d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.774+0000 7fe952a4a700 1 -- 172.21.15.2:0/4214316253 >> 172.21.15.2:0/4214316253 conn(0x7fe94c0fa090 msgr2=0x7fe94c0fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:55.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.775+0000 7fe952a4a700 1 -- 172.21.15.2:0/4214316253 shutdown_connections 2023-06-23T18:42:55.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.775+0000 7fe952a4a700 1 -- 172.21.15.2:0/4214316253 wait complete. 2023-06-23T18:42:55.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.775+0000 7fe952a4a700 1 Processor -- start 2023-06-23T18:42:55.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.775+0000 7fe952a4a700 1 -- start start 2023-06-23T18:42:55.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe952a4a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0fea90 0x7fe94c197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe952a4a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c0ffc80 0x7fe94c197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe952a4a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe94c100610 0x7fe94c19f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe952a4a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19f600 con 0x7fe94c100610 2023-06-23T18:42:55.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe952a4a700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19f740 con 0x7fe94c0fea90 2023-06-23T18:42:55.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe952a4a700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19f8d0 con 0x7fe94c0ffc80 2023-06-23T18:42:55.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe94bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0fea90 0x7fe94c197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe94bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0fea90 0x7fe94c197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48568/0 (socket says 172.21.15.2:48568) 2023-06-23T18:42:55.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe94bfff700 1 -- 172.21.15.2:0/3963718659 learned_addr learned my addr 172.21.15.2:0/3963718659 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:55.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe950fe7700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe94c100610 0x7fe94c19f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe94bfff700 1 -- 172.21.15.2:0/3963718659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c0ffc80 msgr2=0x7fe94c197fc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-06-23T18:42:55.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe94bfff700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c0ffc80 0x7fe94c197fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe94bfff700 1 -- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe94c100610 msgr2=0x7fe94c19f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:55.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe94bfff700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe94c100610 0x7fe94c19f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe94bfff700 1 -- 172.21.15.2:0/3963718659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400095f0 con 0x7fe94c0fea90 2023-06-23T18:42:55.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.776+0000 7fe950fe7700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe94c100610 0x7fe94c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:42:55.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.777+0000 7fe94bfff700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0fea90 0x7fe94c197a80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00b500 tx=0x7fe93c00b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:55.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.777+0000 7fe9497fa700 1 -- 172.21.15.2:0/3963718659 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011830 con 0x7fe94c0fea90 2023-06-23T18:42:55.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.777+0000 7fe9497fa700 1 -- 172.21.15.2:0/3963718659 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe93c011e20 con 0x7fe94c0fea90 2023-06-23T18:42:55.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.777+0000 7fe9497fa700 1 -- 172.21.15.2:0/3963718659 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c010dd0 con 0x7fe94c0fea90 2023-06-23T18:42:55.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.777+0000 7fe952a4a700 1 -- 172.21.15.2:0/3963718659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c19fbb0 con 0x7fe94c0fea90 2023-06-23T18:42:55.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.777+0000 7fe952a4a700 1 -- 172.21.15.2:0/3963718659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c1a00c0 con 0x7fe94c0fea90 2023-06-23T18:42:55.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.780+0000 7fe9497fa700 1 -- 172.21.15.2:0/3963718659 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe93c011990 con 0x7fe94c0fea90 2023-06-23T18:42:55.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.781+0000 7fe9497fa700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9340a11d0 0x7fe9340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.781+0000 7fe9497fa700 1 -- 172.21.15.2:0/3963718659 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe93c0cc340 con 0x7fe94c0fea90 2023-06-23T18:42:55.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.781+0000 7fe94b7fe700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9340a11d0 0x7fe9340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe95ebcfb80 1 Processor -- start 2023-06-23T18:42:55.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe95ebcfb80 1 -- start start 2023-06-23T18:42:55.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 0x558a0cafc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94b7fe700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9340a11d0 0x7fe9340a3680 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe940005910 tx=0x7fe940000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:55.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafcde0 0x558a0cafd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cafdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0c9f8cb0 con 0x558a0cafd810 2023-06-23T18:42:55.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0ca4ba40 con 0x558a0cafcde0 2023-06-23T18:42:55.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0ca44ed0 con 0x558a0cafc490 2023-06-23T18:42:55.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafcde0 0x558a0cafd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafcde0 0x558a0cafd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48574/0 (socket says 172.21.15.2:48574) 2023-06-23T18:42:55.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94b7fe700 1 -- 172.21.15.2:0/2847581430 learned_addr learned my addr 172.21.15.2:0/2847581430 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:55.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94bfff700 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 0x558a0cafc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe950fe7700 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cafdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94b7fe700 1 -- 172.21.15.2:0/2847581430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 msgr2=0x558a0cafc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:55.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94b7fe700 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 0x558a0cafc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94b7fe700 1 -- 172.21.15.2:0/2847581430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 msgr2=0x558a0cafdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:55.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.782+0000 7fe94b7fe700 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cafdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.783+0000 7fe94b7fe700 1 -- 172.21.15.2:0/2847581430 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0c8c6630 con 0x558a0cafcde0 2023-06-23T18:42:55.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.783+0000 7fe94b7fe700 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafcde0 0x558a0cafd280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe9400051b0 tx=0x7fe94001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37778483315c54b5 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.783+0000 7fe9327fc700 1 -- 172.21.15.2:0/2847581430 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940029070 con 0x558a0cafcde0 2023-06-23T18:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.783+0000 7fe9327fc700 1 -- 172.21.15.2:0/2847581430 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe940003b90 con 0x558a0cafcde0 2023-06-23T18:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.783+0000 7fe9327fc700 1 -- 172.21.15.2:0/2847581430 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94001d070 con 0x558a0cafcde0 2023-06-23T18:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2847581430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafcde0 msgr2=0x558a0cafd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:55.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafcde0 0x558a0cafd280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe9400051b0 tx=0x7fe94001d000 comp rx=0 tx=0).stop 2023-06-23T18:42:55.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2847581430 shutdown_connections 2023-06-23T18:42:55.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cafdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 0x558a0cafc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2847581430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafcde0 0x558a0cafd280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2847581430 >> 172.21.15.2:0/2847581430 conn(0x558a0cae93f0 msgr2=0x558a0caee520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:55.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2847581430 shutdown_connections 2023-06-23T18:42:55.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.784+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2847581430 wait complete. 2023-06-23T18:42:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.785+0000 7fe95ebcfb80 1 -- start start 2023-06-23T18:42:55.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.785+0000 7fe95ebcfb80 1 Processor -- start 2023-06-23T18:42:55.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 0x558a0cb16d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb17240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafd810 0x558a0cb1e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0c9e4ac0 con 0x558a0cafcde0 2023-06-23T18:42:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0c9e47c0 con 0x558a0cafd810 2023-06-23T18:42:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0c8c8920 con 0x558a0cafc490 2023-06-23T18:42:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe95ebcfb80 10 client.0 _ll_register_callbacks cb 0x558a0c9ea260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:42:55.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb17240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb17240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:57988/0 (socket says 172.21.15.2:57988) 2023-06-23T18:42:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe94b7fe700 1 -- 172.21.15.2:0/3813470911 learned_addr learned my addr 172.21.15.2:0/3813470911 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe950fe7700 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafd810 0x558a0cb1e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe94bfff700 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 0x558a0cb16d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe950fe7700 1 -- 172.21.15.2:0/3813470911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 msgr2=0x558a0cb16d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe950fe7700 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 0x558a0cb16d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe950fe7700 1 -- 172.21.15.2:0/3813470911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 msgr2=0x558a0cb17240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:55.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe950fe7700 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb17240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.790+0000 7fe950fe7700 1 -- 172.21.15.2:0/3813470911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c068680 con 0x558a0cafd810 2023-06-23T18:42:55.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.791+0000 7fe950fe7700 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafd810 0x558a0cb1e280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe9440042f0 tx=0x7fe944004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0f89bfc5fb20f35 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:55.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.791+0000 7fe92b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:42:55.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.794+0000 7fe92b7fe700 1 -- 172.21.15.2:0/3813470911 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9440048c0 con 0x558a0cafd810 2023-06-23T18:42:55.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.794+0000 7fe92b7fe700 1 -- 172.21.15.2:0/3813470911 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe944004bb0 con 0x558a0cafd810 2023-06-23T18:42:55.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.794+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/3813470911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a0cae6bb0 con 0x558a0cafd810 2023-06-23T18:42:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.794+0000 7fe92b7fe700 1 -- 172.21.15.2:0/3813470911 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe944004d70 con 0x558a0cafd810 2023-06-23T18:42:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.794+0000 7fe95ebcfb80 10 client.26516 fetch_fsmap learned FSMap version 41 2023-06-23T18:42:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.794+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/3813470911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558a0c872de0 con 0x558a0cafd810 2023-06-23T18:42:55.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.795+0000 7fe92b7fe700 1 -- 172.21.15.2:0/3813470911 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558a0cae6bb0 con 0x558a0cafd810 2023-06-23T18:42:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.796+0000 7fe92b7fe700 1 -- 172.21.15.2:0/3813470911 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe94400e580 con 0x558a0cafd810 2023-06-23T18:42:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.796+0000 7fe95ebcfb80 10 client.26516 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:42:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.796+0000 7fe95ebcfb80 10 client.26516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:42:55.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.796+0000 7fe95ebcfb80 20 client.26516 populate_metadata read hostname 'smithi002' 2023-06-23T18:42:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.796+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558a0caf4dd0 0x558a0caf71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.796+0000 7fe95ebcfb80 4 client.26516 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:42:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.796+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/3813470911 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a0ca3deb0 con 0x558a0caf4dd0 2023-06-23T18:42:55.802 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.796+0000 7fe94bfff700 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558a0caf4dd0 0x558a0caf71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.797+0000 7fe92b7fe700 10 client.26516 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:42:55.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.797+0000 7fe94bfff700 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558a0caf4dd0 0x558a0caf71e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de0e81f19cb45d9 server_cookie=6899d6db756c217d in_seq=0 out_seq=0 2023-06-23T18:42:55.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.798+0000 7fe92b7fe700 1 -- 172.21.15.2:0/3813470911 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a0ca3deb0 con 0x558a0caf4dd0 2023-06-23T18:42:55.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.798+0000 7fe92b7fe700 10 client.26516 handle_command_reply: tid=0 2023-06-23T18:42:55.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.798+0000 7fe95ebcfb80 1 client.26516 shutdown 2023-06-23T18:42:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.799+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafd810 msgr2=0x558a0cb1e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:55.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.799+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafd810 0x558a0cb1e280 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe9440042f0 tx=0x7fe944004320 comp rx=0 tx=0).stop 2023-06-23T18:42:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/3813470911 shutdown_connections 2023-06-23T18:42:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558a0caf4dd0 0x558a0caf71e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafd810 0x558a0cb1e280 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafc490 0x558a0cb16d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/3813470911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb17240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/3813470911 >> 172.21.15.2:0/3813470911 conn(0x558a0cae93f0 msgr2=0x558a0caedc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe92b7fe700 0 client.26516 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:42:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/3813470911 shutdown_connections 2023-06-23T18:42:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/3813470911 wait complete. 2023-06-23T18:42:55.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.800+0000 7fe95ebcfb80 20 client.26516 trim_cache size 0 max 16384 2023-06-23T18:42:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.827+0000 7fe95ebcfb80 1 Processor -- start 2023-06-23T18:42:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe95ebcfb80 1 -- start start 2023-06-23T18:42:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0caee3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb1e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafd810 0x558a0cb03ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0cb161b0 con 0x558a0cafcde0 2023-06-23T18:42:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0caef680 con 0x558a0cafc490 2023-06-23T18:42:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0cae8f70 con 0x558a0cafd810 2023-06-23T18:42:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe950fe7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafd810 0x558a0cb03ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe94bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0caee3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb1e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe950fe7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafd810 0x558a0cb03ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58338/0 (socket says 172.21.15.2:58338) 2023-06-23T18:42:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe950fe7700 1 -- 172.21.15.2:0/2606249892 learned_addr learned my addr 172.21.15.2:0/2606249892 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe950fe7700 1 -- 172.21.15.2:0/2606249892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 msgr2=0x558a0caee3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe950fe7700 1 --2- 172.21.15.2:0/2606249892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0caee3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe950fe7700 1 -- 172.21.15.2:0/2606249892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 msgr2=0x558a0cb1e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe950fe7700 1 --2- 172.21.15.2:0/2606249892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb1e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.828+0000 7fe950fe7700 1 -- 172.21.15.2:0/2606249892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0cafdeb0 con 0x558a0cafd810 2023-06-23T18:42:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.829+0000 7fe950fe7700 1 --2- 172.21.15.2:0/2606249892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafd810 0x558a0cb03ba0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe944000f80 tx=0x7fe944009ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43bd0065cbd759eb server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.829+0000 7fe932ffd700 1 -- 172.21.15.2:0/2606249892 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400eb70 con 0x558a0cafd810 2023-06-23T18:42:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.829+0000 7fe932ffd700 1 -- 172.21.15.2:0/2606249892 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe94400f040 con 0x558a0cafd810 2023-06-23T18:42:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.829+0000 7fe932ffd700 1 -- 172.21.15.2:0/2606249892 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944007710 con 0x558a0cafd810 2023-06-23T18:42:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2606249892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafd810 msgr2=0x558a0cb03ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2606249892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafd810 0x558a0cb03ba0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe944000f80 tx=0x7fe944009ab0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2606249892 shutdown_connections 2023-06-23T18:42:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2606249892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafd810 0x558a0cb03ba0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2606249892 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0caee3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2606249892 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafcde0 0x558a0cb1e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2606249892 >> 172.21.15.2:0/2606249892 conn(0x558a0cae93f0 msgr2=0x558a0cafbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2606249892 shutdown_connections 2023-06-23T18:42:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2606249892 wait complete. 2023-06-23T18:42:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 -- start start 2023-06-23T18:42:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.830+0000 7fe95ebcfb80 1 Processor -- start 2023-06-23T18:42:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0cb1df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafcde0 0x558a0cb1a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe95ebcfb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cb1aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0cb1b420 con 0x558a0cafd810 2023-06-23T18:42:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0cb1b590 con 0x558a0cafc490 2023-06-23T18:42:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe95ebcfb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0cb7e920 con 0x558a0cafcde0 2023-06-23T18:42:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe95ebcfb80 10 client.0 _ll_register_callbacks cb 0x558a0c9ea260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:42:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafcde0 0x558a0cb1a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0cb1df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0cb1df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48594/0 (socket says 172.21.15.2:48594) 2023-06-23T18:42:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafcde0 0x558a0cb1a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58342/0 (socket says 172.21.15.2:58342) 2023-06-23T18:42:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94bfff700 1 -- 172.21.15.2:0/2359256866 learned_addr learned my addr 172.21.15.2:0/2359256866 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:42:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe950fe7700 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cb1aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94b7fe700 1 -- 172.21.15.2:0/2359256866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 msgr2=0x558a0cb1df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94b7fe700 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0cb1df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94b7fe700 1 -- 172.21.15.2:0/2359256866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 msgr2=0x558a0cb1aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94b7fe700 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cb1aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94b7fe700 1 -- 172.21.15.2:0/2359256866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a0c872de0 con 0x558a0cafcde0 2023-06-23T18:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94bfff700 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0cb1df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe950fe7700 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cb1aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe94b7fe700 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafcde0 0x558a0cb1a9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe940009ce0 tx=0x7fe940003a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b13eb610570ca9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.831+0000 7fe92b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.832+0000 7fe92b7fe700 1 -- 172.21.15.2:0/2359256866 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940033030 con 0x558a0cafcde0 2023-06-23T18:42:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.832+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2359256866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe940003b90 con 0x558a0cafcde0 2023-06-23T18:42:55.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.832+0000 7fe92b7fe700 1 -- 172.21.15.2:0/2359256866 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe940004620 con 0x558a0cafcde0 2023-06-23T18:42:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.832+0000 7fe92b7fe700 1 -- 172.21.15.2:0/2359256866 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe940029070 con 0x558a0cafcde0 2023-06-23T18:42:55.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.832+0000 7fe95ebcfb80 10 client.26746 fetch_fsmap learned FSMap version 41 2023-06-23T18:42:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.832+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2359256866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x558a0c85bc60 con 0x558a0cafcde0 2023-06-23T18:42:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.832+0000 7fe92b7fe700 1 -- 172.21.15.2:0/2359256866 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe940003b90 con 0x558a0cafcde0 2023-06-23T18:42:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe92b7fe700 1 -- 172.21.15.2:0/2359256866 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe94000f760 con 0x558a0cafcde0 2023-06-23T18:42:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe95ebcfb80 10 client.26746 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:42:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe95ebcfb80 10 client.26746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:42:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe95ebcfb80 20 client.26746 populate_metadata read hostname 'smithi002' 2023-06-23T18:42:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558a0cb7fe70 0x558a0cb82280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:42:55.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe95ebcfb80 4 client.26746 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:42:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2359256866 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558a0ca00050 con 0x558a0cb7fe70 2023-06-23T18:42:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe94bfff700 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558a0cb7fe70 0x558a0cb82280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:42:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe92b7fe700 10 client.26746 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:42:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.833+0000 7fe94bfff700 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558a0cb7fe70 0x558a0cb82280 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0ce82c528a054ad server_cookie=ae7fc480d27f9079 in_seq=0 out_seq=0 2023-06-23T18:42:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe92b7fe700 1 -- 172.21.15.2:0/2359256866 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558a0ca00050 con 0x558a0cb7fe70 2023-06-23T18:42:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe92b7fe700 10 client.26746 handle_command_reply: tid=0 2023-06-23T18:42:55.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 client.26746 shutdown 2023-06-23T18:42:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2359256866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafcde0 msgr2=0x558a0cb1a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafcde0 0x558a0cb1a9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe940009ce0 tx=0x7fe940003a70 comp rx=0 tx=0).stop 2023-06-23T18:42:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2359256866 shutdown_connections 2023-06-23T18:42:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558a0cb7fe70 0x558a0cb82280 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558a0cafd810 0x558a0cb1aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558a0cafc490 0x558a0cb1df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 --2- 172.21.15.2:0/2359256866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a0cafcde0 0x558a0cb1a9a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe92b7fe700 0 client.26746 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:42:55.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2359256866 >> 172.21.15.2:0/2359256866 conn(0x558a0cae93f0 msgr2=0x558a0cb03ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:55.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2359256866 shutdown_connections 2023-06-23T18:42:55.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.834+0000 7fe95ebcfb80 1 -- 172.21.15.2:0/2359256866 wait complete. 2023-06-23T18:42:55.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe95ebcfb80 20 client.26746 trim_cache size 0 max 16384 2023-06-23T18:42:55.848 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:42:55.848 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:42:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:42:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:42:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:42:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:42:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:42:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:42:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:42:55.850 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:42:55.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 -- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9340a11d0 msgr2=0x7fe9340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:55.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9340a11d0 0x7fe9340a3680 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe940005910 tx=0x7fe940000bc0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 -- 172.21.15.2:0/3963718659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0fea90 msgr2=0x7fe94c197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:42:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0fea90 0x7fe94c197a80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00b500 tx=0x7fe93c00b810 comp rx=0 tx=0).stop 2023-06-23T18:42:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 -- 172.21.15.2:0/3963718659 shutdown_connections 2023-06-23T18:42:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe94c100610 0x7fe94c19f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9340a11d0 0x7fe9340a3680 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe94c0fea90 0x7fe94c197a80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 --2- 172.21.15.2:0/3963718659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe94c0ffc80 0x7fe94c197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:42:55.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 -- 172.21.15.2:0/3963718659 >> 172.21.15.2:0/3963718659 conn(0x7fe94c0fa090 msgr2=0x7fe94c0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:42:55.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.835+0000 7fe952a4a700 1 -- 172.21.15.2:0/3963718659 shutdown_connections 2023-06-23T18:42:55.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:42:55.836+0000 7fe952a4a700 1 -- 172.21.15.2:0/3963718659 wait complete. 2023-06-23T18:42:55.857 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:42:56.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:56 smithi017 ceph-mon[97472]: pgmap v923: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2023-06-23T18:42:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 545.0 s, 352.6 tps, lat 284.205 ms stddev 49.686 2023-06-23T18:42:57.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:56 smithi002 ceph-mon[95947]: pgmap v923: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2023-06-23T18:42:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:56 smithi082 ceph-mon[97889]: pgmap v923: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 10 MiB/s wr, 1.08k op/s 2023-06-23T18:42:58.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:42:58 smithi017 ceph-mon[97472]: pgmap v924: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.11k op/s 2023-06-23T18:42:59.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:42:58 smithi002 ceph-mon[95947]: pgmap v924: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.11k op/s 2023-06-23T18:42:59.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:42:58 smithi082 ceph-mon[97889]: pgmap v924: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 1.11k op/s 2023-06-23T18:43:00.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:00 smithi017 ceph-mon[97472]: pgmap v925: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.18k op/s 2023-06-23T18:43:01.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:00 smithi002 ceph-mon[95947]: pgmap v925: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.18k op/s 2023-06-23T18:43:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:00 smithi082 ceph-mon[97889]: pgmap v925: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 1.18k op/s 2023-06-23T18:43:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 550.0 s, 360.8 tps, lat 277.016 ms stddev 52.786 2023-06-23T18:43:02.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:02 smithi017 ceph-mon[97472]: pgmap v926: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-06-23T18:43:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:02 smithi002 ceph-mon[95947]: pgmap v926: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-06-23T18:43:03.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:02 smithi082 ceph-mon[97889]: pgmap v926: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 13 MiB/s wr, 1.18k op/s 2023-06-23T18:43:04.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:04 smithi017 ceph-mon[97472]: pgmap v927: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:43:05.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:04 smithi002 ceph-mon[95947]: pgmap v927: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:43:05.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:04 smithi082 ceph-mon[97889]: pgmap v927: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 11 MiB/s wr, 1.10k op/s 2023-06-23T18:43:06.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:06 smithi017 ceph-mon[97472]: pgmap v928: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 555.0 s, 347.6 tps, lat 286.697 ms stddev 119.810 2023-06-23T18:43:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:06 smithi002 ceph-mon[95947]: pgmap v928: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:06 smithi082 ceph-mon[97889]: pgmap v928: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:08 smithi017 ceph-mon[97472]: pgmap v929: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:43:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:08 smithi002 ceph-mon[95947]: pgmap v929: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:43:09.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:08 smithi082 ceph-mon[97889]: pgmap v929: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:43:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:10 smithi017 ceph-mon[97472]: pgmap v930: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:10 smithi002 ceph-mon[95947]: pgmap v930: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:10 smithi082 ceph-mon[97889]: pgmap v930: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 560.0 s, 339.0 tps, lat 295.454 ms stddev 40.989 2023-06-23T18:43:12.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:12 smithi017 ceph-mon[97472]: pgmap v931: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 968 op/s 2023-06-23T18:43:13.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:12 smithi002 ceph-mon[95947]: pgmap v931: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 968 op/s 2023-06-23T18:43:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:12 smithi082 ceph-mon[97889]: pgmap v931: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 968 op/s 2023-06-23T18:43:14.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:14 smithi017 ceph-mon[97472]: pgmap v932: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 937 op/s 2023-06-23T18:43:15.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:14 smithi002 ceph-mon[95947]: pgmap v932: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 937 op/s 2023-06-23T18:43:15.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:14 smithi082 ceph-mon[97889]: pgmap v932: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.2 MiB/s wr, 937 op/s 2023-06-23T18:43:16.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:16 smithi017 ceph-mon[97472]: pgmap v933: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.1 MiB/s wr, 916 op/s 2023-06-23T18:43:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 565.0 s, 334.8 tps, lat 299.796 ms stddev 47.863 2023-06-23T18:43:17.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:16 smithi002 ceph-mon[95947]: pgmap v933: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.1 MiB/s wr, 916 op/s 2023-06-23T18:43:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:16 smithi082 ceph-mon[97889]: pgmap v933: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.1 MiB/s wr, 916 op/s 2023-06-23T18:43:18.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:18 smithi017 ceph-mon[97472]: pgmap v934: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 961 op/s 2023-06-23T18:43:19.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:18 smithi002 ceph-mon[95947]: pgmap v934: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 961 op/s 2023-06-23T18:43:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:18 smithi082 ceph-mon[97889]: pgmap v934: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 961 op/s 2023-06-23T18:43:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:20 smithi017 ceph-mon[97472]: pgmap v935: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:21.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:20 smithi002 ceph-mon[95947]: pgmap v935: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:20 smithi082 ceph-mon[97889]: pgmap v935: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 570.0 s, 353.2 tps, lat 282.567 ms stddev 48.333 2023-06-23T18:43:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:22 smithi017 ceph-mon[97472]: pgmap v936: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:23.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:22 smithi002 ceph-mon[95947]: pgmap v936: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:22 smithi082 ceph-mon[97889]: pgmap v936: 97 pgs: 97 active+clean; 8.4 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:24 smithi017 ceph-mon[97472]: pgmap v937: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-06-23T18:43:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:24 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:43:25.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:24 smithi002 ceph-mon[95947]: pgmap v937: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-06-23T18:43:25.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:24 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:43:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:24 smithi082 ceph-mon[97889]: pgmap v937: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-06-23T18:43:25.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:24 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:43:25.858 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:43:26.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 -- 172.21.15.2:0/331264460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100ffcc0 msgr2=0x7f9810100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 --2- 172.21.15.2:0/331264460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100ffcc0 0x7f9810100130 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f98000098f0 tx=0x7f9800009c00 comp rx=0 tx=0).stop 2023-06-23T18:43:26.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 -- 172.21.15.2:0/331264460 shutdown_connections 2023-06-23T18:43:26.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 --2- 172.21.15.2:0/331264460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9810100670 0x7f9810106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 --2- 172.21.15.2:0/331264460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98100fea30 0x7f98100fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 --2- 172.21.15.2:0/331264460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100ffcc0 0x7f9810100130 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 -- 172.21.15.2:0/331264460 >> 172.21.15.2:0/331264460 conn(0x7f98100fa030 msgr2=0x7f98100fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:26.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 -- 172.21.15.2:0/331264460 shutdown_connections 2023-06-23T18:43:26.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.036+0000 7f9818183700 1 -- 172.21.15.2:0/331264460 wait complete. 2023-06-23T18:43:26.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9818183700 1 Processor -- start 2023-06-23T18:43:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9818183700 1 -- start start 2023-06-23T18:43:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9818183700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100fea30 0x7f9810102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9818183700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98100ffcc0 0x7f98101010a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9818183700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9810100670 0x7f98101015e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9818183700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98101996e0 con 0x7f98100ffcc0 2023-06-23T18:43:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9818183700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810101b20 con 0x7f98100fea30 2023-06-23T18:43:26.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9818183700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9810101ce0 con 0x7f9810100670 2023-06-23T18:43:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9815f1f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100fea30 0x7f9810102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9815f1f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100fea30 0x7f9810102a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48608/0 (socket says 172.21.15.2:48608) 2023-06-23T18:43:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9816720700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9810100670 0x7f98101015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f981571e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98100ffcc0 0x7f98101010a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.037+0000 7f9815f1f700 1 -- 172.21.15.2:0/493096175 learned_addr learned my addr 172.21.15.2:0/493096175 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9815f1f700 1 -- 172.21.15.2:0/493096175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9810100670 msgr2=0x7f98101015e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9815f1f700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9810100670 0x7f98101015e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9815f1f700 1 -- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98100ffcc0 msgr2=0x7f98101010a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9815f1f700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98100ffcc0 0x7f98101010a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9815f1f700 1 -- 172.21.15.2:0/493096175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9800009520 con 0x7f98100fea30 2023-06-23T18:43:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9815f1f700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100fea30 0x7f9810102a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e9c0 tx=0x7f980c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9806ffd700 1 -- 172.21.15.2:0/493096175 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c004a30 con 0x7f98100fea30 2023-06-23T18:43:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9806ffd700 1 -- 172.21.15.2:0/493096175 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f980c00cc50 con 0x7f98100fea30 2023-06-23T18:43:26.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.038+0000 7f9806ffd700 1 -- 172.21.15.2:0/493096175 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c005410 con 0x7f98100fea30 2023-06-23T18:43:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.040+0000 7f9818183700 1 -- 172.21.15.2:0/493096175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98101a4090 con 0x7f98100fea30 2023-06-23T18:43:26.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.040+0000 7f9818183700 1 -- 172.21.15.2:0/493096175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98101a4460 con 0x7f98100fea30 2023-06-23T18:43:26.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.043+0000 7f9806ffd700 1 -- 172.21.15.2:0/493096175 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f980c005570 con 0x7f98100fea30 2023-06-23T18:43:26.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.044+0000 7f9806ffd700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f97fc0a11d0 0x7f97fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.044+0000 7f9824308b80 1 Processor -- start 2023-06-23T18:43:26.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.044+0000 7f9824308b80 1 -- start start 2023-06-23T18:43:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.044+0000 7f9806ffd700 1 -- 172.21.15.2:0/493096175 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f980c014070 con 0x7f98100fea30 2023-06-23T18:43:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.044+0000 7f981571e700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f97fc0a11d0 0x7f97fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f9824308b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b483090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f9824308b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b483a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f9824308b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b484470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f9824308b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b387cb0 con 0x563c6b4835d0 2023-06-23T18:43:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f9824308b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b3daa40 con 0x563c6b484000 2023-06-23T18:43:26.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f9824308b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b3d3ed0 con 0x563c6b482c80 2023-06-23T18:43:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b483a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b483a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58026/0 (socket says 172.21.15.2:58026) 2023-06-23T18:43:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 -- 172.21.15.2:0/1176941071 learned_addr learned my addr 172.21.15.2:0/1176941071 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f9816720700 1 --2- 172.21.15.2:0/1176941071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b484470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f97fc0a11d0 0x7f97fc0a3680 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9800009810 tx=0x7f980000f760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 -- 172.21.15.2:0/1176941071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 msgr2=0x563c6b483090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 --2- 172.21.15.2:0/1176941071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b483090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 -- 172.21.15.2:0/1176941071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 msgr2=0x563c6b484470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 --2- 172.21.15.2:0/1176941071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b484470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.045+0000 7f981571e700 1 -- 172.21.15.2:0/1176941071 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c6b255630 con 0x563c6b4835d0 2023-06-23T18:43:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.046+0000 7f981571e700 1 --2- 172.21.15.2:0/1176941071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b483a70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f98000098f0 tx=0x7f9800027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dd997b61999dd7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.046+0000 7f97fbfff700 1 -- 172.21.15.2:0/1176941071 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800005040 con 0x563c6b4835d0 2023-06-23T18:43:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.046+0000 7f97fbfff700 1 -- 172.21.15.2:0/1176941071 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f980001d070 con 0x563c6b4835d0 2023-06-23T18:43:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.046+0000 7f97fbfff700 1 -- 172.21.15.2:0/1176941071 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9800003cc0 con 0x563c6b4835d0 2023-06-23T18:43:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 -- 172.21.15.2:0/1176941071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 msgr2=0x563c6b483a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 --2- 172.21.15.2:0/1176941071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b483a70 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f98000098f0 tx=0x7f9800027000 comp rx=0 tx=0).stop 2023-06-23T18:43:26.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 -- 172.21.15.2:0/1176941071 shutdown_connections 2023-06-23T18:43:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 --2- 172.21.15.2:0/1176941071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b484470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 --2- 172.21.15.2:0/1176941071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b483090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 --2- 172.21.15.2:0/1176941071 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b483a70 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 -- 172.21.15.2:0/1176941071 >> 172.21.15.2:0/1176941071 conn(0x563c6b4783f0 msgr2=0x563c6b47d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 -- 172.21.15.2:0/1176941071 shutdown_connections 2023-06-23T18:43:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 -- 172.21.15.2:0/1176941071 wait complete. 2023-06-23T18:43:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.047+0000 7f9824308b80 1 -- start start 2023-06-23T18:43:26.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9824308b80 1 Processor -- start 2023-06-23T18:43:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9824308b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b482c80 0x563c6b48c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9824308b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b4835d0 0x563c6b48ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9824308b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b493ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9824308b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b373ac0 con 0x563c6b484000 2023-06-23T18:43:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9824308b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b3737c0 con 0x563c6b482c80 2023-06-23T18:43:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9824308b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b257920 con 0x563c6b4835d0 2023-06-23T18:43:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9824308b80 10 client.0 _ll_register_callbacks cb 0x563c6b379260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:43:26.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9816720700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b493ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9816720700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b493ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58030/0 (socket says 172.21.15.2:58030) 2023-06-23T18:43:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9816720700 1 -- 172.21.15.2:0/1992983454 learned_addr learned my addr 172.21.15.2:0/1992983454 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f9815f1f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b482c80 0x563c6b48c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.048+0000 7f981571e700 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b4835d0 0x563c6b48ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f9815f1f700 1 -- 172.21.15.2:0/1992983454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b4835d0 msgr2=0x563c6b48ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f9815f1f700 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b4835d0 0x563c6b48ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f9815f1f700 1 -- 172.21.15.2:0/1992983454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 msgr2=0x563c6b493ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f9815f1f700 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b493ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f9815f1f700 1 -- 172.21.15.2:0/1992983454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9810107780 con 0x563c6b482c80 2023-06-23T18:43:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f981571e700 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b4835d0 0x563c6b48ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:43:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f9816720700 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b493ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:43:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f9815f1f700 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b482c80 0x563c6b48c540 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f980c00f010 tx=0x7f980c098430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9eafa672c0f5648f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:26.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f97f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:43:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f97f97fa700 1 -- 172.21.15.2:0/1992983454 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c09a2e0 con 0x563c6b482c80 2023-06-23T18:43:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f9824308b80 1 -- 172.21.15.2:0/1992983454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c6b475bb0 con 0x563c6b482c80 2023-06-23T18:43:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f97f97fa700 1 -- 172.21.15.2:0/1992983454 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f980c09deb0 con 0x563c6b482c80 2023-06-23T18:43:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.049+0000 7f97f97fa700 1 -- 172.21.15.2:0/1992983454 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f980c0a22d0 con 0x563c6b482c80 2023-06-23T18:43:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.050+0000 7f9824308b80 10 client.26534 fetch_fsmap learned FSMap version 41 2023-06-23T18:43:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.050+0000 7f9824308b80 1 -- 172.21.15.2:0/1992983454 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563c6b201de0 con 0x563c6b482c80 2023-06-23T18:43:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.050+0000 7f97f97fa700 1 -- 172.21.15.2:0/1992983454 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f980c0c8080 con 0x563c6b482c80 2023-06-23T18:43:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f97f97fa700 1 -- 172.21.15.2:0/1992983454 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f980c0a1e40 con 0x563c6b482c80 2023-06-23T18:43:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f9824308b80 10 client.26534 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:43:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f9824308b80 10 client.26534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:43:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f9824308b80 20 client.26534 populate_metadata read hostname 'smithi002' 2023-06-23T18:43:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f9824308b80 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563c6b4982a0 0x563c6b49a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f9824308b80 4 client.26534 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:43:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f9824308b80 1 -- 172.21.15.2:0/1992983454 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c6b3cceb0 con 0x563c6b4982a0 2023-06-23T18:43:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f981571e700 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563c6b4982a0 0x563c6b49a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.052+0000 7f981571e700 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563c6b4982a0 0x563c6b49a6b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ddfa946b6575f7 server_cookie=e1442a68ff6bb68e in_seq=0 out_seq=0 2023-06-23T18:43:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.053+0000 7f97f97fa700 10 client.26534 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:43:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.053+0000 7f97f97fa700 1 -- 172.21.15.2:0/1992983454 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563c6b3cceb0 con 0x563c6b4982a0 2023-06-23T18:43:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f97f97fa700 10 client.26534 handle_command_reply: tid=0 2023-06-23T18:43:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 client.26534 shutdown 2023-06-23T18:43:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 -- 172.21.15.2:0/1992983454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b482c80 msgr2=0x563c6b48c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b482c80 0x563c6b48c540 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f980c00f010 tx=0x7f980c098430 comp rx=0 tx=0).stop 2023-06-23T18:43:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 -- 172.21.15.2:0/1992983454 shutdown_connections 2023-06-23T18:43:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563c6b4982a0 0x563c6b49a6b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b493ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f97f97fa700 0 client.26534 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:43:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b482c80 0x563c6b48c540 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 --2- 172.21.15.2:0/1992983454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b4835d0 0x563c6b48ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 -- 172.21.15.2:0/1992983454 >> 172.21.15.2:0/1992983454 conn(0x563c6b4783f0 msgr2=0x563c6b47cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 -- 172.21.15.2:0/1992983454 shutdown_connections 2023-06-23T18:43:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 1 -- 172.21.15.2:0/1992983454 wait complete. 2023-06-23T18:43:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.054+0000 7f9824308b80 20 client.26534 trim_cache size 0 max 16384 2023-06-23T18:43:26.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9824308b80 1 Processor -- start 2023-06-23T18:43:26.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9824308b80 1 -- start start 2023-06-23T18:43:26.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9824308b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b486ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9824308b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b493a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9824308b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b487400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9824308b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b3cceb0 con 0x563c6b4835d0 2023-06-23T18:43:26.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9824308b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b47e810 con 0x563c6b484000 2023-06-23T18:43:26.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9824308b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b48b9f0 con 0x563c6b482c80 2023-06-23T18:43:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9816720700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b487400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f981571e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b493a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9816720700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b487400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48634/0 (socket says 172.21.15.2:48634) 2023-06-23T18:43:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9815f1f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b486ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9816720700 1 -- 172.21.15.2:0/1239917641 learned_addr learned my addr 172.21.15.2:0/1239917641 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9815f1f700 1 -- 172.21.15.2:0/1239917641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 msgr2=0x563c6b487400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:26.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9815f1f700 1 --2- 172.21.15.2:0/1239917641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b487400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9815f1f700 1 -- 172.21.15.2:0/1239917641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 msgr2=0x563c6b493a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9815f1f700 1 --2- 172.21.15.2:0/1239917641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b493a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.080+0000 7f9815f1f700 1 -- 172.21.15.2:0/1239917641 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c6b1eac60 con 0x563c6b482c80 2023-06-23T18:43:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.081+0000 7f9815f1f700 1 --2- 172.21.15.2:0/1239917641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b486ec0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f980c005950 tx=0x7f980c009550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c80e17290a017ab server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.081+0000 7f9804ff9700 1 -- 172.21.15.2:0/1239917641 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0adc20 con 0x563c6b482c80 2023-06-23T18:43:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.081+0000 7f9804ff9700 1 -- 172.21.15.2:0/1239917641 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563c6b475bb0 con 0x563c6b482c80 2023-06-23T18:43:26.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.081+0000 7f9804ff9700 1 -- 172.21.15.2:0/1239917641 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0ca3c0 con 0x563c6b482c80 2023-06-23T18:43:26.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.082+0000 7f9824308b80 1 -- 172.21.15.2:0/1239917641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 msgr2=0x563c6b486ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.082+0000 7f9824308b80 1 --2- 172.21.15.2:0/1239917641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b486ec0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f980c005950 tx=0x7f980c009550 comp rx=0 tx=0).stop 2023-06-23T18:43:26.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.082+0000 7f9824308b80 1 -- 172.21.15.2:0/1239917641 shutdown_connections 2023-06-23T18:43:26.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.082+0000 7f9824308b80 1 --2- 172.21.15.2:0/1239917641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b484000 0x563c6b487400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.082+0000 7f9824308b80 1 --2- 172.21.15.2:0/1239917641 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b486ec0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.082+0000 7f9824308b80 1 --2- 172.21.15.2:0/1239917641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b4835d0 0x563c6b493a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.082+0000 7f9824308b80 1 -- 172.21.15.2:0/1239917641 >> 172.21.15.2:0/1239917641 conn(0x563c6b4783f0 msgr2=0x563c6b48b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:26.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.083+0000 7f9824308b80 1 -- 172.21.15.2:0/1239917641 shutdown_connections 2023-06-23T18:43:26.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.085+0000 7f9824308b80 1 -- 172.21.15.2:0/1239917641 wait complete. 2023-06-23T18:43:26.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.085+0000 7f9824308b80 1 -- start start 2023-06-23T18:43:26.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.085+0000 7f9824308b80 1 Processor -- start 2023-06-23T18:43:26.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9824308b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b493410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9824308b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b4835d0 0x563c6b493950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9824308b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b48feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9824308b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b4905c0 con 0x563c6b484000 2023-06-23T18:43:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9824308b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b490730 con 0x563c6b4835d0 2023-06-23T18:43:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9824308b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c6b4908f0 con 0x563c6b482c80 2023-06-23T18:43:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9824308b80 10 client.0 _ll_register_callbacks cb 0x563c6b379260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:43:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9816720700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b48feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9816720700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b48feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58046/0 (socket says 172.21.15.2:58046) 2023-06-23T18:43:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.086+0000 7f9816720700 1 -- 172.21.15.2:0/95855146 learned_addr learned my addr 172.21.15.2:0/95855146 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.087+0000 7f9815f1f700 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b493410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.087+0000 7f9816720700 1 -- 172.21.15.2:0/95855146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 msgr2=0x563c6b493410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.087+0000 7f9816720700 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b493410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.087+0000 7f9816720700 1 -- 172.21.15.2:0/95855146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b4835d0 msgr2=0x563c6b493950 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:43:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.087+0000 7f9816720700 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b4835d0 0x563c6b493950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.087+0000 7f9816720700 1 -- 172.21.15.2:0/95855146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c6b201de0 con 0x563c6b484000 2023-06-23T18:43:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.087+0000 7f9816720700 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b48feb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9808004710 tx=0x7f98080047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f91fbc1f408c6c4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:26.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.087+0000 7f97f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:43:26.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.088+0000 7f97f97fa700 1 -- 172.21.15.2:0/95855146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980801c070 con 0x563c6b484000 2023-06-23T18:43:26.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.088+0000 7f97f97fa700 1 -- 172.21.15.2:0/95855146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9808004bb0 con 0x563c6b484000 2023-06-23T18:43:26.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.088+0000 7f9824308b80 1 -- 172.21.15.2:0/95855146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f980001d070 con 0x563c6b484000 2023-06-23T18:43:26.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.088+0000 7f97f97fa700 1 -- 172.21.15.2:0/95855146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f980800e040 con 0x563c6b484000 2023-06-23T18:43:26.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.088+0000 7f9824308b80 10 client.17028 fetch_fsmap learned FSMap version 41 2023-06-23T18:43:26.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.088+0000 7f9824308b80 1 -- 172.21.15.2:0/95855146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563c6b228250 con 0x563c6b484000 2023-06-23T18:43:26.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.088+0000 7f97f97fa700 1 -- 172.21.15.2:0/95855146 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f980001d070 con 0x563c6b484000 2023-06-23T18:43:26.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.088+0000 7f97f97fa700 1 -- 172.21.15.2:0/95855146 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9808021940 con 0x563c6b484000 2023-06-23T18:43:26.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.089+0000 7f9824308b80 10 client.17028 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:43:26.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.089+0000 7f9824308b80 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:43:26.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.089+0000 7f9824308b80 20 client.17028 populate_metadata read hostname 'smithi002' 2023-06-23T18:43:26.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.089+0000 7f9824308b80 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563c6b4ed870 0x563c6b4efc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:26.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.089+0000 7f9824308b80 4 client.17028 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:43:26.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.089+0000 7f9824308b80 1 -- 172.21.15.2:0/95855146 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563c6b38f050 con 0x563c6b4ed870 2023-06-23T18:43:26.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.089+0000 7f9815f1f700 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563c6b4ed870 0x563c6b4efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:26.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.089+0000 7f97f97fa700 10 client.17028 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:43:26.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9815f1f700 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563c6b4ed870 0x563c6b4efc80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cf5268fb19d3f0 server_cookie=2e7e15237543f9c0 in_seq=0 out_seq=0 2023-06-23T18:43:26.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f97f97fa700 1 -- 172.21.15.2:0/95855146 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563c6b38f050 con 0x563c6b4ed870 2023-06-23T18:43:26.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f97f97fa700 10 client.17028 handle_command_reply: tid=0 2023-06-23T18:43:26.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 client.17028 shutdown 2023-06-23T18:43:26.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 -- 172.21.15.2:0/95855146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 msgr2=0x563c6b48feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b48feb0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9808004710 tx=0x7f98080047f0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 -- 172.21.15.2:0/95855146 shutdown_connections 2023-06-23T18:43:26.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x563c6b4ed870 0x563c6b4efc80 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x563c6b484000 0x563c6b48feb0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f97f97fa700 0 client.17028 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:43:26.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563c6b482c80 0x563c6b493410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 --2- 172.21.15.2:0/95855146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x563c6b4835d0 0x563c6b493950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 -- 172.21.15.2:0/95855146 >> 172.21.15.2:0/95855146 conn(0x563c6b4783f0 msgr2=0x563c6b486ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:26.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 -- 172.21.15.2:0/95855146 shutdown_connections 2023-06-23T18:43:26.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 1 -- 172.21.15.2:0/95855146 wait complete. 2023-06-23T18:43:26.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.090+0000 7f9824308b80 20 client.17028 trim_cache size 0 max 16384 2023-06-23T18:43:26.100 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:43:26.100 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:43:26.100 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:43:26.100 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:43:26.100 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:43:26.100 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:43:26.101 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:43:26.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:43:26.101 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:43:26.101 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:43:26.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 -- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f97fc0a11d0 msgr2=0x7f97fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f97fc0a11d0 0x7f97fc0a3680 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9800009810 tx=0x7f980000f760 comp rx=0 tx=0).stop 2023-06-23T18:43:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 -- 172.21.15.2:0/493096175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100fea30 msgr2=0x7f9810102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100fea30 0x7f9810102a60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e9c0 tx=0x7f980c00ecd0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 -- 172.21.15.2:0/493096175 shutdown_connections 2023-06-23T18:43:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9810100670 0x7f98101015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f97fc0a11d0 0x7f97fc0a3680 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98100fea30 0x7f9810102a60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 --2- 172.21.15.2:0/493096175 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f98100ffcc0 0x7f98101010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:26.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 -- 172.21.15.2:0/493096175 >> 172.21.15.2:0/493096175 conn(0x7f98100fa030 msgr2=0x7f98100fa9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:26.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 -- 172.21.15.2:0/493096175 shutdown_connections 2023-06-23T18:43:26.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:26.091+0000 7f9818183700 1 -- 172.21.15.2:0/493096175 wait complete. 2023-06-23T18:43:26.113 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:43:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:26 smithi017 ceph-mon[97472]: pgmap v938: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 990 op/s 2023-06-23T18:43:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 575.0 s, 336.8 tps, lat 295.847 ms stddev 47.693 2023-06-23T18:43:27.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:26 smithi002 ceph-mon[95947]: pgmap v938: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 990 op/s 2023-06-23T18:43:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:26 smithi082 ceph-mon[97889]: pgmap v938: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 990 op/s 2023-06-23T18:43:28.946 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:28 smithi017 ceph-mon[97472]: pgmap v939: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:43:28.947 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:28 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:28.947 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:28 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:29.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:28 smithi002 ceph-mon[95947]: pgmap v939: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:43:29.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:28 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:29.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:28 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:28 smithi082 ceph-mon[97889]: pgmap v939: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:43:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:28 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:28 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:30.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:30 smithi002 ceph-mon[95947]: pgmap v940: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:30.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:30.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:30 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:30 smithi082 ceph-mon[97889]: pgmap v940: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:30.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:30 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:30 smithi017 ceph-mon[97472]: pgmap v940: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:30.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:30 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 580.0 s, 337.2 tps, lat 297.304 ms stddev 99.454 2023-06-23T18:43:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:32 smithi002 ceph-mon[95947]: pgmap v941: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 992 op/s 2023-06-23T18:43:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:43:32.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:43:32.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:32 smithi082 ceph-mon[97889]: pgmap v941: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 992 op/s 2023-06-23T18:43:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:43:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:43:32.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:32 smithi017 ceph-mon[97472]: pgmap v941: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.6 MiB/s wr, 992 op/s 2023-06-23T18:43:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:43:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:43:32.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:43:34.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:34 smithi002 ceph-mon[95947]: pgmap v942: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:43:34.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:34 smithi082 ceph-mon[97889]: pgmap v942: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:43:34.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:34 smithi017 ceph-mon[97472]: pgmap v942: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:43:36.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:36 smithi017 ceph-mon[97472]: pgmap v943: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 957 op/s 2023-06-23T18:43:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:36 smithi002 ceph-mon[95947]: pgmap v943: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 957 op/s 2023-06-23T18:43:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:36 smithi082 ceph-mon[97889]: pgmap v943: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 957 op/s 2023-06-23T18:43:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 585.0 s, 351.2 tps, lat 284.844 ms stddev 43.355 2023-06-23T18:43:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:38 smithi017 ceph-mon[97472]: pgmap v944: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:43:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:38 smithi002 ceph-mon[95947]: pgmap v944: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:43:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:38 smithi082 ceph-mon[97889]: pgmap v944: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:43:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:40 smithi017 ceph-mon[97472]: pgmap v945: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-06-23T18:43:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:40 smithi002 ceph-mon[95947]: pgmap v945: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-06-23T18:43:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:40 smithi082 ceph-mon[97889]: pgmap v945: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.05k op/s 2023-06-23T18:43:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 590.0 s, 339.6 tps, lat 296.631 ms stddev 51.247 2023-06-23T18:43:42.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:42 smithi017 ceph-mon[97472]: pgmap v946: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:42 smithi002 ceph-mon[95947]: pgmap v946: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:42 smithi082 ceph-mon[97889]: pgmap v946: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:44 smithi017 ceph-mon[97472]: pgmap v947: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:43:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:44 smithi002 ceph-mon[95947]: pgmap v947: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:43:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:44 smithi082 ceph-mon[97889]: pgmap v947: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-06-23T18:43:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:46 smithi017 ceph-mon[97472]: pgmap v948: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:43:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:46 smithi002 ceph-mon[95947]: pgmap v948: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:43:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:46 smithi082 ceph-mon[97889]: pgmap v948: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 10 MiB/s wr, 1.00k op/s 2023-06-23T18:43:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 595.0 s, 346.0 tps, lat 286.793 ms stddev 45.059 2023-06-23T18:43:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:48 smithi017 ceph-mon[97472]: pgmap v949: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:48 smithi002 ceph-mon[95947]: pgmap v949: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:48 smithi082 ceph-mon[97889]: pgmap v949: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-06-23T18:43:50.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:50 smithi017 ceph-mon[97472]: pgmap v950: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.08k op/s 2023-06-23T18:43:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:50 smithi002 ceph-mon[95947]: pgmap v950: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.08k op/s 2023-06-23T18:43:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:50 smithi082 ceph-mon[97889]: pgmap v950: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.08k op/s 2023-06-23T18:43:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 600.0 s, 313.0 tps, lat 317.762 ms stddev 65.842 2023-06-23T18:43:52.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:52 smithi017 ceph-mon[97472]: pgmap v951: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:52.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:52 smithi002 ceph-mon[95947]: pgmap v951: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:52 smithi082 ceph-mon[97889]: pgmap v951: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 11 MiB/s wr, 1.04k op/s 2023-06-23T18:43:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:54 smithi017 ceph-mon[97472]: pgmap v952: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:54.806 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:54 smithi002 ceph-mon[95947]: pgmap v952: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:54 smithi082 ceph-mon[97889]: pgmap v952: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 1.03k op/s 2023-06-23T18:43:56.114 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:43:56.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/1638200401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fea60 msgr2=0x7f2fdc0fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/1638200401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fea60 0x7f2fdc0fee70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009990 tx=0x7f2fd0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/1638200401 shutdown_connections 2023-06-23T18:43:56.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/1638200401 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fdc1006a0 0x7f2fdc106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/1638200401 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0fea60 0x7f2fdc0fee70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/1638200401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fdc0ffcf0 0x7f2fdc100160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/1638200401 >> 172.21.15.2:0/1638200401 conn(0x7f2fdc0fa080 msgr2=0x7f2fdc0fc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:56.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/1638200401 shutdown_connections 2023-06-23T18:43:56.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.353+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/1638200401 wait complete. 2023-06-23T18:43:56.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.354+0000 7f2fe2b5b700 1 Processor -- start 2023-06-23T18:43:56.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.354+0000 7f2fe2b5b700 1 -- start start 2023-06-23T18:43:56.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.354+0000 7f2fe2b5b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fdc0fea60 0x7f2fdc104c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.354+0000 7f2fe2b5b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0ffcf0 0x7f2fdc1051a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.354+0000 7f2fe2b5b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fdc1006a0 0x7f2fdc1017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.354+0000 7f2fe2b5b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc101f90 con 0x7f2fdc0fea60 2023-06-23T18:43:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.354+0000 7f2fe2b5b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc102100 con 0x7f2fdc1006a0 2023-06-23T18:43:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.354+0000 7f2fe2b5b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1022c0 con 0x7f2fdc0ffcf0 2023-06-23T18:43:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0ffcf0 0x7f2fdc1051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe08f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fdc0fea60 0x7f2fdc104c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe10f8700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fdc1006a0 0x7f2fdc1017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0ffcf0 0x7f2fdc1051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58396/0 (socket says 172.21.15.2:58396) 2023-06-23T18:43:56.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fdbfff700 1 -- 172.21.15.2:0/2709231903 learned_addr learned my addr 172.21.15.2:0/2709231903 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:56.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fdc0fea60 0x7f2fdc104c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58056/0 (socket says 172.21.15.2:58056) 2023-06-23T18:43:56.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe10f8700 1 -- 172.21.15.2:0/2709231903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0ffcf0 msgr2=0x7f2fdc1051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0ffcf0 0x7f2fdc1051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe10f8700 1 -- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fdc0fea60 msgr2=0x7f2fdc104c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fdc0fea60 0x7f2fdc104c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe10f8700 1 -- 172.21.15.2:0/2709231903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00095f0 con 0x7f2fdc1006a0 2023-06-23T18:43:56.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.355+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fdc1006a0 0x7f2fdc1017f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4007f10 tx=0x7f2fd400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:56.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.358+0000 7f2fd9ffb700 1 -- 172.21.15.2:0/2709231903 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4004db0 con 0x7f2fdc1006a0 2023-06-23T18:43:56.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.358+0000 7f2fd9ffb700 1 -- 172.21.15.2:0/2709231903 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fd400f040 con 0x7f2fdc1006a0 2023-06-23T18:43:56.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.358+0000 7f2fd9ffb700 1 -- 172.21.15.2:0/2709231903 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4005960 con 0x7f2fdc1006a0 2023-06-23T18:43:56.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.358+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/2709231903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc1a3ff0 con 0x7f2fdc1006a0 2023-06-23T18:43:56.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.359+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/2709231903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc1a4480 con 0x7f2fdc1006a0 2023-06-23T18:43:56.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.361+0000 7f2fd9ffb700 1 -- 172.21.15.2:0/2709231903 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2fd400db20 con 0x7f2fdc1006a0 2023-06-23T18:43:56.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.362+0000 7f2fd9ffb700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2fc40a12a0 0x7f2fc40a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.362+0000 7f2fd9ffb700 1 -- 172.21.15.2:0/2709231903 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2fd40cbff0 con 0x7f2fdc1006a0 2023-06-23T18:43:56.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.363+0000 7f2feece0b80 1 Processor -- start 2023-06-23T18:43:56.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2feece0b80 1 -- start start 2023-06-23T18:43:56.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 0x55f1d2033090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d20335d0 0x55f1d2033a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2034000 0x55f1d2034470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2feece0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1f37cb0 con 0x55f1d2032c80 2023-06-23T18:43:56.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2feece0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1f8aa40 con 0x55f1d2034000 2023-06-23T18:43:56.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2feece0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1f83ed0 con 0x55f1d20335d0 2023-06-23T18:43:56.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d20335d0 0x55f1d2033a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d20335d0 0x55f1d2033a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58400/0 (socket says 172.21.15.2:58400) 2023-06-23T18:43:56.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2fdbfff700 1 -- 172.21.15.2:0/3816778096 learned_addr learned my addr 172.21.15.2:0/3816778096 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:56.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2034000 0x55f1d2034470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2fc40a12a0 0x7f2fc40a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.364+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 0x55f1d2033090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fdbfff700 1 -- 172.21.15.2:0/3816778096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2034000 msgr2=0x55f1d2034470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:56.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2034000 0x55f1d2034470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fdbfff700 1 -- 172.21.15.2:0/3816778096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 msgr2=0x55f1d2033090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:56.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 0x55f1d2033090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fdbfff700 1 -- 172.21.15.2:0/3816778096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1d1e05630 con 0x55f1d20335d0 2023-06-23T18:43:56.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2fc40a12a0 0x7f2fc40a3750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006010 tx=0x7f2fd001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:56.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d20335d0 0x55f1d2033a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009bb0 tx=0x7f2fcc009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eca2f9ceccf4a60e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:56.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fc2ffd700 1 -- 172.21.15.2:0/3816778096 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0096b0 con 0x55f1d20335d0 2023-06-23T18:43:56.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fc2ffd700 1 -- 172.21.15.2:0/3816778096 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc00ee20 con 0x55f1d20335d0 2023-06-23T18:43:56.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.365+0000 7f2fc2ffd700 1 -- 172.21.15.2:0/3816778096 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0123d0 con 0x55f1d20335d0 2023-06-23T18:43:56.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 -- 172.21.15.2:0/3816778096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d20335d0 msgr2=0x55f1d2033a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d20335d0 0x55f1d2033a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009bb0 tx=0x7f2fcc009ec0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 -- 172.21.15.2:0/3816778096 shutdown_connections 2023-06-23T18:43:56.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2034000 0x55f1d2034470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 0x55f1d2033090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 --2- 172.21.15.2:0/3816778096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d20335d0 0x55f1d2033a70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 -- 172.21.15.2:0/3816778096 >> 172.21.15.2:0/3816778096 conn(0x55f1d20283f0 msgr2=0x55f1d202d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:56.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 -- 172.21.15.2:0/3816778096 shutdown_connections 2023-06-23T18:43:56.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.366+0000 7f2feece0b80 1 -- 172.21.15.2:0/3816778096 wait complete. 2023-06-23T18:43:56.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 1 -- start start 2023-06-23T18:43:56.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 1 Processor -- start 2023-06-23T18:43:56.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2032c80 0x55f1d204d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d20335d0 0x55f1d204daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 0x55f1d2054ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1f23ac0 con 0x55f1d20335d0 2023-06-23T18:43:56.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1f237c0 con 0x55f1d2032c80 2023-06-23T18:43:56.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1e07920 con 0x55f1d2034000 2023-06-23T18:43:56.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2feece0b80 10 client.0 _ll_register_callbacks cb 0x55f1d1f29260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:43:56.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2fe08f7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2032c80 0x55f1d204d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2fe08f7700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2032c80 0x55f1d204d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48662/0 (socket says 172.21.15.2:48662) 2023-06-23T18:43:56.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.367+0000 7f2fe08f7700 1 -- 172.21.15.2:0/1997552538 learned_addr learned my addr 172.21.15.2:0/1997552538 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:56.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 0x55f1d2054ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fe08f7700 1 -- 172.21.15.2:0/1997552538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 msgr2=0x55f1d2054ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:56.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 0x55f1d2054ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fe08f7700 1 -- 172.21.15.2:0/1997552538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d20335d0 msgr2=0x55f1d204daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:56.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d20335d0 0x55f1d204daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fe08f7700 1 -- 172.21.15.2:0/1997552538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fcc009810 con 0x55f1d2032c80 2023-06-23T18:43:56.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 0x55f1d2054ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:43:56.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2032c80 0x55f1d204d560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009990 tx=0x7f2fd0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e13ddfb4c55fc0da server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:56.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:43:56.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fbbfff700 1 -- 172.21.15.2:0/1997552538 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0004b70 con 0x55f1d2032c80 2023-06-23T18:43:56.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2feece0b80 1 -- 172.21.15.2:0/1997552538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1d2025bb0 con 0x55f1d2032c80 2023-06-23T18:43:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fbbfff700 1 -- 172.21.15.2:0/1997552538 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2fd0004cd0 con 0x55f1d2032c80 2023-06-23T18:43:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.368+0000 7f2fbbfff700 1 -- 172.21.15.2:0/1997552538 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fd0004e90 con 0x55f1d2032c80 2023-06-23T18:43:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2feece0b80 10 client.26546 fetch_fsmap learned FSMap version 41 2023-06-23T18:43:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2feece0b80 1 -- 172.21.15.2:0/1997552538 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55f1d1db1de0 con 0x55f1d2032c80 2023-06-23T18:43:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2fbbfff700 1 -- 172.21.15.2:0/1997552538 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f1d2025bb0 con 0x55f1d2032c80 2023-06-23T18:43:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2fbbfff700 1 -- 172.21.15.2:0/1997552538 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2fd000f580 con 0x55f1d2032c80 2023-06-23T18:43:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2feece0b80 10 client.26546 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:43:56.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2feece0b80 10 client.26546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:43:56.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2feece0b80 20 client.26546 populate_metadata read hostname 'smithi002' 2023-06-23T18:43:56.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2feece0b80 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f1d2059330 0x55f1d205b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2feece0b80 4 client.26546 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:43:56.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.369+0000 7f2feece0b80 1 -- 172.21.15.2:0/1997552538 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1d1f7ceb0 con 0x55f1d2059330 2023-06-23T18:43:56.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.371+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f1d2059330 0x55f1d205b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.371+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f1d2059330 0x55f1d205b740 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5a009a9e56f366 server_cookie=4f4d38d3f0684139 in_seq=0 out_seq=0 2023-06-23T18:43:56.382 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.371+0000 7f2fbbfff700 10 client.26546 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:43:56.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2fbbfff700 1 -- 172.21.15.2:0/1997552538 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f1d1f7ceb0 con 0x55f1d2059330 2023-06-23T18:43:56.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2fbbfff700 10 client.26546 handle_command_reply: tid=0 2023-06-23T18:43:56.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 client.26546 shutdown 2023-06-23T18:43:56.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 -- 172.21.15.2:0/1997552538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2032c80 msgr2=0x55f1d204d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2032c80 0x55f1d204d560 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009990 tx=0x7f2fd0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 -- 172.21.15.2:0/1997552538 shutdown_connections 2023-06-23T18:43:56.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f1d2059330 0x55f1d205b740 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 0x55f1d2054ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2fbbfff700 0 client.26546 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:43:56.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d2032c80 0x55f1d204d560 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 --2- 172.21.15.2:0/1997552538 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d20335d0 0x55f1d204daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 -- 172.21.15.2:0/1997552538 >> 172.21.15.2:0/1997552538 conn(0x55f1d20283f0 msgr2=0x55f1d202cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:56.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 -- 172.21.15.2:0/1997552538 shutdown_connections 2023-06-23T18:43:56.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.372+0000 7f2feece0b80 1 -- 172.21.15.2:0/1997552538 wait complete. 2023-06-23T18:43:56.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.373+0000 7f2feece0b80 20 client.26546 trim_cache size 0 max 16384 2023-06-23T18:43:56.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.394+0000 7f2feece0b80 1 Processor -- start 2023-06-23T18:43:56.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.394+0000 7f2feece0b80 1 -- start start 2023-06-23T18:43:56.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.394+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2032c80 0x55f1d20553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2054aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2034000 0x55f1d204d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2feece0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d20567a0 con 0x55f1d2034000 2023-06-23T18:43:56.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2feece0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d202de20 con 0x55f1d20335d0 2023-06-23T18:43:56.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2feece0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d203a0e0 con 0x55f1d2032c80 2023-06-23T18:43:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2054aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe10f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2034000 0x55f1d204d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe08f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2032c80 0x55f1d20553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2054aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48672/0 (socket says 172.21.15.2:48672) 2023-06-23T18:43:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe08f7700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2032c80 0x55f1d20553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58416/0 (socket says 172.21.15.2:58416) 2023-06-23T18:43:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fdbfff700 1 -- 172.21.15.2:0/3314349857 learned_addr learned my addr 172.21.15.2:0/3314349857 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:56.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe10f8700 1 -- 172.21.15.2:0/3314349857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2032c80 msgr2=0x55f1d20553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/3314349857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2032c80 0x55f1d20553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe10f8700 1 -- 172.21.15.2:0/3314349857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 msgr2=0x55f1d2054aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/3314349857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2054aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe10f8700 1 -- 172.21.15.2:0/3314349857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1d1d9ac60 con 0x55f1d2034000 2023-06-23T18:43:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/3314349857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2054aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:43:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.395+0000 7f2fe10f8700 1 --2- 172.21.15.2:0/3314349857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2034000 0x55f1d204d310 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4007ee0 tx=0x7f2fd40b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5d27ed285c72ed server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:56.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.396+0000 7f2fc37fe700 1 -- 172.21.15.2:0/3314349857 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4005ac0 con 0x55f1d2034000 2023-06-23T18:43:56.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.396+0000 7f2fc37fe700 1 -- 172.21.15.2:0/3314349857 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fd4005000 con 0x55f1d2034000 2023-06-23T18:43:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.396+0000 7f2fc37fe700 1 -- 172.21.15.2:0/3314349857 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd409d160 con 0x55f1d2034000 2023-06-23T18:43:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 -- 172.21.15.2:0/3314349857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2034000 msgr2=0x55f1d204d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 --2- 172.21.15.2:0/3314349857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2034000 0x55f1d204d310 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4007ee0 tx=0x7f2fd40b9040 comp rx=0 tx=0).stop 2023-06-23T18:43:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 -- 172.21.15.2:0/3314349857 shutdown_connections 2023-06-23T18:43:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 --2- 172.21.15.2:0/3314349857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2034000 0x55f1d204d310 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 --2- 172.21.15.2:0/3314349857 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2032c80 0x55f1d20553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 --2- 172.21.15.2:0/3314349857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2054aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 -- 172.21.15.2:0/3314349857 >> 172.21.15.2:0/3314349857 conn(0x55f1d20283f0 msgr2=0x55f1d20600a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 -- 172.21.15.2:0/3314349857 shutdown_connections 2023-06-23T18:43:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 -- 172.21.15.2:0/3314349857 wait complete. 2023-06-23T18:43:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.397+0000 7f2feece0b80 1 -- start start 2023-06-23T18:43:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2feece0b80 1 Processor -- start 2023-06-23T18:43:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 0x55f1d2054880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2051320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2feece0b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 0x55f1d2051860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2feece0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d2051e30 con 0x55f1d2032c80 2023-06-23T18:43:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2feece0b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d2051fa0 con 0x55f1d20335d0 2023-06-23T18:43:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2feece0b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d20ad260 con 0x55f1d2034000 2023-06-23T18:43:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2feece0b80 10 client.0 _ll_register_callbacks cb 0x55f1d1f29260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:43:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2051320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fe08f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 0x55f1d2054880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fdbfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2051320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48680/0 (socket says 172.21.15.2:48680) 2023-06-23T18:43:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fdbfff700 1 -- 172.21.15.2:0/2288731460 learned_addr learned my addr 172.21.15.2:0/2288731460 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:43:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fdbfff700 1 -- 172.21.15.2:0/2288731460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 msgr2=0x55f1d2051860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 0x55f1d2051860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fdbfff700 1 -- 172.21.15.2:0/2288731460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 msgr2=0x55f1d2054880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:43:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 0x55f1d2054880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.398+0000 7f2fdbfff700 1 -- 172.21.15.2:0/2288731460 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1d1e05630 con 0x55f1d20335d0 2023-06-23T18:43:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2fdbfff700 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2051320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009bb0 tx=0x7f2fcc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea44599ab19fb898 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:43:56.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2fbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:43:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2fbbfff700 1 -- 172.21.15.2:0/2288731460 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc024520 con 0x55f1d20335d0 2023-06-23T18:43:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2fbbfff700 1 -- 172.21.15.2:0/2288731460 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2fcc021050 con 0x55f1d20335d0 2023-06-23T18:43:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2feece0b80 1 -- 172.21.15.2:0/2288731460 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fcc00ee20 con 0x55f1d20335d0 2023-06-23T18:43:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2fbbfff700 1 -- 172.21.15.2:0/2288731460 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fcc0248d0 con 0x55f1d20335d0 2023-06-23T18:43:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2feece0b80 10 client.26552 fetch_fsmap learned FSMap version 41 2023-06-23T18:43:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2feece0b80 1 -- 172.21.15.2:0/2288731460 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f2fdc1a4480 con 0x55f1d20335d0 2023-06-23T18:43:56.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2fbbfff700 1 -- 172.21.15.2:0/2288731460 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc00ee20 con 0x55f1d20335d0 2023-06-23T18:43:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2fbbfff700 1 -- 172.21.15.2:0/2288731460 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2fcc011070 con 0x55f1d20335d0 2023-06-23T18:43:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2feece0b80 10 client.26552 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:43:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.399+0000 7f2feece0b80 10 client.26552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:43:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2feece0b80 20 client.26552 populate_metadata read hostname 'smithi002' 2023-06-23T18:43:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2feece0b80 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f1d2059330 0x55f1d20b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:43:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2feece0b80 4 client.26552 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:43:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2feece0b80 1 -- 172.21.15.2:0/2288731460 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f1d1f3f050 con 0x55f1d2059330 2023-06-23T18:43:56.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f1d2059330 0x55f1d20b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:43:56.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2fbbfff700 10 client.26552 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:43:56.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2fe08f7700 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f1d2059330 0x55f1d20b0b10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d4349d1e45dd19 server_cookie=778aab8103da5927 in_seq=0 out_seq=0 2023-06-23T18:43:56.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2fbbfff700 1 -- 172.21.15.2:0/2288731460 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f1d1f3f050 con 0x55f1d2059330 2023-06-23T18:43:56.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2fbbfff700 10 client.26552 handle_command_reply: tid=0 2023-06-23T18:43:56.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.400+0000 7f2feece0b80 1 client.26552 shutdown 2023-06-23T18:43:56.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 -- 172.21.15.2:0/2288731460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 msgr2=0x55f1d2051320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2051320 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009bb0 tx=0x7f2fcc00ef90 comp rx=0 tx=0).stop 2023-06-23T18:43:56.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 -- 172.21.15.2:0/2288731460 shutdown_connections 2023-06-23T18:43:56.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55f1d2059330 0x55f1d20b0b10 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f1d2034000 0x55f1d2051860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55f1d2032c80 0x55f1d2054880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 --2- 172.21.15.2:0/2288731460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55f1d20335d0 0x55f1d2051320 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 -- 172.21.15.2:0/2288731460 >> 172.21.15.2:0/2288731460 conn(0x55f1d20283f0 msgr2=0x55f1d2036e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:56.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2fbbfff700 0 client.26552 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:43:56.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 -- 172.21.15.2:0/2288731460 shutdown_connections 2023-06-23T18:43:56.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 1 -- 172.21.15.2:0/2288731460 wait complete. 2023-06-23T18:43:56.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.401+0000 7f2feece0b80 20 client.26552 trim_cache size 0 max 16384 2023-06-23T18:43:56.415 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:43:56.415 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:43:56.415 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:43:56.415 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:43:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:43:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:43:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:43:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:43:56.416 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:43:56.416 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:43:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2fc40a12a0 msgr2=0x7f2fc40a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2fc40a12a0 0x7f2fc40a3750 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0006010 tx=0x7f2fd001a040 comp rx=0 tx=0).stop 2023-06-23T18:43:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/2709231903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fdc1006a0 msgr2=0x7f2fdc1017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:43:56.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fdc1006a0 0x7f2fdc1017f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4007f10 tx=0x7f2fd400d3b0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/2709231903 shutdown_connections 2023-06-23T18:43:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f2fc40a12a0 0x7f2fc40a3750 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fdc1006a0 0x7f2fdc1017f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fdc0fea60 0x7f2fdc104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 --2- 172.21.15.2:0/2709231903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2fdc0ffcf0 0x7f2fdc1051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:43:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/2709231903 >> 172.21.15.2:0/2709231903 conn(0x7f2fdc0fa080 msgr2=0x7f2fdc105700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:43:56.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/2709231903 shutdown_connections 2023-06-23T18:43:56.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:43:56.402+0000 7f2fe2b5b700 1 -- 172.21.15.2:0/2709231903 wait complete. 2023-06-23T18:43:56.425 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:43:56.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:56 smithi002 ceph-mon[95947]: pgmap v953: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:43:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:56 smithi017 ceph-mon[97472]: pgmap v953: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:43:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:56 smithi082 ceph-mon[97889]: pgmap v953: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:43:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 605.0 s, 337.8 tps, lat 296.679 ms stddev 54.557 2023-06-23T18:43:58.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:43:58 smithi017 ceph-mon[97472]: pgmap v954: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:43:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:43:58 smithi002 ceph-mon[95947]: pgmap v954: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:43:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:43:58 smithi082 ceph-mon[97889]: pgmap v954: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:44:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:00 smithi017 ceph-mon[97472]: pgmap v955: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:44:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:00 smithi002 ceph-mon[95947]: pgmap v955: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:44:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:00 smithi082 ceph-mon[97889]: pgmap v955: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:44:02.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 610.0 s, 336.4 tps, lat 255.412 ms stddev 89.463 2023-06-23T18:44:02.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:02 smithi017 ceph-mon[97472]: pgmap v956: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.4 MiB/s wr, 965 op/s 2023-06-23T18:44:02.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:02 smithi002 ceph-mon[95947]: pgmap v956: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.4 MiB/s wr, 965 op/s 2023-06-23T18:44:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:02 smithi082 ceph-mon[97889]: pgmap v956: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.4 MiB/s wr, 965 op/s 2023-06-23T18:44:04.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:04 smithi017 ceph-mon[97472]: pgmap v957: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.2 MiB/s wr, 960 op/s 2023-06-23T18:44:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:04 smithi002 ceph-mon[95947]: pgmap v957: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.2 MiB/s wr, 960 op/s 2023-06-23T18:44:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:04 smithi082 ceph-mon[97889]: pgmap v957: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.2 MiB/s wr, 960 op/s 2023-06-23T18:44:06.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:06 smithi017 ceph-mon[97472]: pgmap v958: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 976 op/s 2023-06-23T18:44:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:06 smithi002 ceph-mon[95947]: pgmap v958: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 976 op/s 2023-06-23T18:44:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:06 smithi082 ceph-mon[97889]: pgmap v958: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 976 op/s 2023-06-23T18:44:07.030 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 615.0 s, 275.4 tps, lat 50.910 ms stddev 13.247 2023-06-23T18:44:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:08 smithi002 ceph-mon[95947]: pgmap v959: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 938 op/s 2023-06-23T18:44:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:08 smithi082 ceph-mon[97889]: pgmap v959: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 938 op/s 2023-06-23T18:44:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:08 smithi017 ceph-mon[97472]: pgmap v959: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 938 op/s 2023-06-23T18:44:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:10 smithi002 ceph-mon[95947]: pgmap v960: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.0 MiB/s wr, 970 op/s 2023-06-23T18:44:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:10 smithi082 ceph-mon[97889]: pgmap v960: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.0 MiB/s wr, 970 op/s 2023-06-23T18:44:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:10 smithi017 ceph-mon[97472]: pgmap v960: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.0 MiB/s wr, 970 op/s 2023-06-23T18:44:12.030 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 620.0 s, 255.2 tps, lat 54.813 ms stddev 20.307 2023-06-23T18:44:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:12 smithi002 ceph-mon[95947]: pgmap v961: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 989 op/s 2023-06-23T18:44:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:12 smithi082 ceph-mon[97889]: pgmap v961: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 989 op/s 2023-06-23T18:44:12.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:12 smithi017 ceph-mon[97472]: pgmap v961: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 989 op/s 2023-06-23T18:44:14.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:14 smithi002 ceph-mon[95947]: pgmap v962: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 945 op/s 2023-06-23T18:44:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:14 smithi082 ceph-mon[97889]: pgmap v962: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 945 op/s 2023-06-23T18:44:14.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:14 smithi017 ceph-mon[97472]: pgmap v962: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 945 op/s 2023-06-23T18:44:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:16 smithi002 ceph-mon[95947]: pgmap v963: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.5 MiB/s wr, 889 op/s 2023-06-23T18:44:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:16 smithi082 ceph-mon[97889]: pgmap v963: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.5 MiB/s wr, 889 op/s 2023-06-23T18:44:16.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:16 smithi017 ceph-mon[97472]: pgmap v963: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.5 MiB/s wr, 889 op/s 2023-06-23T18:44:17.033 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 625.0 s, 253.6 tps, lat 55.032 ms stddev 16.412 2023-06-23T18:44:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:18 smithi002 ceph-mon[95947]: pgmap v964: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 829 op/s 2023-06-23T18:44:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:18 smithi082 ceph-mon[97889]: pgmap v964: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 829 op/s 2023-06-23T18:44:18.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:18 smithi017 ceph-mon[97472]: pgmap v964: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 829 op/s 2023-06-23T18:44:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:20 smithi002 ceph-mon[95947]: pgmap v965: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 925 op/s 2023-06-23T18:44:20.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:20 smithi082 ceph-mon[97889]: pgmap v965: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 925 op/s 2023-06-23T18:44:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:20 smithi017 ceph-mon[97472]: pgmap v965: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 925 op/s 2023-06-23T18:44:22.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 630.0 s, 283.0 tps, lat 1317.921 ms stddev 4550.431 2023-06-23T18:44:22.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:22 smithi002 ceph-mon[95947]: pgmap v966: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.1 MiB/s wr, 906 op/s 2023-06-23T18:44:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:22 smithi082 ceph-mon[97889]: pgmap v966: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.1 MiB/s wr, 906 op/s 2023-06-23T18:44:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:22 smithi017 ceph-mon[97472]: pgmap v966: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.1 MiB/s wr, 906 op/s 2023-06-23T18:44:24.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:24 smithi002 ceph-mon[95947]: pgmap v967: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 885 op/s 2023-06-23T18:44:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:24 smithi082 ceph-mon[97889]: pgmap v967: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 885 op/s 2023-06-23T18:44:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:24 smithi017 ceph-mon[97472]: pgmap v967: 97 pgs: 97 active+clean; 8.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.2 MiB/s wr, 885 op/s 2023-06-23T18:44:26.426 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:44:26.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/4024472387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00fea80 msgr2=0x7fc9f00fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/4024472387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00fea80 0x7fc9f00fee90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009990 tx=0x7fc9e0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/4024472387 shutdown_connections 2023-06-23T18:44:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/4024472387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f0100600 0x7fc9f0106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/4024472387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00fea80 0x7fc9f00fee90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/4024472387 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9f00ffc70 0x7fc9f01000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/4024472387 >> 172.21.15.2:0/4024472387 conn(0x7fc9f00fa0e0 msgr2=0x7fc9f00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/4024472387 shutdown_connections 2023-06-23T18:44:26.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.615+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/4024472387 wait complete. 2023-06-23T18:44:26.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9f4ae0700 1 Processor -- start 2023-06-23T18:44:26.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9f4ae0700 1 -- start start 2023-06-23T18:44:26.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9f4ae0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9f00fea80 0x7fc9f0193580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9f4ae0700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00ffc70 0x7fc9f0193ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9f4ae0700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f0100600 0x7fc9f019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9f4ae0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f019b100 con 0x7fc9f00fea80 2023-06-23T18:44:26.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9f4ae0700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f019b240 con 0x7fc9f0100600 2023-06-23T18:44:26.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9f4ae0700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f019b3d0 con 0x7fc9f00ffc70 2023-06-23T18:44:26.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9f00fea80 0x7fc9f0193580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9f00fea80 0x7fc9f0193580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58098/0 (socket says 172.21.15.2:58098) 2023-06-23T18:44:26.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9ee59c700 1 -- 172.21.15.2:0/1287001713 learned_addr learned my addr 172.21.15.2:0/1287001713 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:26.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.616+0000 7fc9eed9d700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f0100600 0x7fc9f019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00ffc70 0x7fc9f0193ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9ee59c700 1 -- 172.21.15.2:0/1287001713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00ffc70 msgr2=0x7fc9f0193ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00ffc70 0x7fc9f0193ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9ee59c700 1 -- 172.21.15.2:0/1287001713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f0100600 msgr2=0x7fc9f019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f0100600 0x7fc9f019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9ee59c700 1 -- 172.21.15.2:0/1287001713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e00095f0 con 0x7fc9f00fea80 2023-06-23T18:44:26.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00ffc70 0x7fc9f0193ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:44:26.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9eed9d700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f0100600 0x7fc9f019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:44:26.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9f00fea80 0x7fc9f0193580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000c00 tx=0x7fc9e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:26.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9df7fe700 1 -- 172.21.15.2:0/1287001713 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e001d070 con 0x7fc9f00fea80 2023-06-23T18:44:26.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9df7fe700 1 -- 172.21.15.2:0/1287001713 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9e000be70 con 0x7fc9f00fea80 2023-06-23T18:44:26.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/1287001713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f019b650 con 0x7fc9f00fea80 2023-06-23T18:44:26.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9df7fe700 1 -- 172.21.15.2:0/1287001713 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0022e40 con 0x7fc9f00fea80 2023-06-23T18:44:26.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.617+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/1287001713 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f019bad0 con 0x7fc9f00fea80 2023-06-23T18:44:26.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.620+0000 7fc9df7fe700 1 -- 172.21.15.2:0/1287001713 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc9e00225e0 con 0x7fc9f00fea80 2023-06-23T18:44:26.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.621+0000 7fc9df7fe700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9d80a1150 0x7fc9d80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.621+0000 7fc9df7fe700 1 -- 172.21.15.2:0/1287001713 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc9e00cde40 con 0x7fc9f00fea80 2023-06-23T18:44:26.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.621+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9d80a1150 0x7fc9d80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.622+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9d80a1150 0x7fc9d80a3600 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009cb0 tx=0x7fc9e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:26.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.622+0000 7fca00c65b80 1 Processor -- start 2023-06-23T18:44:26.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.622+0000 7fca00c65b80 1 -- start start 2023-06-23T18:44:26.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.622+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b3144ec80 0x560b3144f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.622+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3144fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.622+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b31450000 0x560b31450470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fca00c65b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31353cb0 con 0x560b3144ec80 2023-06-23T18:44:26.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fca00c65b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b313a6a40 con 0x560b3144f5d0 2023-06-23T18:44:26.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fca00c65b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3139fed0 con 0x560b31450000 2023-06-23T18:44:26.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b3144ec80 0x560b3144f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b3144ec80 0x560b3144f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58106/0 (socket says 172.21.15.2:58106) 2023-06-23T18:44:26.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 -- 172.21.15.2:0/3226751994 learned_addr learned my addr 172.21.15.2:0/3226751994 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:26.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9eed9d700 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b31450000 0x560b31450470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3144fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 -- 172.21.15.2:0/3226751994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b31450000 msgr2=0x560b31450470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:26.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b31450000 0x560b31450470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 -- 172.21.15.2:0/3226751994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 msgr2=0x560b3144fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:26.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3144fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 -- 172.21.15.2:0/3226751994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b31221630 con 0x560b3144ec80 2023-06-23T18:44:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9eed9d700 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b31450000 0x560b31450470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:44:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3144fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:44:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.623+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b3144ec80 0x560b3144f090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc9e009afe0 tx=0x7fc9e009b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a80e56f5ed6350 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fc9dcff9700 1 -- 172.21.15.2:0/3226751994 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e009ac40 con 0x560b3144ec80 2023-06-23T18:44:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fc9dcff9700 1 -- 172.21.15.2:0/3226751994 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9e009b2e0 con 0x560b3144ec80 2023-06-23T18:44:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fc9dcff9700 1 -- 172.21.15.2:0/3226751994 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e009a7d0 con 0x560b3144ec80 2023-06-23T18:44:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fca00c65b80 1 -- 172.21.15.2:0/3226751994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b3144ec80 msgr2=0x560b3144f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b3144ec80 0x560b3144f090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc9e009afe0 tx=0x7fc9e009b0b0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fca00c65b80 1 -- 172.21.15.2:0/3226751994 shutdown_connections 2023-06-23T18:44:26.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b31450000 0x560b31450470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b3144ec80 0x560b3144f090 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3226751994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3144fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.624+0000 7fca00c65b80 1 -- 172.21.15.2:0/3226751994 >> 172.21.15.2:0/3226751994 conn(0x560b314443f0 msgr2=0x560b31449520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:26.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 -- 172.21.15.2:0/3226751994 shutdown_connections 2023-06-23T18:44:26.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 -- 172.21.15.2:0/3226751994 wait complete. 2023-06-23T18:44:26.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 -- start start 2023-06-23T18:44:26.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 Processor -- start 2023-06-23T18:44:26.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b314541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b314551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3133fac0 con 0x560b31450000 2023-06-23T18:44:26.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3133f7c0 con 0x560b3144ec80 2023-06-23T18:44:26.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31223920 con 0x560b3144f5d0 2023-06-23T18:44:26.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.625+0000 7fca00c65b80 10 client.0 _ll_register_callbacks cb 0x560b31345260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:44:26.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9edd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b314541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9edd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b314541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58452/0 (socket says 172.21.15.2:58452) 2023-06-23T18:44:26.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9edd9b700 1 -- 172.21.15.2:0/3859347708 learned_addr learned my addr 172.21.15.2:0/3859347708 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:26.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9eed9d700 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b314551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9ee59c700 1 -- 172.21.15.2:0/3859347708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 msgr2=0x560b314541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:26.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b314541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9ee59c700 1 -- 172.21.15.2:0/3859347708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 msgr2=0x560b314551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:26.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b314551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9ee59c700 1 -- 172.21.15.2:0/3859347708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e800b100 con 0x560b3144ec80 2023-06-23T18:44:26.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9eed9d700 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b314551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:44:26.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00a3170 tx=0x7fc9e00a3440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c72aca567b38a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:26.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:44:26.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/3859347708 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e009f310 con 0x560b3144ec80 2023-06-23T18:44:26.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.626+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/3859347708 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc9e00a36a0 con 0x560b3144ec80 2023-06-23T18:44:26.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 1 -- 172.21.15.2:0/3859347708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b31441bb0 con 0x560b3144ec80 2023-06-23T18:44:26.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/3859347708 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e009f140 con 0x560b3144ec80 2023-06-23T18:44:26.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 10 client.26564 fetch_fsmap learned FSMap version 41 2023-06-23T18:44:26.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 1 -- 172.21.15.2:0/3859347708 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fc9f019bad0 con 0x560b3144ec80 2023-06-23T18:44:26.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/3859347708 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560b31441bb0 con 0x560b3144ec80 2023-06-23T18:44:26.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/3859347708 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc9e00970d0 con 0x560b3144ec80 2023-06-23T18:44:26.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 10 client.26564 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:44:26.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 10 client.26564 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:44:26.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 20 client.26564 populate_metadata read hostname 'smithi002' 2023-06-23T18:44:26.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560b31463b20 0x560b31465f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 4 client.26564 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:44:26.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.627+0000 7fca00c65b80 1 -- 172.21.15.2:0/3859347708 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b31398eb0 con 0x560b31463b20 2023-06-23T18:44:26.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.628+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560b31463b20 0x560b31465f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.628+0000 7fc9d5ffb700 10 client.26564 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:44:26.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.628+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560b31463b20 0x560b31465f00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e478073576ed16 server_cookie=851bc6c4b0c05661 in_seq=0 out_seq=0 2023-06-23T18:44:26.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.629+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/3859347708 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560b31398eb0 con 0x560b31463b20 2023-06-23T18:44:26.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fc9d5ffb700 10 client.26564 handle_command_reply: tid=0 2023-06-23T18:44:26.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 client.26564 shutdown 2023-06-23T18:44:26.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 -- 172.21.15.2:0/3859347708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 msgr2=0x560b31453c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00a3170 tx=0x7fc9e00a3440 comp rx=0 tx=0).stop 2023-06-23T18:44:26.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 -- 172.21.15.2:0/3859347708 shutdown_connections 2023-06-23T18:44:26.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560b31463b20 0x560b31465f00 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b314551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fc9d5ffb700 0 client.26564 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:44:26.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453c80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 --2- 172.21.15.2:0/3859347708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b314541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 -- 172.21.15.2:0/3859347708 >> 172.21.15.2:0/3859347708 conn(0x560b314443f0 msgr2=0x560b31448c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:26.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 -- 172.21.15.2:0/3859347708 shutdown_connections 2023-06-23T18:44:26.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 1 -- 172.21.15.2:0/3859347708 wait complete. 2023-06-23T18:44:26.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.630+0000 7fca00c65b80 20 client.26564 trim_cache size 0 max 16384 2023-06-23T18:44:26.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fca00c65b80 1 Processor -- start 2023-06-23T18:44:26.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fca00c65b80 1 -- start start 2023-06-23T18:44:26.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b31453f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b31450c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fca00c65b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31454e70 con 0x560b31450000 2023-06-23T18:44:26.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fca00c65b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31454cc0 con 0x560b3144ec80 2023-06-23T18:44:26.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fca00c65b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b31453130 con 0x560b3144f5d0 2023-06-23T18:44:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fc9edd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b31453f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fc9edd9b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b31453f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58458/0 (socket says 172.21.15.2:58458) 2023-06-23T18:44:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.657+0000 7fc9edd9b700 1 -- 172.21.15.2:0/737836152 learned_addr learned my addr 172.21.15.2:0/737836152 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9eed9d700 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b31450c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9edd9b700 1 -- 172.21.15.2:0/737836152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 msgr2=0x560b31453a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9edd9b700 1 -- 172.21.15.2:0/737836152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 msgr2=0x560b31450c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b31450c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9edd9b700 1 -- 172.21.15.2:0/737836152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b311cdde0 con 0x560b3144f5d0 2023-06-23T18:44:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b31453f70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4011a50 tx=0x7fc9e4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1185f4d29f761e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:26.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9dd7fa700 1 -- 172.21.15.2:0/737836152 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4027070 con 0x560b3144f5d0 2023-06-23T18:44:26.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9dd7fa700 1 -- 172.21.15.2:0/737836152 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9e4003960 con 0x560b3144f5d0 2023-06-23T18:44:26.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.658+0000 7fc9dd7fa700 1 -- 172.21.15.2:0/737836152 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4049cd0 con 0x560b3144f5d0 2023-06-23T18:44:26.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.659+0000 7fca00c65b80 1 -- 172.21.15.2:0/737836152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 msgr2=0x560b31453f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.659+0000 7fca00c65b80 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b31453f70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4011a50 tx=0x7fc9e4014000 comp rx=0 tx=0).stop 2023-06-23T18:44:26.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.659+0000 7fca00c65b80 1 -- 172.21.15.2:0/737836152 shutdown_connections 2023-06-23T18:44:26.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.659+0000 7fca00c65b80 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b31450c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.659+0000 7fca00c65b80 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144ec80 0x560b31453a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.659+0000 7fca00c65b80 1 --2- 172.21.15.2:0/737836152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144f5d0 0x560b31453f70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.659+0000 7fca00c65b80 1 -- 172.21.15.2:0/737836152 >> 172.21.15.2:0/737836152 conn(0x560b314443f0 msgr2=0x560b314590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.660+0000 7fca00c65b80 1 -- 172.21.15.2:0/737836152 shutdown_connections 2023-06-23T18:44:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.660+0000 7fca00c65b80 1 -- 172.21.15.2:0/737836152 wait complete. 2023-06-23T18:44:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.660+0000 7fca00c65b80 1 -- start start 2023-06-23T18:44:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.660+0000 7fca00c65b80 1 Processor -- start 2023-06-23T18:44:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144ec80 0x560b314609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3145d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fca00c65b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b3145d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fca00c65b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3145def0 con 0x560b31450000 2023-06-23T18:44:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fca00c65b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b3145e060 con 0x560b3144f5d0 2023-06-23T18:44:26.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fca00c65b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b314fb900 con 0x560b3144ec80 2023-06-23T18:44:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fca00c65b80 10 client.0 _ll_register_callbacks cb 0x560b31345260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:44:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144ec80 0x560b314609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9ee59c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144ec80 0x560b314609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58464/0 (socket says 172.21.15.2:58464) 2023-06-23T18:44:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9ee59c700 1 -- 172.21.15.2:0/2201221598 learned_addr learned my addr 172.21.15.2:0/2201221598 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3145d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9eed9d700 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b3145d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9ee59c700 1 -- 172.21.15.2:0/2201221598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 msgr2=0x560b3145d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3145d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9ee59c700 1 -- 172.21.15.2:0/2201221598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 msgr2=0x560b3145d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b3145d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.661+0000 7fc9ee59c700 1 -- 172.21.15.2:0/2201221598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e4011c60 con 0x560b3144ec80 2023-06-23T18:44:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fc9ee59c700 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144ec80 0x560b314609d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0022930 tx=0x7fc9e00a12f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=254952164c33a5e7 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fc9d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:44:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/2201221598 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e009f140 con 0x560b3144ec80 2023-06-23T18:44:26.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fca00c65b80 1 -- 172.21.15.2:0/2201221598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9e009b2e0 con 0x560b3144ec80 2023-06-23T18:44:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/2201221598 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc9e0084d40 con 0x560b3144ec80 2023-06-23T18:44:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/2201221598 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e009f310 con 0x560b3144ec80 2023-06-23T18:44:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fca00c65b80 10 client.26791 fetch_fsmap learned FSMap version 41 2023-06-23T18:44:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fca00c65b80 1 -- 172.21.15.2:0/2201221598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x560b311b6c60 con 0x560b3144ec80 2023-06-23T18:44:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.662+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/2201221598 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9e009b2e0 con 0x560b3144ec80 2023-06-23T18:44:26.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/2201221598 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc9e00b0d90 con 0x560b3144ec80 2023-06-23T18:44:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fca00c65b80 10 client.26791 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:44:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fca00c65b80 10 client.26791 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:44:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fca00c65b80 20 client.26791 populate_metadata read hostname 'smithi002' 2023-06-23T18:44:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fca00c65b80 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560b314fcd20 0x560b314ff130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fca00c65b80 4 client.26791 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:44:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fca00c65b80 1 -- 172.21.15.2:0/2201221598 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b3135b050 con 0x560b314fcd20 2023-06-23T18:44:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560b314fcd20 0x560b314ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:26.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fc9d5ffb700 10 client.26791 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:44:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.663+0000 7fc9edd9b700 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560b314fcd20 0x560b314ff130 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ecc5c941a9bb69 server_cookie=7ba5b6108a12d958 in_seq=0 out_seq=0 2023-06-23T18:44:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fc9d5ffb700 1 -- 172.21.15.2:0/2201221598 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560b3135b050 con 0x560b314fcd20 2023-06-23T18:44:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fc9d5ffb700 10 client.26791 handle_command_reply: tid=0 2023-06-23T18:44:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 client.26791 shutdown 2023-06-23T18:44:26.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 -- 172.21.15.2:0/2201221598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144ec80 msgr2=0x560b314609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144ec80 0x560b314609d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0022930 tx=0x7fc9e00a12f0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 -- 172.21.15.2:0/2201221598 shutdown_connections 2023-06-23T18:44:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x560b314fcd20 0x560b314ff130 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x560b31450000 0x560b3145d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560b3144ec80 0x560b314609d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 --2- 172.21.15.2:0/2201221598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560b3144f5d0 0x560b3145d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fca00c65b80 1 -- 172.21.15.2:0/2201221598 >> 172.21.15.2:0/2201221598 conn(0x560b314443f0 msgr2=0x560b31453a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:26.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.665+0000 7fc9d5ffb700 0 client.26791 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:44:26.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.666+0000 7fca00c65b80 1 -- 172.21.15.2:0/2201221598 shutdown_connections 2023-06-23T18:44:26.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.666+0000 7fca00c65b80 1 -- 172.21.15.2:0/2201221598 wait complete. 2023-06-23T18:44:26.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.666+0000 7fca00c65b80 20 client.26791 trim_cache size 0 max 16384 2023-06-23T18:44:26.680 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:44:26.680 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:44:26.681 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:44:26.681 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:44:26.681 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:44:26.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:44:26.681 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:44:26.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:44:26.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:44:26.682 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:44:26.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9d80a1150 msgr2=0x7fc9d80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9d80a1150 0x7fc9d80a3600 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009cb0 tx=0x7fc9e4009260 comp rx=0 tx=0).stop 2023-06-23T18:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9f00fea80 msgr2=0x7fc9f0193580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9f00fea80 0x7fc9f0193580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000c00 tx=0x7fc9e0004730 comp rx=0 tx=0).stop 2023-06-23T18:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/1287001713 shutdown_connections 2023-06-23T18:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fc9d80a1150 0x7fc9d80a3600 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9f0100600 0x7fc9f019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc9f00fea80 0x7fc9f0193580 secure :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0000c00 tx=0x7fc9e0004730 comp rx=0 tx=0).stop 2023-06-23T18:44:26.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 --2- 172.21.15.2:0/1287001713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9f00ffc70 0x7fc9f0193ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/1287001713 >> 172.21.15.2:0/1287001713 conn(0x7fc9f00fa0e0 msgr2=0x7fc9f00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/1287001713 shutdown_connections 2023-06-23T18:44:26.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:26.667+0000 7fc9f4ae0700 1 -- 172.21.15.2:0/1287001713 wait complete. 2023-06-23T18:44:26.690 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:44:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:26 smithi002 ceph-mon[95947]: pgmap v968: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.7 MiB/s wr, 894 op/s 2023-06-23T18:44:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:26 smithi082 ceph-mon[97889]: pgmap v968: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.7 MiB/s wr, 894 op/s 2023-06-23T18:44:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:26 smithi017 ceph-mon[97472]: pgmap v968: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.7 MiB/s wr, 894 op/s 2023-06-23T18:44:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 635.0 s, 296.4 tps, lat 336.270 ms stddev 66.032 2023-06-23T18:44:28.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:28 smithi002 ceph-mon[95947]: pgmap v969: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 912 op/s 2023-06-23T18:44:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:28 smithi082 ceph-mon[97889]: pgmap v969: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 912 op/s 2023-06-23T18:44:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:28 smithi017 ceph-mon[97472]: pgmap v969: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 912 op/s 2023-06-23T18:44:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:30 smithi002 ceph-mon[95947]: pgmap v970: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:44:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:30 smithi082 ceph-mon[97889]: pgmap v970: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:44:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:30 smithi017 ceph-mon[97472]: pgmap v970: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 1.02k op/s 2023-06-23T18:44:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 640.0 s, 297.0 tps, lat 337.529 ms stddev 71.134 2023-06-23T18:44:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:32 smithi002 ceph-mon[95947]: pgmap v971: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 985 op/s 2023-06-23T18:44:32.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:32 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:44:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:32 smithi082 ceph-mon[97889]: pgmap v971: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 985 op/s 2023-06-23T18:44:32.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:32 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:44:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:32 smithi017 ceph-mon[97472]: pgmap v971: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 985 op/s 2023-06-23T18:44:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:32 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:44:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:34 smithi002 ceph-mon[95947]: pgmap v972: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:44:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:44:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:44:34.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:44:34.856 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:34 smithi082 ceph-mon[97889]: pgmap v972: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:44:34.856 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:44:34.857 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:44:34.857 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:44:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:34 smithi017 ceph-mon[97472]: pgmap v972: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:44:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:44:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:44:34.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:44:36.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:36 smithi002 ceph-mon[95947]: pgmap v973: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 958 op/s 2023-06-23T18:44:36.860 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:36 smithi082 ceph-mon[97889]: pgmap v973: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 958 op/s 2023-06-23T18:44:36.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:36 smithi017 ceph-mon[97472]: pgmap v973: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.6 MiB/s wr, 958 op/s 2023-06-23T18:44:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 645.0 s, 324.2 tps, lat 308.283 ms stddev 67.636 2023-06-23T18:44:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:38 smithi002 ceph-mon[95947]: pgmap v974: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.5 MiB/s wr, 928 op/s 2023-06-23T18:44:38.879 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:38 smithi082 ceph-mon[97889]: pgmap v974: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.5 MiB/s wr, 928 op/s 2023-06-23T18:44:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:38 smithi017 ceph-mon[97472]: pgmap v974: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.5 MiB/s wr, 928 op/s 2023-06-23T18:44:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:40 smithi002 ceph-mon[95947]: pgmap v975: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:44:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:40 smithi082 ceph-mon[97889]: pgmap v975: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:44:40.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:40 smithi017 ceph-mon[97472]: pgmap v975: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:44:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 650.0 s, 327.4 tps, lat 307.435 ms stddev 65.273 2023-06-23T18:44:42.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:42 smithi002 ceph-mon[95947]: pgmap v976: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 898 op/s 2023-06-23T18:44:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:42 smithi082 ceph-mon[97889]: pgmap v976: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 898 op/s 2023-06-23T18:44:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:42 smithi017 ceph-mon[97472]: pgmap v976: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 898 op/s 2023-06-23T18:44:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:44 smithi002 ceph-mon[95947]: pgmap v977: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 877 op/s 2023-06-23T18:44:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:44 smithi082 ceph-mon[97889]: pgmap v977: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 877 op/s 2023-06-23T18:44:44.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:44 smithi017 ceph-mon[97472]: pgmap v977: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 877 op/s 2023-06-23T18:44:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:46 smithi002 ceph-mon[95947]: pgmap v978: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 836 op/s 2023-06-23T18:44:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:46 smithi082 ceph-mon[97889]: pgmap v978: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 836 op/s 2023-06-23T18:44:46.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:46 smithi017 ceph-mon[97472]: pgmap v978: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.4 MiB/s wr, 836 op/s 2023-06-23T18:44:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 655.0 s, 326.2 tps, lat 303.394 ms stddev 50.114 2023-06-23T18:44:48.809 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:48 smithi002 ceph-mon[95947]: pgmap v979: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 845 op/s 2023-06-23T18:44:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:48 smithi082 ceph-mon[97889]: pgmap v979: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 845 op/s 2023-06-23T18:44:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:48 smithi017 ceph-mon[97472]: pgmap v979: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 845 op/s 2023-06-23T18:44:50.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:50 smithi002 ceph-mon[95947]: pgmap v980: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 944 op/s 2023-06-23T18:44:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:50 smithi082 ceph-mon[97889]: pgmap v980: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 944 op/s 2023-06-23T18:44:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:50 smithi017 ceph-mon[97472]: pgmap v980: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 944 op/s 2023-06-23T18:44:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 660.0 s, 328.2 tps, lat 306.183 ms stddev 44.551 2023-06-23T18:44:52.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:52 smithi002 ceph-mon[95947]: pgmap v981: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 928 op/s 2023-06-23T18:44:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:52 smithi082 ceph-mon[97889]: pgmap v981: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 928 op/s 2023-06-23T18:44:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:52 smithi017 ceph-mon[97472]: pgmap v981: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 928 op/s 2023-06-23T18:44:54.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:54 smithi002 ceph-mon[95947]: pgmap v982: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 966 op/s 2023-06-23T18:44:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:54 smithi082 ceph-mon[97889]: pgmap v982: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 966 op/s 2023-06-23T18:44:54.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:54 smithi017 ceph-mon[97472]: pgmap v982: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 966 op/s 2023-06-23T18:44:56.691 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:44:56.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:56 smithi002 ceph-mon[95947]: pgmap v983: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 955 op/s 2023-06-23T18:44:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:56 smithi082 ceph-mon[97889]: pgmap v983: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 955 op/s 2023-06-23T18:44:56.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 -- 172.21.15.2:0/1052876073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b20108090 msgr2=0x7f4b20071580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1052876073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b20108090 0x7f4b20071580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b1800b0a0 tx=0x7f4b1800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 -- 172.21.15.2:0/1052876073 shutdown_connections 2023-06-23T18:44:56.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1052876073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b20071bd0 0x7f4b2010c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1052876073 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20107740 0x7f4b20107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1052876073 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b20108090 0x7f4b20071580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 -- 172.21.15.2:0/1052876073 >> 172.21.15.2:0/1052876073 conn(0x7f4b2006d3b0 msgr2=0x7f4b2006f800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:56.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 -- 172.21.15.2:0/1052876073 shutdown_connections 2023-06-23T18:44:56.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.877+0000 7f4b26d89700 1 -- 172.21.15.2:0/1052876073 wait complete. 2023-06-23T18:44:56.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.878+0000 7f4b26d89700 1 Processor -- start 2023-06-23T18:44:56.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.878+0000 7f4b26d89700 1 -- start start 2023-06-23T18:44:56.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.878+0000 7f4b26d89700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b20071bd0 0x7f4b201a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.878+0000 7f4b26d89700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b20107740 0x7f4b201a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.878+0000 7f4b26d89700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20108090 0x7f4b201abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.878+0000 7f4b26d89700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201ac330 con 0x7f4b20108090 2023-06-23T18:44:56.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.878+0000 7f4b26d89700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201ac470 con 0x7f4b20107740 2023-06-23T18:44:56.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.878+0000 7f4b26d89700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b201ac600 con 0x7f4b20071bd0 2023-06-23T18:44:56.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20108090 0x7f4b201abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20108090 0x7f4b201abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58154/0 (socket says 172.21.15.2:58154) 2023-06-23T18:44:56.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 -- 172.21.15.2:0/1749239473 learned_addr learned my addr 172.21.15.2:0/1749239473 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:56.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b24b25700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b20071bd0 0x7f4b201a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b1ffff700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b20107740 0x7f4b201a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 -- 172.21.15.2:0/1749239473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b20071bd0 msgr2=0x7f4b201a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b20071bd0 0x7f4b201a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 -- 172.21.15.2:0/1749239473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b20107740 msgr2=0x7f4b201a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b20107740 0x7f4b201a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 -- 172.21.15.2:0/1749239473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b18009d00 con 0x7f4b20108090 2023-06-23T18:44:56.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b25326700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20108090 0x7f4b201abd60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000e940 tx=0x7f4b1000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:56.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b1dffb700 1 -- 172.21.15.2:0/1749239473 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b100097b0 con 0x7f4b20108090 2023-06-23T18:44:56.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b26d89700 1 -- 172.21.15.2:0/1749239473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b201ac8e0 con 0x7f4b20108090 2023-06-23T18:44:56.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b1dffb700 1 -- 172.21.15.2:0/1749239473 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b10005020 con 0x7f4b20108090 2023-06-23T18:44:56.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.879+0000 7f4b1dffb700 1 -- 172.21.15.2:0/1749239473 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000cb70 con 0x7f4b20108090 2023-06-23T18:44:56.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.880+0000 7f4b26d89700 1 -- 172.21.15.2:0/1749239473 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b201acdf0 con 0x7f4b20108090 2023-06-23T18:44:56.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.882+0000 7f4b1dffb700 1 -- 172.21.15.2:0/1749239473 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4b10004a30 con 0x7f4b20108090 2023-06-23T18:44:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.884+0000 7f4b1dffb700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b080a1150 0x7f4b080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.884+0000 7f4b24b25700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b080a1150 0x7f4b080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.884+0000 7f4b1dffb700 1 -- 172.21.15.2:0/1749239473 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b10014070 con 0x7f4b20108090 2023-06-23T18:44:56.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.884+0000 7f4b24b25700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b080a1150 0x7f4b080a3600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4b14005950 tx=0x7f4b1400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:56.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b32f0eb80 1 Processor -- start 2023-06-23T18:44:56.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b32f0eb80 1 -- start start 2023-06-23T18:44:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc520f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc520fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 0x558dc5210470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc5113cb0 con 0x558dc5210000 2023-06-23T18:44:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc5166a40 con 0x558dc520ec80 2023-06-23T18:44:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc515fed0 con 0x558dc520f5d0 2023-06-23T18:44:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b24b25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc520f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b1ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc520fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b25326700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 0x558dc5210470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b24b25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc520f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48750/0 (socket says 172.21.15.2:48750) 2023-06-23T18:44:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b24b25700 1 -- 172.21.15.2:0/4245455259 learned_addr learned my addr 172.21.15.2:0/4245455259 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.885+0000 7f4b1ffff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc520fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58498/0 (socket says 172.21.15.2:58498) 2023-06-23T18:44:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b24b25700 1 -- 172.21.15.2:0/4245455259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 msgr2=0x558dc520fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b24b25700 1 --2- 172.21.15.2:0/4245455259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc520fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b24b25700 1 -- 172.21.15.2:0/4245455259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 msgr2=0x558dc5210470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b24b25700 1 --2- 172.21.15.2:0/4245455259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 0x558dc5210470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b24b25700 1 -- 172.21.15.2:0/4245455259 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc4fe1630 con 0x558dc520ec80 2023-06-23T18:44:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b1ffff700 1 --2- 172.21.15.2:0/4245455259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc520fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:44:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b24b25700 1 --2- 172.21.15.2:0/4245455259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc520f090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4b14010e60 tx=0x7f4b14013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac121faf037903d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b06ffd700 1 -- 172.21.15.2:0/4245455259 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b14011db0 con 0x558dc520ec80 2023-06-23T18:44:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b06ffd700 1 -- 172.21.15.2:0/4245455259 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b14015070 con 0x558dc520ec80 2023-06-23T18:44:56.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.886+0000 7f4b06ffd700 1 -- 172.21.15.2:0/4245455259 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1401c610 con 0x558dc520ec80 2023-06-23T18:44:56.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/4245455259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 msgr2=0x558dc520f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/4245455259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc520f090 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4b14010e60 tx=0x7f4b14013000 comp rx=0 tx=0).stop 2023-06-23T18:44:56.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/4245455259 shutdown_connections 2023-06-23T18:44:56.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/4245455259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 0x558dc5210470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/4245455259 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc520f090 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/4245455259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc520fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/4245455259 >> 172.21.15.2:0/4245455259 conn(0x558dc52043f0 msgr2=0x558dc5209520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:56.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/4245455259 shutdown_connections 2023-06-23T18:44:56.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.887+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/4245455259 wait complete. 2023-06-23T18:44:56.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 1 -- start start 2023-06-23T18:44:56.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 1 Processor -- start 2023-06-23T18:44:56.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc5218af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc5210000 0x558dc521fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc50ffac0 con 0x558dc520f5d0 2023-06-23T18:44:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc50ff7c0 con 0x558dc520ec80 2023-06-23T18:44:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc4fe3920 con 0x558dc5210000 2023-06-23T18:44:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.888+0000 7f4b32f0eb80 10 client.0 _ll_register_callbacks cb 0x558dc5105260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:44:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b1ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc5218af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b24b25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b24b25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48756/0 (socket says 172.21.15.2:48756) 2023-06-23T18:44:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b1ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc5218af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58166/0 (socket says 172.21.15.2:58166) 2023-06-23T18:44:56.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b24b25700 1 -- 172.21.15.2:0/3786714655 learned_addr learned my addr 172.21.15.2:0/3786714655 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:56.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b25326700 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc5210000 0x558dc521fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b25326700 1 -- 172.21.15.2:0/3786714655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 msgr2=0x558dc52185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:56.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b25326700 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b25326700 1 -- 172.21.15.2:0/3786714655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 msgr2=0x558dc5218af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:56.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b25326700 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc5218af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b25326700 1 -- 172.21.15.2:0/3786714655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2010cff0 con 0x558dc5210000 2023-06-23T18:44:56.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b25326700 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc5210000 0x558dc521fb30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b10099c50 tx=0x7f4b1000b2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29f08a97b674449c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:56.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4afffff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:44:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4afffff700 1 -- 172.21.15.2:0/3786714655 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10099f50 con 0x558dc5210000 2023-06-23T18:44:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4afffff700 1 -- 172.21.15.2:0/3786714655 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b1009df80 con 0x558dc5210000 2023-06-23T18:44:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.889+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/3786714655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558dc5201bb0 con 0x558dc5210000 2023-06-23T18:44:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4afffff700 1 -- 172.21.15.2:0/3786714655 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b100a23a0 con 0x558dc5210000 2023-06-23T18:44:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b32f0eb80 10 client.26803 fetch_fsmap learned FSMap version 41 2023-06-23T18:44:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/3786714655 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f4b14010b10 con 0x558dc5210000 2023-06-23T18:44:56.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4afffff700 1 -- 172.21.15.2:0/3786714655 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558dc5201bb0 con 0x558dc5210000 2023-06-23T18:44:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4afffff700 1 -- 172.21.15.2:0/3786714655 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b10099440 con 0x558dc5210000 2023-06-23T18:44:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b32f0eb80 10 client.26803 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:44:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b32f0eb80 10 client.26803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:44:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b32f0eb80 20 client.26803 populate_metadata read hostname 'smithi002' 2023-06-23T18:44:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558dc5224360 0x558dc5226770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b32f0eb80 4 client.26803 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:44:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/3786714655 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558dc5158eb0 con 0x558dc5224360 2023-06-23T18:44:56.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.890+0000 7f4b24b25700 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558dc5224360 0x558dc5226770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.891+0000 7f4afffff700 10 client.26803 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:44:56.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.891+0000 7f4b24b25700 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558dc5224360 0x558dc5226770 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae128a54ce283b51 server_cookie=8aced087f480393d in_seq=0 out_seq=0 2023-06-23T18:44:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4afffff700 1 -- 172.21.15.2:0/3786714655 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558dc5158eb0 con 0x558dc5224360 2023-06-23T18:44:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4afffff700 10 client.26803 handle_command_reply: tid=0 2023-06-23T18:44:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 client.26803 shutdown 2023-06-23T18:44:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/3786714655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc5210000 msgr2=0x558dc521fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc5210000 0x558dc521fb30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4b10099c50 tx=0x7f4b1000b2c0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/3786714655 shutdown_connections 2023-06-23T18:44:56.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558dc5224360 0x558dc5226770 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc5210000 0x558dc521fb30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/3786714655 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc5218af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/3786714655 >> 172.21.15.2:0/3786714655 conn(0x558dc52043f0 msgr2=0x558dc5208c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4afffff700 0 client.26803 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:44:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/3786714655 shutdown_connections 2023-06-23T18:44:56.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.892+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/3786714655 wait complete. 2023-06-23T18:44:56.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.893+0000 7f4b32f0eb80 20 client.26803 trim_cache size 0 max 16384 2023-06-23T18:44:56.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.920+0000 7f4b32f0eb80 1 Processor -- start 2023-06-23T18:44:56.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.920+0000 7f4b32f0eb80 1 -- start start 2023-06-23T18:44:56.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.920+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc521faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc5210000 0x558dc5218af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc520b010 con 0x558dc520f5d0 2023-06-23T18:44:56.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc5158eb0 con 0x558dc5210000 2023-06-23T18:44:56.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc52194b0 con 0x558dc520ec80 2023-06-23T18:44:56.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b24b25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b1ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc521faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b1ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc521faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58174/0 (socket says 172.21.15.2:58174) 2023-06-23T18:44:56.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b24b25700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58510/0 (socket says 172.21.15.2:58510) 2023-06-23T18:44:56.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b1ffff700 1 -- 172.21.15.2:0/1896560507 learned_addr learned my addr 172.21.15.2:0/1896560507 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:56.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b25326700 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc5210000 0x558dc5218af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b25326700 1 -- 172.21.15.2:0/1896560507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520ec80 msgr2=0x558dc52185b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:56.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b25326700 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b25326700 1 -- 172.21.15.2:0/1896560507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 msgr2=0x558dc521faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:56.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b25326700 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc521faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b25326700 1 -- 172.21.15.2:0/1896560507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558dc5217530 con 0x558dc5210000 2023-06-23T18:44:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b1ffff700 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc521faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:44:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.921+0000 7f4b25326700 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc5210000 0x558dc5218af0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b100a1e70 tx=0x7f4b10005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90b4016670c23b1a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.922+0000 7f4b077fe700 1 -- 172.21.15.2:0/1896560507 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1009df80 con 0x558dc5210000 2023-06-23T18:44:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.922+0000 7f4b077fe700 1 -- 172.21.15.2:0/1896560507 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b100a1f10 con 0x558dc5210000 2023-06-23T18:44:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.922+0000 7f4b077fe700 1 -- 172.21.15.2:0/1896560507 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10099440 con 0x558dc5210000 2023-06-23T18:44:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/1896560507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc5210000 msgr2=0x558dc5218af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc5210000 0x558dc5218af0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4b100a1e70 tx=0x7f4b10005700 comp rx=0 tx=0).stop 2023-06-23T18:44:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/1896560507 shutdown_connections 2023-06-23T18:44:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc5210000 0x558dc5218af0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520ec80 0x558dc52185b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/1896560507 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc520f5d0 0x558dc521faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/1896560507 >> 172.21.15.2:0/1896560507 conn(0x558dc52043f0 msgr2=0x558dc5220300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/1896560507 shutdown_connections 2023-06-23T18:44:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/1896560507 wait complete. 2023-06-23T18:44:56.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 -- start start 2023-06-23T18:44:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.923+0000 7f4b32f0eb80 1 Processor -- start 2023-06-23T18:44:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc521f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc521f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b32f0eb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 0x558dc521be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc521c470 con 0x558dc5210000 2023-06-23T18:44:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc521c5e0 con 0x558dc520ec80 2023-06-23T18:44:56.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b32f0eb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558dc521c7a0 con 0x558dc520f5d0 2023-06-23T18:44:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b32f0eb80 10 client.0 _ll_register_callbacks cb 0x558dc5105260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:44:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b24b25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc521f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b25326700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 0x558dc521be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b24b25700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc521f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48770/0 (socket says 172.21.15.2:48770) 2023-06-23T18:44:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b24b25700 1 -- 172.21.15.2:0/806672305 learned_addr learned my addr 172.21.15.2:0/806672305 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:44:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b1ffff700 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc521f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b24b25700 1 -- 172.21.15.2:0/806672305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 msgr2=0x558dc521f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:56.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b24b25700 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc521f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b24b25700 1 -- 172.21.15.2:0/806672305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 msgr2=0x558dc521be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:44:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b24b25700 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 0x558dc521be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b24b25700 1 -- 172.21.15.2:0/806672305 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b18007b90 con 0x558dc520ec80 2023-06-23T18:44:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.924+0000 7f4b1ffff700 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc521f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:44:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4b24b25700 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc521f3b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x558dc5227080 tx=0x7f4b14010e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ed082b5fb5bc1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:44:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4afffff700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:44:56.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4afffff700 1 -- 172.21.15.2:0/806672305 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1402c040 con 0x558dc520ec80 2023-06-23T18:44:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4afffff700 1 -- 172.21.15.2:0/806672305 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4b1401ee40 con 0x558dc520ec80 2023-06-23T18:44:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/806672305 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b14015070 con 0x558dc520ec80 2023-06-23T18:44:56.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4afffff700 1 -- 172.21.15.2:0/806672305 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b140057a0 con 0x558dc520ec80 2023-06-23T18:44:56.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4b32f0eb80 10 client.26588 fetch_fsmap learned FSMap version 41 2023-06-23T18:44:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/806672305 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f4b14010b10 con 0x558dc520ec80 2023-06-23T18:44:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.925+0000 7f4afffff700 1 -- 172.21.15.2:0/806672305 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b14015070 con 0x558dc520ec80 2023-06-23T18:44:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4afffff700 1 -- 172.21.15.2:0/806672305 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4b1403b020 con 0x558dc520ec80 2023-06-23T18:44:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4b32f0eb80 10 client.26588 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:44:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4b32f0eb80 10 client.26588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:44:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4b32f0eb80 20 client.26588 populate_metadata read hostname 'smithi002' 2023-06-23T18:44:56.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558dc52796c0 0x558dc527bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:44:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4b32f0eb80 4 client.26588 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:44:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/806672305 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558dc511b050 con 0x558dc52796c0 2023-06-23T18:44:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4b1ffff700 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558dc52796c0 0x558dc527bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:44:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4afffff700 10 client.26588 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:44:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.926+0000 7f4b1ffff700 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558dc52796c0 0x558dc527bad0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=317d5362efe679ae server_cookie=ea074bc0e1e36afb in_seq=0 out_seq=0 2023-06-23T18:44:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.927+0000 7f4afffff700 1 -- 172.21.15.2:0/806672305 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558dc511b050 con 0x558dc52796c0 2023-06-23T18:44:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.927+0000 7f4afffff700 10 client.26588 handle_command_reply: tid=0 2023-06-23T18:44:56.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.927+0000 7f4b32f0eb80 1 client.26588 shutdown 2023-06-23T18:44:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.927+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/806672305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 msgr2=0x558dc521f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.927+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc521f3b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x558dc5227080 tx=0x7f4b14010e50 comp rx=0 tx=0).stop 2023-06-23T18:44:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.927+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/806672305 shutdown_connections 2023-06-23T18:44:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.927+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x558dc52796c0 0x558dc527bad0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.927+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x558dc5210000 0x558dc521be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.928+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x558dc520ec80 0x558dc521f3b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.928+0000 7f4b32f0eb80 1 --2- 172.21.15.2:0/806672305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558dc520f5d0 0x558dc521f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.928+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/806672305 >> 172.21.15.2:0/806672305 conn(0x558dc52043f0 msgr2=0x558dc5218550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:56.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.928+0000 7f4afffff700 0 client.26588 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:44:56.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.928+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/806672305 shutdown_connections 2023-06-23T18:44:56.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.928+0000 7f4b32f0eb80 1 -- 172.21.15.2:0/806672305 wait complete. 2023-06-23T18:44:56.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.928+0000 7f4b32f0eb80 20 client.26588 trim_cache size 0 max 16384 2023-06-23T18:44:56.940 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:44:56.941 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:44:56.941 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:44:56.941 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:44:56.941 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:44:56.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:44:56.941 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:44:56.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:44:56.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:44:56.942 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:44:56.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:56 smithi017 ceph-mon[97472]: pgmap v983: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 955 op/s 2023-06-23T18:44:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 -- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b080a1150 msgr2=0x7f4b080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b080a1150 0x7f4b080a3600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4b14005950 tx=0x7f4b1400b410 comp rx=0 tx=0).stop 2023-06-23T18:44:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 -- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20108090 msgr2=0x7f4b201abd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:44:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20108090 0x7f4b201abd60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000e940 tx=0x7f4b1000ec50 comp rx=0 tx=0).stop 2023-06-23T18:44:56.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 -- 172.21.15.2:0/1749239473 shutdown_connections 2023-06-23T18:44:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4b20108090 0x7f4b201abd60 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b20071bd0 0x7f4b201a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f4b080a1150 0x7f4b080a3600 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 --2- 172.21.15.2:0/1749239473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b20107740 0x7f4b201a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:44:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.929+0000 7f4b26d89700 1 -- 172.21.15.2:0/1749239473 >> 172.21.15.2:0/1749239473 conn(0x7f4b2006d3b0 msgr2=0x7f4b2006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:44:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.931+0000 7f4b26d89700 1 -- 172.21.15.2:0/1749239473 shutdown_connections 2023-06-23T18:44:56.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:44:56.931+0000 7f4b26d89700 1 -- 172.21.15.2:0/1749239473 wait complete. 2023-06-23T18:44:56.951 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:44:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 665.0 s, 331.2 tps, lat 303.644 ms stddev 46.119 2023-06-23T18:44:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:44:58 smithi002 ceph-mon[95947]: pgmap v984: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 950 op/s 2023-06-23T18:44:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:44:58 smithi082 ceph-mon[97889]: pgmap v984: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 950 op/s 2023-06-23T18:44:58.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:44:58 smithi017 ceph-mon[97472]: pgmap v984: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 950 op/s 2023-06-23T18:45:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:00 smithi002 ceph-mon[95947]: pgmap v985: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:45:00.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:00 smithi082 ceph-mon[97889]: pgmap v985: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:45:00.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:00 smithi017 ceph-mon[97472]: pgmap v985: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 1.01k op/s 2023-06-23T18:45:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 670.0 s, 333.2 tps, lat 297.534 ms stddev 47.033 2023-06-23T18:45:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:02 smithi002 ceph-mon[95947]: pgmap v986: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 982 op/s 2023-06-23T18:45:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:02 smithi082 ceph-mon[97889]: pgmap v986: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 982 op/s 2023-06-23T18:45:02.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:02 smithi017 ceph-mon[97472]: pgmap v986: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 982 op/s 2023-06-23T18:45:04.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:04 smithi002 ceph-mon[95947]: pgmap v987: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 961 op/s 2023-06-23T18:45:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:04 smithi082 ceph-mon[97889]: pgmap v987: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 961 op/s 2023-06-23T18:45:04.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:04 smithi017 ceph-mon[97472]: pgmap v987: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 961 op/s 2023-06-23T18:45:06.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:06 smithi002 ceph-mon[95947]: pgmap v988: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 917 op/s 2023-06-23T18:45:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:06 smithi082 ceph-mon[97889]: pgmap v988: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 917 op/s 2023-06-23T18:45:06.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:06 smithi017 ceph-mon[97472]: pgmap v988: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 917 op/s 2023-06-23T18:45:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 675.0 s, 328.6 tps, lat 308.087 ms stddev 102.547 2023-06-23T18:45:08.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:08 smithi002 ceph-mon[95947]: pgmap v989: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 913 op/s 2023-06-23T18:45:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:08 smithi082 ceph-mon[97889]: pgmap v989: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 913 op/s 2023-06-23T18:45:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:08 smithi017 ceph-mon[97472]: pgmap v989: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.0 MiB/s wr, 913 op/s 2023-06-23T18:45:10.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:10 smithi017 ceph-mon[97472]: pgmap v990: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 987 op/s 2023-06-23T18:45:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:10 smithi002 ceph-mon[95947]: pgmap v990: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 987 op/s 2023-06-23T18:45:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:10 smithi082 ceph-mon[97889]: pgmap v990: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 987 op/s 2023-06-23T18:45:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 680.0 s, 330.6 tps, lat 299.157 ms stddev 55.490 2023-06-23T18:45:12.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:12 smithi002 ceph-mon[95947]: pgmap v991: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1k op/s 2023-06-23T18:45:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:12 smithi082 ceph-mon[97889]: pgmap v991: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1k op/s 2023-06-23T18:45:12.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:12 smithi017 ceph-mon[97472]: pgmap v991: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1k op/s 2023-06-23T18:45:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:14 smithi082 ceph-mon[97889]: pgmap v992: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 971 op/s 2023-06-23T18:45:14.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:14 smithi017 ceph-mon[97472]: pgmap v992: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 971 op/s 2023-06-23T18:45:15.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:14 smithi002 ceph-mon[95947]: pgmap v992: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.6 MiB/s wr, 971 op/s 2023-06-23T18:45:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:16 smithi082 ceph-mon[97889]: pgmap v993: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 971 op/s 2023-06-23T18:45:16.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:16 smithi017 ceph-mon[97472]: pgmap v993: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 971 op/s 2023-06-23T18:45:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 685.0 s, 350.4 tps, lat 285.660 ms stddev 52.958 2023-06-23T18:45:17.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:16 smithi002 ceph-mon[95947]: pgmap v993: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 971 op/s 2023-06-23T18:45:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:18 smithi082 ceph-mon[97889]: pgmap v994: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:45:18.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:18 smithi017 ceph-mon[97472]: pgmap v994: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:45:19.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:18 smithi002 ceph-mon[95947]: pgmap v994: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 982 op/s 2023-06-23T18:45:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:20 smithi082 ceph-mon[97889]: pgmap v995: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:45:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:20 smithi017 ceph-mon[97472]: pgmap v995: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:45:21.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:20 smithi002 ceph-mon[95947]: pgmap v995: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-06-23T18:45:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 690.0 s, 344.8 tps, lat 291.065 ms stddev 50.035 2023-06-23T18:45:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:22 smithi082 ceph-mon[97889]: pgmap v996: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:45:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:22 smithi017 ceph-mon[97472]: pgmap v996: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:45:23.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:22 smithi002 ceph-mon[95947]: pgmap v996: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:45:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:24 smithi082 ceph-mon[97889]: pgmap v997: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 957 op/s 2023-06-23T18:45:24.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:24 smithi017 ceph-mon[97472]: pgmap v997: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 957 op/s 2023-06-23T18:45:25.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:24 smithi002 ceph-mon[95947]: pgmap v997: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.6 MiB/s wr, 957 op/s 2023-06-23T18:45:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:26 smithi082 ceph-mon[97889]: pgmap v998: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 945 op/s 2023-06-23T18:45:26.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:26 smithi017 ceph-mon[97472]: pgmap v998: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 945 op/s 2023-06-23T18:45:26.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:45:26.991 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:26 smithi002 ceph-mon[95947]: pgmap v998: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 945 op/s 2023-06-23T18:45:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 695.0 s, 348.4 tps, lat 286.960 ms stddev 50.122 2023-06-23T18:45:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 -- 172.21.15.2:0/201250063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00fea80 msgr2=0x7f14e00fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 --2- 172.21.15.2:0/201250063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00fea80 0x7f14e00fee90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0098f0 tx=0x7f14dc009c00 comp rx=0 tx=0).stop 2023-06-23T18:45:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 -- 172.21.15.2:0/201250063 shutdown_connections 2023-06-23T18:45:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 --2- 172.21.15.2:0/201250063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e01006c0 0x7f14e0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 --2- 172.21.15.2:0/201250063 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00fea80 0x7f14e00fee90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 --2- 172.21.15.2:0/201250063 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00ffd10 0x7f14e0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 -- 172.21.15.2:0/201250063 >> 172.21.15.2:0/201250063 conn(0x7f14e00fa0c0 msgr2=0x7f14e00fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:27.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 -- 172.21.15.2:0/201250063 shutdown_connections 2023-06-23T18:45:27.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.133+0000 7f14e827d700 1 -- 172.21.15.2:0/201250063 wait complete. 2023-06-23T18:45:27.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.134+0000 7f14e827d700 1 Processor -- start 2023-06-23T18:45:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.134+0000 7f14e827d700 1 -- start start 2023-06-23T18:45:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e827d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00fea80 0x7f14e0194720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e827d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00ffd10 0x7f14e0194c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e827d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e01006c0 0x7f14e019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e827d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e019c2a0 con 0x7f14e00ffd10 2023-06-23T18:45:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e827d700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e0069ef0 con 0x7f14e00fea80 2023-06-23T18:45:27.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e827d700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e006a080 con 0x7f14e01006c0 2023-06-23T18:45:27.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e6019700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00fea80 0x7f14e0194720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e5818700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00ffd10 0x7f14e0194c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e681a700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e01006c0 0x7f14e019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e5818700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00ffd10 0x7f14e0194c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58194/0 (socket says 172.21.15.2:58194) 2023-06-23T18:45:27.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e5818700 1 -- 172.21.15.2:0/2445624205 learned_addr learned my addr 172.21.15.2:0/2445624205 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:27.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e6019700 1 -- 172.21.15.2:0/2445624205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e01006c0 msgr2=0x7f14e019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e6019700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e01006c0 0x7f14e019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e6019700 1 -- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00ffd10 msgr2=0x7f14e0194c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e6019700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00ffd10 0x7f14e0194c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e6019700 1 -- 172.21.15.2:0/2445624205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc009520 con 0x7f14e00fea80 2023-06-23T18:45:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.135+0000 7f14e681a700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e01006c0 0x7f14e019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:45:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.136+0000 7f14e6019700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00fea80 0x7f14e0194720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f14dc000c00 tx=0x7f14dc00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.136+0000 7f14d77fe700 1 -- 172.21.15.2:0/2445624205 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc01c070 con 0x7f14e00fea80 2023-06-23T18:45:27.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.136+0000 7f14e827d700 1 -- 172.21.15.2:0/2445624205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e006a300 con 0x7f14e00fea80 2023-06-23T18:45:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.136+0000 7f14d77fe700 1 -- 172.21.15.2:0/2445624205 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14dc00f7d0 con 0x7f14e00fea80 2023-06-23T18:45:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.136+0000 7f14d77fe700 1 -- 172.21.15.2:0/2445624205 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc0039b0 con 0x7f14e00fea80 2023-06-23T18:45:27.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.136+0000 7f14e827d700 1 -- 172.21.15.2:0/2445624205 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e006a7b0 con 0x7f14e00fea80 2023-06-23T18:45:27.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.139+0000 7f14d77fe700 1 -- 172.21.15.2:0/2445624205 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f14dc020030 con 0x7f14e00fea80 2023-06-23T18:45:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.140+0000 7f14d77fe700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14cc0a1150 0x7f14cc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.140+0000 7f14d77fe700 1 -- 172.21.15.2:0/2445624205 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f14dc0cd930 con 0x7f14e00fea80 2023-06-23T18:45:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.140+0000 7f14e5818700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14cc0a1150 0x7f14cc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14f4402b80 1 Processor -- start 2023-06-23T18:45:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14f4402b80 1 -- start start 2023-06-23T18:45:27.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14e5818700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14cc0a1150 0x7f14cc0a3600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14d00044f0 tx=0x7f14d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x561244083090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5612440835d0 0x561244083a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244084000 0x561244084470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14f4402b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561243f87cb0 con 0x561244084000 2023-06-23T18:45:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14f4402b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561243fdaa40 con 0x5612440835d0 2023-06-23T18:45:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14f4402b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561243fd3ed0 con 0x561244082c80 2023-06-23T18:45:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14e5818700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5612440835d0 0x561244083a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14e5818700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5612440835d0 0x561244083a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48792/0 (socket says 172.21.15.2:48792) 2023-06-23T18:45:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14e6019700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x561244083090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.141+0000 7f14e5818700 1 -- 172.21.15.2:0/752984282 learned_addr learned my addr 172.21.15.2:0/752984282 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14e681a700 1 --2- 172.21.15.2:0/752984282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244084000 0x561244084470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14e5818700 1 -- 172.21.15.2:0/752984282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 msgr2=0x561244083090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14e5818700 1 --2- 172.21.15.2:0/752984282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x561244083090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14e5818700 1 -- 172.21.15.2:0/752984282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244084000 msgr2=0x561244084470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:27.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14e5818700 1 --2- 172.21.15.2:0/752984282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244084000 0x561244084470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14e5818700 1 -- 172.21.15.2:0/752984282 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561243e55630 con 0x5612440835d0 2023-06-23T18:45:27.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14e5818700 1 --2- 172.21.15.2:0/752984282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5612440835d0 0x561244083a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f14d0003860 tx=0x7f14d0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=413d2bd7a91f3c4f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:27.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14d4dac700 1 -- 172.21.15.2:0/752984282 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d003e790 con 0x5612440835d0 2023-06-23T18:45:27.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14d4dac700 1 -- 172.21.15.2:0/752984282 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14d003ed80 con 0x5612440835d0 2023-06-23T18:45:27.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.142+0000 7f14d4dac700 1 -- 172.21.15.2:0/752984282 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0047e10 con 0x5612440835d0 2023-06-23T18:45:27.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 -- 172.21.15.2:0/752984282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5612440835d0 msgr2=0x561244083a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 --2- 172.21.15.2:0/752984282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5612440835d0 0x561244083a70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f14d0003860 tx=0x7f14d0009f90 comp rx=0 tx=0).stop 2023-06-23T18:45:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 -- 172.21.15.2:0/752984282 shutdown_connections 2023-06-23T18:45:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 --2- 172.21.15.2:0/752984282 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244084000 0x561244084470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 --2- 172.21.15.2:0/752984282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x561244083090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 --2- 172.21.15.2:0/752984282 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5612440835d0 0x561244083a70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 -- 172.21.15.2:0/752984282 >> 172.21.15.2:0/752984282 conn(0x5612440783f0 msgr2=0x56124407d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 -- 172.21.15.2:0/752984282 shutdown_connections 2023-06-23T18:45:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.143+0000 7f14f4402b80 1 -- 172.21.15.2:0/752984282 wait complete. 2023-06-23T18:45:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 1 -- start start 2023-06-23T18:45:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 1 Processor -- start 2023-06-23T18:45:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x56124408c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x56124408cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x561244093b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561243f73ac0 con 0x5612440835d0 2023-06-23T18:45:27.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561243f737c0 con 0x561244084000 2023-06-23T18:45:27.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561243e57920 con 0x561244082c80 2023-06-23T18:45:27.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.144+0000 7f14f4402b80 10 client.0 _ll_register_callbacks cb 0x561243f79260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:45:27.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e6019700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x56124408c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e681a700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x561244093b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e6019700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x56124408c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58542/0 (socket says 172.21.15.2:58542) 2023-06-23T18:45:27.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e6019700 1 -- 172.21.15.2:0/1544402614 learned_addr learned my addr 172.21.15.2:0/1544402614 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:27.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e5818700 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x56124408cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e681a700 1 -- 172.21.15.2:0/1544402614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 msgr2=0x56124408c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:27.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e681a700 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x56124408c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e681a700 1 -- 172.21.15.2:0/1544402614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 msgr2=0x56124408cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:27.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e681a700 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x56124408cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e681a700 1 -- 172.21.15.2:0/1544402614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d0039070 con 0x561244084000 2023-06-23T18:45:27.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e6019700 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x56124408c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:45:27.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e5818700 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x56124408cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:45:27.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14e681a700 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x561244093b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f14d8007bb0 tx=0x7f14d8007be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5962e5abf21eca01 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:27.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:45:27.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1544402614 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d8004db0 con 0x561244084000 2023-06-23T18:45:27.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1544402614 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f14d80043d0 con 0x561244084000 2023-06-23T18:45:27.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.145+0000 7f14f4402b80 1 -- 172.21.15.2:0/1544402614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561244075bb0 con 0x561244084000 2023-06-23T18:45:27.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1544402614 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d8004590 con 0x561244084000 2023-06-23T18:45:27.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14f4402b80 10 client.26600 fetch_fsmap learned FSMap version 41 2023-06-23T18:45:27.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14f4402b80 1 -- 172.21.15.2:0/1544402614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561243e01de0 con 0x561244084000 2023-06-23T18:45:27.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1544402614 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561244075bb0 con 0x561244084000 2023-06-23T18:45:27.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1544402614 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14d8016b80 con 0x561244084000 2023-06-23T18:45:27.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14f4402b80 10 client.26600 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:45:27.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14f4402b80 10 client.26600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:45:27.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14f4402b80 20 client.26600 populate_metadata read hostname 'smithi002' 2023-06-23T18:45:27.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5612440983b0 0x56124409a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14f4402b80 4 client.26600 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:45:27.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14f4402b80 1 -- 172.21.15.2:0/1544402614 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561243fcceb0 con 0x5612440983b0 2023-06-23T18:45:27.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.146+0000 7f14e6019700 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5612440983b0 0x56124409a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.147+0000 7f14e6019700 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5612440983b0 0x56124409a7c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c4d82819855ac8 server_cookie=52d3648b1d8f8cbc in_seq=0 out_seq=0 2023-06-23T18:45:27.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.147+0000 7f14c9ffb700 10 client.26600 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:45:27.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.147+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1544402614 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561243fcceb0 con 0x5612440983b0 2023-06-23T18:45:27.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.147+0000 7f14c9ffb700 10 client.26600 handle_command_reply: tid=0 2023-06-23T18:45:27.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.147+0000 7f14f4402b80 1 client.26600 shutdown 2023-06-23T18:45:27.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 -- 172.21.15.2:0/1544402614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 msgr2=0x561244093b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x561244093b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f14d8007bb0 tx=0x7f14d8007be0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 -- 172.21.15.2:0/1544402614 shutdown_connections 2023-06-23T18:45:27.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5612440983b0 0x56124409a7c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x561244093b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244082c80 0x56124408c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1544402614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x56124408cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 -- 172.21.15.2:0/1544402614 >> 172.21.15.2:0/1544402614 conn(0x5612440783f0 msgr2=0x56124407cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:27.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14c9ffb700 0 client.26600 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:45:27.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 -- 172.21.15.2:0/1544402614 shutdown_connections 2023-06-23T18:45:27.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 1 -- 172.21.15.2:0/1544402614 wait complete. 2023-06-23T18:45:27.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.148+0000 7f14f4402b80 20 client.26600 trim_cache size 0 max 16384 2023-06-23T18:45:27.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.175+0000 7f14f4402b80 1 Processor -- start 2023-06-23T18:45:27.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.175+0000 7f14f4402b80 1 -- start start 2023-06-23T18:45:27.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.175+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244082c80 0x56124408a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.175+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612440835d0 0x561244093ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.175+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x56124408c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.175+0000 7f14f4402b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124408c0f0 con 0x561244082c80 2023-06-23T18:45:27.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.175+0000 7f14f4402b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124407ed50 con 0x561244084000 2023-06-23T18:45:27.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.175+0000 7f14f4402b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56124407e3d0 con 0x5612440835d0 2023-06-23T18:45:27.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612440835d0 0x561244093ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612440835d0 0x561244093ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58552/0 (socket says 172.21.15.2:58552) 2023-06-23T18:45:27.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 -- 172.21.15.2:0/369038681 learned_addr learned my addr 172.21.15.2:0/369038681 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:27.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e6019700 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244082c80 0x56124408a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e681a700 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x56124408c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 -- 172.21.15.2:0/369038681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 msgr2=0x56124408c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:27.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x56124408c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 -- 172.21.15.2:0/369038681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244082c80 msgr2=0x56124408a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:27.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244082c80 0x56124408a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 -- 172.21.15.2:0/369038681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561243deac60 con 0x5612440835d0 2023-06-23T18:45:27.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e681a700 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x56124408c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:45:27.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14e5818700 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612440835d0 0x561244093ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f14d0003e50 tx=0x7f14d003ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ad4f4175a784b1f server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:27.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.176+0000 7f14d55ad700 1 -- 172.21.15.2:0/369038681 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0047480 con 0x5612440835d0 2023-06-23T18:45:27.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.177+0000 7f14d55ad700 1 -- 172.21.15.2:0/369038681 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14d003ea20 con 0x5612440835d0 2023-06-23T18:45:27.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.177+0000 7f14d55ad700 1 -- 172.21.15.2:0/369038681 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d00052a0 con 0x5612440835d0 2023-06-23T18:45:27.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.177+0000 7f14f4402b80 1 -- 172.21.15.2:0/369038681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612440835d0 msgr2=0x561244093ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.177+0000 7f14f4402b80 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612440835d0 0x561244093ac0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f14d0003e50 tx=0x7f14d003ef30 comp rx=0 tx=0).stop 2023-06-23T18:45:27.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 -- 172.21.15.2:0/369038681 shutdown_connections 2023-06-23T18:45:27.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244084000 0x56124408c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561244082c80 0x56124408a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 --2- 172.21.15.2:0/369038681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5612440835d0 0x561244093ac0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 -- 172.21.15.2:0/369038681 >> 172.21.15.2:0/369038681 conn(0x5612440783f0 msgr2=0x56124408b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:27.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 -- 172.21.15.2:0/369038681 shutdown_connections 2023-06-23T18:45:27.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 -- 172.21.15.2:0/369038681 wait complete. 2023-06-23T18:45:27.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 -- start start 2023-06-23T18:45:27.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.178+0000 7f14f4402b80 1 Processor -- start 2023-06-23T18:45:27.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244082c80 0x5612440934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x561244093a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14f4402b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244084000 0x56124408ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14f4402b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612440906e0 con 0x5612440835d0 2023-06-23T18:45:27.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14f4402b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561244090850 con 0x561244082c80 2023-06-23T18:45:27.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14f4402b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561244090a10 con 0x561244084000 2023-06-23T18:45:27.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14f4402b80 10 client.0 _ll_register_callbacks cb 0x561243f79260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:45:27.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e6019700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244082c80 0x5612440934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e5818700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x561244093a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e6019700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244082c80 0x5612440934f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48810/0 (socket says 172.21.15.2:48810) 2023-06-23T18:45:27.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e6019700 1 -- 172.21.15.2:0/1415164213 learned_addr learned my addr 172.21.15.2:0/1415164213 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:27.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e681a700 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244084000 0x56124408ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e5818700 1 -- 172.21.15.2:0/1415164213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244084000 msgr2=0x56124408ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:27.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e5818700 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244084000 0x56124408ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e5818700 1 -- 172.21.15.2:0/1415164213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244082c80 msgr2=0x5612440934f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e5818700 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244082c80 0x5612440934f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e5818700 1 -- 172.21.15.2:0/1415164213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561243e55630 con 0x5612440835d0 2023-06-23T18:45:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e681a700 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244084000 0x56124408ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:45:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.179+0000 7f14e6019700 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244082c80 0x5612440934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:45:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14e5818700 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x561244093a30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f14d0003830 tx=0x7f14d0009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763d2647784568b2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:27.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:45:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1415164213 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0005450 con 0x5612440835d0 2023-06-23T18:45:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14f4402b80 1 -- 172.21.15.2:0/1415164213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14d003ed80 con 0x5612440835d0 2023-06-23T18:45:27.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1415164213 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f14d003eb90 con 0x5612440835d0 2023-06-23T18:45:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1415164213 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14d00055b0 con 0x5612440835d0 2023-06-23T18:45:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14f4402b80 10 client.17097 fetch_fsmap learned FSMap version 41 2023-06-23T18:45:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14f4402b80 1 -- 172.21.15.2:0/1415164213 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561243e28250 con 0x5612440835d0 2023-06-23T18:45:27.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1415164213 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14d006b020 con 0x5612440835d0 2023-06-23T18:45:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1415164213 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f14d0033550 con 0x5612440835d0 2023-06-23T18:45:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14f4402b80 10 client.17097 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:45:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.180+0000 7f14f4402b80 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:45:27.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.181+0000 7f14f4402b80 20 client.17097 populate_metadata read hostname 'smithi002' 2023-06-23T18:45:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.181+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5612440ed3e0 0x5612440ef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.181+0000 7f14f4402b80 4 client.17097 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:45:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.181+0000 7f14f4402b80 1 -- 172.21.15.2:0/1415164213 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561243f8f050 con 0x5612440ed3e0 2023-06-23T18:45:27.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.181+0000 7f14e6019700 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5612440ed3e0 0x5612440ef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:27.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.181+0000 7f14c9ffb700 10 client.17097 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:45:27.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.181+0000 7f14e6019700 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5612440ed3e0 0x5612440ef7c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d955e993c9fa11 server_cookie=3c1f5cffeda8a2f8 in_seq=0 out_seq=0 2023-06-23T18:45:27.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14c9ffb700 1 -- 172.21.15.2:0/1415164213 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561243f8f050 con 0x5612440ed3e0 2023-06-23T18:45:27.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14c9ffb700 10 client.17097 handle_command_reply: tid=0 2023-06-23T18:45:27.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 client.17097 shutdown 2023-06-23T18:45:27.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 -- 172.21.15.2:0/1415164213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 msgr2=0x561244093a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x561244093a30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f14d0003830 tx=0x7f14d0009f60 comp rx=0 tx=0).stop 2023-06-23T18:45:27.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 -- 172.21.15.2:0/1415164213 shutdown_connections 2023-06-23T18:45:27.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5612440ed3e0 0x5612440ef7c0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561244084000 0x56124408ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14c9ffb700 0 client.17097 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:45:27.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561244082c80 0x5612440934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 --2- 172.21.15.2:0/1415164213 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5612440835d0 0x561244093a30 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 -- 172.21.15.2:0/1415164213 >> 172.21.15.2:0/1415164213 conn(0x5612440783f0 msgr2=0x56124408a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:27.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 -- 172.21.15.2:0/1415164213 shutdown_connections 2023-06-23T18:45:27.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 1 -- 172.21.15.2:0/1415164213 wait complete. 2023-06-23T18:45:27.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.182+0000 7f14f4402b80 20 client.17097 trim_cache size 0 max 16384 2023-06-23T18:45:27.205 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:45:27.205 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:45:27.205 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:45:27.206 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:45:27.206 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:45:27.206 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:45:27.206 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:45:27.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:45:27.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:45:27.207 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:45:27.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 -- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14cc0a1150 msgr2=0x7f14cc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14cc0a1150 0x7f14cc0a3600 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14d00044f0 tx=0x7f14d0009260 comp rx=0 tx=0).stop 2023-06-23T18:45:27.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 -- 172.21.15.2:0/2445624205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00fea80 msgr2=0x7f14e0194720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:27.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00fea80 0x7f14e0194720 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f14dc000c00 tx=0x7f14dc00b790 comp rx=0 tx=0).stop 2023-06-23T18:45:27.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 -- 172.21.15.2:0/2445624205 shutdown_connections 2023-06-23T18:45:27.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14e01006c0 0x7f14e019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f14cc0a1150 0x7f14cc0a3600 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f14e00fea80 0x7f14e0194720 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 --2- 172.21.15.2:0/2445624205 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14e00ffd10 0x7f14e0194c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:27.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 -- 172.21.15.2:0/2445624205 >> 172.21.15.2:0/2445624205 conn(0x7f14e00fa0c0 msgr2=0x7f14e00fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:27.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 -- 172.21.15.2:0/2445624205 shutdown_connections 2023-06-23T18:45:27.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:27.183+0000 7f14e827d700 1 -- 172.21.15.2:0/2445624205 wait complete. 2023-06-23T18:45:27.212 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:45:28.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:28 smithi082 ceph-mon[97889]: pgmap v999: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.1 MiB/s wr, 917 op/s 2023-06-23T18:45:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:28 smithi017 ceph-mon[97472]: pgmap v999: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.1 MiB/s wr, 917 op/s 2023-06-23T18:45:29.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:28 smithi002 ceph-mon[95947]: pgmap v999: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.1 MiB/s wr, 917 op/s 2023-06-23T18:45:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:30 smithi082 ceph-mon[97889]: pgmap v1000: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 982 op/s 2023-06-23T18:45:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:30 smithi017 ceph-mon[97472]: pgmap v1000: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 982 op/s 2023-06-23T18:45:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:30 smithi002 ceph-mon[95947]: pgmap v1000: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 982 op/s 2023-06-23T18:45:32.026 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 700.0 s, 350.0 tps, lat 284.786 ms stddev 46.243 2023-06-23T18:45:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:32 smithi082 ceph-mon[97889]: pgmap v1001: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 988 op/s 2023-06-23T18:45:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:32 smithi017 ceph-mon[97472]: pgmap v1001: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 988 op/s 2023-06-23T18:45:33.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:32 smithi002 ceph-mon[95947]: pgmap v1001: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 988 op/s 2023-06-23T18:45:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:34 smithi082 ceph-mon[97889]: pgmap v1002: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 969 op/s 2023-06-23T18:45:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:34 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:45:34.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:34 smithi017 ceph-mon[97472]: pgmap v1002: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 969 op/s 2023-06-23T18:45:34.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:34 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:45:35.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:34 smithi002 ceph-mon[95947]: pgmap v1002: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 969 op/s 2023-06-23T18:45:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:34 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:45:36.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:36 smithi082 ceph-mon[97889]: pgmap v1003: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 968 op/s 2023-06-23T18:45:36.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.351 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:36 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.440 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:36 smithi017 ceph-mon[97472]: pgmap v1003: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 968 op/s 2023-06-23T18:45:36.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.441 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:36 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:36 smithi002 ceph-mon[95947]: pgmap v1003: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 10 MiB/s wr, 968 op/s 2023-06-23T18:45:36.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:36.552 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:36 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 705.0 s, 323.0 tps, lat 310.120 ms stddev 65.458 2023-06-23T18:45:37.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.598 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:45:37.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:45:37.599 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:45:37.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:45:37.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:45:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:45:37.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:45:38.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:38 smithi017 ceph-mon[97472]: pgmap v1004: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 957 op/s 2023-06-23T18:45:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:38 smithi002 ceph-mon[95947]: pgmap v1004: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 957 op/s 2023-06-23T18:45:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:38 smithi082 ceph-mon[97889]: pgmap v1004: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 957 op/s 2023-06-23T18:45:40.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:40 smithi017 ceph-mon[97472]: pgmap v1005: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:45:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:40 smithi002 ceph-mon[95947]: pgmap v1005: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:45:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:40 smithi082 ceph-mon[97889]: pgmap v1005: 97 pgs: 97 active+clean; 8.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 1.00k op/s 2023-06-23T18:45:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 710.0 s, 327.6 tps, lat 307.467 ms stddev 51.221 2023-06-23T18:45:42.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:42 smithi017 ceph-mon[97472]: pgmap v1006: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 989 op/s 2023-06-23T18:45:42.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:42 smithi002 ceph-mon[95947]: pgmap v1006: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 989 op/s 2023-06-23T18:45:42.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:42 smithi082 ceph-mon[97889]: pgmap v1006: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 989 op/s 2023-06-23T18:45:44.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:44 smithi017 ceph-mon[97472]: pgmap v1007: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 965 op/s 2023-06-23T18:45:44.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:44 smithi002 ceph-mon[95947]: pgmap v1007: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 965 op/s 2023-06-23T18:45:44.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:44 smithi082 ceph-mon[97889]: pgmap v1007: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 10 MiB/s wr, 965 op/s 2023-06-23T18:45:46.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:46 smithi017 ceph-mon[97472]: pgmap v1008: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 932 op/s 2023-06-23T18:45:46.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:46 smithi002 ceph-mon[95947]: pgmap v1008: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 932 op/s 2023-06-23T18:45:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:46 smithi082 ceph-mon[97889]: pgmap v1008: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 932 op/s 2023-06-23T18:45:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 715.0 s, 327.2 tps, lat 303.243 ms stddev 63.653 2023-06-23T18:45:48.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:48 smithi017 ceph-mon[97472]: pgmap v1009: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 927 op/s 2023-06-23T18:45:48.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:48 smithi002 ceph-mon[95947]: pgmap v1009: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 927 op/s 2023-06-23T18:45:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:48 smithi082 ceph-mon[97889]: pgmap v1009: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 927 op/s 2023-06-23T18:45:50.691 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:50 smithi017 ceph-mon[97472]: pgmap v1010: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 970 op/s 2023-06-23T18:45:50.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:50 smithi002 ceph-mon[95947]: pgmap v1010: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 970 op/s 2023-06-23T18:45:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:50 smithi082 ceph-mon[97889]: pgmap v1010: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 970 op/s 2023-06-23T18:45:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 720.0 s, 308.2 tps, lat 321.517 ms stddev 61.613 2023-06-23T18:45:52.692 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:52 smithi017 ceph-mon[97472]: pgmap v1011: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:45:52.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:52 smithi002 ceph-mon[95947]: pgmap v1011: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:45:52.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:52 smithi082 ceph-mon[97889]: pgmap v1011: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 979 op/s 2023-06-23T18:45:54.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:54 smithi017 ceph-mon[97472]: pgmap v1012: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 951 op/s 2023-06-23T18:45:54.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:54 smithi002 ceph-mon[95947]: pgmap v1012: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 951 op/s 2023-06-23T18:45:54.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:54 smithi082 ceph-mon[97889]: pgmap v1012: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 951 op/s 2023-06-23T18:45:56.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:56 smithi017 ceph-mon[97472]: pgmap v1013: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 942 op/s 2023-06-23T18:45:56.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:56 smithi002 ceph-mon[95947]: pgmap v1013: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 942 op/s 2023-06-23T18:45:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:56 smithi082 ceph-mon[97889]: pgmap v1013: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 942 op/s 2023-06-23T18:45:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 725.0 s, 321.4 tps, lat 315.340 ms stddev 52.334 2023-06-23T18:45:57.213 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:45:57.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.609+0000 7f6dec3b5700 1 -- 172.21.15.2:0/3403916929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de41066a0 msgr2=0x7f6de4106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.609+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/3403916929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de41066a0 0x7f6de4106ab0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6de00098f0 tx=0x7f6de0009c00 comp rx=0 tx=0).stop 2023-06-23T18:45:57.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.612+0000 7f6dec3b5700 1 -- 172.21.15.2:0/3403916929 shutdown_connections 2023-06-23T18:45:57.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.612+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/3403916929 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de4107d60 0x7f6de410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.612+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/3403916929 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de41066a0 0x7f6de4106ab0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.612+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/3403916929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6de41073b0 0x7f6de4107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.612+0000 7f6dec3b5700 1 -- 172.21.15.2:0/3403916929 >> 172.21.15.2:0/3403916929 conn(0x7f6de40751c0 msgr2=0x7f6de4077610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:57.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.612+0000 7f6dec3b5700 1 -- 172.21.15.2:0/3403916929 shutdown_connections 2023-06-23T18:45:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.612+0000 7f6dec3b5700 1 -- 172.21.15.2:0/3403916929 wait complete. 2023-06-23T18:45:57.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.613+0000 7f6dec3b5700 1 Processor -- start 2023-06-23T18:45:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.613+0000 7f6dec3b5700 1 -- start start 2023-06-23T18:45:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.613+0000 7f6dec3b5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de41066a0 0x7f6de407dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.613+0000 7f6dec3b5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6de41073b0 0x7f6de407e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.613+0000 7f6dec3b5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de4107d60 0x7f6de41abff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.613+0000 7f6dec3b5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40811c0 con 0x7f6de4107d60 2023-06-23T18:45:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.613+0000 7f6dec3b5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4081300 con 0x7f6de41073b0 2023-06-23T18:45:57.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.613+0000 7f6dec3b5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4069ef0 con 0x7f6de41066a0 2023-06-23T18:45:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de41066a0 0x7f6de407dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de41066a0 0x7f6de407dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58570/0 (socket says 172.21.15.2:58570) 2023-06-23T18:45:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 -- 172.21.15.2:0/1134071536 learned_addr learned my addr 172.21.15.2:0/1134071536 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6de9950700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6de41073b0 0x7f6de407e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 -- 172.21.15.2:0/1134071536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6de41073b0 msgr2=0x7f6de407e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6de41073b0 0x7f6de407e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 -- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de4107d60 msgr2=0x7f6de41abff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de4107d60 0x7f6de41abff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 -- 172.21.15.2:0/1134071536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd40095f0 con 0x7f6de41066a0 2023-06-23T18:45:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.614+0000 7f6dea151700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de41066a0 0x7f6de407dba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b790 tx=0x7f6de000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:57.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.615+0000 7f6ddb7fe700 1 -- 172.21.15.2:0/1134071536 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de001c070 con 0x7f6de41066a0 2023-06-23T18:45:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.615+0000 7f6ddb7fe700 1 -- 172.21.15.2:0/1134071536 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6de000f7d0 con 0x7f6de41066a0 2023-06-23T18:45:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.615+0000 7f6ddb7fe700 1 -- 172.21.15.2:0/1134071536 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00049c0 con 0x7f6de41066a0 2023-06-23T18:45:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.615+0000 7f6dec3b5700 1 -- 172.21.15.2:0/1134071536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de0009520 con 0x7f6de41066a0 2023-06-23T18:45:57.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.615+0000 7f6dec3b5700 1 -- 172.21.15.2:0/1134071536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de406a350 con 0x7f6de41066a0 2023-06-23T18:45:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.619+0000 7f6ddb7fe700 1 -- 172.21.15.2:0/1134071536 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6de0020070 con 0x7f6de41066a0 2023-06-23T18:45:57.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6df853ab80 1 Processor -- start 2023-06-23T18:45:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6df853ab80 1 -- start start 2023-06-23T18:45:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6ddb7fe700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6dd00a1220 0x7f6dd00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 0x559838a5c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5c5d0 0x559838a5ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5d000 0x559838a5d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6df853ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559838960cb0 con 0x559838a5bc80 2023-06-23T18:45:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6df853ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598389b3a40 con 0x559838a5d000 2023-06-23T18:45:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6df853ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598389aced0 con 0x559838a5c5d0 2023-06-23T18:45:57.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6dea952700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6dd00a1220 0x7f6dd00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6dea952700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5d000 0x559838a5d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6dea952700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5d000 0x559838a5d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48836/0 (socket says 172.21.15.2:48836) 2023-06-23T18:45:57.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6dea952700 1 -- 172.21.15.2:0/109007545 learned_addr learned my addr 172.21.15.2:0/109007545 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:57.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.620+0000 7f6de9950700 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5c5d0 0x559838a5ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dea151700 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 0x559838a5c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dea952700 1 -- 172.21.15.2:0/109007545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5c5d0 msgr2=0x559838a5ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dea952700 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5c5d0 0x559838a5ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dea952700 1 -- 172.21.15.2:0/109007545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 msgr2=0x559838a5c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dea952700 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 0x559838a5c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dea952700 1 -- 172.21.15.2:0/109007545 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55983882e630 con 0x559838a5d000 2023-06-23T18:45:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6de9950700 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5c5d0 0x559838a5ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:45:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dea952700 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5d000 0x559838a5d470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc013dd0 tx=0x7f6ddc0175d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58023f8697299140 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6ddb7fe700 1 -- 172.21.15.2:0/1134071536 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6de00cd660 con 0x7f6de41066a0 2023-06-23T18:45:57.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dea952700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6dd00a1220 0x7f6dd00a36d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc017920 tx=0x7f6ddc008000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dd8dac700 1 -- 172.21.15.2:0/109007545 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc009ce0 con 0x559838a5d000 2023-06-23T18:45:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.621+0000 7f6dd8dac700 1 -- 172.21.15.2:0/109007545 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ddc003e90 con 0x559838a5d000 2023-06-23T18:45:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.622+0000 7f6dd8dac700 1 -- 172.21.15.2:0/109007545 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc007810 con 0x559838a5d000 2023-06-23T18:45:57.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.622+0000 7f6df853ab80 1 -- 172.21.15.2:0/109007545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5d000 msgr2=0x559838a5d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.622+0000 7f6df853ab80 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5d000 0x559838a5d470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc013dd0 tx=0x7f6ddc0175d0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.622+0000 7f6df853ab80 1 -- 172.21.15.2:0/109007545 shutdown_connections 2023-06-23T18:45:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.622+0000 7f6df853ab80 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5d000 0x559838a5d470 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.622+0000 7f6df853ab80 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 0x559838a5c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.622+0000 7f6df853ab80 1 --2- 172.21.15.2:0/109007545 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5c5d0 0x559838a5ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.622+0000 7f6df853ab80 1 -- 172.21.15.2:0/109007545 >> 172.21.15.2:0/109007545 conn(0x559838a513f0 msgr2=0x559838a56520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:57.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.623+0000 7f6df853ab80 1 -- 172.21.15.2:0/109007545 shutdown_connections 2023-06-23T18:45:57.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.623+0000 7f6df853ab80 1 -- 172.21.15.2:0/109007545 wait complete. 2023-06-23T18:45:57.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.623+0000 7f6df853ab80 1 -- start start 2023-06-23T18:45:57.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.623+0000 7f6df853ab80 1 Processor -- start 2023-06-23T18:45:57.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.623+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a65590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.623+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a65ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.623+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6df853ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983894cac0 con 0x559838a5c5d0 2023-06-23T18:45:57.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6df853ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55983894c7c0 con 0x559838a5bc80 2023-06-23T18:45:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6df853ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559838830920 con 0x559838a5d000 2023-06-23T18:45:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6df853ab80 10 client.0 _ll_register_callbacks cb 0x559838952260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:45:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6de9950700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a65ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6de9950700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a65ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58246/0 (socket says 172.21.15.2:58246) 2023-06-23T18:45:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6de9950700 1 -- 172.21.15.2:0/727020709 learned_addr learned my addr 172.21.15.2:0/727020709 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dea952700 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dea151700 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a65590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dea952700 1 -- 172.21.15.2:0/727020709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 msgr2=0x559838a65590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dea952700 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a65590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dea952700 1 -- 172.21.15.2:0/727020709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 msgr2=0x559838a65ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dea952700 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a65ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dea952700 1 -- 172.21.15.2:0/727020709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd4009ad0 con 0x559838a5d000 2023-06-23T18:45:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6de9950700 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a65ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:45:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dea952700 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6cb10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc013760 tx=0x7f6ddc0091f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea54140b42932ead server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.624+0000 7f6dcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:45:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.625+0000 7f6dcdffb700 1 -- 172.21.15.2:0/727020709 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0098a0 con 0x559838a5d000 2023-06-23T18:45:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.625+0000 7f6dcdffb700 1 -- 172.21.15.2:0/727020709 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6ddc009ce0 con 0x559838a5d000 2023-06-23T18:45:57.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.625+0000 7f6df853ab80 1 -- 172.21.15.2:0/727020709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559838a4ebb0 con 0x559838a5d000 2023-06-23T18:45:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.625+0000 7f6dcdffb700 1 -- 172.21.15.2:0/727020709 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ddc003c10 con 0x559838a5d000 2023-06-23T18:45:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.625+0000 7f6df853ab80 10 client.26839 fetch_fsmap learned FSMap version 41 2023-06-23T18:45:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.625+0000 7f6df853ab80 1 -- 172.21.15.2:0/727020709 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f6ddc011040 con 0x559838a5d000 2023-06-23T18:45:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.625+0000 7f6dcdffb700 1 -- 172.21.15.2:0/727020709 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559838a4ebb0 con 0x559838a5d000 2023-06-23T18:45:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.625+0000 7f6dcdffb700 1 -- 172.21.15.2:0/727020709 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6ddc033070 con 0x559838a5d000 2023-06-23T18:45:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6df853ab80 10 client.26839 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:45:57.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6df853ab80 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:45:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6df853ab80 20 client.26839 populate_metadata read hostname 'smithi002' 2023-06-23T18:45:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6df853ab80 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559838a71240 0x559838a73650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6df853ab80 4 client.26839 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:45:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6df853ab80 1 -- 172.21.15.2:0/727020709 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598389a5eb0 con 0x559838a71240 2023-06-23T18:45:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6dea151700 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559838a71240 0x559838a73650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6dcdffb700 10 client.26839 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:45:57.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.626+0000 7f6dea151700 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559838a71240 0x559838a73650 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5da516028bfd4c86 server_cookie=12374da090b5e63b in_seq=0 out_seq=0 2023-06-23T18:45:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6dcdffb700 1 -- 172.21.15.2:0/727020709 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5598389a5eb0 con 0x559838a71240 2023-06-23T18:45:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6dcdffb700 10 client.26839 handle_command_reply: tid=0 2023-06-23T18:45:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 client.26839 shutdown 2023-06-23T18:45:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 -- 172.21.15.2:0/727020709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 msgr2=0x559838a6cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6cb10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc013760 tx=0x7f6ddc0091f0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 -- 172.21.15.2:0/727020709 shutdown_connections 2023-06-23T18:45:57.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559838a71240 0x559838a73650 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6cb10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a65590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6dcdffb700 0 client.26839 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:45:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 --2- 172.21.15.2:0/727020709 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a65ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.627+0000 7f6df853ab80 1 -- 172.21.15.2:0/727020709 >> 172.21.15.2:0/727020709 conn(0x559838a513f0 msgr2=0x559838a55c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.628+0000 7f6df853ab80 1 -- 172.21.15.2:0/727020709 shutdown_connections 2023-06-23T18:45:57.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.628+0000 7f6df853ab80 1 -- 172.21.15.2:0/727020709 wait complete. 2023-06-23T18:45:57.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.628+0000 7f6df853ab80 20 client.26839 trim_cache size 0 max 16384 2023-06-23T18:45:57.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6df853ab80 1 Processor -- start 2023-06-23T18:45:57.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6df853ab80 1 -- start start 2023-06-23T18:45:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 0x559838a64230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5c5d0 0x559838a6cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6df853ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559838a57630 con 0x559838a5bc80 2023-06-23T18:45:57.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6df853ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559838a571f0 con 0x559838a5c5d0 2023-06-23T18:45:57.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6df853ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559838a64a40 con 0x559838a5d000 2023-06-23T18:45:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6dea952700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6dea952700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58594/0 (socket says 172.21.15.2:58594) 2023-06-23T18:45:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.656+0000 7f6dea952700 1 -- 172.21.15.2:0/1095975317 learned_addr learned my addr 172.21.15.2:0/1095975317 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dea952700 1 -- 172.21.15.2:0/1095975317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5c5d0 msgr2=0x559838a6cad0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:45:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dea952700 1 --2- 172.21.15.2:0/1095975317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5c5d0 0x559838a6cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dea952700 1 -- 172.21.15.2:0/1095975317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 msgr2=0x559838a64230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dea952700 1 --2- 172.21.15.2:0/1095975317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 0x559838a64230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dea952700 1 -- 172.21.15.2:0/1095975317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598387dade0 con 0x559838a5d000 2023-06-23T18:45:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dea952700 1 --2- 172.21.15.2:0/1095975317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6d400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc004c30 tx=0x7f6ddc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6477aabb46f6ba5a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dd95ad700 1 -- 172.21.15.2:0/1095975317 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc01e040 con 0x559838a5d000 2023-06-23T18:45:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dd95ad700 1 -- 172.21.15.2:0/1095975317 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ddc0048e0 con 0x559838a5d000 2023-06-23T18:45:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.657+0000 7f6dd95ad700 1 -- 172.21.15.2:0/1095975317 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0046a0 con 0x559838a5d000 2023-06-23T18:45:57.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.658+0000 7f6df853ab80 1 -- 172.21.15.2:0/1095975317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 msgr2=0x559838a6d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.658+0000 7f6df853ab80 1 --2- 172.21.15.2:0/1095975317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6d400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc004c30 tx=0x7f6ddc004ad0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.660+0000 7f6df853ab80 1 -- 172.21.15.2:0/1095975317 shutdown_connections 2023-06-23T18:45:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.660+0000 7f6df853ab80 1 --2- 172.21.15.2:0/1095975317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6d400 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.660+0000 7f6df853ab80 1 --2- 172.21.15.2:0/1095975317 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5bc80 0x559838a64230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.660+0000 7f6df853ab80 1 --2- 172.21.15.2:0/1095975317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5c5d0 0x559838a6cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.660+0000 7f6df853ab80 1 -- 172.21.15.2:0/1095975317 >> 172.21.15.2:0/1095975317 conn(0x559838a513f0 msgr2=0x559838a77f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:57.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.661+0000 7f6df853ab80 1 -- 172.21.15.2:0/1095975317 shutdown_connections 2023-06-23T18:45:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.661+0000 7f6df853ab80 1 -- 172.21.15.2:0/1095975317 wait complete. 2023-06-23T18:45:57.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 1 -- start start 2023-06-23T18:45:57.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 1 Processor -- start 2023-06-23T18:45:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a68a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a68f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559838a6c4b0 con 0x559838a5c5d0 2023-06-23T18:45:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559838a6c620 con 0x559838a5bc80 2023-06-23T18:45:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559838a6c7e0 con 0x559838a5d000 2023-06-23T18:45:57.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.662+0000 7f6df853ab80 10 client.0 _ll_register_callbacks cb 0x559838952260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:45:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a68a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a68a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48850/0 (socket says 172.21.15.2:48850) 2023-06-23T18:45:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 -- 172.21.15.2:0/3643677358 learned_addr learned my addr 172.21.15.2:0/3643677358 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:45:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea952700 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6de9950700 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a68f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 -- 172.21.15.2:0/3643677358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 msgr2=0x559838a6bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 -- 172.21.15.2:0/3643677358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 msgr2=0x559838a68f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:45:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a68f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 -- 172.21.15.2:0/3643677358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc011040 con 0x559838a5bc80 2023-06-23T18:45:57.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea952700 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:45:57.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dea151700 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a68a10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6de0097000 tx=0x7f6de00092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62589682c2a1f658 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:45:57.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dcdffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:45:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.663+0000 7f6dcdffb700 1 -- 172.21.15.2:0/3643677358 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de00a2ec0 con 0x559838a5bc80 2023-06-23T18:45:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6dcdffb700 1 -- 172.21.15.2:0/3643677358 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6de00cb270 con 0x559838a5bc80 2023-06-23T18:45:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 1 -- 172.21.15.2:0/3643677358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ddc003e90 con 0x559838a5bc80 2023-06-23T18:45:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6dcdffb700 1 -- 172.21.15.2:0/3643677358 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6de00b3140 con 0x559838a5bc80 2023-06-23T18:45:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 10 client.26618 fetch_fsmap learned FSMap version 41 2023-06-23T18:45:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 1 -- 172.21.15.2:0/3643677358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5598387c3c60 con 0x559838a5bc80 2023-06-23T18:45:57.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6dcdffb700 1 -- 172.21.15.2:0/3643677358 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ddc003e90 con 0x559838a5bc80 2023-06-23T18:45:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6dcdffb700 1 -- 172.21.15.2:0/3643677358 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6de0099ff0 con 0x559838a5bc80 2023-06-23T18:45:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 10 client.26618 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:45:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 10 client.26618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:45:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 20 client.26618 populate_metadata read hostname 'smithi002' 2023-06-23T18:45:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559838ac5d40 0x559838ac8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:45:57.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 4 client.26618 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:45:57.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6df853ab80 1 -- 172.21.15.2:0/3643677358 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559838968050 con 0x559838ac5d40 2023-06-23T18:45:57.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.664+0000 7f6de9950700 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559838ac5d40 0x559838ac8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:45:57.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.668+0000 7f6dcdffb700 10 client.26618 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:45:57.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.668+0000 7f6de9950700 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559838ac5d40 0x559838ac8150 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6e57a7eec21671 server_cookie=84e39757e03dd9a5 in_seq=0 out_seq=0 2023-06-23T18:45:57.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.668+0000 7f6dcdffb700 1 -- 172.21.15.2:0/3643677358 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559838968050 con 0x559838ac5d40 2023-06-23T18:45:57.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.668+0000 7f6dcdffb700 10 client.26618 handle_command_reply: tid=0 2023-06-23T18:45:57.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.668+0000 7f6df853ab80 1 client.26618 shutdown 2023-06-23T18:45:57.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 -- 172.21.15.2:0/3643677358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 msgr2=0x559838a68a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a68a10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6de0097000 tx=0x7f6de00092d0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 -- 172.21.15.2:0/3643677358 shutdown_connections 2023-06-23T18:45:57.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559838ac5d40 0x559838ac8150 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559838a5d000 0x559838a6bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x559838a5bc80 0x559838a68a10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 --2- 172.21.15.2:0/3643677358 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559838a5c5d0 0x559838a68f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 -- 172.21.15.2:0/3643677358 >> 172.21.15.2:0/3643677358 conn(0x559838a513f0 msgr2=0x559838a64230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:57.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6dcdffb700 0 client.26618 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:45:57.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 -- 172.21.15.2:0/3643677358 shutdown_connections 2023-06-23T18:45:57.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 1 -- 172.21.15.2:0/3643677358 wait complete. 2023-06-23T18:45:57.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6df853ab80 20 client.26618 trim_cache size 0 max 16384 2023-06-23T18:45:57.680 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:45:57.681 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:45:57.681 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:45:57.681 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:45:57.681 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:45:57.681 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:45:57.681 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:45:57.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:45:57.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:45:57.682 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:45:57.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6dec3b5700 1 -- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6dd00a1220 msgr2=0x7f6dd00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6dd00a1220 0x7f6dd00a36d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc017920 tx=0x7f6ddc008000 comp rx=0 tx=0).stop 2023-06-23T18:45:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6dec3b5700 1 -- 172.21.15.2:0/1134071536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de41066a0 msgr2=0x7f6de407dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:45:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.669+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de41066a0 0x7f6de407dba0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6de000b790 tx=0x7f6de000b870 comp rx=0 tx=0).stop 2023-06-23T18:45:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.670+0000 7f6dec3b5700 1 -- 172.21.15.2:0/1134071536 shutdown_connections 2023-06-23T18:45:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.670+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6de4107d60 0x7f6de41abff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.670+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de41066a0 0x7f6de407dba0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.670+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f6dd00a1220 0x7f6dd00a36d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.670+0000 7f6dec3b5700 1 --2- 172.21.15.2:0/1134071536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6de41073b0 0x7f6de407e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:45:57.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.670+0000 7f6dec3b5700 1 -- 172.21.15.2:0/1134071536 >> 172.21.15.2:0/1134071536 conn(0x7f6de40751c0 msgr2=0x7f6de410a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:45:57.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.670+0000 7f6dec3b5700 1 -- 172.21.15.2:0/1134071536 shutdown_connections 2023-06-23T18:45:57.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:45:57.670+0000 7f6dec3b5700 1 -- 172.21.15.2:0/1134071536 wait complete. 2023-06-23T18:45:57.690 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:45:58.693 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:45:58 smithi017 ceph-mon[97472]: pgmap v1014: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 964 op/s 2023-06-23T18:45:58.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:45:58 smithi002 ceph-mon[95947]: pgmap v1014: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 964 op/s 2023-06-23T18:45:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:45:58 smithi082 ceph-mon[97889]: pgmap v1014: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 964 op/s 2023-06-23T18:46:00.690 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:00 smithi017 ceph-mon[97472]: pgmap v1015: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 999 op/s 2023-06-23T18:46:00.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:00 smithi002 ceph-mon[95947]: pgmap v1015: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 999 op/s 2023-06-23T18:46:00.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:00 smithi082 ceph-mon[97889]: pgmap v1015: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 999 op/s 2023-06-23T18:46:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 730.0 s, 319.8 tps, lat 311.089 ms stddev 59.810 2023-06-23T18:46:02.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:02 smithi002 ceph-mon[95947]: pgmap v1016: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1k op/s 2023-06-23T18:46:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:02 smithi082 ceph-mon[97889]: pgmap v1016: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1k op/s 2023-06-23T18:46:02.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:02 smithi017 ceph-mon[97472]: pgmap v1016: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 1k op/s 2023-06-23T18:46:04.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:04 smithi002 ceph-mon[95947]: pgmap v1017: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 998 op/s 2023-06-23T18:46:04.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:04 smithi082 ceph-mon[97889]: pgmap v1017: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 998 op/s 2023-06-23T18:46:04.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:04 smithi017 ceph-mon[97472]: pgmap v1017: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 998 op/s 2023-06-23T18:46:06.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:06 smithi002 ceph-mon[95947]: pgmap v1018: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 979 op/s 2023-06-23T18:46:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:06 smithi082 ceph-mon[97889]: pgmap v1018: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 979 op/s 2023-06-23T18:46:06.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:06 smithi017 ceph-mon[97472]: pgmap v1018: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 979 op/s 2023-06-23T18:46:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 735.0 s, 348.2 tps, lat 287.175 ms stddev 48.809 2023-06-23T18:46:08.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:08 smithi002 ceph-mon[95947]: pgmap v1019: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.9 MiB/s wr, 947 op/s 2023-06-23T18:46:08.850 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:08 smithi082 ceph-mon[97889]: pgmap v1019: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.9 MiB/s wr, 947 op/s 2023-06-23T18:46:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:08 smithi017 ceph-mon[97472]: pgmap v1019: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.9 MiB/s wr, 947 op/s 2023-06-23T18:46:10.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:10 smithi002 ceph-mon[95947]: pgmap v1020: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.6 MiB/s wr, 950 op/s 2023-06-23T18:46:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:10 smithi082 ceph-mon[97889]: pgmap v1020: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.6 MiB/s wr, 950 op/s 2023-06-23T18:46:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:10 smithi017 ceph-mon[97472]: pgmap v1020: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.6 MiB/s wr, 950 op/s 2023-06-23T18:46:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 740.0 s, 346.6 tps, lat 255.648 ms stddev 73.152 2023-06-23T18:46:12.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:12 smithi002 ceph-mon[95947]: pgmap v1021: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 944 op/s 2023-06-23T18:46:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:12 smithi082 ceph-mon[97889]: pgmap v1021: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 944 op/s 2023-06-23T18:46:12.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:12 smithi017 ceph-mon[97472]: pgmap v1021: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 944 op/s 2023-06-23T18:46:14.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:14 smithi002 ceph-mon[95947]: pgmap v1022: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 892 op/s 2023-06-23T18:46:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:14 smithi082 ceph-mon[97889]: pgmap v1022: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 892 op/s 2023-06-23T18:46:14.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:14 smithi017 ceph-mon[97472]: pgmap v1022: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.8 MiB/s wr, 892 op/s 2023-06-23T18:46:16.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:16 smithi002 ceph-mon[95947]: pgmap v1023: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 823 op/s 2023-06-23T18:46:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:16 smithi082 ceph-mon[97889]: pgmap v1023: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 823 op/s 2023-06-23T18:46:16.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:16 smithi017 ceph-mon[97472]: pgmap v1023: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 823 op/s 2023-06-23T18:46:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 745.0 s, 215.6 tps, lat 55.893 ms stddev 15.487 2023-06-23T18:46:18.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:18 smithi002 ceph-mon[95947]: pgmap v1024: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.3 MiB/s wr, 791 op/s 2023-06-23T18:46:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:18 smithi082 ceph-mon[97889]: pgmap v1024: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.3 MiB/s wr, 791 op/s 2023-06-23T18:46:18.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:18 smithi017 ceph-mon[97472]: pgmap v1024: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.3 MiB/s wr, 791 op/s 2023-06-23T18:46:20.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:20 smithi002 ceph-mon[95947]: pgmap v1025: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 842 op/s 2023-06-23T18:46:20.853 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:20 smithi082 ceph-mon[97889]: pgmap v1025: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 842 op/s 2023-06-23T18:46:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:20 smithi017 ceph-mon[97472]: pgmap v1025: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 842 op/s 2023-06-23T18:46:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 750.0 s, 266.0 tps, lat 45.037 ms stddev 11.422 2023-06-23T18:46:22.804 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:22 smithi002 ceph-mon[95947]: pgmap v1026: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 816 op/s 2023-06-23T18:46:22.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:22 smithi082 ceph-mon[97889]: pgmap v1026: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 816 op/s 2023-06-23T18:46:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:22 smithi017 ceph-mon[97472]: pgmap v1026: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.3 MiB/s wr, 816 op/s 2023-06-23T18:46:24.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:24 smithi002 ceph-mon[95947]: pgmap v1027: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.4 MiB/s wr, 770 op/s 2023-06-23T18:46:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:24 smithi082 ceph-mon[97889]: pgmap v1027: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.4 MiB/s wr, 770 op/s 2023-06-23T18:46:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:24 smithi017 ceph-mon[97472]: pgmap v1027: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.4 MiB/s wr, 770 op/s 2023-06-23T18:46:26.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:26 smithi002 ceph-mon[95947]: pgmap v1028: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.5 MiB/s wr, 770 op/s 2023-06-23T18:46:26.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:26 smithi082 ceph-mon[97889]: pgmap v1028: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.5 MiB/s wr, 770 op/s 2023-06-23T18:46:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:26 smithi017 ceph-mon[97472]: pgmap v1028: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.5 MiB/s wr, 770 op/s 2023-06-23T18:46:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 755.0 s, 241.2 tps, lat 49.865 ms stddev 21.517 2023-06-23T18:46:27.691 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:46:27.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.889+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/686402487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb340fea80 msgr2=0x7fcb340fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.889+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/686402487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb340fea80 0x7fcb340fee90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009990 tx=0x7fcb30009ca0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.890+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/686402487 shutdown_connections 2023-06-23T18:46:27.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.890+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/686402487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb34100600 0x7fcb340677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.890+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/686402487 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb340fea80 0x7fcb340fee90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.890+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/686402487 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb340ffc70 0x7fcb341000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.890+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/686402487 >> 172.21.15.2:0/686402487 conn(0x7fcb340fa0e0 msgr2=0x7fcb340fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:27.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.890+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/686402487 shutdown_connections 2023-06-23T18:46:27.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.890+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/686402487 wait complete. 2023-06-23T18:46:27.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3c3c1700 1 Processor -- start 2023-06-23T18:46:27.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3c3c1700 1 -- start start 2023-06-23T18:46:27.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3c3c1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb340fea80 0x7fcb3406ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3c3c1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb340ffc70 0x7fcb3406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3c3c1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb34100600 0x7fcb341a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3c3c1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb34072330 con 0x7fcb34100600 2023-06-23T18:46:27.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3c3c1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340724a0 con 0x7fcb340fea80 2023-06-23T18:46:27.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3c3c1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb341a8120 con 0x7fcb340ffc70 2023-06-23T18:46:27.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3a15d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb340fea80 0x7fcb3406ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3a15d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb340fea80 0x7fcb3406ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48866/0 (socket says 172.21.15.2:48866) 2023-06-23T18:46:27.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3a15d700 1 -- 172.21.15.2:0/3938262677 learned_addr learned my addr 172.21.15.2:0/3938262677 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:27.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.891+0000 7fcb3a95e700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb34100600 0x7fcb341a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3a95e700 1 -- 172.21.15.2:0/3938262677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb340ffc70 msgr2=0x7fcb3406f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3a95e700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb340ffc70 0x7fcb3406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3a95e700 1 -- 172.21.15.2:0/3938262677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb340fea80 msgr2=0x7fcb3406ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3a95e700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb340fea80 0x7fcb3406ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3a95e700 1 -- 172.21.15.2:0/3938262677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb300095f0 con 0x7fcb34100600 2023-06-23T18:46:27.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb340fea80 0x7fcb3406ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:46:27.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3a95e700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb34100600 0x7fcb341a7be0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00efc0 tx=0x7fcb2c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:27.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb2b7fe700 1 -- 172.21.15.2:0/3938262677 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c018070 con 0x7fcb34100600 2023-06-23T18:46:27.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/3938262677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb341a8320 con 0x7fcb34100600 2023-06-23T18:46:27.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb2b7fe700 1 -- 172.21.15.2:0/3938262677 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb2c00f040 con 0x7fcb34100600 2023-06-23T18:46:27.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb2b7fe700 1 -- 172.21.15.2:0/3938262677 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c00cbf0 con 0x7fcb34100600 2023-06-23T18:46:27.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.892+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/3938262677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb341a8830 con 0x7fcb34100600 2023-06-23T18:46:27.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.897+0000 7fcb48546b80 1 Processor -- start 2023-06-23T18:46:27.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.897+0000 7fcb2b7fe700 1 -- 172.21.15.2:0/3938262677 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcb2c003a00 con 0x7fcb34100600 2023-06-23T18:46:27.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.897+0000 7fcb2b7fe700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcb200a11d0 0x7fcb200a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb2b7fe700 1 -- 172.21.15.2:0/3938262677 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcb2c0cc4d0 con 0x7fcb34100600 2023-06-23T18:46:27.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb3995c700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcb200a11d0 0x7fcb200a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb48546b80 1 -- start start 2023-06-23T18:46:27.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592ec80 0x55908592f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592f5d0 0x55908592fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x559085930470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb48546b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085833cb0 con 0x559085930000 2023-06-23T18:46:27.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb48546b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085886a40 con 0x55908592ec80 2023-06-23T18:46:27.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb48546b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908587fed0 con 0x55908592f5d0 2023-06-23T18:46:27.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb3a15d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592ec80 0x55908592f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb3a15d700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592ec80 0x55908592f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48874/0 (socket says 172.21.15.2:48874) 2023-06-23T18:46:27.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb3a15d700 1 -- 172.21.15.2:0/2618084937 learned_addr learned my addr 172.21.15.2:0/2618084937 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:27.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb3995c700 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592f5d0 0x55908592fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb3a95e700 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x559085930470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.898+0000 7fcb3995c700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcb200a11d0 0x7fcb200a3680 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcb24005950 tx=0x7fcb240058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:27.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb3a15d700 1 -- 172.21.15.2:0/2618084937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592f5d0 msgr2=0x55908592fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:27.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592f5d0 0x55908592fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb3a15d700 1 -- 172.21.15.2:0/2618084937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 msgr2=0x559085930470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:27.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x559085930470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb3a15d700 1 -- 172.21.15.2:0/2618084937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559085701630 con 0x55908592ec80 2023-06-23T18:46:27.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592ec80 0x55908592f090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb30000c00 tx=0x7fcb30005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6e19ec3b184319 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:27.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb28dac700 1 -- 172.21.15.2:0/2618084937 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3001d050 con 0x55908592ec80 2023-06-23T18:46:27.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb28dac700 1 -- 172.21.15.2:0/2618084937 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb3001e040 con 0x55908592ec80 2023-06-23T18:46:27.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.899+0000 7fcb28dac700 1 -- 172.21.15.2:0/2618084937 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30005250 con 0x55908592ec80 2023-06-23T18:46:27.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 -- 172.21.15.2:0/2618084937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592ec80 msgr2=0x55908592f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592ec80 0x55908592f090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcb30000c00 tx=0x7fcb30005c00 comp rx=0 tx=0).stop 2023-06-23T18:46:27.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 -- 172.21.15.2:0/2618084937 shutdown_connections 2023-06-23T18:46:27.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x559085930470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592ec80 0x55908592f090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 --2- 172.21.15.2:0/2618084937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592f5d0 0x55908592fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 -- 172.21.15.2:0/2618084937 >> 172.21.15.2:0/2618084937 conn(0x5590859243f0 msgr2=0x559085929520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:27.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 -- 172.21.15.2:0/2618084937 shutdown_connections 2023-06-23T18:46:27.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.900+0000 7fcb48546b80 1 -- 172.21.15.2:0/2618084937 wait complete. 2023-06-23T18:46:27.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 1 -- start start 2023-06-23T18:46:27.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 1 Processor -- start 2023-06-23T18:46:27.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 0x5590859384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x5590859389f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 0x55908593fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908581fac0 con 0x55908592ec80 2023-06-23T18:46:27.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908581f7c0 con 0x55908592f5d0 2023-06-23T18:46:27.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085703920 con 0x559085930000 2023-06-23T18:46:27.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb48546b80 10 client.0 _ll_register_callbacks cb 0x559085825260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:46:27.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb3995c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x5590859389f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb3995c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x5590859389f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48882/0 (socket says 172.21.15.2:48882) 2023-06-23T18:46:27.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.901+0000 7fcb3995c700 1 -- 172.21.15.2:0/667879645 learned_addr learned my addr 172.21.15.2:0/667879645 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:27.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.902+0000 7fcb3a95e700 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 0x55908593fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.902+0000 7fcb3995c700 1 -- 172.21.15.2:0/667879645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 msgr2=0x55908593fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:27.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.902+0000 7fcb3995c700 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 0x55908593fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.902+0000 7fcb3995c700 1 -- 172.21.15.2:0/667879645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 msgr2=0x5590859384b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:27.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.902+0000 7fcb3995c700 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 0x5590859384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.902+0000 7fcb3995c700 1 -- 172.21.15.2:0/667879645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb340681d0 con 0x55908592f5d0 2023-06-23T18:46:27.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb3995c700 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x5590859389f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401d710 tx=0x7fcb24004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d20a849f18eb1ae6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:27.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:46:27.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb1dffb700 1 -- 172.21.15.2:0/667879645 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24004a30 con 0x55908592f5d0 2023-06-23T18:46:27.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb1dffb700 1 -- 172.21.15.2:0/667879645 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcb24004b90 con 0x55908592f5d0 2023-06-23T18:46:27.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb48546b80 1 -- 172.21.15.2:0/667879645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559085921bb0 con 0x55908592f5d0 2023-06-23T18:46:27.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb1dffb700 1 -- 172.21.15.2:0/667879645 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb24004d50 con 0x55908592f5d0 2023-06-23T18:46:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb48546b80 10 client.26630 fetch_fsmap learned FSMap version 41 2023-06-23T18:46:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb48546b80 1 -- 172.21.15.2:0/667879645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x5590856adde0 con 0x55908592f5d0 2023-06-23T18:46:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.904+0000 7fcb1dffb700 1 -- 172.21.15.2:0/667879645 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559085921bb0 con 0x55908592f5d0 2023-06-23T18:46:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.905+0000 7fcb1dffb700 1 -- 172.21.15.2:0/667879645 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb24010990 con 0x55908592f5d0 2023-06-23T18:46:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.905+0000 7fcb48546b80 10 client.26630 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:46:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.905+0000 7fcb48546b80 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:46:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.905+0000 7fcb48546b80 20 client.26630 populate_metadata read hostname 'smithi002' 2023-06-23T18:46:27.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.905+0000 7fcb48546b80 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559085944310 0x559085946720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.905+0000 7fcb48546b80 4 client.26630 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:46:27.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.905+0000 7fcb48546b80 1 -- 172.21.15.2:0/667879645 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559085878eb0 con 0x559085944310 2023-06-23T18:46:27.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.905+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559085944310 0x559085946720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.906+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559085944310 0x559085946720 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34aad01ba1def3c5 server_cookie=c512d27719271609 in_seq=0 out_seq=0 2023-06-23T18:46:27.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.906+0000 7fcb1dffb700 10 client.26630 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:46:27.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.906+0000 7fcb1dffb700 1 -- 172.21.15.2:0/667879645 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559085878eb0 con 0x559085944310 2023-06-23T18:46:27.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.906+0000 7fcb1dffb700 10 client.26630 handle_command_reply: tid=0 2023-06-23T18:46:27.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.906+0000 7fcb48546b80 1 client.26630 shutdown 2023-06-23T18:46:27.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 -- 172.21.15.2:0/667879645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 msgr2=0x5590859389f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x5590859389f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcb2401d710 tx=0x7fcb24004750 comp rx=0 tx=0).stop 2023-06-23T18:46:27.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 -- 172.21.15.2:0/667879645 shutdown_connections 2023-06-23T18:46:27.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559085944310 0x559085946720 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 0x55908593fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 0x5590859384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb1dffb700 0 client.26630 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:46:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 --2- 172.21.15.2:0/667879645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x5590859389f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 -- 172.21.15.2:0/667879645 >> 172.21.15.2:0/667879645 conn(0x5590859243f0 msgr2=0x559085928c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 -- 172.21.15.2:0/667879645 shutdown_connections 2023-06-23T18:46:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 1 -- 172.21.15.2:0/667879645 wait complete. 2023-06-23T18:46:27.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.907+0000 7fcb48546b80 20 client.26630 trim_cache size 0 max 16384 2023-06-23T18:46:27.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.929+0000 7fcb48546b80 1 Processor -- start 2023-06-23T18:46:27.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb48546b80 1 -- start start 2023-06-23T18:46:27.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592ec80 0x559085937190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x5590859384b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb48546b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590859393b0 con 0x559085930000 2023-06-23T18:46:27.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb48546b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559085937960 con 0x55908592f5d0 2023-06-23T18:46:27.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb48546b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908592b050 con 0x55908592ec80 2023-06-23T18:46:27.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a15d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592ec80 0x559085937190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3995c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a15d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592ec80 0x559085937190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58634/0 (socket says 172.21.15.2:58634) 2023-06-23T18:46:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3995c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48888/0 (socket says 172.21.15.2:48888) 2023-06-23T18:46:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a15d700 1 -- 172.21.15.2:0/1700111687 learned_addr learned my addr 172.21.15.2:0/1700111687 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a95e700 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x5590859384b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a15d700 1 -- 172.21.15.2:0/1700111687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 msgr2=0x55908593f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a15d700 1 -- 172.21.15.2:0/1700111687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 msgr2=0x5590859384b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x5590859384b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a15d700 1 -- 172.21.15.2:0/1700111687 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559085696c60 con 0x55908592ec80 2023-06-23T18:46:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3995c700 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:46:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.930+0000 7fcb3a95e700 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x5590859384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:46:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.931+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592ec80 0x559085937190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb30000c00 tx=0x7fcb30009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4de0acc47b83945 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.931+0000 7fcb295ad700 1 -- 172.21.15.2:0/1700111687 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3001d040 con 0x55908592ec80 2023-06-23T18:46:27.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.931+0000 7fcb295ad700 1 -- 172.21.15.2:0/1700111687 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb3000b310 con 0x55908592ec80 2023-06-23T18:46:27.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.931+0000 7fcb295ad700 1 -- 172.21.15.2:0/1700111687 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3000b710 con 0x55908592ec80 2023-06-23T18:46:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 -- 172.21.15.2:0/1700111687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592ec80 msgr2=0x559085937190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592ec80 0x559085937190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb30000c00 tx=0x7fcb30009310 comp rx=0 tx=0).stop 2023-06-23T18:46:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 -- 172.21.15.2:0/1700111687 shutdown_connections 2023-06-23T18:46:27.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x559085930000 0x5590859384b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55908592ec80 0x559085937190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 --2- 172.21.15.2:0/1700111687 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 -- 172.21.15.2:0/1700111687 >> 172.21.15.2:0/1700111687 conn(0x5590859243f0 msgr2=0x55908594b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 -- 172.21.15.2:0/1700111687 shutdown_connections 2023-06-23T18:46:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 -- 172.21.15.2:0/1700111687 wait complete. 2023-06-23T18:46:27.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.932+0000 7fcb48546b80 1 -- start start 2023-06-23T18:46:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb48546b80 1 Processor -- start 2023-06-23T18:46:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 0x55908593b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb48546b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 0x55908593ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb48546b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908593f190 con 0x55908592ec80 2023-06-23T18:46:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb48546b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908593f300 con 0x55908592f5d0 2023-06-23T18:46:27.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb48546b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55908593f4c0 con 0x559085930000 2023-06-23T18:46:27.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb48546b80 10 client.0 _ll_register_callbacks cb 0x559085825260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:46:27.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb3995c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb3a95e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 0x55908593ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb3995c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48896/0 (socket says 172.21.15.2:48896) 2023-06-23T18:46:27.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb3995c700 1 -- 172.21.15.2:0/954990219 learned_addr learned my addr 172.21.15.2:0/954990219 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:27.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 0x55908593b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.933+0000 7fcb3995c700 1 -- 172.21.15.2:0/954990219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 msgr2=0x55908593ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:27.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb3995c700 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 0x55908593ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb3995c700 1 -- 172.21.15.2:0/954990219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 msgr2=0x55908593b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:27.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb3995c700 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 0x55908593b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb3995c700 1 -- 172.21.15.2:0/954990219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590856adde0 con 0x55908592f5d0 2023-06-23T18:46:27.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 0x55908593b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:46:27.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb3995c700 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593bc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcb24010d70 tx=0x7fcb24009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7574f18d4b6afc21 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:27.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:46:27.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb1dffb700 1 -- 172.21.15.2:0/954990219 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24004a30 con 0x55908592f5d0 2023-06-23T18:46:27.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb48546b80 1 -- 172.21.15.2:0/954990219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55908583b050 con 0x55908592f5d0 2023-06-23T18:46:27.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb1dffb700 1 -- 172.21.15.2:0/954990219 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcb24004b90 con 0x55908592f5d0 2023-06-23T18:46:27.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb1dffb700 1 -- 172.21.15.2:0/954990219 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb24004d50 con 0x55908592f5d0 2023-06-23T18:46:27.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb48546b80 10 client.26636 fetch_fsmap learned FSMap version 41 2023-06-23T18:46:27.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.934+0000 7fcb48546b80 1 -- 172.21.15.2:0/954990219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7fcb341a8830 con 0x55908592f5d0 2023-06-23T18:46:27.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb1dffb700 1 -- 172.21.15.2:0/954990219 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55908583b050 con 0x55908592f5d0 2023-06-23T18:46:27.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb1dffb700 1 -- 172.21.15.2:0/954990219 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcb2403a030 con 0x55908592f5d0 2023-06-23T18:46:27.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb48546b80 10 client.26636 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:46:27.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb48546b80 10 client.26636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:46:27.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb48546b80 20 client.26636 populate_metadata read hostname 'smithi002' 2023-06-23T18:46:27.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb48546b80 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5590859a1930 0x5590859a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:27.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb48546b80 4 client.26636 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:46:27.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb48546b80 1 -- 172.21.15.2:0/954990219 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5590856cef70 con 0x5590859a1930 2023-06-23T18:46:27.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.935+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5590859a1930 0x5590859a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:27.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.936+0000 7fcb1dffb700 10 client.26636 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:46:27.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.936+0000 7fcb3a15d700 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5590859a1930 0x5590859a3d40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6be067cf0428f5c server_cookie=7deca3fefe54ae1f in_seq=0 out_seq=0 2023-06-23T18:46:27.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.936+0000 7fcb1dffb700 1 -- 172.21.15.2:0/954990219 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5590856cef70 con 0x5590859a1930 2023-06-23T18:46:27.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.936+0000 7fcb1dffb700 10 client.26636 handle_command_reply: tid=0 2023-06-23T18:46:27.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.936+0000 7fcb48546b80 1 client.26636 shutdown 2023-06-23T18:46:27.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.936+0000 7fcb48546b80 1 -- 172.21.15.2:0/954990219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 msgr2=0x55908593bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593bc30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fcb24010d70 tx=0x7fcb24009920 comp rx=0 tx=0).stop 2023-06-23T18:46:27.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 -- 172.21.15.2:0/954990219 shutdown_connections 2023-06-23T18:46:27.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5590859a1930 0x5590859a3d40 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559085930000 0x55908593ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55908592ec80 0x55908593b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 --2- 172.21.15.2:0/954990219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55908592f5d0 0x55908593bc30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 -- 172.21.15.2:0/954990219 >> 172.21.15.2:0/954990219 conn(0x5590859243f0 msgr2=0x559085937190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:27.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb1dffb700 0 client.26636 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:46:27.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 -- 172.21.15.2:0/954990219 shutdown_connections 2023-06-23T18:46:27.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 1 -- 172.21.15.2:0/954990219 wait complete. 2023-06-23T18:46:27.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb48546b80 20 client.26636 trim_cache size 0 max 16384 2023-06-23T18:46:27.954 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:46:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:46:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:46:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:46:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:46:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:46:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:46:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:46:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:46:27.956 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:46:27.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcb200a11d0 msgr2=0x7fcb200a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcb200a11d0 0x7fcb200a3680 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcb24005950 tx=0x7fcb240058e0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb34100600 msgr2=0x7fcb341a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb34100600 0x7fcb341a7be0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c00efc0 tx=0x7fcb2c00c3a0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/3938262677 shutdown_connections 2023-06-23T18:46:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb34100600 0x7fcb341a7be0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcb200a11d0 0x7fcb200a3680 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb340fea80 0x7fcb3406ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 --2- 172.21.15.2:0/3938262677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb340ffc70 0x7fcb3406f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.937+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/3938262677 >> 172.21.15.2:0/3938262677 conn(0x7fcb340fa0e0 msgr2=0x7fcb340fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.938+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/3938262677 shutdown_connections 2023-06-23T18:46:27.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:27.938+0000 7fcb3c3c1700 1 -- 172.21.15.2:0/3938262677 wait complete. 2023-06-23T18:46:27.961 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:46:28.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:28 smithi002 ceph-mon[95947]: pgmap v1029: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.7 MiB/s wr, 795 op/s 2023-06-23T18:46:28.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:28 smithi082 ceph-mon[97889]: pgmap v1029: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.7 MiB/s wr, 795 op/s 2023-06-23T18:46:28.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:28 smithi017 ceph-mon[97472]: pgmap v1029: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.7 MiB/s wr, 795 op/s 2023-06-23T18:46:30.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:30 smithi002 ceph-mon[95947]: pgmap v1030: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 798 op/s 2023-06-23T18:46:30.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:30 smithi082 ceph-mon[97889]: pgmap v1030: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 798 op/s 2023-06-23T18:46:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:30 smithi017 ceph-mon[97472]: pgmap v1030: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.1 MiB/s wr, 798 op/s 2023-06-23T18:46:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 760.0 s, 316.0 tps, lat 1188.184 ms stddev 3800.985 2023-06-23T18:46:32.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:32 smithi002 ceph-mon[95947]: pgmap v1031: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.1 MiB/s wr, 789 op/s 2023-06-23T18:46:32.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:32 smithi082 ceph-mon[97889]: pgmap v1031: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.1 MiB/s wr, 789 op/s 2023-06-23T18:46:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:32 smithi017 ceph-mon[97472]: pgmap v1031: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.1 MiB/s wr, 789 op/s 2023-06-23T18:46:34.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:34 smithi002 ceph-mon[95947]: pgmap v1032: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 831 op/s 2023-06-23T18:46:34.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:34 smithi082 ceph-mon[97889]: pgmap v1032: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 831 op/s 2023-06-23T18:46:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:34 smithi017 ceph-mon[97472]: pgmap v1032: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 831 op/s 2023-06-23T18:46:36.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:36 smithi002 ceph-mon[95947]: pgmap v1033: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 823 op/s 2023-06-23T18:46:36.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:36 smithi082 ceph-mon[97889]: pgmap v1033: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 823 op/s 2023-06-23T18:46:36.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:36 smithi017 ceph-mon[97472]: pgmap v1033: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.2 MiB/s wr, 823 op/s 2023-06-23T18:46:37.028 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 765.0 s, 309.5 tps, lat 323.741 ms stddev 67.784 2023-06-23T18:46:37.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:46:37.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:37.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:37 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:37.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:46:37.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:37.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:37 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:37.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:46:37.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:37.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:37 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:38.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:38 smithi002 ceph-mon[95947]: pgmap v1034: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.0 MiB/s wr, 820 op/s 2023-06-23T18:46:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:38.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:38 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:38 smithi082 ceph-mon[97889]: pgmap v1034: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.0 MiB/s wr, 820 op/s 2023-06-23T18:46:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:38.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:38 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:38 smithi017 ceph-mon[97472]: pgmap v1034: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.0 MiB/s wr, 820 op/s 2023-06-23T18:46:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:38.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:38 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:39.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:46:39.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:46:39.802 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:39.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:46:39.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:46:39.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:39.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:46:39.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:46:39.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:46:40.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:40 smithi002 ceph-mon[95947]: pgmap v1035: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.3 MiB/s wr, 843 op/s 2023-06-23T18:46:40.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:40 smithi082 ceph-mon[97889]: pgmap v1035: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.3 MiB/s wr, 843 op/s 2023-06-23T18:46:40.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:40 smithi017 ceph-mon[97472]: pgmap v1035: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.3 MiB/s wr, 843 op/s 2023-06-23T18:46:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 770.0 s, 248.5 tps, lat 398.265 ms stddev 80.861 2023-06-23T18:46:42.814 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:42 smithi002 ceph-mon[95947]: pgmap v1036: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 880 op/s 2023-06-23T18:46:42.855 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:42 smithi082 ceph-mon[97889]: pgmap v1036: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 880 op/s 2023-06-23T18:46:42.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:42 smithi017 ceph-mon[97472]: pgmap v1036: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.3 MiB/s wr, 880 op/s 2023-06-23T18:46:44.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:44 smithi002 ceph-mon[95947]: pgmap v1037: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 896 op/s 2023-06-23T18:46:44.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:44 smithi082 ceph-mon[97889]: pgmap v1037: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 896 op/s 2023-06-23T18:46:44.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:44 smithi017 ceph-mon[97472]: pgmap v1037: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 896 op/s 2023-06-23T18:46:46.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:46 smithi002 ceph-mon[95947]: pgmap v1038: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 844 op/s 2023-06-23T18:46:46.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:46 smithi082 ceph-mon[97889]: pgmap v1038: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 844 op/s 2023-06-23T18:46:46.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:46 smithi017 ceph-mon[97472]: pgmap v1038: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 844 op/s 2023-06-23T18:46:47.027 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 775.0 s, 187.7 tps, lat 376.727 ms stddev 145.813 2023-06-23T18:46:48.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:48 smithi002 ceph-mon[95947]: pgmap v1039: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.5 MiB/s wr, 841 op/s 2023-06-23T18:46:48.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:48 smithi082 ceph-mon[97889]: pgmap v1039: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.5 MiB/s wr, 841 op/s 2023-06-23T18:46:48.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:48 smithi017 ceph-mon[97472]: pgmap v1039: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.5 MiB/s wr, 841 op/s 2023-06-23T18:46:50.803 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:50 smithi002 ceph-mon[95947]: pgmap v1040: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 838 op/s 2023-06-23T18:46:50.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:50 smithi082 ceph-mon[97889]: pgmap v1040: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 838 op/s 2023-06-23T18:46:50.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:50 smithi017 ceph-mon[97472]: pgmap v1040: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 838 op/s 2023-06-23T18:46:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 780.0 s, 307.3 tps, lat 424.174 ms stddev 716.353 2023-06-23T18:46:52.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:52 smithi002 ceph-mon[95947]: pgmap v1041: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.3 MiB/s wr, 764 op/s 2023-06-23T18:46:52.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:52 smithi082 ceph-mon[97889]: pgmap v1041: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.3 MiB/s wr, 764 op/s 2023-06-23T18:46:52.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:52 smithi017 ceph-mon[97472]: pgmap v1041: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.3 MiB/s wr, 764 op/s 2023-06-23T18:46:54.800 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:54 smithi002 ceph-mon[95947]: pgmap v1042: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 721 op/s 2023-06-23T18:46:54.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:54 smithi082 ceph-mon[97889]: pgmap v1042: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 721 op/s 2023-06-23T18:46:54.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:54 smithi017 ceph-mon[97472]: pgmap v1042: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.6 MiB/s wr, 721 op/s 2023-06-23T18:46:56.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:56 smithi082 ceph-mon[97889]: pgmap v1043: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 685 op/s 2023-06-23T18:46:56.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:56 smithi017 ceph-mon[97472]: pgmap v1043: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 685 op/s 2023-06-23T18:46:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 785.0 s, 327.0 tps, lat 305.129 ms stddev 70.218 2023-06-23T18:46:57.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:56 smithi002 ceph-mon[95947]: pgmap v1043: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.1 MiB/s wr, 685 op/s 2023-06-23T18:46:57.961 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:46:58.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.130+0000 7f22cdb65700 1 -- 172.21.15.2:0/89638787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8102f80 msgr2=0x7f22c81033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.130+0000 7f22cdb65700 1 --2- 172.21.15.2:0/89638787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8102f80 0x7f22c81033d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009990 tx=0x7f22b0009ca0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.130+0000 7f22cdb65700 1 -- 172.21.15.2:0/89638787 shutdown_connections 2023-06-23T18:46:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.130+0000 7f22cdb65700 1 --2- 172.21.15.2:0/89638787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22c8103910 0x7f22c810a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.130+0000 7f22cdb65700 1 --2- 172.21.15.2:0/89638787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c8101d90 0x7f22c81021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.130+0000 7f22cdb65700 1 --2- 172.21.15.2:0/89638787 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8102f80 0x7f22c81033d0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.130+0000 7f22cdb65700 1 -- 172.21.15.2:0/89638787 >> 172.21.15.2:0/89638787 conn(0x7f22c80fd3b0 msgr2=0x7f22c80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:58.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.130+0000 7f22cdb65700 1 -- 172.21.15.2:0/89638787 shutdown_connections 2023-06-23T18:46:58.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.131+0000 7f22cdb65700 1 -- 172.21.15.2:0/89638787 wait complete. 2023-06-23T18:46:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.131+0000 7f22cdb65700 1 Processor -- start 2023-06-23T18:46:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.131+0000 7f22cdb65700 1 -- start start 2023-06-23T18:46:58.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22cdb65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8101d90 0x7f22c81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22cdb65700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22c8102f80 0x7f22c8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22cdb65700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c8103910 0x7f22c819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22cdb65700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c819f570 con 0x7f22c8101d90 2023-06-23T18:46:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22cdb65700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c819f6b0 con 0x7f22c8102f80 2023-06-23T18:46:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22cdb65700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c819f840 con 0x7f22c8103910 2023-06-23T18:46:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8101d90 0x7f22c81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8101d90 0x7f22c81979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58328/0 (socket says 172.21.15.2:58328) 2023-06-23T18:46:58.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c6ffd700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22c8102f80 0x7f22c8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c77fe700 1 -- 172.21.15.2:0/1261207470 learned_addr learned my addr 172.21.15.2:0/1261207470 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:58.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c7fff700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c8103910 0x7f22c819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c6ffd700 1 -- 172.21.15.2:0/1261207470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c8103910 msgr2=0x7f22c819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c6ffd700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c8103910 0x7f22c819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c6ffd700 1 -- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8101d90 msgr2=0x7f22c81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c6ffd700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8101d90 0x7f22c81979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c6ffd700 1 -- 172.21.15.2:0/1261207470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b00095f0 con 0x7f22c8102f80 2023-06-23T18:46:58.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.132+0000 7f22c6ffd700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22c8102f80 0x7f22c8197f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22b0000c00 tx=0x7f22b0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:58.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.133+0000 7f22c4ff9700 1 -- 172.21.15.2:0/1261207470 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b001d070 con 0x7f22c8102f80 2023-06-23T18:46:58.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.133+0000 7f22cdb65700 1 -- 172.21.15.2:0/1261207470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c819fac0 con 0x7f22c8102f80 2023-06-23T18:46:58.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.133+0000 7f22cdb65700 1 -- 172.21.15.2:0/1261207470 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c819ff40 con 0x7f22c8102f80 2023-06-23T18:46:58.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.133+0000 7f22c4ff9700 1 -- 172.21.15.2:0/1261207470 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22b000bd50 con 0x7f22c8102f80 2023-06-23T18:46:58.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.133+0000 7f22c4ff9700 1 -- 172.21.15.2:0/1261207470 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0022cb0 con 0x7f22c8102f80 2023-06-23T18:46:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.136+0000 7f22c4ff9700 1 -- 172.21.15.2:0/1261207470 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f22b0022470 con 0x7f22c8102f80 2023-06-23T18:46:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.137+0000 7f22c4ff9700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f22b40a1150 0x7f22b40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.137+0000 7f22c4ff9700 1 -- 172.21.15.2:0/1261207470 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f22b00cdb00 con 0x7f22c8102f80 2023-06-23T18:46:58.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.137+0000 7f22c77fe700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f22b40a1150 0x7f22b40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.144 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22d9ceab80 1 Processor -- start 2023-06-23T18:46:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22d9ceab80 1 -- start start 2023-06-23T18:46:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071bc80 0x55955071c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 0x55955071ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071d000 0x55955071d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550620cb0 con 0x55955071bc80 2023-06-23T18:46:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550673a40 con 0x55955071c5d0 2023-06-23T18:46:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955066ced0 con 0x55955071d000 2023-06-23T18:46:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22c77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071bc80 0x55955071c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22c77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071bc80 0x55955071c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58336/0 (socket says 172.21.15.2:58336) 2023-06-23T18:46:58.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22c77fe700 1 -- 172.21.15.2:0/504657146 learned_addr learned my addr 172.21.15.2:0/504657146 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22c77fe700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f22b40a1150 0x7f22b40a3600 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f22b8005950 tx=0x7f22b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.138+0000 7f22c7fff700 1 --2- 172.21.15.2:0/504657146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071d000 0x55955071d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22c77fe700 1 -- 172.21.15.2:0/504657146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071d000 msgr2=0x55955071d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22c77fe700 1 --2- 172.21.15.2:0/504657146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071d000 0x55955071d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22c77fe700 1 -- 172.21.15.2:0/504657146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 msgr2=0x55955071ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22c77fe700 1 --2- 172.21.15.2:0/504657146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 0x55955071ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22c77fe700 1 -- 172.21.15.2:0/504657146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595504ee630 con 0x55955071bc80 2023-06-23T18:46:58.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22c77fe700 1 --2- 172.21.15.2:0/504657146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071bc80 0x55955071c090 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f22b800e4b0 tx=0x7f22b8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b0054cf598acb4 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:58.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22bdffb700 1 -- 172.21.15.2:0/504657146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b800ee50 con 0x55955071bc80 2023-06-23T18:46:58.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22bdffb700 1 -- 172.21.15.2:0/504657146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22b8021070 con 0x55955071bc80 2023-06-23T18:46:58.148 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.139+0000 7f22bdffb700 1 -- 172.21.15.2:0/504657146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b801d410 con 0x55955071bc80 2023-06-23T18:46:58.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 -- 172.21.15.2:0/504657146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071bc80 msgr2=0x55955071c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/504657146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071bc80 0x55955071c090 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f22b800e4b0 tx=0x7f22b8014000 comp rx=0 tx=0).stop 2023-06-23T18:46:58.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 -- 172.21.15.2:0/504657146 shutdown_connections 2023-06-23T18:46:58.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/504657146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071d000 0x55955071d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/504657146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071bc80 0x55955071c090 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/504657146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 0x55955071ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 -- 172.21.15.2:0/504657146 >> 172.21.15.2:0/504657146 conn(0x5595507113f0 msgr2=0x559550716520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:58.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 -- 172.21.15.2:0/504657146 shutdown_connections 2023-06-23T18:46:58.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.140+0000 7f22d9ceab80 1 -- 172.21.15.2:0/504657146 wait complete. 2023-06-23T18:46:58.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 1 -- start start 2023-06-23T18:46:58.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 1 Processor -- start 2023-06-23T18:46:58.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550725590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 0x559550725ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x55955072cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955060cac0 con 0x55955071d000 2023-06-23T18:46:58.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955060c7c0 con 0x55955071c5d0 2023-06-23T18:46:58.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595504f0920 con 0x55955071bc80 2023-06-23T18:46:58.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22d9ceab80 10 client.0 _ll_register_callbacks cb 0x559550612260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:46:58.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22c77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550725590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22c7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x55955072cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22c77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550725590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58680/0 (socket says 172.21.15.2:58680) 2023-06-23T18:46:58.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22c77fe700 1 -- 172.21.15.2:0/3914608032 learned_addr learned my addr 172.21.15.2:0/3914608032 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:58.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.141+0000 7f22c6ffd700 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 0x559550725ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22c7fff700 1 -- 172.21.15.2:0/3914608032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 msgr2=0x559550725590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:58.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22c7fff700 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550725590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22c7fff700 1 -- 172.21.15.2:0/3914608032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 msgr2=0x559550725ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22c7fff700 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 0x559550725ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22c7fff700 1 -- 172.21.15.2:0/3914608032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b8017070 con 0x55955071d000 2023-06-23T18:46:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22c7fff700 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x55955072cb10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f22c0011e90 tx=0x7f22c0014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee8dcc2d84a6a62 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:46:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22a6ffd700 1 -- 172.21.15.2:0/3914608032 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c000e040 con 0x55955071d000 2023-06-23T18:46:58.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22a6ffd700 1 -- 172.21.15.2:0/3914608032 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f22c00093a0 con 0x55955071d000 2023-06-23T18:46:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22d9ceab80 1 -- 172.21.15.2:0/3914608032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55955070ebb0 con 0x55955071d000 2023-06-23T18:46:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22a6ffd700 1 -- 172.21.15.2:0/3914608032 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22c0004db0 con 0x55955071d000 2023-06-23T18:46:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22d9ceab80 10 client.17136 fetch_fsmap learned FSMap version 41 2023-06-23T18:46:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22d9ceab80 1 -- 172.21.15.2:0/3914608032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55955049ade0 con 0x55955071d000 2023-06-23T18:46:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.142+0000 7f22a6ffd700 1 -- 172.21.15.2:0/3914608032 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55955070ebb0 con 0x55955071d000 2023-06-23T18:46:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22a6ffd700 1 -- 172.21.15.2:0/3914608032 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f22c001c030 con 0x55955071d000 2023-06-23T18:46:58.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22d9ceab80 10 client.17136 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:46:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22d9ceab80 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:46:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22d9ceab80 20 client.17136 populate_metadata read hostname 'smithi002' 2023-06-23T18:46:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559550731530 0x559550733940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22d9ceab80 4 client.17136 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:46:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22d9ceab80 1 -- 172.21.15.2:0/3914608032 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559550665eb0 con 0x559550731530 2023-06-23T18:46:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22c77fe700 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559550731530 0x559550733940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22a6ffd700 10 client.17136 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:46:58.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.143+0000 7f22c77fe700 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559550731530 0x559550733940 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c816b44474d28708 server_cookie=e800b49deca4cfc9 in_seq=0 out_seq=0 2023-06-23T18:46:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22a6ffd700 1 -- 172.21.15.2:0/3914608032 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559550665eb0 con 0x559550731530 2023-06-23T18:46:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22a6ffd700 10 client.17136 handle_command_reply: tid=0 2023-06-23T18:46:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 client.17136 shutdown 2023-06-23T18:46:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 -- 172.21.15.2:0/3914608032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 msgr2=0x55955072cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x55955072cb10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f22c0011e90 tx=0x7f22c0014600 comp rx=0 tx=0).stop 2023-06-23T18:46:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 -- 172.21.15.2:0/3914608032 shutdown_connections 2023-06-23T18:46:58.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559550731530 0x559550733940 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x55955072cb10 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22a6ffd700 0 client.17136 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:46:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550725590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/3914608032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071c5d0 0x559550725ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 -- 172.21.15.2:0/3914608032 >> 172.21.15.2:0/3914608032 conn(0x5595507113f0 msgr2=0x559550715c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 -- 172.21.15.2:0/3914608032 shutdown_connections 2023-06-23T18:46:58.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 1 -- 172.21.15.2:0/3914608032 wait complete. 2023-06-23T18:46:58.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.144+0000 7f22d9ceab80 20 client.17136 trim_cache size 0 max 16384 2023-06-23T18:46:58.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.171+0000 7f22d9ceab80 1 Processor -- start 2023-06-23T18:46:58.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.171+0000 7f22d9ceab80 1 -- start start 2023-06-23T18:46:58.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.171+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550723310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.171+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071c5d0 0x559550723850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.171+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071d000 0x55955072cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595507240b0 con 0x55955071c5d0 2023-06-23T18:46:58.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550717d50 con 0x55955071d000 2023-06-23T18:46:58.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550717fb0 con 0x55955071bc80 2023-06-23T18:46:58.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071c5d0 0x559550723850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c77fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550723310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071c5d0 0x559550723850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58352/0 (socket says 172.21.15.2:58352) 2023-06-23T18:46:58.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 -- 172.21.15.2:0/2141873761 learned_addr learned my addr 172.21.15.2:0/2141873761 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:58.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c7fff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071d000 0x55955072cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 -- 172.21.15.2:0/2141873761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 msgr2=0x559550723310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:58.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 --2- 172.21.15.2:0/2141873761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550723310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 -- 172.21.15.2:0/2141873761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071d000 msgr2=0x55955072cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:58.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 --2- 172.21.15.2:0/2141873761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071d000 0x55955072cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 -- 172.21.15.2:0/2141873761 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559550483c60 con 0x55955071c5d0 2023-06-23T18:46:58.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c7fff700 1 --2- 172.21.15.2:0/2141873761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071d000 0x55955072cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:46:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22c6ffd700 1 --2- 172.21.15.2:0/2141873761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071c5d0 0x559550723850 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f22b00979d0 tx=0x7f22b009ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8152844a9dc1d2 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22be7fc700 1 -- 172.21.15.2:0/2141873761 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b009af60 con 0x55955071c5d0 2023-06-23T18:46:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22be7fc700 1 -- 172.21.15.2:0/2141873761 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22b009ef90 con 0x55955071c5d0 2023-06-23T18:46:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.172+0000 7f22be7fc700 1 -- 172.21.15.2:0/2141873761 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b00a30f0 con 0x55955071c5d0 2023-06-23T18:46:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 -- 172.21.15.2:0/2141873761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071c5d0 msgr2=0x559550723850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/2141873761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071c5d0 0x559550723850 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f22b00979d0 tx=0x7f22b009ab90 comp rx=0 tx=0).stop 2023-06-23T18:46:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 -- 172.21.15.2:0/2141873761 shutdown_connections 2023-06-23T18:46:58.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/2141873761 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071d000 0x55955072cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/2141873761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071bc80 0x559550723310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/2141873761 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071c5d0 0x559550723850 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 -- 172.21.15.2:0/2141873761 >> 172.21.15.2:0/2141873761 conn(0x5595507113f0 msgr2=0x55955072d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 -- 172.21.15.2:0/2141873761 shutdown_connections 2023-06-23T18:46:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 -- 172.21.15.2:0/2141873761 wait complete. 2023-06-23T18:46:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.173+0000 7f22d9ceab80 1 -- start start 2023-06-23T18:46:58.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22d9ceab80 1 Processor -- start 2023-06-23T18:46:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071bc80 0x55955072c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071c5d0 0x55955072c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22d9ceab80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x559550728ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550729530 con 0x55955071d000 2023-06-23T18:46:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595507296a0 con 0x55955071bc80 2023-06-23T18:46:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22d9ceab80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550729860 con 0x55955071c5d0 2023-06-23T18:46:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22d9ceab80 10 client.0 _ll_register_callbacks cb 0x559550612260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:46:58.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x559550728ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c6ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071c5d0 0x55955072c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x559550728ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58360/0 (socket says 172.21.15.2:58360) 2023-06-23T18:46:58.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c7fff700 1 -- 172.21.15.2:0/4182037061 learned_addr learned my addr 172.21.15.2:0/4182037061 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:46:58.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c77fe700 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071bc80 0x55955072c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c7fff700 1 -- 172.21.15.2:0/4182037061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071c5d0 msgr2=0x55955072c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:58.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c7fff700 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071c5d0 0x55955072c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c7fff700 1 -- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071bc80 msgr2=0x55955072c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:46:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c7fff700 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071bc80 0x55955072c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.174+0000 7f22c7fff700 1 -- 172.21.15.2:0/4182037061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55955049ade0 con 0x55955071d000 2023-06-23T18:46:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22c77fe700 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071bc80 0x55955072c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:46:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22c7fff700 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x559550728ea0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f22c0004160 tx=0x7f22c0003c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6aa1642569245e0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:46:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22a6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:46:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22a6ffd700 1 -- 172.21.15.2:0/4182037061 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c000e070 con 0x55955071d000 2023-06-23T18:46:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22a6ffd700 1 -- 172.21.15.2:0/4182037061 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f22c0014d60 con 0x55955071d000 2023-06-23T18:46:58.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22d9ceab80 1 -- 172.21.15.2:0/4182037061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22b8021070 con 0x55955071d000 2023-06-23T18:46:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22a6ffd700 1 -- 172.21.15.2:0/4182037061 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22c0009410 con 0x55955071d000 2023-06-23T18:46:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22d9ceab80 10 client.17142 fetch_fsmap learned FSMap version 41 2023-06-23T18:46:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.175+0000 7f22d9ceab80 1 -- 172.21.15.2:0/4182037061 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f22c819ff40 con 0x55955071d000 2023-06-23T18:46:58.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22a6ffd700 1 -- 172.21.15.2:0/4182037061 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22b8021070 con 0x55955071d000 2023-06-23T18:46:58.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22a6ffd700 1 -- 172.21.15.2:0/4182037061 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f22c0014320 con 0x55955071d000 2023-06-23T18:46:58.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22d9ceab80 10 client.17142 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:46:58.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22d9ceab80 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:46:58.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22d9ceab80 20 client.17142 populate_metadata read hostname 'smithi002' 2023-06-23T18:46:58.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559550786950 0x559550788d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:46:58.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22d9ceab80 4 client.17142 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:46:58.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22d9ceab80 1 -- 172.21.15.2:0/4182037061 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559550628050 con 0x559550786950 2023-06-23T18:46:58.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.176+0000 7f22c77fe700 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559550786950 0x559550788d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:46:58.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.177+0000 7f22c77fe700 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559550786950 0x559550788d60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ba93d74d976fb0 server_cookie=c224ea59829ae201 in_seq=0 out_seq=0 2023-06-23T18:46:58.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.177+0000 7f22a6ffd700 10 client.17142 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:46:58.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.177+0000 7f22a6ffd700 1 -- 172.21.15.2:0/4182037061 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559550628050 con 0x559550786950 2023-06-23T18:46:58.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.177+0000 7f22a6ffd700 10 client.17142 handle_command_reply: tid=0 2023-06-23T18:46:58.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.177+0000 7f22d9ceab80 1 client.17142 shutdown 2023-06-23T18:46:58.193 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:46:58.193 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:46:58.193 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:46:58.193 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:46:58.193 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:46:58.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:46:58.194 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:46:58.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:46:58.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:46:58.194 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:46:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 -- 172.21.15.2:0/4182037061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 msgr2=0x559550728ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x559550728ea0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f22c0004160 tx=0x7f22c0003c30 comp rx=0 tx=0).stop 2023-06-23T18:46:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 -- 172.21.15.2:0/4182037061 shutdown_connections 2023-06-23T18:46:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x559550786950 0x559550788d60 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55955071d000 0x559550728ea0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55955071bc80 0x55955072c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 --2- 172.21.15.2:0/4182037061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55955071c5d0 0x55955072c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 -- 172.21.15.2:0/4182037061 >> 172.21.15.2:0/4182037061 conn(0x5595507113f0 msgr2=0x559550723310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22a6ffd700 0 client.17142 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:46:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 -- 172.21.15.2:0/4182037061 shutdown_connections 2023-06-23T18:46:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 1 -- 172.21.15.2:0/4182037061 wait complete. 2023-06-23T18:46:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.179+0000 7f22d9ceab80 20 client.17142 trim_cache size 0 max 16384 2023-06-23T18:46:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 -- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f22b40a1150 msgr2=0x7f22b40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f22b40a1150 0x7f22b40a3600 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f22b8005950 tx=0x7f22b8009260 comp rx=0 tx=0).stop 2023-06-23T18:46:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 -- 172.21.15.2:0/1261207470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22c8102f80 msgr2=0x7f22c8197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:46:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22c8102f80 0x7f22c8197f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22b0000c00 tx=0x7f22b0004730 comp rx=0 tx=0).stop 2023-06-23T18:46:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 -- 172.21.15.2:0/1261207470 shutdown_connections 2023-06-23T18:46:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22c8103910 0x7f22c819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22c8101d90 0x7f22c81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f22b40a1150 0x7f22b40a3600 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 --2- 172.21.15.2:0/1261207470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22c8102f80 0x7f22c8197f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:46:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 -- 172.21.15.2:0/1261207470 >> 172.21.15.2:0/1261207470 conn(0x7f22c80fd3b0 msgr2=0x7f22c80ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:46:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 -- 172.21.15.2:0/1261207470 shutdown_connections 2023-06-23T18:46:58.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:46:58.183+0000 7f22cdb65700 1 -- 172.21.15.2:0/1261207470 wait complete. 2023-06-23T18:46:58.203 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:46:58.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:46:58 smithi082 ceph-mon[97889]: pgmap v1044: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.0 MiB/s wr, 716 op/s 2023-06-23T18:46:58.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:46:58 smithi017 ceph-mon[97472]: pgmap v1044: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.0 MiB/s wr, 716 op/s 2023-06-23T18:46:59.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:46:58 smithi002 ceph-mon[95947]: pgmap v1044: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.0 MiB/s wr, 716 op/s 2023-06-23T18:47:00.849 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:00 smithi082 ceph-mon[97889]: pgmap v1045: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2023-06-23T18:47:00.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:00 smithi017 ceph-mon[97472]: pgmap v1045: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2023-06-23T18:47:01.056 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:00 smithi002 ceph-mon[95947]: pgmap v1045: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.1 MiB/s wr, 748 op/s 2023-06-23T18:47:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 790.0 s, 318.8 tps, lat 312.563 ms stddev 56.221 2023-06-23T18:47:02.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:02 smithi082 ceph-mon[97889]: pgmap v1046: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 795 op/s 2023-06-23T18:47:02.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:02 smithi017 ceph-mon[97472]: pgmap v1046: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 795 op/s 2023-06-23T18:47:03.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:02 smithi002 ceph-mon[95947]: pgmap v1046: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.5 MiB/s wr, 795 op/s 2023-06-23T18:47:04.851 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:04 smithi082 ceph-mon[97889]: pgmap v1047: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 890 op/s 2023-06-23T18:47:04.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:04 smithi017 ceph-mon[97472]: pgmap v1047: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 890 op/s 2023-06-23T18:47:05.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:04 smithi002 ceph-mon[95947]: pgmap v1047: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 890 op/s 2023-06-23T18:47:06.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:06 smithi082 ceph-mon[97889]: pgmap v1048: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 908 op/s 2023-06-23T18:47:06.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:06 smithi017 ceph-mon[97472]: pgmap v1048: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 908 op/s 2023-06-23T18:47:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 795.0 s, 326.2 tps, lat 308.344 ms stddev 55.556 2023-06-23T18:47:07.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:06 smithi002 ceph-mon[95947]: pgmap v1048: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 11 MiB/s wr, 908 op/s 2023-06-23T18:47:08.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:08 smithi082 ceph-mon[97889]: pgmap v1049: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 916 op/s 2023-06-23T18:47:08.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:08 smithi017 ceph-mon[97472]: pgmap v1049: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 916 op/s 2023-06-23T18:47:09.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:08 smithi002 ceph-mon[95947]: pgmap v1049: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.8 MiB/s wr, 916 op/s 2023-06-23T18:47:10.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:10 smithi082 ceph-mon[97889]: pgmap v1050: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.0 MiB/s wr, 917 op/s 2023-06-23T18:47:10.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:10 smithi017 ceph-mon[97472]: pgmap v1050: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.0 MiB/s wr, 917 op/s 2023-06-23T18:47:11.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:10 smithi002 ceph-mon[95947]: pgmap v1050: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.0 MiB/s wr, 917 op/s 2023-06-23T18:47:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 800.0 s, 326.8 tps, lat 303.387 ms stddev 61.554 2023-06-23T18:47:12.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:12 smithi082 ceph-mon[97889]: pgmap v1051: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 922 op/s 2023-06-23T18:47:12.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:12 smithi017 ceph-mon[97472]: pgmap v1051: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 922 op/s 2023-06-23T18:47:13.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:12 smithi002 ceph-mon[95947]: pgmap v1051: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 922 op/s 2023-06-23T18:47:14.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:14 smithi082 ceph-mon[97889]: pgmap v1052: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 918 op/s 2023-06-23T18:47:14.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:14 smithi017 ceph-mon[97472]: pgmap v1052: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 918 op/s 2023-06-23T18:47:15.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:14 smithi002 ceph-mon[95947]: pgmap v1052: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 918 op/s 2023-06-23T18:47:16.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:16 smithi082 ceph-mon[97889]: pgmap v1053: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.4 MiB/s wr, 876 op/s 2023-06-23T18:47:16.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:16 smithi017 ceph-mon[97472]: pgmap v1053: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.4 MiB/s wr, 876 op/s 2023-06-23T18:47:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 805.0 s, 360.8 tps, lat 279.674 ms stddev 47.057 2023-06-23T18:47:17.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:16 smithi002 ceph-mon[95947]: pgmap v1053: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.4 MiB/s wr, 876 op/s 2023-06-23T18:47:18.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:18 smithi082 ceph-mon[97889]: pgmap v1054: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.2 MiB/s wr, 885 op/s 2023-06-23T18:47:18.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:18 smithi017 ceph-mon[97472]: pgmap v1054: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.2 MiB/s wr, 885 op/s 2023-06-23T18:47:19.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:18 smithi002 ceph-mon[95947]: pgmap v1054: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.2 MiB/s wr, 885 op/s 2023-06-23T18:47:20.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:20 smithi082 ceph-mon[97889]: pgmap v1055: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 921 op/s 2023-06-23T18:47:20.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:20 smithi017 ceph-mon[97472]: pgmap v1055: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 921 op/s 2023-06-23T18:47:21.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:20 smithi002 ceph-mon[95947]: pgmap v1055: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 921 op/s 2023-06-23T18:47:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 810.0 s, 324.6 tps, lat 306.493 ms stddev 46.825 2023-06-23T18:47:22.854 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:22 smithi082 ceph-mon[97889]: pgmap v1056: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 914 op/s 2023-06-23T18:47:22.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:22 smithi017 ceph-mon[97472]: pgmap v1056: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 914 op/s 2023-06-23T18:47:23.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:22 smithi002 ceph-mon[95947]: pgmap v1056: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 914 op/s 2023-06-23T18:47:24.848 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:24 smithi082 ceph-mon[97889]: pgmap v1057: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:47:24.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:24 smithi017 ceph-mon[97472]: pgmap v1057: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:47:25.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:24 smithi002 ceph-mon[95947]: pgmap v1057: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:47:26.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:26 smithi017 ceph-mon[97472]: pgmap v1058: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.1 MiB/s wr, 899 op/s 2023-06-23T18:47:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 815.0 s, 338.8 tps, lat 297.154 ms stddev 49.563 2023-06-23T18:47:27.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:26 smithi002 ceph-mon[95947]: pgmap v1058: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.1 MiB/s wr, 899 op/s 2023-06-23T18:47:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:26 smithi082 ceph-mon[97889]: pgmap v1058: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.1 MiB/s wr, 899 op/s 2023-06-23T18:47:28.204 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:47:28.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 -- 172.21.15.2:0/1521803870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8105850 msgr2=0x7f96f8107cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 --2- 172.21.15.2:0/1521803870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8105850 0x7f96f8107cb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f96f000b0a0 tx=0x7f96f000b3b0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 -- 172.21.15.2:0/1521803870 shutdown_connections 2023-06-23T18:47:28.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 --2- 172.21.15.2:0/1521803870 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8105850 0x7f96f8107cb0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 --2- 172.21.15.2:0/1521803870 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f8100610 0x7f96f81029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 --2- 172.21.15.2:0/1521803870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96f8102f30 0x7f96f8105310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 -- 172.21.15.2:0/1521803870 >> 172.21.15.2:0/1521803870 conn(0x7f96f80fa090 msgr2=0x7f96f80fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:28.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 -- 172.21.15.2:0/1521803870 shutdown_connections 2023-06-23T18:47:28.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.550+0000 7f96ff668700 1 -- 172.21.15.2:0/1521803870 wait complete. 2023-06-23T18:47:28.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96ff668700 1 Processor -- start 2023-06-23T18:47:28.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96ff668700 1 -- start start 2023-06-23T18:47:28.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96ff668700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8100610 0x7f96f80fff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96ff668700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f8102f30 0x7f96f80fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96ff668700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96f8105850 0x7f96f80feb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96ff668700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80ff2a0 con 0x7f96f8100610 2023-06-23T18:47:28.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96ff668700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80ff410 con 0x7f96f8102f30 2023-06-23T18:47:28.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96ff668700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f819f980 con 0x7f96f8105850 2023-06-23T18:47:28.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fd404700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8100610 0x7f96f80fff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fd404700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8100610 0x7f96f80fff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58372/0 (socket says 172.21.15.2:58372) 2023-06-23T18:47:28.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fd404700 1 -- 172.21.15.2:0/1998284974 learned_addr learned my addr 172.21.15.2:0/1998284974 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:28.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fdc05700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96f8105850 0x7f96f80feb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fd404700 1 -- 172.21.15.2:0/1998284974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96f8105850 msgr2=0x7f96f80feb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fd404700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96f8105850 0x7f96f80feb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fd404700 1 -- 172.21.15.2:0/1998284974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f8102f30 msgr2=0x7f96f80fe5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fd404700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f8102f30 0x7f96f80fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.551+0000 7f96fd404700 1 -- 172.21.15.2:0/1998284974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0009d00 con 0x7f96f8100610 2023-06-23T18:47:28.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.552+0000 7f96fd404700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8100610 0x7f96f80fff80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f96f400b530 tx=0x7f96f400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:28.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.552+0000 7f96ee7fc700 1 -- 172.21.15.2:0/1998284974 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4011830 con 0x7f96f8100610 2023-06-23T18:47:28.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.552+0000 7f96ff668700 1 -- 172.21.15.2:0/1998284974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f819fc60 con 0x7f96f8100610 2023-06-23T18:47:28.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.552+0000 7f96ee7fc700 1 -- 172.21.15.2:0/1998284974 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96f4011e20 con 0x7f96f8100610 2023-06-23T18:47:28.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.552+0000 7f96ee7fc700 1 -- 172.21.15.2:0/1998284974 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f400f2e0 con 0x7f96f8100610 2023-06-23T18:47:28.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.552+0000 7f96ff668700 1 -- 172.21.15.2:0/1998284974 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f81a0140 con 0x7f96f8100610 2023-06-23T18:47:28.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.555+0000 7f96ee7fc700 1 -- 172.21.15.2:0/1998284974 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f96f4011990 con 0x7f96f8100610 2023-06-23T18:47:28.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.556+0000 7f96ee7fc700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f96e40a1150 0x7f96e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.556+0000 7f96ee7fc700 1 -- 172.21.15.2:0/1998284974 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96f40cc450 con 0x7f96f8100610 2023-06-23T18:47:28.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.557+0000 7f970b7edb80 1 Processor -- start 2023-06-23T18:47:28.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.557+0000 7f970b7edb80 1 -- start start 2023-06-23T18:47:28.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.557+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd867090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.557+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd8675d0 0x561cfd867a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.557+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd868470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.557+0000 7f970b7edb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd76bcb0 con 0x561cfd868000 2023-06-23T18:47:28.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.557+0000 7f970b7edb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd7bea40 con 0x561cfd8675d0 2023-06-23T18:47:28.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.557+0000 7f970b7edb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd7b7ed0 con 0x561cfd866c80 2023-06-23T18:47:28.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fd404700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd867090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fdc05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd868470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fd404700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd867090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58716/0 (socket says 172.21.15.2:58716) 2023-06-23T18:47:28.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fd404700 1 -- 172.21.15.2:0/2031924823 learned_addr learned my addr 172.21.15.2:0/2031924823 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:28.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fdc05700 1 -- 172.21.15.2:0/2031924823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 msgr2=0x561cfd867090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:28.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fdc05700 1 --2- 172.21.15.2:0/2031924823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd867090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fdc05700 1 -- 172.21.15.2:0/2031924823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd8675d0 msgr2=0x561cfd867a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:47:28.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fdc05700 1 --2- 172.21.15.2:0/2031924823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd8675d0 0x561cfd867a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fdc05700 1 -- 172.21.15.2:0/2031924823 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cfd639630 con 0x561cfd868000 2023-06-23T18:47:28.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96fdc05700 1 --2- 172.21.15.2:0/2031924823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd868470 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f96f000b070 tx=0x7f96f00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325797e4600c8c7d server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:28.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96e37fe700 1 -- 172.21.15.2:0/2031924823 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000e040 con 0x561cfd868000 2023-06-23T18:47:28.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96e37fe700 1 -- 172.21.15.2:0/2031924823 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96f00076e0 con 0x561cfd868000 2023-06-23T18:47:28.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.558+0000 7f96e37fe700 1 -- 172.21.15.2:0/2031924823 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f0003e40 con 0x561cfd868000 2023-06-23T18:47:28.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.559+0000 7f970b7edb80 1 -- 172.21.15.2:0/2031924823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 msgr2=0x561cfd868470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.559+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2031924823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd868470 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f96f000b070 tx=0x7f96f00093f0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.559+0000 7f970b7edb80 1 -- 172.21.15.2:0/2031924823 shutdown_connections 2023-06-23T18:47:28.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.559+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2031924823 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd868470 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.559+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2031924823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd867090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.559+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2031924823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd8675d0 0x561cfd867a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.559+0000 7f970b7edb80 1 -- 172.21.15.2:0/2031924823 >> 172.21.15.2:0/2031924823 conn(0x561cfd85c3f0 msgr2=0x561cfd861520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:28.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.561+0000 7f970b7edb80 1 -- 172.21.15.2:0/2031924823 shutdown_connections 2023-06-23T18:47:28.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.561+0000 7f96fcc03700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f96e40a1150 0x7f96e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.562+0000 7f96fcc03700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f96e40a1150 0x7f96e40a3600 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x561cfd863650 tx=0x7f96e80095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:28.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.564+0000 7f970b7edb80 1 -- 172.21.15.2:0/2031924823 wait complete. 2023-06-23T18:47:28.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 1 -- start start 2023-06-23T18:47:28.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 1 Processor -- start 2023-06-23T18:47:28.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd866c80 0x561cfd878e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd868000 0x561cfd879340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd87e370 0x561cfd880750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd757ac0 con 0x561cfd866c80 2023-06-23T18:47:28.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd7577c0 con 0x561cfd868000 2023-06-23T18:47:28.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd63b920 con 0x561cfd87e370 2023-06-23T18:47:28.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f970b7edb80 10 client.0 _ll_register_callbacks cb 0x561cfd75d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:47:28.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f96fcc03700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd868000 0x561cfd879340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f96fcc03700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd868000 0x561cfd879340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:48978/0 (socket says 172.21.15.2:48978) 2023-06-23T18:47:28.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f96fcc03700 1 -- 172.21.15.2:0/2105664306 learned_addr learned my addr 172.21.15.2:0/2105664306 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:28.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.565+0000 7f96fd404700 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd866c80 0x561cfd878e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96fdc05700 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd87e370 0x561cfd880750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96fcc03700 1 -- 172.21.15.2:0/2105664306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd87e370 msgr2=0x561cfd880750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:28.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96fcc03700 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd87e370 0x561cfd880750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96fcc03700 1 -- 172.21.15.2:0/2105664306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd866c80 msgr2=0x561cfd878e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:28.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96fcc03700 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd866c80 0x561cfd878e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96fcc03700 1 -- 172.21.15.2:0/2105664306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f81088a0 con 0x561cfd868000 2023-06-23T18:47:28.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96fdc05700 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd87e370 0x561cfd880750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:47:28.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96fcc03700 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd868000 0x561cfd879340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f96e800ee70 tx=0x7f96e8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d77d52916b9a371 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:28.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:47:28.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96e0ff9700 1 -- 172.21.15.2:0/2105664306 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8004220 con 0x561cfd868000 2023-06-23T18:47:28.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f970b7edb80 1 -- 172.21.15.2:0/2105664306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cfd859bb0 con 0x561cfd868000 2023-06-23T18:47:28.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96e0ff9700 1 -- 172.21.15.2:0/2105664306 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96e8009750 con 0x561cfd868000 2023-06-23T18:47:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f96e0ff9700 1 -- 172.21.15.2:0/2105664306 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e8010e60 con 0x561cfd868000 2023-06-23T18:47:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f970b7edb80 10 client.26654 fetch_fsmap learned FSMap version 41 2023-06-23T18:47:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.566+0000 7f970b7edb80 1 -- 172.21.15.2:0/2105664306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561cfd5e5de0 con 0x561cfd868000 2023-06-23T18:47:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f96e0ff9700 1 -- 172.21.15.2:0/2105664306 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561cfd859bb0 con 0x561cfd868000 2023-06-23T18:47:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f96e0ff9700 1 -- 172.21.15.2:0/2105664306 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f96e8018070 con 0x561cfd868000 2023-06-23T18:47:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f970b7edb80 10 client.26654 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:47:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f970b7edb80 10 client.26654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:47:28.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f970b7edb80 20 client.26654 populate_metadata read hostname 'smithi002' 2023-06-23T18:47:28.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561cfd874600 0x561cfd876a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f970b7edb80 4 client.26654 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:47:28.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f970b7edb80 1 -- 172.21.15.2:0/2105664306 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cfd7b0eb0 con 0x561cfd874600 2023-06-23T18:47:28.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f96fd404700 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561cfd874600 0x561cfd876a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f96fd404700 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561cfd874600 0x561cfd876a10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09697901b50a7 server_cookie=6b01eed0ebc5b16c in_seq=0 out_seq=0 2023-06-23T18:47:28.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.567+0000 7f96e0ff9700 10 client.26654 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:47:28.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f96e0ff9700 1 -- 172.21.15.2:0/2105664306 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561cfd7b0eb0 con 0x561cfd874600 2023-06-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f96e0ff9700 10 client.26654 handle_command_reply: tid=0 2023-06-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 client.26654 shutdown 2023-06-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 -- 172.21.15.2:0/2105664306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd868000 msgr2=0x561cfd879340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd868000 0x561cfd879340 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f96e800ee70 tx=0x7f96e8004750 comp rx=0 tx=0).stop 2023-06-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 -- 172.21.15.2:0/2105664306 shutdown_connections 2023-06-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561cfd874600 0x561cfd876a10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd87e370 0x561cfd880750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f96e0ff9700 0 client.26654 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:47:28.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd866c80 0x561cfd878e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 --2- 172.21.15.2:0/2105664306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd868000 0x561cfd879340 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 -- 172.21.15.2:0/2105664306 >> 172.21.15.2:0/2105664306 conn(0x561cfd85c3f0 msgr2=0x561cfd860c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:28.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 -- 172.21.15.2:0/2105664306 shutdown_connections 2023-06-23T18:47:28.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.568+0000 7f970b7edb80 1 -- 172.21.15.2:0/2105664306 wait complete. 2023-06-23T18:47:28.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.569+0000 7f970b7edb80 20 client.26654 trim_cache size 0 max 16384 2023-06-23T18:47:28.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f970b7edb80 1 Processor -- start 2023-06-23T18:47:28.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f970b7edb80 1 -- start start 2023-06-23T18:47:28.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd866c80 0x561cfd895cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd874600 0x561cfd86f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f970b7edb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd862980 con 0x561cfd868000 2023-06-23T18:47:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f970b7edb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd85bf70 con 0x561cfd866c80 2023-06-23T18:47:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f970b7edb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd86bae0 con 0x561cfd874600 2023-06-23T18:47:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f96fcc03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f96fcc03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58394/0 (socket says 172.21.15.2:58394) 2023-06-23T18:47:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f96fcc03700 1 -- 172.21.15.2:0/3991099204 learned_addr learned my addr 172.21.15.2:0/3991099204 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f96fd404700 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd866c80 0x561cfd895cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.593+0000 7f96fdc05700 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd874600 0x561cfd86f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96fd404700 1 -- 172.21.15.2:0/3991099204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd874600 msgr2=0x561cfd86f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96fd404700 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd874600 0x561cfd86f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96fd404700 1 -- 172.21.15.2:0/3991099204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 msgr2=0x561cfd89ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96fd404700 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96fd404700 1 -- 172.21.15.2:0/3991099204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cfd5cec60 con 0x561cfd866c80 2023-06-23T18:47:28.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96fcc03700 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:47:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96fd404700 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd866c80 0x561cfd895cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f96f40997f0 tx=0x7f96f40c9e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e9c1254b68595e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96e3fff700 1 -- 172.21.15.2:0/3991099204 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4099860 con 0x561cfd866c80 2023-06-23T18:47:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96e3fff700 1 -- 172.21.15.2:0/3991099204 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96f40b7e90 con 0x561cfd866c80 2023-06-23T18:47:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.594+0000 7f96e3fff700 1 -- 172.21.15.2:0/3991099204 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4097c00 con 0x561cfd866c80 2023-06-23T18:47:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 -- 172.21.15.2:0/3991099204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd866c80 msgr2=0x561cfd895cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd866c80 0x561cfd895cc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f96f40997f0 tx=0x7f96f40c9e20 comp rx=0 tx=0).stop 2023-06-23T18:47:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 -- 172.21.15.2:0/3991099204 shutdown_connections 2023-06-23T18:47:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd874600 0x561cfd86f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd866c80 0x561cfd895cc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3991099204 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 -- 172.21.15.2:0/3991099204 >> 172.21.15.2:0/3991099204 conn(0x561cfd85c3f0 msgr2=0x561cfd85d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 -- 172.21.15.2:0/3991099204 shutdown_connections 2023-06-23T18:47:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 -- 172.21.15.2:0/3991099204 wait complete. 2023-06-23T18:47:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 -- start start 2023-06-23T18:47:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.595+0000 7f970b7edb80 1 Processor -- start 2023-06-23T18:47:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd89e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f970b7edb80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd874600 0x561cfd89b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f970b7edb80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd89bc30 con 0x561cfd868000 2023-06-23T18:47:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f970b7edb80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd89bda0 con 0x561cfd874600 2023-06-23T18:47:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f970b7edb80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfd89bf60 con 0x561cfd866c80 2023-06-23T18:47:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f970b7edb80 10 client.0 _ll_register_callbacks cb 0x561cfd75d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:47:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58400/0 (socket says 172.21.15.2:58400) 2023-06-23T18:47:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fd404700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd89e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 -- 172.21.15.2:0/3616243669 learned_addr learned my addr 172.21.15.2:0/3616243669 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fdc05700 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd874600 0x561cfd89b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 -- 172.21.15.2:0/3616243669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 msgr2=0x561cfd89e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd89e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 -- 172.21.15.2:0/3616243669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd874600 msgr2=0x561cfd89b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd874600 0x561cfd89b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 -- 172.21.15.2:0/3616243669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cfd5e5de0 con 0x561cfd868000 2023-06-23T18:47:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96fcc03700 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ed90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f96e800ee70 tx=0x7f96e801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ec18ff9b0f218 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.596+0000 7f96e0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:47:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.597+0000 7f96e0ff9700 1 -- 172.21.15.2:0/3616243669 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8034070 con 0x561cfd868000 2023-06-23T18:47:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.597+0000 7f970b7edb80 1 -- 172.21.15.2:0/3616243669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96f00076e0 con 0x561cfd868000 2023-06-23T18:47:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.597+0000 7f96e0ff9700 1 -- 172.21.15.2:0/3616243669 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96e802f460 con 0x561cfd868000 2023-06-23T18:47:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.597+0000 7f96e0ff9700 1 -- 172.21.15.2:0/3616243669 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96e8009890 con 0x561cfd868000 2023-06-23T18:47:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.597+0000 7f970b7edb80 10 client.17160 fetch_fsmap learned FSMap version 41 2023-06-23T18:47:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.597+0000 7f970b7edb80 1 -- 172.21.15.2:0/3616243669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7f96f81a0140 con 0x561cfd868000 2023-06-23T18:47:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f96e0ff9700 1 -- 172.21.15.2:0/3616243669 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f96f00076e0 con 0x561cfd868000 2023-06-23T18:47:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f96e0ff9700 1 -- 172.21.15.2:0/3616243669 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f96e8010430 con 0x561cfd868000 2023-06-23T18:47:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f970b7edb80 10 client.17160 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:47:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f970b7edb80 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:47:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f970b7edb80 20 client.17160 populate_metadata read hostname 'smithi002' 2023-06-23T18:47:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561cfd87e370 0x561cfd8e4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f970b7edb80 4 client.17160 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:47:28.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f970b7edb80 1 -- 172.21.15.2:0/3616243669 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561cfd773050 con 0x561cfd87e370 2023-06-23T18:47:28.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f96fd404700 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561cfd87e370 0x561cfd8e4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:28.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f96fd404700 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561cfd87e370 0x561cfd8e4cc0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe95981e2422a6f server_cookie=5b250ba74dad7902 in_seq=0 out_seq=0 2023-06-23T18:47:28.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.598+0000 7f96e0ff9700 10 client.17160 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:47:28.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.599+0000 7f96e0ff9700 1 -- 172.21.15.2:0/3616243669 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561cfd773050 con 0x561cfd87e370 2023-06-23T18:47:28.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.599+0000 7f96e0ff9700 10 client.17160 handle_command_reply: tid=0 2023-06-23T18:47:28.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.599+0000 7f970b7edb80 1 client.17160 shutdown 2023-06-23T18:47:28.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.599+0000 7f970b7edb80 1 -- 172.21.15.2:0/3616243669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 msgr2=0x561cfd89ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.599+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ed90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f96e800ee70 tx=0x7f96e801cc80 comp rx=0 tx=0).stop 2023-06-23T18:47:28.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 1 -- 172.21.15.2:0/3616243669 shutdown_connections 2023-06-23T18:47:28.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x561cfd87e370 0x561cfd8e4cc0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x561cfd874600 0x561cfd89b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561cfd866c80 0x561cfd89e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f96e0ff9700 0 client.17160 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:47:28.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 1 --2- 172.21.15.2:0/3616243669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x561cfd868000 0x561cfd89ed90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 1 -- 172.21.15.2:0/3616243669 >> 172.21.15.2:0/3616243669 conn(0x561cfd85c3f0 msgr2=0x561cfd86f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:28.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 1 -- 172.21.15.2:0/3616243669 shutdown_connections 2023-06-23T18:47:28.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 1 -- 172.21.15.2:0/3616243669 wait complete. 2023-06-23T18:47:28.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f970b7edb80 20 client.17160 trim_cache size 0 max 16384 2023-06-23T18:47:28.612 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:47:28.612 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:47:28.612 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:47:28.613 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:47:28.613 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:47:28.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:47:28.613 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:47:28.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:47:28.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:47:28.613 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:47:28.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f96ff668700 1 -- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f96e40a1150 msgr2=0x7f96e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f96ff668700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f96e40a1150 0x7f96e40a3600 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x561cfd863650 tx=0x7f96e80095c0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f96ff668700 1 -- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8100610 msgr2=0x7f96f80fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:28.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.601+0000 7f96ff668700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8100610 0x7f96f80fff80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f96f400b530 tx=0x7f96f400b840 comp rx=0 tx=0).stop 2023-06-23T18:47:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.602+0000 7f96ff668700 1 -- 172.21.15.2:0/1998284974 shutdown_connections 2023-06-23T18:47:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.602+0000 7f96ff668700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f96f8105850 0x7f96f80feb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.602+0000 7f96ff668700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f8100610 0x7f96f80fff80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.602+0000 7f96ff668700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f96e40a1150 0x7f96e40a3600 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.602+0000 7f96ff668700 1 --2- 172.21.15.2:0/1998284974 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96f8102f30 0x7f96f80fe5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.602+0000 7f96ff668700 1 -- 172.21.15.2:0/1998284974 >> 172.21.15.2:0/1998284974 conn(0x7f96f80fa090 msgr2=0x7f96f8101270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:28.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.602+0000 7f96ff668700 1 -- 172.21.15.2:0/1998284974 shutdown_connections 2023-06-23T18:47:28.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:28.602+0000 7f96ff668700 1 -- 172.21.15.2:0/1998284974 wait complete. 2023-06-23T18:47:28.621 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:47:28.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:28 smithi017 ceph-mon[97472]: pgmap v1059: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 912 op/s 2023-06-23T18:47:29.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:28 smithi002 ceph-mon[95947]: pgmap v1059: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 912 op/s 2023-06-23T18:47:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:28 smithi082 ceph-mon[97889]: pgmap v1059: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 912 op/s 2023-06-23T18:47:30.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:30 smithi017 ceph-mon[97472]: pgmap v1060: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.9 MiB/s wr, 944 op/s 2023-06-23T18:47:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:30 smithi002 ceph-mon[95947]: pgmap v1060: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.9 MiB/s wr, 944 op/s 2023-06-23T18:47:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:30 smithi082 ceph-mon[97889]: pgmap v1060: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.9 MiB/s wr, 944 op/s 2023-06-23T18:47:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 820.0 s, 326.6 tps, lat 305.358 ms stddev 59.134 2023-06-23T18:47:32.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:32 smithi017 ceph-mon[97472]: pgmap v1061: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.6 MiB/s wr, 912 op/s 2023-06-23T18:47:33.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:32 smithi002 ceph-mon[95947]: pgmap v1061: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.6 MiB/s wr, 912 op/s 2023-06-23T18:47:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:32 smithi082 ceph-mon[97889]: pgmap v1061: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.6 MiB/s wr, 912 op/s 2023-06-23T18:47:34.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:34 smithi017 ceph-mon[97472]: pgmap v1062: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.8 MiB/s wr, 919 op/s 2023-06-23T18:47:35.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:34 smithi002 ceph-mon[95947]: pgmap v1062: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.8 MiB/s wr, 919 op/s 2023-06-23T18:47:35.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:34 smithi082 ceph-mon[97889]: pgmap v1062: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.8 MiB/s wr, 919 op/s 2023-06-23T18:47:36.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:36 smithi017 ceph-mon[97472]: pgmap v1063: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.5 MiB/s wr, 856 op/s 2023-06-23T18:47:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 825.0 s, 300.0 tps, lat 330.858 ms stddev 58.001 2023-06-23T18:47:37.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:36 smithi002 ceph-mon[95947]: pgmap v1063: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.5 MiB/s wr, 856 op/s 2023-06-23T18:47:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:36 smithi082 ceph-mon[97889]: pgmap v1063: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.5 MiB/s wr, 856 op/s 2023-06-23T18:47:38.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:38 smithi017 ceph-mon[97472]: pgmap v1064: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.8 MiB/s wr, 893 op/s 2023-06-23T18:47:39.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:38 smithi002 ceph-mon[95947]: pgmap v1064: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.8 MiB/s wr, 893 op/s 2023-06-23T18:47:39.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:38 smithi082 ceph-mon[97889]: pgmap v1064: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.8 MiB/s wr, 893 op/s 2023-06-23T18:47:39.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:39 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:47:40.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:39 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:47:40.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:39 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:47:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:40 smithi017 ceph-mon[97472]: pgmap v1065: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 928 op/s 2023-06-23T18:47:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:47:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:47:40.941 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:47:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:40 smithi002 ceph-mon[95947]: pgmap v1065: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 928 op/s 2023-06-23T18:47:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:47:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:47:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:47:41.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:40 smithi082 ceph-mon[97889]: pgmap v1065: 97 pgs: 97 active+clean; 8.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 10 MiB/s wr, 928 op/s 2023-06-23T18:47:41.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:47:41.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:47:41.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:47:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 830.0 s, 268.2 tps, lat 375.896 ms stddev 100.295 2023-06-23T18:47:42.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:42 smithi017 ceph-mon[97472]: pgmap v1066: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.9 MiB/s wr, 923 op/s 2023-06-23T18:47:43.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:42 smithi002 ceph-mon[95947]: pgmap v1066: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.9 MiB/s wr, 923 op/s 2023-06-23T18:47:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:42 smithi082 ceph-mon[97889]: pgmap v1066: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.9 MiB/s wr, 923 op/s 2023-06-23T18:47:44.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:44 smithi017 ceph-mon[97472]: pgmap v1067: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-06-23T18:47:45.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:44 smithi002 ceph-mon[95947]: pgmap v1067: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-06-23T18:47:45.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:44 smithi082 ceph-mon[97889]: pgmap v1067: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-06-23T18:47:46.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:46 smithi017 ceph-mon[97472]: pgmap v1068: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-06-23T18:47:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 835.0 s, 336.4 tps, lat 296.304 ms stddev 66.417 2023-06-23T18:47:47.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:46 smithi002 ceph-mon[95947]: pgmap v1068: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-06-23T18:47:47.102 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:46 smithi082 ceph-mon[97889]: pgmap v1068: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 916 op/s 2023-06-23T18:47:48.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:48 smithi017 ceph-mon[97472]: pgmap v1069: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.9 MiB/s wr, 921 op/s 2023-06-23T18:47:49.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:48 smithi002 ceph-mon[95947]: pgmap v1069: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.9 MiB/s wr, 921 op/s 2023-06-23T18:47:49.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:48 smithi082 ceph-mon[97889]: pgmap v1069: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.9 MiB/s wr, 921 op/s 2023-06-23T18:47:50.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:50 smithi017 ceph-mon[97472]: pgmap v1070: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 946 op/s 2023-06-23T18:47:51.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:50 smithi002 ceph-mon[95947]: pgmap v1070: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 946 op/s 2023-06-23T18:47:51.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:50 smithi082 ceph-mon[97889]: pgmap v1070: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.7 MiB/s wr, 946 op/s 2023-06-23T18:47:52.029 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 840.0 s, 368.0 tps, lat 273.328 ms stddev 51.984 2023-06-23T18:47:52.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:52 smithi017 ceph-mon[97472]: pgmap v1071: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 935 op/s 2023-06-23T18:47:53.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:52 smithi002 ceph-mon[95947]: pgmap v1071: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 935 op/s 2023-06-23T18:47:53.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:52 smithi082 ceph-mon[97889]: pgmap v1071: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 935 op/s 2023-06-23T18:47:54.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:54 smithi017 ceph-mon[97472]: pgmap v1072: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.9 MiB/s wr, 917 op/s 2023-06-23T18:47:55.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:54 smithi002 ceph-mon[95947]: pgmap v1072: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.9 MiB/s wr, 917 op/s 2023-06-23T18:47:55.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:54 smithi082 ceph-mon[97889]: pgmap v1072: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.9 MiB/s wr, 917 op/s 2023-06-23T18:47:56.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:56 smithi017 ceph-mon[97472]: pgmap v1073: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 861 op/s 2023-06-23T18:47:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 845.0 s, 366.0 tps, lat 271.717 ms stddev 53.491 2023-06-23T18:47:57.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:56 smithi002 ceph-mon[95947]: pgmap v1073: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 861 op/s 2023-06-23T18:47:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:56 smithi082 ceph-mon[97889]: pgmap v1073: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.5 MiB/s wr, 861 op/s 2023-06-23T18:47:58.623 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:47:58.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:47:58 smithi017 ceph-mon[97472]: pgmap v1074: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 854 op/s 2023-06-23T18:47:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.936+0000 7fcfcadd1700 1 -- 172.21.15.2:0/1086521361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc40fea90 msgr2=0x7fcfc40feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:58.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.936+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/1086521361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc40fea90 0x7fcfc40feea0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009990 tx=0x7fcfb8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.936+0000 7fcfcadd1700 1 -- 172.21.15.2:0/1086521361 shutdown_connections 2023-06-23T18:47:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.936+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/1086521361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfc4100610 0x7fcfc4106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.936+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/1086521361 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc40fea90 0x7fcfc40feea0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.936+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/1086521361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc40ffc80 0x7fcfc41000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.936+0000 7fcfcadd1700 1 -- 172.21.15.2:0/1086521361 >> 172.21.15.2:0/1086521361 conn(0x7fcfc40fa090 msgr2=0x7fcfc40fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:58.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.937+0000 7fcfcadd1700 1 -- 172.21.15.2:0/1086521361 shutdown_connections 2023-06-23T18:47:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.937+0000 7fcfcadd1700 1 -- 172.21.15.2:0/1086521361 wait complete. 2023-06-23T18:47:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.937+0000 7fcfcadd1700 1 Processor -- start 2023-06-23T18:47:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfcadd1700 1 -- start start 2023-06-23T18:47:58.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfcadd1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc40fea90 0x7fcfc406c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfcadd1700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc40ffc80 0x7fcfc406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfcadd1700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfc4100610 0x7fcfc4071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfcadd1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc4072190 con 0x7fcfc40fea90 2023-06-23T18:47:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfcadd1700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc40722d0 con 0x7fcfc4100610 2023-06-23T18:47:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfcadd1700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc4072460 con 0x7fcfc40ffc80 2023-06-23T18:47:58.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfc936e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfc4100610 0x7fcfc4071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfc936e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfc4100610 0x7fcfc4071bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:49004/0 (socket says 172.21.15.2:49004) 2023-06-23T18:47:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfc936e700 1 -- 172.21.15.2:0/3916464177 learned_addr learned my addr 172.21.15.2:0/3916464177 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc40ffc80 0x7fcfc406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc40fea90 0x7fcfc406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.938+0000 7fcfc936e700 1 -- 172.21.15.2:0/3916464177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc40ffc80 msgr2=0x7fcfc406cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.939+0000 7fcfc936e700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc40ffc80 0x7fcfc406cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.939+0000 7fcfc936e700 1 -- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc40fea90 msgr2=0x7fcfc406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.939+0000 7fcfc936e700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc40fea90 0x7fcfc406c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.939+0000 7fcfc936e700 1 -- 172.21.15.2:0/3916464177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb80095f0 con 0x7fcfc4100610 2023-06-23T18:47:58.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.939+0000 7fcfc936e700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfc4100610 0x7fcfc4071bc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00ef30 tx=0x7fcfbc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.939+0000 7fcfc1ffb700 1 -- 172.21.15.2:0/3916464177 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc018070 con 0x7fcfc4100610 2023-06-23T18:47:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.939+0000 7fcfc1ffb700 1 -- 172.21.15.2:0/3916464177 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfbc00f040 con 0x7fcfc4100610 2023-06-23T18:47:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.939+0000 7fcfc1ffb700 1 -- 172.21.15.2:0/3916464177 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00cbc0 con 0x7fcfc4100610 2023-06-23T18:47:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.940+0000 7fcfcadd1700 1 -- 172.21.15.2:0/3916464177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc410c180 con 0x7fcfc4100610 2023-06-23T18:47:58.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.940+0000 7fcfcadd1700 1 -- 172.21.15.2:0/3916464177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc410c640 con 0x7fcfc4100610 2023-06-23T18:47:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.942+0000 7fcfc1ffb700 1 -- 172.21.15.2:0/3916464177 <== mon.2 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcfbc007500 con 0x7fcfc4100610 2023-06-23T18:47:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.943+0000 7fcfc1ffb700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfb40a11b0 0x7fcfb40a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.943+0000 7fcfc1ffb700 1 -- 172.21.15.2:0/3916464177 <== mon.2 v2:172.21.15.17:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfbc0cc380 con 0x7fcfc4100610 2023-06-23T18:47:58.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.944+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfb40a11b0 0x7fcfb40a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.944+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfb40a11b0 0x7fcfb40a3660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8006010 tx=0x7fcfb80042e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.944+0000 7fcfd6f56b80 1 Processor -- start 2023-06-23T18:47:58.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.944+0000 7fcfd6f56b80 1 -- start start 2023-06-23T18:47:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ef000 0x564f079ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f078f2cb0 con 0x564f079ef000 2023-06-23T18:47:58.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f07945a40 con 0x564f079ee5d0 2023-06-23T18:47:58.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f0793eed0 con 0x564f079edc80 2023-06-23T18:47:58.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc8b6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc8b6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079ee090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58756/0 (socket says 172.21.15.2:58756) 2023-06-23T18:47:58.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/3723158227 learned_addr learned my addr 172.21.15.2:0/3723158227 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:58.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc936e700 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ef000 0x564f079ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/3723158227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 msgr2=0x564f079eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/3723158227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ef000 msgr2=0x564f079ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:58.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ef000 0x564f079ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.945+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/3723158227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f077c0630 con 0x564f079edc80 2023-06-23T18:47:58.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.946+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079ee090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800b3e0 tx=0x7fcfb8020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbacc87d2a8cfa7a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:58.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.946+0000 7fcfaeffd700 1 -- 172.21.15.2:0/3723158227 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8003ce0 con 0x564f079edc80 2023-06-23T18:47:58.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.946+0000 7fcfaeffd700 1 -- 172.21.15.2:0/3723158227 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfb801e040 con 0x564f079edc80 2023-06-23T18:47:58.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.946+0000 7fcfaeffd700 1 -- 172.21.15.2:0/3723158227 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8004900 con 0x564f079edc80 2023-06-23T18:47:58.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.947+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3723158227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 msgr2=0x564f079ee090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:58.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.947+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079ee090 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800b3e0 tx=0x7fcfb8020000 comp rx=0 tx=0).stop 2023-06-23T18:47:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3723158227 shutdown_connections 2023-06-23T18:47:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ef000 0x564f079ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079ee090 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3723158227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3723158227 >> 172.21.15.2:0/3723158227 conn(0x564f079e33f0 msgr2=0x564f079e8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3723158227 shutdown_connections 2023-06-23T18:47:58.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3723158227 wait complete. 2023-06-23T18:47:58.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 -- start start 2023-06-23T18:47:58.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.948+0000 7fcfd6f56b80 1 Processor -- start 2023-06-23T18:47:58.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079f7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ee5d0 0x564f079f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f078deac0 con 0x564f079ee5d0 2023-06-23T18:47:58.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f078de7c0 con 0x564f079ef000 2023-06-23T18:47:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f077c2920 con 0x564f079edc80 2023-06-23T18:47:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfd6f56b80 10 client.0 _ll_register_callbacks cb 0x564f078e4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:47:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ee5d0 0x564f079f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc8b6d700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ee5d0 0x564f079f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58426/0 (socket says 172.21.15.2:58426) 2023-06-23T18:47:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc3fff700 1 -- 172.21.15.2:0/1397112238 learned_addr learned my addr 172.21.15.2:0/1397112238 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:58.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc936e700 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 msgr2=0x564f079feaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079feaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/1397112238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ee5d0 msgr2=0x564f079f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ee5d0 0x564f079f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/1397112238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc4107800 con 0x564f079edc80 2023-06-23T18:47:58.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc936e700 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079feaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:47:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ee5d0 0x564f079f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:47:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079f7520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8000c00 tx=0x7fcfb8005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4bae67341f732ff server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.949+0000 7fcfa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:47:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.950+0000 7fcfa3fff700 1 -- 172.21.15.2:0/1397112238 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8005a00 con 0x564f079edc80 2023-06-23T18:47:58.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.950+0000 7fcfa3fff700 1 -- 172.21.15.2:0/1397112238 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfb800b6b0 con 0x564f079edc80 2023-06-23T18:47:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.950+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/1397112238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564f079e0bb0 con 0x564f079edc80 2023-06-23T18:47:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.950+0000 7fcfa3fff700 1 -- 172.21.15.2:0/1397112238 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfb800b870 con 0x564f079edc80 2023-06-23T18:47:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.950+0000 7fcfd6f56b80 10 client.26908 fetch_fsmap learned FSMap version 41 2023-06-23T18:47:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.950+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/1397112238 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564f0776cde0 con 0x564f079edc80 2023-06-23T18:47:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.950+0000 7fcfa3fff700 1 -- 172.21.15.2:0/1397112238 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564f079e0bb0 con 0x564f079edc80 2023-06-23T18:47:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfa3fff700 1 -- 172.21.15.2:0/1397112238 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcfb800fc60 con 0x564f079edc80 2023-06-23T18:47:58.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfd6f56b80 10 client.26908 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:47:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfd6f56b80 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:47:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfd6f56b80 20 client.26908 populate_metadata read hostname 'smithi002' 2023-06-23T18:47:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f07a02d10 0x564f07a050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfd6f56b80 4 client.26908 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:47:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/1397112238 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564f07937eb0 con 0x564f07a02d10 2023-06-23T18:47:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f07a02d10 0x564f07a050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f07a02d10 0x564f07a050f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60390a3900ed55d server_cookie=5145d7753b72c26e in_seq=0 out_seq=0 2023-06-23T18:47:58.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.951+0000 7fcfa3fff700 10 client.26908 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:47:58.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfa3fff700 1 -- 172.21.15.2:0/1397112238 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564f07937eb0 con 0x564f07a02d10 2023-06-23T18:47:58.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfa3fff700 10 client.26908 handle_command_reply: tid=0 2023-06-23T18:47:58.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 client.26908 shutdown 2023-06-23T18:47:58.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/1397112238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 msgr2=0x564f079f7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:58.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079f7520 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8000c00 tx=0x7fcfb8005f10 comp rx=0 tx=0).stop 2023-06-23T18:47:58.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/1397112238 shutdown_connections 2023-06-23T18:47:58.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f07a02d10 0x564f07a050f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079feaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079edc80 0x564f079f7520 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/1397112238 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079ee5d0 0x564f079f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/1397112238 >> 172.21.15.2:0/1397112238 conn(0x564f079e33f0 msgr2=0x564f079e7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.952+0000 7fcfa3fff700 0 client.26908 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:47:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.953+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/1397112238 shutdown_connections 2023-06-23T18:47:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.953+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/1397112238 wait complete. 2023-06-23T18:47:58.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.953+0000 7fcfd6f56b80 20 client.26908 trim_cache size 0 max 16384 2023-06-23T18:47:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.980+0000 7fcfd6f56b80 1 Processor -- start 2023-06-23T18:47:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.980+0000 7fcfd6f56b80 1 -- start start 2023-06-23T18:47:58.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.980+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079ff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.980+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ee5d0 0x564f079fea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.980+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.980+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f079f70f0 con 0x564f079edc80 2023-06-23T18:47:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.980+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f079f69d0 con 0x564f079ef000 2023-06-23T18:47:58.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.980+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f079f86a0 con 0x564f079ee5d0 2023-06-23T18:47:58.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079ff390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58432/0 (socket says 172.21.15.2:58432) 2023-06-23T18:47:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/2583384208 learned_addr learned my addr 172.21.15.2:0/2583384208 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc936e700 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/2583384208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ee5d0 msgr2=0x564f079fea60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:47:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ee5d0 0x564f079fea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/2583384208 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 msgr2=0x564f079f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079f1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/2583384208 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f079ef6a0 con 0x564f079edc80 2023-06-23T18:47:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc936e700 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:47:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.981+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079ff390 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800e450 tx=0x7fcfb800e3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c4324018719c3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.982+0000 7fcfaf7fe700 1 -- 172.21.15.2:0/2583384208 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8024600 con 0x564f079edc80 2023-06-23T18:47:58.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.982+0000 7fcfaf7fe700 1 -- 172.21.15.2:0/2583384208 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfb8004890 con 0x564f079edc80 2023-06-23T18:47:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.982+0000 7fcfaf7fe700 1 -- 172.21.15.2:0/2583384208 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800bad0 con 0x564f079edc80 2023-06-23T18:47:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/2583384208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 msgr2=0x564f079ff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079ff390 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800e450 tx=0x7fcfb800e3e0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/2583384208 shutdown_connections 2023-06-23T18:47:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ef000 0x564f079f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079ff390 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/2583384208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ee5d0 0x564f079fea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/2583384208 >> 172.21.15.2:0/2583384208 conn(0x564f079e33f0 msgr2=0x564f079f6230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:58.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/2583384208 shutdown_connections 2023-06-23T18:47:58.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/2583384208 wait complete. 2023-06-23T18:47:58.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 -- start start 2023-06-23T18:47:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.983+0000 7fcfd6f56b80 1 Processor -- start 2023-06-23T18:47:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfd6f56b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ef000 0x564f079fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f079fda10 con 0x564f079edc80 2023-06-23T18:47:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f079fdb80 con 0x564f079ee5d0 2023-06-23T18:47:58.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfd6f56b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f079fdd40 con 0x564f079ef000 2023-06-23T18:47:58.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfd6f56b80 10 client.0 _ll_register_callbacks cb 0x564f078e4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:47:58.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079f9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58436/0 (socket says 172.21.15.2:58436) 2023-06-23T18:47:58.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/3280801380 learned_addr learned my addr 172.21.15.2:0/3280801380 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:47:58.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc936e700 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ef000 0x564f079fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/3280801380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ef000 msgr2=0x564f079fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ef000 0x564f079fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 msgr2=0x564f079fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:47:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079fa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 -- 172.21.15.2:0/3280801380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc410c640 con 0x564f079edc80 2023-06-23T18:47:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:47:58.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfc8b6d700 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079f9f70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8005910 tx=0x7fcfb80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a880d9a6a4a1d7a9 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:47:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.984+0000 7fcfa3fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:47:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfa3fff700 1 -- 172.21.15.2:0/3280801380 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8024600 con 0x564f079edc80 2023-06-23T18:47:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfa3fff700 1 -- 172.21.15.2:0/3280801380 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcfb8024760 con 0x564f079edc80 2023-06-23T18:47:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3280801380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfb801e040 con 0x564f079edc80 2023-06-23T18:47:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfa3fff700 1 -- 172.21.15.2:0/3280801380 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfb8024920 con 0x564f079edc80 2023-06-23T18:47:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 10 client.17178 fetch_fsmap learned FSMap version 41 2023-06-23T18:47:58.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3280801380 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x564f0776cde0 con 0x564f079edc80 2023-06-23T18:47:58.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfa3fff700 1 -- 172.21.15.2:0/3280801380 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfb801e040 con 0x564f079edc80 2023-06-23T18:47:58.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfa3fff700 1 -- 172.21.15.2:0/3280801380 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcfb800fc00 con 0x564f079edc80 2023-06-23T18:47:58.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 10 client.17178 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:47:58.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:47:58.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 20 client.17178 populate_metadata read hostname 'smithi002' 2023-06-23T18:47:58.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f07a56d90 0x564f07a591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:47:58.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 4 client.17178 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:47:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.985+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3280801380 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564f078fa050 con 0x564f07a56d90 2023-06-23T18:47:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.986+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f07a56d90 0x564f07a591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:47:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.986+0000 7fcfa3fff700 10 client.17178 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:47:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.986+0000 7fcfc3fff700 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f07a56d90 0x564f07a591a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc6aab258c91e88 server_cookie=d9aa02e0eaa167cc in_seq=0 out_seq=0 2023-06-23T18:47:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.986+0000 7fcfa3fff700 1 -- 172.21.15.2:0/3280801380 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564f078fa050 con 0x564f07a56d90 2023-06-23T18:47:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.986+0000 7fcfa3fff700 10 client.17178 handle_command_reply: tid=0 2023-06-23T18:47:58.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.986+0000 7fcfd6f56b80 1 client.17178 shutdown 2023-06-23T18:47:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3280801380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 msgr2=0x564f079f9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079f9f70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8005910 tx=0x7fcfb80045f0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3280801380 shutdown_connections 2023-06-23T18:47:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x564f07a56d90 0x564f07a591a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564f079ef000 0x564f079fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfa3fff700 0 client.17178 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:47:59.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x564f079edc80 0x564f079f9f70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 --2- 172.21.15.2:0/3280801380 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x564f079ee5d0 0x564f079fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3280801380 >> 172.21.15.2:0/3280801380 conn(0x564f079e33f0 msgr2=0x564f079f5360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3280801380 shutdown_connections 2023-06-23T18:47:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 1 -- 172.21.15.2:0/3280801380 wait complete. 2023-06-23T18:47:59.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfd6f56b80 20 client.17178 trim_cache size 0 max 16384 2023-06-23T18:47:59.002 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:47:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:47:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:47:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:47:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:47:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:47:59.002 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:47:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:47:59.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:47:59.003 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:47:59.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfcadd1700 1 -- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfb40a11b0 msgr2=0x7fcfb40a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:59.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfb40a11b0 0x7fcfb40a3660 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8006010 tx=0x7fcfb80042e0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfcadd1700 1 -- 172.21.15.2:0/3916464177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfc4100610 msgr2=0x7fcfc4071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:47:59.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfc4100610 0x7fcfc4071bc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00ef30 tx=0x7fcfbc00c3a0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfcadd1700 1 -- 172.21.15.2:0/3916464177 shutdown_connections 2023-06-23T18:47:59.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcfb40a11b0 0x7fcfb40a3660 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.987+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcfc4100610 0x7fcfc4071bc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.988+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcfc40fea90 0x7fcfc406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.988+0000 7fcfcadd1700 1 --2- 172.21.15.2:0/3916464177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfc40ffc80 0x7fcfc406cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:47:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.988+0000 7fcfcadd1700 1 -- 172.21.15.2:0/3916464177 >> 172.21.15.2:0/3916464177 conn(0x7fcfc40fa090 msgr2=0x7fcfc410b610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:47:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.992+0000 7fcfcadd1700 1 -- 172.21.15.2:0/3916464177 shutdown_connections 2023-06-23T18:47:59.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:47:58.992+0000 7fcfcadd1700 1 -- 172.21.15.2:0/3916464177 wait complete. 2023-06-23T18:47:59.012 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:47:58 smithi002 ceph-mon[95947]: pgmap v1074: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 854 op/s 2023-06-23T18:47:59.013 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:47:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:47:58 smithi082 ceph-mon[97889]: pgmap v1074: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.3 MiB/s wr, 854 op/s 2023-06-23T18:48:00.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:00 smithi017 ceph-mon[97472]: pgmap v1075: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 879 op/s 2023-06-23T18:48:01.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:00 smithi002 ceph-mon[95947]: pgmap v1075: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 879 op/s 2023-06-23T18:48:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:00 smithi082 ceph-mon[97889]: pgmap v1075: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.9 MiB/s wr, 879 op/s 2023-06-23T18:48:02.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 850.0 s, 284.2 tps, lat 348.422 ms stddev 66.633 2023-06-23T18:48:02.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:02 smithi017 ceph-mon[97472]: pgmap v1076: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 861 op/s 2023-06-23T18:48:03.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:02 smithi002 ceph-mon[95947]: pgmap v1076: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 861 op/s 2023-06-23T18:48:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:02 smithi082 ceph-mon[97889]: pgmap v1076: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 861 op/s 2023-06-23T18:48:04.942 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:04 smithi017 ceph-mon[97472]: pgmap v1077: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 841 op/s 2023-06-23T18:48:05.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:04 smithi002 ceph-mon[95947]: pgmap v1077: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 841 op/s 2023-06-23T18:48:05.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:04 smithi082 ceph-mon[97889]: pgmap v1077: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.8 MiB/s wr, 841 op/s 2023-06-23T18:48:06.939 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:06 smithi017 ceph-mon[97472]: pgmap v1078: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 827 op/s 2023-06-23T18:48:07.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 855.0 s, 308.0 tps, lat 326.635 ms stddev 51.877 2023-06-23T18:48:07.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:06 smithi002 ceph-mon[95947]: pgmap v1078: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 827 op/s 2023-06-23T18:48:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:06 smithi082 ceph-mon[97889]: pgmap v1078: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.2 MiB/s wr, 827 op/s 2023-06-23T18:48:08.940 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:08 smithi017 ceph-mon[97472]: pgmap v1079: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.9 MiB/s wr, 840 op/s 2023-06-23T18:48:09.055 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:08 smithi002 ceph-mon[95947]: pgmap v1079: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.9 MiB/s wr, 840 op/s 2023-06-23T18:48:09.102 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:08 smithi082 ceph-mon[97889]: pgmap v1079: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.9 MiB/s wr, 840 op/s 2023-06-23T18:48:11.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:10 smithi002 ceph-mon[95947]: pgmap v1080: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 875 op/s 2023-06-23T18:48:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:10 smithi082 ceph-mon[97889]: pgmap v1080: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 875 op/s 2023-06-23T18:48:11.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:10 smithi017 ceph-mon[97472]: pgmap v1080: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.5 MiB/s wr, 875 op/s 2023-06-23T18:48:12.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 860.0 s, 310.2 tps, lat 319.872 ms stddev 55.066 2023-06-23T18:48:13.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:12 smithi002 ceph-mon[95947]: pgmap v1081: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 857 op/s 2023-06-23T18:48:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:12 smithi082 ceph-mon[97889]: pgmap v1081: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 857 op/s 2023-06-23T18:48:13.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:12 smithi017 ceph-mon[97472]: pgmap v1081: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.4 MiB/s wr, 857 op/s 2023-06-23T18:48:15.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:14 smithi002 ceph-mon[95947]: pgmap v1082: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 842 op/s 2023-06-23T18:48:15.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:14 smithi082 ceph-mon[97889]: pgmap v1082: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 842 op/s 2023-06-23T18:48:15.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:14 smithi017 ceph-mon[97472]: pgmap v1082: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 8.7 MiB/s wr, 842 op/s 2023-06-23T18:48:17.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 865.0 s, 314.8 tps, lat 318.772 ms stddev 56.246 2023-06-23T18:48:17.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:16 smithi002 ceph-mon[95947]: pgmap v1083: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.0 MiB/s wr, 827 op/s 2023-06-23T18:48:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:16 smithi082 ceph-mon[97889]: pgmap v1083: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.0 MiB/s wr, 827 op/s 2023-06-23T18:48:17.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:16 smithi017 ceph-mon[97472]: pgmap v1083: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.0 MiB/s wr, 827 op/s 2023-06-23T18:48:19.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:18 smithi002 ceph-mon[95947]: pgmap v1084: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.0 MiB/s wr, 836 op/s 2023-06-23T18:48:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:18 smithi082 ceph-mon[97889]: pgmap v1084: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.0 MiB/s wr, 836 op/s 2023-06-23T18:48:19.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:18 smithi017 ceph-mon[97472]: pgmap v1084: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.0 MiB/s wr, 836 op/s 2023-06-23T18:48:21.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:20 smithi002 ceph-mon[95947]: pgmap v1085: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.4 MiB/s wr, 871 op/s 2023-06-23T18:48:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:20 smithi082 ceph-mon[97889]: pgmap v1085: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.4 MiB/s wr, 871 op/s 2023-06-23T18:48:21.195 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:20 smithi017 ceph-mon[97472]: pgmap v1085: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.4 MiB/s wr, 871 op/s 2023-06-23T18:48:22.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 870.0 s, 322.4 tps, lat 313.995 ms stddev 96.945 2023-06-23T18:48:23.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:22 smithi002 ceph-mon[95947]: pgmap v1086: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 864 op/s 2023-06-23T18:48:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:22 smithi082 ceph-mon[97889]: pgmap v1086: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 864 op/s 2023-06-23T18:48:23.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:22 smithi017 ceph-mon[97472]: pgmap v1086: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 8.6 MiB/s wr, 864 op/s 2023-06-23T18:48:25.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:24 smithi002 ceph-mon[95947]: pgmap v1087: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 933 op/s 2023-06-23T18:48:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:24 smithi082 ceph-mon[97889]: pgmap v1087: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 933 op/s 2023-06-23T18:48:25.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:24 smithi017 ceph-mon[97472]: pgmap v1087: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 933 op/s 2023-06-23T18:48:27.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 875.0 s, 374.8 tps, lat 267.311 ms stddev 45.726 2023-06-23T18:48:27.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:26 smithi002 ceph-mon[95947]: pgmap v1088: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 920 op/s 2023-06-23T18:48:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:26 smithi082 ceph-mon[97889]: pgmap v1088: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 920 op/s 2023-06-23T18:48:27.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:26 smithi017 ceph-mon[97472]: pgmap v1088: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 10 MiB/s wr, 920 op/s 2023-06-23T18:48:29.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:48:29.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:28 smithi002 ceph-mon[95947]: pgmap v1089: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.4 MiB/s wr, 906 op/s 2023-06-23T18:48:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:28 smithi082 ceph-mon[97889]: pgmap v1089: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.4 MiB/s wr, 906 op/s 2023-06-23T18:48:29.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:28 smithi017 ceph-mon[97472]: pgmap v1089: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.4 MiB/s wr, 906 op/s 2023-06-23T18:48:29.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 -- 172.21.15.2:0/4292620988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcec1066a0 msgr2=0x7efcec106ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 --2- 172.21.15.2:0/4292620988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcec1066a0 0x7efcec106ab0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7efcdc009990 tx=0x7efcdc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 -- 172.21.15.2:0/4292620988 shutdown_connections 2023-06-23T18:48:29.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 --2- 172.21.15.2:0/4292620988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcec107d60 0x7efcec10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 --2- 172.21.15.2:0/4292620988 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcec1066a0 0x7efcec106ab0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 --2- 172.21.15.2:0/4292620988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec1073b0 0x7efcec107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 -- 172.21.15.2:0/4292620988 >> 172.21.15.2:0/4292620988 conn(0x7efcec0751f0 msgr2=0x7efcec077620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:29.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 -- 172.21.15.2:0/4292620988 shutdown_connections 2023-06-23T18:48:29.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.210+0000 7efcf161f700 1 -- 172.21.15.2:0/4292620988 wait complete. 2023-06-23T18:48:29.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcf161f700 1 Processor -- start 2023-06-23T18:48:29.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcf161f700 1 -- start start 2023-06-23T18:48:29.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcf161f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcec1066a0 0x7efcec19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcf161f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec1073b0 0x7efcec19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcf161f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcec107d60 0x7efcec1a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcf161f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec1a3a30 con 0x7efcec107d60 2023-06-23T18:48:29.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcf161f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec069ef0 con 0x7efcec1066a0 2023-06-23T18:48:29.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcf161f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcec06a080 con 0x7efcec1073b0 2023-06-23T18:48:29.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcea7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec1073b0 0x7efcec19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcea7fc700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec1073b0 0x7efcec19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58792/0 (socket says 172.21.15.2:58792) 2023-06-23T18:48:29.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efcea7fc700 1 -- 172.21.15.2:0/4087063940 learned_addr learned my addr 172.21.15.2:0/4087063940 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:29.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efceaffd700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcec1066a0 0x7efcec19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.211+0000 7efceb7fe700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcec107d60 0x7efcec1a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcea7fc700 1 -- 172.21.15.2:0/4087063940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcec1066a0 msgr2=0x7efcec19beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcea7fc700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcec1066a0 0x7efcec19beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcea7fc700 1 -- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcec107d60 msgr2=0x7efcec1a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcea7fc700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcec107d60 0x7efcec1a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcea7fc700 1 -- 172.21.15.2:0/4087063940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0095f0 con 0x7efcec1073b0 2023-06-23T18:48:29.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcea7fc700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec1073b0 0x7efcec19c3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efce000b530 tx=0x7efce000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:29.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcd3fff700 1 -- 172.21.15.2:0/4087063940 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0011830 con 0x7efcec1073b0 2023-06-23T18:48:29.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcf161f700 1 -- 172.21.15.2:0/4087063940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcec06a360 con 0x7efcec1073b0 2023-06-23T18:48:29.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcd3fff700 1 -- 172.21.15.2:0/4087063940 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efce0011e20 con 0x7efcec1073b0 2023-06-23T18:48:29.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcd3fff700 1 -- 172.21.15.2:0/4087063940 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0010dd0 con 0x7efcec1073b0 2023-06-23T18:48:29.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.212+0000 7efcf161f700 1 -- 172.21.15.2:0/4087063940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec06a940 con 0x7efcec1073b0 2023-06-23T18:48:29.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.215+0000 7efcd3fff700 1 -- 172.21.15.2:0/4087063940 <== mon.1 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7efce0011990 con 0x7efcec1073b0 2023-06-23T18:48:29.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.216+0000 7efcd3fff700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efcd40a1150 0x7efcd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.216+0000 7efceaffd700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efcd40a1150 0x7efcd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.216+0000 7efcd3fff700 1 -- 172.21.15.2:0/4087063940 <== mon.1 v2:172.21.15.82:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efce00cc2f0 con 0x7efcec1073b0 2023-06-23T18:48:29.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.217+0000 7efceaffd700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efcd40a1150 0x7efcd40a3600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00b5c0 tx=0x7efcdc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:29.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.217+0000 7efcfd7a4b80 1 Processor -- start 2023-06-23T18:48:29.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.217+0000 7efcfd7a4b80 1 -- start start 2023-06-23T18:48:29.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe129090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe129a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe12a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe02dcb0 con 0x55dbbe1295d0 2023-06-23T18:48:29.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe080a40 con 0x55dbbe12a000 2023-06-23T18:48:29.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe079ed0 con 0x55dbbe128c80 2023-06-23T18:48:29.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe129090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceaffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe129090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58798/0 (socket says 172.21.15.2:58798) 2023-06-23T18:48:29.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceaffd700 1 -- 172.21.15.2:0/2920447560 learned_addr learned my addr 172.21.15.2:0/2920447560 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:29.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efcea7fc700 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe129a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceb7fe700 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe12a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceb7fe700 1 -- 172.21.15.2:0/2920447560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 msgr2=0x55dbbe129090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:29.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceb7fe700 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe129090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceb7fe700 1 -- 172.21.15.2:0/2920447560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 msgr2=0x55dbbe129a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:29.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceb7fe700 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe129a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efceb7fe700 1 -- 172.21.15.2:0/2920447560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbdefb630 con 0x55dbbe12a000 2023-06-23T18:48:29.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.218+0000 7efcea7fc700 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe129a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:48:29.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.219+0000 7efceb7fe700 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe12a470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efce4009f70 tx=0x7efce4009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c94a9b0f5e8f61 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:29.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.219+0000 7efcd17fa700 1 -- 172.21.15.2:0/2920447560 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4010040 con 0x55dbbe12a000 2023-06-23T18:48:29.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.219+0000 7efcd17fa700 1 -- 172.21.15.2:0/2920447560 <== mon.2 v2:172.21.15.17:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efce4014ab0 con 0x55dbbe12a000 2023-06-23T18:48:29.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.219+0000 7efcd17fa700 1 -- 172.21.15.2:0/2920447560 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce40138f0 con 0x55dbbe12a000 2023-06-23T18:48:29.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/2920447560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 msgr2=0x55dbbe12a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe12a470 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efce4009f70 tx=0x7efce4009650 comp rx=0 tx=0).stop 2023-06-23T18:48:29.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/2920447560 shutdown_connections 2023-06-23T18:48:29.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe12a470 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe129090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/2920447560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe129a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/2920447560 >> 172.21.15.2:0/2920447560 conn(0x55dbbe11e3f0 msgr2=0x55dbbe123520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:29.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/2920447560 shutdown_connections 2023-06-23T18:48:29.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.220+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/2920447560 wait complete. 2023-06-23T18:48:29.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 1 -- start start 2023-06-23T18:48:29.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 1 Processor -- start 2023-06-23T18:48:29.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe132550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe132a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe139ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe019ac0 con 0x55dbbe1295d0 2023-06-23T18:48:29.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe0197c0 con 0x55dbbe12a000 2023-06-23T18:48:29.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbdefd920 con 0x55dbbe128c80 2023-06-23T18:48:29.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.221+0000 7efcfd7a4b80 10 client.0 _ll_register_callbacks cb 0x55dbbe01f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:48:29.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efcea7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe132a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efceb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe139ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efcea7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe132a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58468/0 (socket says 172.21.15.2:58468) 2023-06-23T18:48:29.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efcea7fc700 1 -- 172.21.15.2:0/3135830291 learned_addr learned my addr 172.21.15.2:0/3135830291 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:29.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efceb7fe700 1 -- 172.21.15.2:0/3135830291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 msgr2=0x55dbbe132550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-06-23T18:48:29.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efceb7fe700 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe132550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efceb7fe700 1 -- 172.21.15.2:0/3135830291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 msgr2=0x55dbbe132a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:29.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efceb7fe700 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe132a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efceb7fe700 1 -- 172.21.15.2:0/3135830291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcec06a940 con 0x55dbbe12a000 2023-06-23T18:48:29.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efceb7fe700 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe139ad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efce4009f70 tx=0x7efce4014650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61ebca23f6c246a server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:29.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efcceff5700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:48:29.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efcceff5700 1 -- 172.21.15.2:0/3135830291 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4010070 con 0x55dbbe12a000 2023-06-23T18:48:29.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efcceff5700 1 -- 172.21.15.2:0/3135830291 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efce4006e40 con 0x55dbbe12a000 2023-06-23T18:48:29.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.222+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/3135830291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dbbe11bbb0 con 0x55dbbe12a000 2023-06-23T18:48:29.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcceff5700 1 -- 172.21.15.2:0/3135830291 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efce40148f0 con 0x55dbbe12a000 2023-06-23T18:48:29.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcfd7a4b80 10 client.26687 fetch_fsmap learned FSMap version 41 2023-06-23T18:48:29.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/3135830291 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x7efce4009ba0 con 0x55dbbe12a000 2023-06-23T18:48:29.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcceff5700 1 -- 172.21.15.2:0/3135830291 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dbbe11bbb0 con 0x55dbbe12a000 2023-06-23T18:48:29.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcceff5700 1 -- 172.21.15.2:0/3135830291 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efce4024020 con 0x55dbbe12a000 2023-06-23T18:48:29.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcfd7a4b80 10 client.26687 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:48:29.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcfd7a4b80 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:48:29.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcfd7a4b80 20 client.26687 populate_metadata read hostname 'smithi002' 2023-06-23T18:48:29.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55dbbe13e200 0x55dbbe140610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcfd7a4b80 4 client.26687 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:48:29.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/3135830291 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dbbe072eb0 con 0x55dbbe13e200 2023-06-23T18:48:29.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.223+0000 7efceaffd700 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55dbbe13e200 0x55dbbe140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.224+0000 7efcceff5700 10 client.26687 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:48:29.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.224+0000 7efceaffd700 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55dbbe13e200 0x55dbbe140610 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdc00df9a0adc818 server_cookie=71f0d61e2d30fbca in_seq=0 out_seq=0 2023-06-23T18:48:29.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.225+0000 7efcceff5700 1 -- 172.21.15.2:0/3135830291 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dbbe072eb0 con 0x55dbbe13e200 2023-06-23T18:48:29.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.225+0000 7efcceff5700 10 client.26687 handle_command_reply: tid=0 2023-06-23T18:48:29.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.225+0000 7efcfd7a4b80 1 client.26687 shutdown 2023-06-23T18:48:29.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/3135830291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 msgr2=0x55dbbe139ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe139ad0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efce4009f70 tx=0x7efce4014650 comp rx=0 tx=0).stop 2023-06-23T18:48:29.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/3135830291 shutdown_connections 2023-06-23T18:48:29.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55dbbe13e200 0x55dbbe140610 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe139ad0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe132550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/3135830291 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe132a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/3135830291 >> 172.21.15.2:0/3135830291 conn(0x55dbbe11e3f0 msgr2=0x55dbbe122c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:29.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcceff5700 0 client.26687 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:48:29.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/3135830291 shutdown_connections 2023-06-23T18:48:29.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/3135830291 wait complete. 2023-06-23T18:48:29.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.226+0000 7efcfd7a4b80 20 client.26687 trim_cache size 0 max 16384 2023-06-23T18:48:29.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.254+0000 7efcfd7a4b80 1 Processor -- start 2023-06-23T18:48:29.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.254+0000 7efcfd7a4b80 1 -- start start 2023-06-23T18:48:29.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe128c80 0x55dbbe140210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe1295d0 0x55dbbe139a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe12a000 0x55dbbe12cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe125010 con 0x55dbbe12a000 2023-06-23T18:48:29.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe13dfe0 con 0x55dbbe128c80 2023-06-23T18:48:29.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe12da70 con 0x55dbbe1295d0 2023-06-23T18:48:29.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efceb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe12a000 0x55dbbe12cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efceb7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe12a000 0x55dbbe12cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58476/0 (socket says 172.21.15.2:58476) 2023-06-23T18:48:29.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efceb7fe700 1 -- 172.21.15.2:0/1769411812 learned_addr learned my addr 172.21.15.2:0/1769411812 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:29.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efcea7fc700 1 --2- 172.21.15.2:0/1769411812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe1295d0 0x55dbbe139a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efceb7fe700 1 -- 172.21.15.2:0/1769411812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe1295d0 msgr2=0x55dbbe139a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:29.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efceb7fe700 1 --2- 172.21.15.2:0/1769411812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe1295d0 0x55dbbe139a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efceb7fe700 1 -- 172.21.15.2:0/1769411812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe128c80 msgr2=0x55dbbe140210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:48:29.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efceb7fe700 1 --2- 172.21.15.2:0/1769411812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe128c80 0x55dbbe140210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.255+0000 7efceb7fe700 1 -- 172.21.15.2:0/1769411812 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbbdea7de0 con 0x55dbbe12a000 2023-06-23T18:48:29.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.256+0000 7efceb7fe700 1 --2- 172.21.15.2:0/1769411812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe12a000 0x55dbbe12cf00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7efce4009f40 tx=0x7efce4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acec5c4fef925db1 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:29.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.256+0000 7efcd1ffb700 1 -- 172.21.15.2:0/1769411812 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4010070 con 0x55dbbe12a000 2023-06-23T18:48:29.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.256+0000 7efcd1ffb700 1 -- 172.21.15.2:0/1769411812 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efce402c030 con 0x55dbbe12a000 2023-06-23T18:48:29.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.256+0000 7efcd1ffb700 1 -- 172.21.15.2:0/1769411812 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4013880 con 0x55dbbe12a000 2023-06-23T18:48:29.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.257+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1769411812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe12a000 msgr2=0x55dbbe12cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.257+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1769411812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe12a000 0x55dbbe12cf00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7efce4009f40 tx=0x7efce4000f40 comp rx=0 tx=0).stop 2023-06-23T18:48:29.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.257+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1769411812 shutdown_connections 2023-06-23T18:48:29.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.257+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1769411812 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe12a000 0x55dbbe12cf00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.257+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1769411812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe128c80 0x55dbbe140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.257+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1769411812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe1295d0 0x55dbbe139a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.257+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1769411812 >> 172.21.15.2:0/1769411812 conn(0x55dbbe11e3f0 msgr2=0x55dbbe131260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:29.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.257+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1769411812 shutdown_connections 2023-06-23T18:48:29.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.262+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1769411812 wait complete. 2023-06-23T18:48:29.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.262+0000 7efcfd7a4b80 1 -- start start 2023-06-23T18:48:29.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.262+0000 7efcfd7a4b80 1 Processor -- start 2023-06-23T18:48:29.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe135ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe136220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efcfd7a4b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe138d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe139240 con 0x55dbbe1295d0 2023-06-23T18:48:29.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe1393b0 con 0x55dbbe12a000 2023-06-23T18:48:29.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efcfd7a4b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbbe139570 con 0x55dbbe128c80 2023-06-23T18:48:29.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efcfd7a4b80 10 client.0 _ll_register_callbacks cb 0x55dbbe01f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:48:29.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efceb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efceb7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe138d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:49072/0 (socket says 172.21.15.2:49072) 2023-06-23T18:48:29.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.263+0000 7efceb7fe700 1 -- 172.21.15.2:0/1698116083 learned_addr learned my addr 172.21.15.2:0/1698116083 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:29.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.264+0000 7efceb7fe700 1 -- 172.21.15.2:0/1698116083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 msgr2=0x55dbbe135ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:29.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.264+0000 7efceb7fe700 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe135ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.264+0000 7efceb7fe700 1 -- 172.21.15.2:0/1698116083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 msgr2=0x55dbbe136220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-06-23T18:48:29.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.264+0000 7efceb7fe700 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.264+0000 7efceb7fe700 1 -- 172.21.15.2:0/1698116083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce4009ba0 con 0x55dbbe12a000 2023-06-23T18:48:29.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.264+0000 7efceb7fe700 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe138d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efce4030ff0 tx=0x7efce4014a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1ca072a463a8ff server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:29.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.266+0000 7efcceff5700 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2023-06-23T18:48:29.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.266+0000 7efcceff5700 1 -- 172.21.15.2:0/1698116083 <== mon.2 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4010070 con 0x55dbbe12a000 2023-06-23T18:48:29.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.266+0000 7efcceff5700 1 -- 172.21.15.2:0/1698116083 <== mon.2 v2:172.21.15.17:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7efce4026770 con 0x55dbbe12a000 2023-06-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.266+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1698116083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efce4014ab0 con 0x55dbbe12a000 2023-06-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.266+0000 7efcceff5700 1 -- 172.21.15.2:0/1698116083 <== mon.2 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7efce4026e20 con 0x55dbbe12a000 2023-06-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.266+0000 7efcfd7a4b80 10 client.26690 fetch_fsmap learned FSMap version 41 2023-06-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.266+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1698116083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55dbbde90c60 con 0x55dbbe12a000 2023-06-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.266+0000 7efcceff5700 1 -- 172.21.15.2:0/1698116083 <== mon.2 v2:172.21.15.17:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efce4014ab0 con 0x55dbbe12a000 2023-06-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efcceff5700 1 -- 172.21.15.2:0/1698116083 <== mon.2 v2:172.21.15.17:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7efce4042070 con 0x55dbbe12a000 2023-06-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efcfd7a4b80 10 client.26690 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:48:29.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efcfd7a4b80 10 client.26690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:48:29.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efcfd7a4b80 20 client.26690 populate_metadata read hostname 'smithi002' 2023-06-23T18:48:29.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55dbbe192b00 0x55dbbe194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:29.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efcfd7a4b80 4 client.26690 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:48:29.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1698116083 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dbbe035050 con 0x55dbbe192b00 2023-06-23T18:48:29.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efceaffd700 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55dbbe192b00 0x55dbbe194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:29.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efcceff5700 10 client.26690 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:48:29.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.267+0000 7efceaffd700 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55dbbe192b00 0x55dbbe194ee0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7124d79a0efbdcd1 server_cookie=fc56f84cf237023b in_seq=0 out_seq=0 2023-06-23T18:48:29.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcceff5700 1 -- 172.21.15.2:0/1698116083 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55dbbe035050 con 0x55dbbe192b00 2023-06-23T18:48:29.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcceff5700 10 client.26690 handle_command_reply: tid=0 2023-06-23T18:48:29.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 client.26690 shutdown 2023-06-23T18:48:29.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1698116083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 msgr2=0x55dbbe138d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe138d00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efce4030ff0 tx=0x7efce4014a10 comp rx=0 tx=0).stop 2023-06-23T18:48:29.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1698116083 shutdown_connections 2023-06-23T18:48:29.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55dbbe192b00 0x55dbbe194ee0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55dbbe12a000 0x55dbbe138d00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbbe128c80 0x55dbbe135ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 --2- 172.21.15.2:0/1698116083 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55dbbe1295d0 0x55dbbe136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcceff5700 0 client.26690 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:48:29.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1698116083 >> 172.21.15.2:0/1698116083 conn(0x55dbbe11e3f0 msgr2=0x55dbbe131260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:29.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1698116083 shutdown_connections 2023-06-23T18:48:29.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 1 -- 172.21.15.2:0/1698116083 wait complete. 2023-06-23T18:48:29.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.268+0000 7efcfd7a4b80 20 client.26690 trim_cache size 0 max 16384 2023-06-23T18:48:29.278 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:48:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:48:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:48:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:48:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:48:29.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:48:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:48:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:48:29.279 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:48:29.279 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:48:29.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 -- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efcd40a1150 msgr2=0x7efcd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efcd40a1150 0x7efcd40a3600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00b5c0 tx=0x7efcdc005fb0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 -- 172.21.15.2:0/4087063940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec1073b0 msgr2=0x7efcec19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:29.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec1073b0 0x7efcec19c3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efce000b530 tx=0x7efce000b840 comp rx=0 tx=0).stop 2023-06-23T18:48:29.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 -- 172.21.15.2:0/4087063940 shutdown_connections 2023-06-23T18:48:29.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efcec107d60 0x7efcec1a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7efcd40a1150 0x7efcd40a3600 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcec1066a0 0x7efcec19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 --2- 172.21.15.2:0/4087063940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcec1073b0 0x7efcec19c3f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:29.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 -- 172.21.15.2:0/4087063940 >> 172.21.15.2:0/4087063940 conn(0x7efcec0751f0 msgr2=0x7efcec077490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:29.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 -- 172.21.15.2:0/4087063940 shutdown_connections 2023-06-23T18:48:29.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:29.269+0000 7efcf161f700 1 -- 172.21.15.2:0/4087063940 wait complete. 2023-06-23T18:48:29.293 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:48:31.054 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:30 smithi002 ceph-mon[95947]: pgmap v1090: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:48:31.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:30 smithi082 ceph-mon[97889]: pgmap v1090: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:48:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:30 smithi017 ceph-mon[97472]: pgmap v1090: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 10 MiB/s wr, 949 op/s 2023-06-23T18:48:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 880.0 s, 307.8 tps, lat 324.148 ms stddev 357.480 2023-06-23T18:48:33.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:32 smithi002 ceph-mon[95947]: pgmap v1091: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 939 op/s 2023-06-23T18:48:33.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:32 smithi082 ceph-mon[97889]: pgmap v1091: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 939 op/s 2023-06-23T18:48:33.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:32 smithi017 ceph-mon[97472]: pgmap v1091: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.5 MiB/s wr, 939 op/s 2023-06-23T18:48:35.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:34 smithi002 ceph-mon[95947]: pgmap v1092: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 969 op/s 2023-06-23T18:48:35.101 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:34 smithi082 ceph-mon[97889]: pgmap v1092: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 969 op/s 2023-06-23T18:48:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:34 smithi017 ceph-mon[97472]: pgmap v1092: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 10 MiB/s wr, 969 op/s 2023-06-23T18:48:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 885.0 s, 299.8 tps, lat 185.868 ms stddev 116.491 2023-06-23T18:48:37.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:36 smithi002 ceph-mon[95947]: pgmap v1093: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.4 MiB/s wr, 909 op/s 2023-06-23T18:48:37.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:36 smithi082 ceph-mon[97889]: pgmap v1093: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.4 MiB/s wr, 909 op/s 2023-06-23T18:48:37.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:36 smithi017 ceph-mon[97472]: pgmap v1093: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 9.4 MiB/s wr, 909 op/s 2023-06-23T18:48:39.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:38 smithi002 ceph-mon[95947]: pgmap v1094: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.9 MiB/s wr, 899 op/s 2023-06-23T18:48:39.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:38 smithi082 ceph-mon[97889]: pgmap v1094: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.9 MiB/s wr, 899 op/s 2023-06-23T18:48:39.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:38 smithi017 ceph-mon[97472]: pgmap v1094: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.9 MiB/s wr, 899 op/s 2023-06-23T18:48:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:40 smithi002 ceph-mon[95947]: pgmap v1095: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 960 op/s 2023-06-23T18:48:41.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:40 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:48:41.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:40 smithi082 ceph-mon[97889]: pgmap v1095: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 960 op/s 2023-06-23T18:48:41.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:40 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:48:41.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:40 smithi017 ceph-mon[97472]: pgmap v1095: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 11 MiB/s wr, 960 op/s 2023-06-23T18:48:41.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:40 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:48:42.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 890.0 s, 241.6 tps, lat 61.857 ms stddev 16.966 2023-06-23T18:48:42.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:48:42.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:48:42.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:48:42.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:48:42.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:48:42.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:48:42.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:48:42.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:48:42.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:48:43.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:42 smithi002 ceph-mon[95947]: pgmap v1096: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 905 op/s 2023-06-23T18:48:43.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:42 smithi082 ceph-mon[97889]: pgmap v1096: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 905 op/s 2023-06-23T18:48:43.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:42 smithi017 ceph-mon[97472]: pgmap v1096: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 9.1 MiB/s wr, 905 op/s 2023-06-23T18:48:45.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:44 smithi002 ceph-mon[95947]: pgmap v1097: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.9 MiB/s wr, 912 op/s 2023-06-23T18:48:45.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:44 smithi082 ceph-mon[97889]: pgmap v1097: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.9 MiB/s wr, 912 op/s 2023-06-23T18:48:45.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:44 smithi017 ceph-mon[97472]: pgmap v1097: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 9.9 MiB/s wr, 912 op/s 2023-06-23T18:48:47.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 895.0 s, 264.2 tps, lat 56.897 ms stddev 14.049 2023-06-23T18:48:47.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:46 smithi002 ceph-mon[95947]: pgmap v1098: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 842 op/s 2023-06-23T18:48:47.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:46 smithi082 ceph-mon[97889]: pgmap v1098: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 842 op/s 2023-06-23T18:48:47.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:46 smithi017 ceph-mon[97472]: pgmap v1098: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 8.8 MiB/s wr, 842 op/s 2023-06-23T18:48:49.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:48 smithi002 ceph-mon[95947]: pgmap v1099: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.1 MiB/s wr, 820 op/s 2023-06-23T18:48:49.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:48 smithi082 ceph-mon[97889]: pgmap v1099: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.1 MiB/s wr, 820 op/s 2023-06-23T18:48:49.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:48 smithi017 ceph-mon[97472]: pgmap v1099: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 8.1 MiB/s wr, 820 op/s 2023-06-23T18:48:51.053 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:50 smithi002 ceph-mon[95947]: pgmap v1100: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 819 op/s 2023-06-23T18:48:51.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:50 smithi082 ceph-mon[97889]: pgmap v1100: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 819 op/s 2023-06-23T18:48:51.192 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:50 smithi017 ceph-mon[97472]: pgmap v1100: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 819 op/s 2023-06-23T18:48:52.025 INFO:teuthology.orchestra.run.smithi002.stderr:progress: 900.0 s, 259.6 tps, lat 57.913 ms stddev 16.201 2023-06-23T18:48:53.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:52 smithi002 ceph-mon[95947]: pgmap v1101: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 778 op/s 2023-06-23T18:48:53.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:52 smithi082 ceph-mon[97889]: pgmap v1101: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 778 op/s 2023-06-23T18:48:53.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:52 smithi017 ceph-mon[97472]: pgmap v1101: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 778 op/s 2023-06-23T18:48:53.248 INFO:teuthology.orchestra.run.smithi002.stdout:transaction type: 2023-06-23T18:48:53.248 INFO:teuthology.orchestra.run.smithi002.stdout:scaling factor: 500 2023-06-23T18:48:53.249 INFO:teuthology.orchestra.run.smithi002.stdout:query mode: simple 2023-06-23T18:48:53.249 INFO:teuthology.orchestra.run.smithi002.stdout:number of clients: 100 2023-06-23T18:48:53.250 INFO:teuthology.orchestra.run.smithi002.stdout:number of threads: 4 2023-06-23T18:48:53.250 INFO:teuthology.orchestra.run.smithi002.stdout:duration: 900 s 2023-06-23T18:48:53.250 INFO:teuthology.orchestra.run.smithi002.stdout:number of transactions actually processed: 305335 2023-06-23T18:48:53.250 INFO:teuthology.orchestra.run.smithi002.stdout:latency average = 294.974 ms 2023-06-23T18:48:53.251 INFO:teuthology.orchestra.run.smithi002.stdout:latency stddev = 933.541 ms 2023-06-23T18:48:53.251 INFO:teuthology.orchestra.run.smithi002.stdout:tps = 338.800897 (including connections establishing) 2023-06-23T18:48:53.251 INFO:teuthology.orchestra.run.smithi002.stdout:tps = 338.805047 (excluding connections establishing) 2023-06-23T18:48:53.253 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-06-23T18:48:55.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:54 smithi002 ceph-mon[95947]: pgmap v1102: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.5 MiB/s wr, 779 op/s 2023-06-23T18:48:55.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:54 smithi082 ceph-mon[97889]: pgmap v1102: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.5 MiB/s wr, 779 op/s 2023-06-23T18:48:55.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:54 smithi017 ceph-mon[97472]: pgmap v1102: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.5 MiB/s wr, 779 op/s 2023-06-23T18:48:57.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:56 smithi002 ceph-mon[95947]: pgmap v1103: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.4 MiB/s wr, 732 op/s 2023-06-23T18:48:57.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:56 smithi082 ceph-mon[97889]: pgmap v1103: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.4 MiB/s wr, 732 op/s 2023-06-23T18:48:57.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:56 smithi017 ceph-mon[97472]: pgmap v1103: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.4 MiB/s wr, 732 op/s 2023-06-23T18:48:59.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:48:58 smithi002 ceph-mon[95947]: pgmap v1104: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.5 MiB/s wr, 729 op/s 2023-06-23T18:48:59.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:48:58 smithi082 ceph-mon[97889]: pgmap v1104: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.5 MiB/s wr, 729 op/s 2023-06-23T18:48:59.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:48:58 smithi017 ceph-mon[97472]: pgmap v1104: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.5 MiB/s wr, 729 op/s 2023-06-23T18:48:59.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:48:59.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 -- 172.21.15.2:0/752915925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0ffc70 msgr2=0x7f007c1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 --2- 172.21.15.2:0/752915925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0ffc70 0x7f007c1000c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f0070009990 tx=0x7f0070009ca0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 -- 172.21.15.2:0/752915925 shutdown_connections 2023-06-23T18:48:59.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 --2- 172.21.15.2:0/752915925 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f007c100600 0x7f007c106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 --2- 172.21.15.2:0/752915925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f007c0fea80 0x7f007c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 --2- 172.21.15.2:0/752915925 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0ffc70 0x7f007c1000c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 -- 172.21.15.2:0/752915925 >> 172.21.15.2:0/752915925 conn(0x7f007c0fa0e0 msgr2=0x7f007c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:59.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 -- 172.21.15.2:0/752915925 shutdown_connections 2023-06-23T18:48:59.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.439+0000 7f00828d3700 1 -- 172.21.15.2:0/752915925 wait complete. 2023-06-23T18:48:59.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f00828d3700 1 Processor -- start 2023-06-23T18:48:59.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f00828d3700 1 -- start start 2023-06-23T18:48:59.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f00828d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0fea80 0x7f007c068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f00828d3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f007c0ffc70 0x7f007c19a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f00828d3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f007c100600 0x7f007c19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f00828d3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c19d330 con 0x7f007c0fea80 2023-06-23T18:48:59.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f00828d3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c19d4a0 con 0x7f007c0ffc70 2023-06-23T18:48:59.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f00828d3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007c19d660 con 0x7f007c100600 2023-06-23T18:48:59.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0fea80 0x7f007c068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0fea80 0x7f007c068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58492/0 (socket says 172.21.15.2:58492) 2023-06-23T18:48:59.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007bfff700 1 -- 172.21.15.2:0/1206380165 learned_addr learned my addr 172.21.15.2:0/1206380165 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:59.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007b7fe700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f007c0ffc70 0x7f007c19a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f0080e70700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f007c100600 0x7f007c19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007bfff700 1 -- 172.21.15.2:0/1206380165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f007c100600 msgr2=0x7f007c19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007bfff700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f007c100600 0x7f007c19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007bfff700 1 -- 172.21.15.2:0/1206380165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f007c0ffc70 msgr2=0x7f007c19a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007bfff700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f007c0ffc70 0x7f007c19a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.440+0000 7f007bfff700 1 -- 172.21.15.2:0/1206380165 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00700095f0 con 0x7f007c0fea80 2023-06-23T18:48:59.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.441+0000 7f007bfff700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0fea80 0x7f007c068330 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f006c00e970 tx=0x7f006c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:59.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.441+0000 7f00797fa700 1 -- 172.21.15.2:0/1206380165 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c009790 con 0x7f007c0fea80 2023-06-23T18:48:59.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.441+0000 7f00828d3700 1 -- 172.21.15.2:0/1206380165 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007c19d940 con 0x7f007c0fea80 2023-06-23T18:48:59.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.441+0000 7f00797fa700 1 -- 172.21.15.2:0/1206380165 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f006c00c7e0 con 0x7f007c0fea80 2023-06-23T18:48:59.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.441+0000 7f00797fa700 1 -- 172.21.15.2:0/1206380165 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c003bd0 con 0x7f007c0fea80 2023-06-23T18:48:59.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.441+0000 7f00828d3700 1 -- 172.21.15.2:0/1206380165 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c19de20 con 0x7f007c0fea80 2023-06-23T18:48:59.451 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.443+0000 7f00797fa700 1 -- 172.21.15.2:0/1206380165 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f006c00c950 con 0x7f007c0fea80 2023-06-23T18:48:59.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.443+0000 7f00797fa700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00640a1150 0x7f00640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.444+0000 7f00797fa700 1 -- 172.21.15.2:0/1206380165 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f006c014070 con 0x7f007c0fea80 2023-06-23T18:48:59.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.444+0000 7f007b7fe700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00640a1150 0x7f00640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.444+0000 7f007b7fe700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00640a1150 0x7f00640a3600 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0070006010 tx=0x7f007000b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:59.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.444+0000 7f008ea58b80 1 Processor -- start 2023-06-23T18:48:59.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.444+0000 7f008ea58b80 1 -- start start 2023-06-23T18:48:59.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a245418c80 0x55a245419090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a245419a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a24541a000 0x55a24541a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f008ea58b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24531dcb0 con 0x55a245418c80 2023-06-23T18:48:59.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f008ea58b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a245370a40 con 0x55a2454195d0 2023-06-23T18:48:59.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f008ea58b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a245369ed0 con 0x55a24541a000 2023-06-23T18:48:59.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a245418c80 0x55a245419090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a245418c80 0x55a245419090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58500/0 (socket says 172.21.15.2:58500) 2023-06-23T18:48:59.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 -- 172.21.15.2:0/3959265258 learned_addr learned my addr 172.21.15.2:0/3959265258 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:59.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f0080e70700 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a24541a000 0x55a24541a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007b7fe700 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a245419a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 -- 172.21.15.2:0/3959265258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a24541a000 msgr2=0x55a24541a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:59.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a24541a000 0x55a24541a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 -- 172.21.15.2:0/3959265258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 msgr2=0x55a245419a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:59.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a245419a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 -- 172.21.15.2:0/3959265258 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2451eb630 con 0x55a245418c80 2023-06-23T18:48:59.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f007bfff700 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a245418c80 0x55a245419090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f006c096980 tx=0x7f006c005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f12610b02f961c3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:59.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f00627fc700 1 -- 172.21.15.2:0/3959265258 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0b1070 con 0x55a245418c80 2023-06-23T18:48:59.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f00627fc700 1 -- 172.21.15.2:0/3959265258 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f006c09df40 con 0x55a245418c80 2023-06-23T18:48:59.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.445+0000 7f00627fc700 1 -- 172.21.15.2:0/3959265258 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c09a170 con 0x55a245418c80 2023-06-23T18:48:59.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 -- 172.21.15.2:0/3959265258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a245418c80 msgr2=0x55a245419090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a245418c80 0x55a245419090 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f006c096980 tx=0x7f006c005910 comp rx=0 tx=0).stop 2023-06-23T18:48:59.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 -- 172.21.15.2:0/3959265258 shutdown_connections 2023-06-23T18:48:59.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a24541a000 0x55a24541a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a245418c80 0x55a245419090 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 --2- 172.21.15.2:0/3959265258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a245419a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 -- 172.21.15.2:0/3959265258 >> 172.21.15.2:0/3959265258 conn(0x55a24540e3f0 msgr2=0x55a245413520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:59.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 -- 172.21.15.2:0/3959265258 shutdown_connections 2023-06-23T18:48:59.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 -- 172.21.15.2:0/3959265258 wait complete. 2023-06-23T18:48:59.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.446+0000 7f008ea58b80 1 -- start start 2023-06-23T18:48:59.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f008ea58b80 1 Processor -- start 2023-06-23T18:48:59.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a245418c80 0x55a2454269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2454195d0 0x55a245426f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f008ea58b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a245309ac0 con 0x55a24541a000 2023-06-23T18:48:59.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f008ea58b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2453097c0 con 0x55a245418c80 2023-06-23T18:48:59.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f008ea58b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2451ed920 con 0x55a2454195d0 2023-06-23T18:48:59.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f008ea58b80 10 client.0 _ll_register_callbacks cb 0x55a24530f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:48:59.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f0080e70700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f007b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2454195d0 0x55a245426f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f007bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a245418c80 0x55a2454269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f0080e70700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58510/0 (socket says 172.21.15.2:58510) 2023-06-23T18:48:59.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f0080e70700 1 -- 172.21.15.2:0/1893755838 learned_addr learned my addr 172.21.15.2:0/1893755838 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:59.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f007bfff700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a245418c80 0x55a2454269c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:49098/0 (socket says 172.21.15.2:49098) 2023-06-23T18:48:59.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.447+0000 7f007b7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2454195d0 0x55a245426f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58846/0 (socket says 172.21.15.2:58846) 2023-06-23T18:48:59.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f0080e70700 1 -- 172.21.15.2:0/1893755838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2454195d0 msgr2=0x55a245426f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:59.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f0080e70700 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2454195d0 0x55a245426f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f0080e70700 1 -- 172.21.15.2:0/1893755838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a245418c80 msgr2=0x55a2454269c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:59.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f0080e70700 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a245418c80 0x55a2454269c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f0080e70700 1 -- 172.21.15.2:0/1893755838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c19de20 con 0x55a24541a000 2023-06-23T18:48:59.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f007bfff700 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a245418c80 0x55a2454269c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:48:59.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f0080e70700 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542df40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0074011f60 tx=0x7f0074014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ec61b79a56e19e server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:59.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f005b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:48:59.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f005b7fe700 1 -- 172.21.15.2:0/1893755838 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007400e040 con 0x55a24541a000 2023-06-23T18:48:59.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f005b7fe700 1 -- 172.21.15.2:0/1893755838 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0074014ba0 con 0x55a24541a000 2023-06-23T18:48:59.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f008ea58b80 1 -- 172.21.15.2:0/1893755838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a24540bbb0 con 0x55a24541a000 2023-06-23T18:48:59.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f005b7fe700 1 -- 172.21.15.2:0/1893755838 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0074014d60 con 0x55a24541a000 2023-06-23T18:48:59.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f008ea58b80 10 client.17205 fetch_fsmap learned FSMap version 41 2023-06-23T18:48:59.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f008ea58b80 1 -- 172.21.15.2:0/1893755838 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a245197de0 con 0x55a24541a000 2023-06-23T18:48:59.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.448+0000 7f005b7fe700 1 -- 172.21.15.2:0/1893755838 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a24540bbb0 con 0x55a24541a000 2023-06-23T18:48:59.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f005b7fe700 1 -- 172.21.15.2:0/1893755838 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f007401c020 con 0x55a24541a000 2023-06-23T18:48:59.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f008ea58b80 10 client.17205 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:48:59.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f008ea58b80 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:48:59.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f008ea58b80 20 client.17205 populate_metadata read hostname 'smithi002' 2023-06-23T18:48:59.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a2454327b0 0x55a245434bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f008ea58b80 4 client.17205 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:48:59.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f008ea58b80 1 -- 172.21.15.2:0/1893755838 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a245362eb0 con 0x55a2454327b0 2023-06-23T18:48:59.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f007bfff700 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a2454327b0 0x55a245434bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f005b7fe700 10 client.17205 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:48:59.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.449+0000 7f007bfff700 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a2454327b0 0x55a245434bc0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9693e57b389436 server_cookie=8634bf41491fe0fe in_seq=0 out_seq=0 2023-06-23T18:48:59.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f005b7fe700 1 -- 172.21.15.2:0/1893755838 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a245362eb0 con 0x55a2454327b0 2023-06-23T18:48:59.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f005b7fe700 10 client.17205 handle_command_reply: tid=0 2023-06-23T18:48:59.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 client.17205 shutdown 2023-06-23T18:48:59.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 -- 172.21.15.2:0/1893755838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 msgr2=0x55a24542df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542df40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0074011f60 tx=0x7f0074014600 comp rx=0 tx=0).stop 2023-06-23T18:48:59.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 -- 172.21.15.2:0/1893755838 shutdown_connections 2023-06-23T18:48:59.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a2454327b0 0x55a245434bc0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542df40 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a245418c80 0x55a2454269c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f005b7fe700 0 client.17205 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:48:59.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1893755838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a2454195d0 0x55a245426f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 -- 172.21.15.2:0/1893755838 >> 172.21.15.2:0/1893755838 conn(0x55a24540e3f0 msgr2=0x55a245412c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:59.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 -- 172.21.15.2:0/1893755838 shutdown_connections 2023-06-23T18:48:59.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 1 -- 172.21.15.2:0/1893755838 wait complete. 2023-06-23T18:48:59.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.450+0000 7f008ea58b80 20 client.17205 trim_cache size 0 max 16384 2023-06-23T18:48:59.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f008ea58b80 1 Processor -- start 2023-06-23T18:48:59.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f008ea58b80 1 -- start start 2023-06-23T18:48:59.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a24541cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24541d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f008ea58b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24542fc30 con 0x55a24541a000 2023-06-23T18:48:59.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f008ea58b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a245421a60 con 0x55a2454195d0 2023-06-23T18:48:59.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f008ea58b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2454143f0 con 0x55a245418c80 2023-06-23T18:48:59.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24541d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a24541cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f0080e70700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007b7fe700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24541d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.2:49108/0 (socket says 172.21.15.2:49108) 2023-06-23T18:48:59.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007b7fe700 1 -- 172.21.15.2:0/1282226444 learned_addr learned my addr 172.21.15.2:0/1282226444 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:59.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007bfff700 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a24541cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58852/0 (socket says 172.21.15.2:58852) 2023-06-23T18:48:59.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007bfff700 1 -- 172.21.15.2:0/1282226444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 msgr2=0x55a24541d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:59.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007bfff700 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24541d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007bfff700 1 -- 172.21.15.2:0/1282226444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 msgr2=0x55a24542de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:59.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007bfff700 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007bfff700 1 -- 172.21.15.2:0/1282226444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a245180c60 con 0x55a245418c80 2023-06-23T18:48:59.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f007b7fe700 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24541d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:48:59.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.470+0000 7f0080e70700 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:48:59.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.471+0000 7f007bfff700 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a24541cec0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f006c00cc00 tx=0x7f006c00cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=621c33014a41ee2b server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:59.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.471+0000 7f0062ffd700 1 -- 172.21.15.2:0/1282226444 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0b1040 con 0x55a245418c80 2023-06-23T18:48:59.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.471+0000 7f0062ffd700 1 -- 172.21.15.2:0/1282226444 <== mon.1 v2:172.21.15.82:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f006c0b6070 con 0x55a245418c80 2023-06-23T18:48:59.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.471+0000 7f0062ffd700 1 -- 172.21.15.2:0/1282226444 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c09a3d0 con 0x55a245418c80 2023-06-23T18:48:59.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.471+0000 7f008ea58b80 1 -- 172.21.15.2:0/1282226444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 msgr2=0x55a24541cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a24541cec0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f006c00cc00 tx=0x7f006c00cf50 comp rx=0 tx=0).stop 2023-06-23T18:48:59.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 -- 172.21.15.2:0/1282226444 shutdown_connections 2023-06-23T18:48:59.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a24541cec0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 --2- 172.21.15.2:0/1282226444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24541d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 -- 172.21.15.2:0/1282226444 >> 172.21.15.2:0/1282226444 conn(0x55a24540e3f0 msgr2=0x55a245421290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 -- 172.21.15.2:0/1282226444 shutdown_connections 2023-06-23T18:48:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 -- 172.21.15.2:0/1282226444 wait complete. 2023-06-23T18:48:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 -- start start 2023-06-23T18:48:59.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.472+0000 7f008ea58b80 1 Processor -- start 2023-06-23T18:48:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a245429e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24542a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f008ea58b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f008ea58b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24542d8b0 con 0x55a24541a000 2023-06-23T18:48:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f008ea58b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24542da20 con 0x55a2454195d0 2023-06-23T18:48:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f008ea58b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a24542dbe0 con 0x55a245418c80 2023-06-23T18:48:59.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f008ea58b80 10 client.0 _ll_register_callbacks cb 0x55a24530f260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:48:59.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a245429e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f0080e70700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a245429e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.2:58858/0 (socket says 172.21.15.2:58858) 2023-06-23T18:48:59.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 -- 172.21.15.2:0/2316696266 learned_addr learned my addr 172.21.15.2:0/2316696266 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:48:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 -- 172.21.15.2:0/2316696266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 msgr2=0x55a24542a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24542a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 -- 172.21.15.2:0/2316696266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 msgr2=0x55a24542d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:48:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 -- 172.21.15.2:0/2316696266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007c1077f0 con 0x55a245418c80 2023-06-23T18:48:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f0080e70700 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:48:59.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f007bfff700 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a245429e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f006c096410 tx=0x7f006c096380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=930941e3bd671a46 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:48:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.473+0000 7f005b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2023-06-23T18:48:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f005b7fe700 1 -- 172.21.15.2:0/2316696266 <== mon.1 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0b1050 con 0x55a245418c80 2023-06-23T18:48:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f005b7fe700 1 -- 172.21.15.2:0/2316696266 <== mon.1 v2:172.21.15.82:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f006c0ca300 con 0x55a245418c80 2023-06-23T18:48:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 1 -- 172.21.15.2:0/2316696266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f006c09df40 con 0x55a245418c80 2023-06-23T18:48:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f005b7fe700 1 -- 172.21.15.2:0/2316696266 <== mon.1 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f006c0a23e0 con 0x55a245418c80 2023-06-23T18:48:59.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 10 client.26938 fetch_fsmap learned FSMap version 41 2023-06-23T18:48:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 1 -- 172.21.15.2:0/2316696266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55a245197de0 con 0x55a245418c80 2023-06-23T18:48:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f005b7fe700 1 -- 172.21.15.2:0/2316696266 <== mon.1 v2:172.21.15.82:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f006c09df40 con 0x55a245418c80 2023-06-23T18:48:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f005b7fe700 1 -- 172.21.15.2:0/2316696266 <== mon.1 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f006c095d00 con 0x55a245418c80 2023-06-23T18:48:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 10 client.26938 fetch_fsmap finished waiting for FSMap version 41 2023-06-23T18:48:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 10 client.26938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:48:59.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 20 client.26938 populate_metadata read hostname 'smithi002' 2023-06-23T18:48:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a2454872c0 0x55a2454896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:48:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 4 client.26938 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:48:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.474+0000 7f008ea58b80 1 -- 172.21.15.2:0/2316696266 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a245325050 con 0x55a2454872c0 2023-06-23T18:48:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.477+0000 7f007b7fe700 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a2454872c0 0x55a2454896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:48:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.477+0000 7f007b7fe700 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a2454872c0 0x55a2454896d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b083dd0e80ee3a server_cookie=297cb286cd675df in_seq=0 out_seq=0 2023-06-23T18:48:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.477+0000 7f005b7fe700 10 client.26938 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:48:59.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f005b7fe700 1 -- 172.21.15.2:0/2316696266 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a245325050 con 0x55a2454872c0 2023-06-23T18:48:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f005b7fe700 10 client.26938 handle_command_reply: tid=0 2023-06-23T18:48:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 client.26938 shutdown 2023-06-23T18:48:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 -- 172.21.15.2:0/2316696266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 msgr2=0x55a245429e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a245429e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f006c096410 tx=0x7f006c096380 comp rx=0 tx=0).stop 2023-06-23T18:48:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 -- 172.21.15.2:0/2316696266 shutdown_connections 2023-06-23T18:48:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x55a2454872c0 0x55a2454896d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55a24541a000 0x55a24542d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f005b7fe700 0 client.26938 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:48:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a245418c80 0x55a245429e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 --2- 172.21.15.2:0/2316696266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55a2454195d0 0x55a24542a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 -- 172.21.15.2:0/2316696266 >> 172.21.15.2:0/2316696266 conn(0x55a24540e3f0 msgr2=0x55a24541cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 -- 172.21.15.2:0/2316696266 shutdown_connections 2023-06-23T18:48:59.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 1 -- 172.21.15.2:0/2316696266 wait complete. 2023-06-23T18:48:59.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.478+0000 7f008ea58b80 20 client.26938 trim_cache size 0 max 16384 2023-06-23T18:48:59.493 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:48:59.493 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:48:59.494 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:48:59.494 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:48:59.494 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:48:59.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:48:59.495 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:48:59.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:48:59.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:48:59.495 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:48:59.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 -- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00640a1150 msgr2=0x7f00640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00640a1150 0x7f00640a3600 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0070006010 tx=0x7f007000b260 comp rx=0 tx=0).stop 2023-06-23T18:48:59.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 -- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0fea80 msgr2=0x7f007c068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:48:59.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0fea80 0x7f007c068330 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f006c00e970 tx=0x7f006c00ec80 comp rx=0 tx=0).stop 2023-06-23T18:48:59.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 -- 172.21.15.2:0/1206380165 shutdown_connections 2023-06-23T18:48:59.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f007c100600 0x7f007c19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f007c0fea80 0x7f007c068330 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f00640a1150 0x7f00640a3600 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 --2- 172.21.15.2:0/1206380165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f007c0ffc70 0x7f007c19a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:48:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 -- 172.21.15.2:0/1206380165 >> 172.21.15.2:0/1206380165 conn(0x7f007c0fa0e0 msgr2=0x7f007c103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:48:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 -- 172.21.15.2:0/1206380165 shutdown_connections 2023-06-23T18:48:59.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:48:59.479+0000 7f00828d3700 1 -- 172.21.15.2:0/1206380165 wait complete. 2023-06-23T18:48:59.499 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:49:01.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:00 smithi002 ceph-mon[95947]: pgmap v1105: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2023-06-23T18:49:01.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:00 smithi082 ceph-mon[97889]: pgmap v1105: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2023-06-23T18:49:01.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:00 smithi017 ceph-mon[97472]: pgmap v1105: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 7.0 MiB/s wr, 675 op/s 2023-06-23T18:49:03.052 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:02 smithi002 ceph-mon[95947]: pgmap v1106: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 6.7 MiB/s wr, 660 op/s 2023-06-23T18:49:03.110 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:02 smithi082 ceph-mon[97889]: pgmap v1106: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 6.7 MiB/s wr, 660 op/s 2023-06-23T18:49:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:02 smithi017 ceph-mon[97472]: pgmap v1106: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 6.7 MiB/s wr, 660 op/s 2023-06-23T18:49:05.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:04 smithi002 ceph-mon[95947]: pgmap v1107: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 5.7 MiB/s wr, 621 op/s 2023-06-23T18:49:05.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:04 smithi082 ceph-mon[97889]: pgmap v1107: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 5.7 MiB/s wr, 621 op/s 2023-06-23T18:49:05.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:04 smithi017 ceph-mon[97472]: pgmap v1107: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 5.7 MiB/s wr, 621 op/s 2023-06-23T18:49:07.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:06 smithi002 ceph-mon[95947]: pgmap v1108: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 4.6 MiB/s wr, 560 op/s 2023-06-23T18:49:07.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:06 smithi082 ceph-mon[97889]: pgmap v1108: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 4.6 MiB/s wr, 560 op/s 2023-06-23T18:49:07.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:06 smithi017 ceph-mon[97472]: pgmap v1108: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 4.6 MiB/s wr, 560 op/s 2023-06-23T18:49:09.100 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:08 smithi082 ceph-mon[97889]: pgmap v1109: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.5 MiB/s wr, 553 op/s 2023-06-23T18:49:09.194 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:08 smithi017 ceph-mon[97472]: pgmap v1109: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.5 MiB/s wr, 553 op/s 2023-06-23T18:49:09.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:08 smithi002 ceph-mon[95947]: pgmap v1109: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.5 MiB/s wr, 553 op/s 2023-06-23T18:49:11.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:10 smithi082 ceph-mon[97889]: pgmap v1110: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 512 op/s 2023-06-23T18:49:11.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:10 smithi017 ceph-mon[97472]: pgmap v1110: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 512 op/s 2023-06-23T18:49:11.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:10 smithi002 ceph-mon[95947]: pgmap v1110: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.1 MiB/s wr, 512 op/s 2023-06-23T18:49:12.596 DEBUG:teuthology.orchestra.run.smithi002:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-06-23T18:49:12.976 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/: 2023-06-23T18:49:12.977 INFO:teuthology.orchestra.run.smithi002.stdout:total 2 2023-06-23T18:49:12.977 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066882 Jun 23 18:22 . 2023-06-23T18:49:12.977 INFO:teuthology.orchestra.run.smithi002.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 8998066882 Jun 23 18:22 .. 2023-06-23T18:49:12.977 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 data 2023-06-23T18:49:12.977 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:12.977 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data: 2023-06-23T18:49:12.978 INFO:teuthology.orchestra.run.smithi002.stdout:total 41 2023-06-23T18:49:12.978 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 . 2023-06-23T18:49:12.978 INFO:teuthology.orchestra.run.smithi002.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066882 Jun 23 18:22 .. 2023-06-23T18:49:12.978 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7923409037 Jun 23 18:32 base 2023-06-23T18:49:12.978 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Jun 23 18:22 current_logfiles 2023-06-23T18:49:12.978 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 587404 Jun 23 18:22 global 2023-06-23T18:49:12.978 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 944 Jun 23 18:22 log 2023-06-23T18:49:12.979 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_commit_ts 2023-06-23T18:49:12.979 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_dynshmem 2023-06-23T18:49:12.979 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jun 23 18:22 pg_hba.conf 2023-06-23T18:49:12.979 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jun 23 18:22 pg_ident.conf 2023-06-23T18:49:12.979 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:49 pg_logical 2023-06-23T18:49:12.979 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 pg_multixact 2023-06-23T18:49:12.979 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 pg_notify 2023-06-23T18:49:12.979 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_replslot 2023-06-23T18:49:12.980 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_serial 2023-06-23T18:49:12.987 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_snapshots 2023-06-23T18:49:12.987 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Jun 23 18:49 pg_stat 2023-06-23T18:49:12.987 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:49 pg_stat_tmp 2023-06-23T18:49:12.988 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:46 pg_subtrans 2023-06-23T18:49:12.988 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_tblspc 2023-06-23T18:49:12.988 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 pg_twophase 2023-06-23T18:49:12.988 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 23 18:22 PG_VERSION 2023-06-23T18:49:12.989 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:49 pg_wal 2023-06-23T18:49:12.989 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 pg_xact 2023-06-23T18:49:12.989 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jun 23 18:22 postgresql.auto.conf 2023-06-23T18:49:12.989 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jun 23 18:22 postgresql.conf 2023-06-23T18:49:12.989 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Jun 23 18:22 postmaster.opts 2023-06-23T18:49:12.990 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:12.990 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base: 2023-06-23T18:49:12.990 INFO:teuthology.orchestra.run.smithi002.stdout:total 3 2023-06-23T18:49:12.990 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7923409037 Jun 23 18:32 . 2023-06-23T18:49:12.990 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:12.990 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 1 2023-06-23T18:49:12.991 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 12346 2023-06-23T18:49:12.991 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7908662407 Jun 23 18:35 12347 2023-06-23T18:49:12.991 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:33 pgsql_tmp 2023-06-23T18:49:12.991 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:12.991 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/1: 2023-06-23T18:49:12.991 INFO:teuthology.orchestra.run.smithi002.stdout:total 7202 2023-06-23T18:49:12.991 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 . 2023-06-23T18:49:12.992 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7923409037 Jun 23 18:32 .. 2023-06-23T18:49:12.992 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 112 2023-06-23T18:49:12.992 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 113 2023-06-23T18:49:12.992 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 23 18:22 12182 2023-06-23T18:49:12.992 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12182_fsm 2023-06-23T18:49:12.992 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12182_vm 2023-06-23T18:49:12.993 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12184 2023-06-23T18:49:12.993 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12186 2023-06-23T18:49:12.993 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187 2023-06-23T18:49:12.993 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12187_fsm 2023-06-23T18:49:12.993 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187_vm 2023-06-23T18:49:12.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12189 2023-06-23T18:49:12.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12191 2023-06-23T18:49:12.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192 2023-06-23T18:49:12.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12192_fsm 2023-06-23T18:49:12.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192_vm 2023-06-23T18:49:12.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12194 2023-06-23T18:49:12.994 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12196 2023-06-23T18:49:12.995 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197 2023-06-23T18:49:12.995 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12197_fsm 2023-06-23T18:49:12.995 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197_vm 2023-06-23T18:49:12.995 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12199 2023-06-23T18:49:12.995 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12201 2023-06-23T18:49:12.995 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202 2023-06-23T18:49:12.995 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12202_fsm 2023-06-23T18:49:12.995 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202_vm 2023-06-23T18:49:12.996 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12204 2023-06-23T18:49:12.996 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12206 2023-06-23T18:49:12.996 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207 2023-06-23T18:49:12.996 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12207_fsm 2023-06-23T18:49:12.996 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207_vm 2023-06-23T18:49:12.996 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12209 2023-06-23T18:49:12.996 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12211 2023-06-23T18:49:12.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12212 2023-06-23T18:49:12.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12214 2023-06-23T18:49:12.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12216 2023-06-23T18:49:12.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jun 23 18:22 1247 2023-06-23T18:49:12.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1247_fsm 2023-06-23T18:49:12.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1247_vm 2023-06-23T18:49:12.997 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jun 23 18:22 1249 2023-06-23T18:49:12.998 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1249_fsm 2023-06-23T18:49:12.998 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1249_vm 2023-06-23T18:49:12.998 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jun 23 18:22 1255 2023-06-23T18:49:12.998 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1255_fsm 2023-06-23T18:49:12.998 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1255_vm 2023-06-23T18:49:12.998 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 23 18:22 1259 2023-06-23T18:49:12.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1259_fsm 2023-06-23T18:49:12.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1259_vm 2023-06-23T18:49:12.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417 2023-06-23T18:49:12.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417_vm 2023-06-23T18:49:12.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418 2023-06-23T18:49:12.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418_vm 2023-06-23T18:49:12.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 174 2023-06-23T18:49:12.999 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 175 2023-06-23T18:49:13.000 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2187 2023-06-23T18:49:13.000 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224 2023-06-23T18:49:13.000 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224_vm 2023-06-23T18:49:13.000 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328 2023-06-23T18:49:13.000 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328_vm 2023-06-23T18:49:13.000 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336 2023-06-23T18:49:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336_vm 2023-06-23T18:49:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2337 2023-06-23T18:49:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2600 2023-06-23T18:49:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2600_fsm 2023-06-23T18:49:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2600_vm 2023-06-23T18:49:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601 2023-06-23T18:49:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2601_fsm 2023-06-23T18:49:13.001 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601_vm 2023-06-23T18:49:13.002 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2602 2023-06-23T18:49:13.002 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2602_fsm 2023-06-23T18:49:13.002 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2602_vm 2023-06-23T18:49:13.002 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2603 2023-06-23T18:49:13.002 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2603_fsm 2023-06-23T18:49:13.002 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2603_vm 2023-06-23T18:49:13.003 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604 2023-06-23T18:49:13.003 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604_vm 2023-06-23T18:49:13.003 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2605 2023-06-23T18:49:13.003 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2605_fsm 2023-06-23T18:49:13.003 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2605_vm 2023-06-23T18:49:13.004 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606 2023-06-23T18:49:13.004 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2606_fsm 2023-06-23T18:49:13.004 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606_vm 2023-06-23T18:49:13.004 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607 2023-06-23T18:49:13.004 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607_fsm 2023-06-23T18:49:13.004 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2607_vm 2023-06-23T18:49:13.005 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jun 23 18:22 2608 2023-06-23T18:49:13.005 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2608_fsm 2023-06-23T18:49:13.005 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2608_vm 2023-06-23T18:49:13.005 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jun 23 18:22 2609 2023-06-23T18:49:13.005 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2609_fsm 2023-06-23T18:49:13.005 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2609_vm 2023-06-23T18:49:13.005 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610 2023-06-23T18:49:13.006 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610_fsm 2023-06-23T18:49:13.006 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2610_vm 2023-06-23T18:49:13.006 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611 2023-06-23T18:49:13.006 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611_vm 2023-06-23T18:49:13.007 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612 2023-06-23T18:49:13.007 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2612_fsm 2023-06-23T18:49:13.007 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612_vm 2023-06-23T18:49:13.008 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613 2023-06-23T18:49:13.008 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613_vm 2023-06-23T18:49:13.008 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615 2023-06-23T18:49:13.009 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2615_fsm 2023-06-23T18:49:13.009 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615_vm 2023-06-23T18:49:13.009 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616 2023-06-23T18:49:13.009 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616_fsm 2023-06-23T18:49:13.010 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2616_vm 2023-06-23T18:49:13.010 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 23 18:22 2617 2023-06-23T18:49:13.010 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2617_fsm 2023-06-23T18:49:13.010 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2617_vm 2023-06-23T18:49:13.011 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 23 18:22 2618 2023-06-23T18:49:13.011 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2618_fsm 2023-06-23T18:49:13.011 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2618_vm 2023-06-23T18:49:13.011 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jun 23 18:22 2619 2023-06-23T18:49:13.012 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2619_fsm 2023-06-23T18:49:13.012 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2619_vm 2023-06-23T18:49:13.012 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620 2023-06-23T18:49:13.012 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620_vm 2023-06-23T18:49:13.013 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2650 2023-06-23T18:49:13.013 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2651 2023-06-23T18:49:13.013 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2652 2023-06-23T18:49:13.013 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2653 2023-06-23T18:49:13.014 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2654 2023-06-23T18:49:13.014 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2655 2023-06-23T18:49:13.014 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2656 2023-06-23T18:49:13.015 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2657 2023-06-23T18:49:13.015 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 23 18:22 2658 2023-06-23T18:49:13.015 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2659 2023-06-23T18:49:13.015 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2660 2023-06-23T18:49:13.016 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2661 2023-06-23T18:49:13.016 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2662 2023-06-23T18:49:13.016 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2663 2023-06-23T18:49:13.016 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2664 2023-06-23T18:49:13.017 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2665 2023-06-23T18:49:13.017 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2666 2023-06-23T18:49:13.017 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2667 2023-06-23T18:49:13.017 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2668 2023-06-23T18:49:13.018 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2669 2023-06-23T18:49:13.018 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2670 2023-06-23T18:49:13.018 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jun 23 18:22 2673 2023-06-23T18:49:13.018 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jun 23 18:22 2674 2023-06-23T18:49:13.019 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:22 2675 2023-06-23T18:49:13.019 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2678 2023-06-23T18:49:13.019 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2679 2023-06-23T18:49:13.020 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2680 2023-06-23T18:49:13.020 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2681 2023-06-23T18:49:13.020 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2682 2023-06-23T18:49:13.020 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2683 2023-06-23T18:49:13.021 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2684 2023-06-23T18:49:13.021 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2685 2023-06-23T18:49:13.021 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2686 2023-06-23T18:49:13.021 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2687 2023-06-23T18:49:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2688 2023-06-23T18:49:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2689 2023-06-23T18:49:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2690 2023-06-23T18:49:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jun 23 18:22 2691 2023-06-23T18:49:13.023 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2692 2023-06-23T18:49:13.023 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2693 2023-06-23T18:49:13.024 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2696 2023-06-23T18:49:13.024 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2699 2023-06-23T18:49:13.024 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2701 2023-06-23T18:49:13.025 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2702 2023-06-23T18:49:13.025 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2703 2023-06-23T18:49:13.025 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2704 2023-06-23T18:49:13.025 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2753 2023-06-23T18:49:13.026 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2753_fsm 2023-06-23T18:49:13.026 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2753_vm 2023-06-23T18:49:13.026 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2754 2023-06-23T18:49:13.026 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2755 2023-06-23T18:49:13.027 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2756 2023-06-23T18:49:13.027 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2757 2023-06-23T18:49:13.027 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830 2023-06-23T18:49:13.027 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830_vm 2023-06-23T18:49:13.028 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2831 2023-06-23T18:49:13.028 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832 2023-06-23T18:49:13.028 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832_vm 2023-06-23T18:49:13.028 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2833 2023-06-23T18:49:13.029 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834 2023-06-23T18:49:13.029 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834_vm 2023-06-23T18:49:13.029 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2835 2023-06-23T18:49:13.029 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836 2023-06-23T18:49:13.030 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836_vm 2023-06-23T18:49:13.030 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2837 2023-06-23T18:49:13.030 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jun 23 18:22 2838 2023-06-23T18:49:13.030 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2838_fsm 2023-06-23T18:49:13.031 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2838_vm 2023-06-23T18:49:13.031 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2839 2023-06-23T18:49:13.031 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840 2023-06-23T18:49:13.032 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840_fsm 2023-06-23T18:49:13.032 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2840_vm 2023-06-23T18:49:13.032 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2841 2023-06-23T18:49:13.032 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995 2023-06-23T18:49:13.033 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995_vm 2023-06-23T18:49:13.033 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2996 2023-06-23T18:49:13.033 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079 2023-06-23T18:49:13.033 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3079_fsm 2023-06-23T18:49:13.034 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079_vm 2023-06-23T18:49:13.034 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3080 2023-06-23T18:49:13.034 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3081 2023-06-23T18:49:13.035 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3085 2023-06-23T18:49:13.035 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118 2023-06-23T18:49:13.035 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118_vm 2023-06-23T18:49:13.035 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3119 2023-06-23T18:49:13.035 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3164 2023-06-23T18:49:13.036 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256 2023-06-23T18:49:13.036 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256_vm 2023-06-23T18:49:13.036 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3257 2023-06-23T18:49:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3258 2023-06-23T18:49:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350 2023-06-23T18:49:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350_vm 2023-06-23T18:49:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3351 2023-06-23T18:49:13.037 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3379 2023-06-23T18:49:13.038 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3380 2023-06-23T18:49:13.038 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381 2023-06-23T18:49:13.038 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381_vm 2023-06-23T18:49:13.039 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3394 2023-06-23T18:49:13.039 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3394_fsm 2023-06-23T18:49:13.039 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3394_vm 2023-06-23T18:49:13.039 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3395 2023-06-23T18:49:13.040 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439 2023-06-23T18:49:13.040 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439_vm 2023-06-23T18:49:13.040 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3440 2023-06-23T18:49:13.040 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 3455 2023-06-23T18:49:13.041 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3456 2023-06-23T18:49:13.041 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3456_fsm 2023-06-23T18:49:13.041 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3456_vm 2023-06-23T18:49:13.041 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466 2023-06-23T18:49:13.041 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466_vm 2023-06-23T18:49:13.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3467 2023-06-23T18:49:13.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3468 2023-06-23T18:49:13.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501 2023-06-23T18:49:13.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501_vm 2023-06-23T18:49:13.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3502 2023-06-23T18:49:13.042 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3503 2023-06-23T18:49:13.044 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3534 2023-06-23T18:49:13.045 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541 2023-06-23T18:49:13.045 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3541_fsm 2023-06-23T18:49:13.045 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541_vm 2023-06-23T18:49:13.045 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3542 2023-06-23T18:49:13.046 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3574 2023-06-23T18:49:13.046 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3575 2023-06-23T18:49:13.046 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576 2023-06-23T18:49:13.046 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576_vm 2023-06-23T18:49:13.047 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596 2023-06-23T18:49:13.047 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596_vm 2023-06-23T18:49:13.047 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3597 2023-06-23T18:49:13.047 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598 2023-06-23T18:49:13.048 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598_vm 2023-06-23T18:49:13.048 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3599 2023-06-23T18:49:13.049 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600 2023-06-23T18:49:13.049 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3600_fsm 2023-06-23T18:49:13.049 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600_vm 2023-06-23T18:49:13.049 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601 2023-06-23T18:49:13.050 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3601_fsm 2023-06-23T18:49:13.050 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601_vm 2023-06-23T18:49:13.050 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602 2023-06-23T18:49:13.050 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3602_fsm 2023-06-23T18:49:13.051 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602_vm 2023-06-23T18:49:13.051 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3603 2023-06-23T18:49:13.051 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3603_fsm 2023-06-23T18:49:13.051 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3603_vm 2023-06-23T18:49:13.052 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3604 2023-06-23T18:49:13.052 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3605 2023-06-23T18:49:13.052 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3606 2023-06-23T18:49:13.052 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3607 2023-06-23T18:49:13.053 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3608 2023-06-23T18:49:13.053 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 3609 2023-06-23T18:49:13.053 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3712 2023-06-23T18:49:13.053 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764 2023-06-23T18:49:13.054 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3764_fsm 2023-06-23T18:49:13.054 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764_vm 2023-06-23T18:49:13.054 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3766 2023-06-23T18:49:13.054 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3767 2023-06-23T18:49:13.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3997 2023-06-23T18:49:13.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 5002 2023-06-23T18:49:13.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 548 2023-06-23T18:49:13.055 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 549 2023-06-23T18:49:13.056 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102 2023-06-23T18:49:13.056 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102_vm 2023-06-23T18:49:13.056 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104 2023-06-23T18:49:13.056 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104_vm 2023-06-23T18:49:13.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106 2023-06-23T18:49:13.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106_vm 2023-06-23T18:49:13.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6110 2023-06-23T18:49:13.057 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6111 2023-06-23T18:49:13.058 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6112 2023-06-23T18:49:13.058 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6113 2023-06-23T18:49:13.058 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6117 2023-06-23T18:49:13.059 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826 2023-06-23T18:49:13.059 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826_vm 2023-06-23T18:49:13.059 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:12 smithi002 ceph-mon[95947]: pgmap v1111: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 624 op/s 2023-06-23T18:49:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:12 smithi082 ceph-mon[97889]: pgmap v1111: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 624 op/s 2023-06-23T18:49:13.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:12 smithi017 ceph-mon[97472]: pgmap v1111: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 4.9 MiB/s wr, 624 op/s 2023-06-23T18:49:13.276 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 827 2023-06-23T18:49:13.276 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 828 2023-06-23T18:49:13.276 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 23 18:22 pg_filenode.map 2023-06-23T18:49:13.276 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 23 18:22 PG_VERSION 2023-06-23T18:49:13.277 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.277 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-06-23T18:49:13.277 INFO:teuthology.orchestra.run.smithi002.stdout:total 7202 2023-06-23T18:49:13.277 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7373315 Jun 23 18:22 . 2023-06-23T18:49:13.277 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7923409037 Jun 23 18:32 .. 2023-06-23T18:49:13.277 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 112 2023-06-23T18:49:13.277 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 113 2023-06-23T18:49:13.278 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 23 18:22 12182 2023-06-23T18:49:13.278 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12182_fsm 2023-06-23T18:49:13.278 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12182_vm 2023-06-23T18:49:13.278 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12184 2023-06-23T18:49:13.278 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12186 2023-06-23T18:49:13.279 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187 2023-06-23T18:49:13.279 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12187_fsm 2023-06-23T18:49:13.279 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187_vm 2023-06-23T18:49:13.279 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12189 2023-06-23T18:49:13.279 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12191 2023-06-23T18:49:13.279 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192 2023-06-23T18:49:13.280 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12192_fsm 2023-06-23T18:49:13.280 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192_vm 2023-06-23T18:49:13.280 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12194 2023-06-23T18:49:13.280 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12196 2023-06-23T18:49:13.280 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197 2023-06-23T18:49:13.280 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12197_fsm 2023-06-23T18:49:13.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197_vm 2023-06-23T18:49:13.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12199 2023-06-23T18:49:13.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12201 2023-06-23T18:49:13.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202 2023-06-23T18:49:13.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12202_fsm 2023-06-23T18:49:13.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202_vm 2023-06-23T18:49:13.281 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12204 2023-06-23T18:49:13.282 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12206 2023-06-23T18:49:13.282 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207 2023-06-23T18:49:13.282 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12207_fsm 2023-06-23T18:49:13.282 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207_vm 2023-06-23T18:49:13.282 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12209 2023-06-23T18:49:13.282 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12211 2023-06-23T18:49:13.283 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12212 2023-06-23T18:49:13.283 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12214 2023-06-23T18:49:13.283 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12216 2023-06-23T18:49:13.283 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jun 23 18:22 1247 2023-06-23T18:49:13.283 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1247_fsm 2023-06-23T18:49:13.283 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1247_vm 2023-06-23T18:49:13.283 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jun 23 18:22 1249 2023-06-23T18:49:13.284 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1249_fsm 2023-06-23T18:49:13.284 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1249_vm 2023-06-23T18:49:13.284 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jun 23 18:22 1255 2023-06-23T18:49:13.284 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1255_fsm 2023-06-23T18:49:13.284 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1255_vm 2023-06-23T18:49:13.284 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 23 18:22 1259 2023-06-23T18:49:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1259_fsm 2023-06-23T18:49:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1259_vm 2023-06-23T18:49:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417 2023-06-23T18:49:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417_vm 2023-06-23T18:49:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418 2023-06-23T18:49:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418_vm 2023-06-23T18:49:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 174 2023-06-23T18:49:13.285 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 175 2023-06-23T18:49:13.286 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2187 2023-06-23T18:49:13.286 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224 2023-06-23T18:49:13.286 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224_vm 2023-06-23T18:49:13.286 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328 2023-06-23T18:49:13.286 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328_vm 2023-06-23T18:49:13.286 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336 2023-06-23T18:49:13.286 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336_vm 2023-06-23T18:49:13.287 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2337 2023-06-23T18:49:13.287 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2600 2023-06-23T18:49:13.287 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2600_fsm 2023-06-23T18:49:13.287 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2600_vm 2023-06-23T18:49:13.287 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601 2023-06-23T18:49:13.287 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2601_fsm 2023-06-23T18:49:13.287 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601_vm 2023-06-23T18:49:13.288 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2602 2023-06-23T18:49:13.288 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2602_fsm 2023-06-23T18:49:13.288 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2602_vm 2023-06-23T18:49:13.288 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2603 2023-06-23T18:49:13.288 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2603_fsm 2023-06-23T18:49:13.288 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2603_vm 2023-06-23T18:49:13.288 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604 2023-06-23T18:49:13.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604_vm 2023-06-23T18:49:13.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2605 2023-06-23T18:49:13.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2605_fsm 2023-06-23T18:49:13.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2605_vm 2023-06-23T18:49:13.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606 2023-06-23T18:49:13.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2606_fsm 2023-06-23T18:49:13.289 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2606_vm 2023-06-23T18:49:13.290 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607 2023-06-23T18:49:13.290 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607_fsm 2023-06-23T18:49:13.290 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2607_vm 2023-06-23T18:49:13.290 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jun 23 18:22 2608 2023-06-23T18:49:13.290 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2608_fsm 2023-06-23T18:49:13.290 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2608_vm 2023-06-23T18:49:13.290 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jun 23 18:22 2609 2023-06-23T18:49:13.291 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2609_fsm 2023-06-23T18:49:13.291 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2609_vm 2023-06-23T18:49:13.291 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610 2023-06-23T18:49:13.291 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2610_fsm 2023-06-23T18:49:13.291 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2610_vm 2023-06-23T18:49:13.291 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611 2023-06-23T18:49:13.291 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611_vm 2023-06-23T18:49:13.292 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612 2023-06-23T18:49:13.292 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2612_fsm 2023-06-23T18:49:13.292 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612_vm 2023-06-23T18:49:13.292 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613 2023-06-23T18:49:13.292 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613_vm 2023-06-23T18:49:13.292 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615 2023-06-23T18:49:13.292 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2615_fsm 2023-06-23T18:49:13.293 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615_vm 2023-06-23T18:49:13.293 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616 2023-06-23T18:49:13.293 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616_fsm 2023-06-23T18:49:13.293 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2616_vm 2023-06-23T18:49:13.293 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 23 18:22 2617 2023-06-23T18:49:13.293 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2617_fsm 2023-06-23T18:49:13.293 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2617_vm 2023-06-23T18:49:13.294 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 23 18:22 2618 2023-06-23T18:49:13.294 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2618_fsm 2023-06-23T18:49:13.294 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2618_vm 2023-06-23T18:49:13.294 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jun 23 18:22 2619 2023-06-23T18:49:13.294 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2619_fsm 2023-06-23T18:49:13.295 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2619_vm 2023-06-23T18:49:13.295 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620 2023-06-23T18:49:13.296 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620_vm 2023-06-23T18:49:13.296 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2650 2023-06-23T18:49:13.296 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2651 2023-06-23T18:49:13.296 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2652 2023-06-23T18:49:13.296 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2653 2023-06-23T18:49:13.296 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2654 2023-06-23T18:49:13.296 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2655 2023-06-23T18:49:13.297 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2656 2023-06-23T18:49:13.297 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2657 2023-06-23T18:49:13.297 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jun 23 18:22 2658 2023-06-23T18:49:13.297 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2659 2023-06-23T18:49:13.297 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2660 2023-06-23T18:49:13.297 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2661 2023-06-23T18:49:13.297 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2662 2023-06-23T18:49:13.298 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2663 2023-06-23T18:49:13.298 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2664 2023-06-23T18:49:13.298 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2665 2023-06-23T18:49:13.298 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2666 2023-06-23T18:49:13.298 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2667 2023-06-23T18:49:13.298 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2668 2023-06-23T18:49:13.298 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2669 2023-06-23T18:49:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2670 2023-06-23T18:49:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jun 23 18:22 2673 2023-06-23T18:49:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jun 23 18:22 2674 2023-06-23T18:49:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:22 2675 2023-06-23T18:49:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2678 2023-06-23T18:49:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2679 2023-06-23T18:49:13.299 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2680 2023-06-23T18:49:13.300 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2681 2023-06-23T18:49:13.300 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2682 2023-06-23T18:49:13.300 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2683 2023-06-23T18:49:13.300 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2684 2023-06-23T18:49:13.300 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2685 2023-06-23T18:49:13.300 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2686 2023-06-23T18:49:13.301 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2687 2023-06-23T18:49:13.301 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2688 2023-06-23T18:49:13.301 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2689 2023-06-23T18:49:13.301 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2690 2023-06-23T18:49:13.301 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jun 23 18:22 2691 2023-06-23T18:49:13.301 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2692 2023-06-23T18:49:13.302 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2693 2023-06-23T18:49:13.302 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2696 2023-06-23T18:49:13.302 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2699 2023-06-23T18:49:13.302 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2701 2023-06-23T18:49:13.302 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2702 2023-06-23T18:49:13.302 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2703 2023-06-23T18:49:13.303 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2704 2023-06-23T18:49:13.303 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2753 2023-06-23T18:49:13.303 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2753_fsm 2023-06-23T18:49:13.303 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2753_vm 2023-06-23T18:49:13.303 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2754 2023-06-23T18:49:13.303 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2755 2023-06-23T18:49:13.303 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2756 2023-06-23T18:49:13.304 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2757 2023-06-23T18:49:13.304 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830 2023-06-23T18:49:13.304 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830_vm 2023-06-23T18:49:13.304 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2831 2023-06-23T18:49:13.304 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832 2023-06-23T18:49:13.304 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832_vm 2023-06-23T18:49:13.304 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2833 2023-06-23T18:49:13.305 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834 2023-06-23T18:49:13.305 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834_vm 2023-06-23T18:49:13.305 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2835 2023-06-23T18:49:13.305 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836 2023-06-23T18:49:13.305 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836_vm 2023-06-23T18:49:13.306 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2837 2023-06-23T18:49:13.306 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jun 23 18:22 2838 2023-06-23T18:49:13.306 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2838_fsm 2023-06-23T18:49:13.306 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2838_vm 2023-06-23T18:49:13.306 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2839 2023-06-23T18:49:13.306 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840 2023-06-23T18:49:13.306 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840_fsm 2023-06-23T18:49:13.307 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2840_vm 2023-06-23T18:49:13.307 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2841 2023-06-23T18:49:13.307 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995 2023-06-23T18:49:13.307 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995_vm 2023-06-23T18:49:13.307 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2996 2023-06-23T18:49:13.307 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079 2023-06-23T18:49:13.307 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3079_fsm 2023-06-23T18:49:13.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079_vm 2023-06-23T18:49:13.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3080 2023-06-23T18:49:13.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3081 2023-06-23T18:49:13.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3085 2023-06-23T18:49:13.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118 2023-06-23T18:49:13.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118_vm 2023-06-23T18:49:13.308 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3119 2023-06-23T18:49:13.309 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3164 2023-06-23T18:49:13.309 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256 2023-06-23T18:49:13.309 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256_vm 2023-06-23T18:49:13.309 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3257 2023-06-23T18:49:13.309 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3258 2023-06-23T18:49:13.309 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350 2023-06-23T18:49:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350_vm 2023-06-23T18:49:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3351 2023-06-23T18:49:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3379 2023-06-23T18:49:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3380 2023-06-23T18:49:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381 2023-06-23T18:49:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381_vm 2023-06-23T18:49:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3394 2023-06-23T18:49:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3394_fsm 2023-06-23T18:49:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3394_vm 2023-06-23T18:49:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3395 2023-06-23T18:49:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439 2023-06-23T18:49:13.311 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439_vm 2023-06-23T18:49:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3440 2023-06-23T18:49:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 3455 2023-06-23T18:49:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3456 2023-06-23T18:49:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3456_fsm 2023-06-23T18:49:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3456_vm 2023-06-23T18:49:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466 2023-06-23T18:49:13.312 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466_vm 2023-06-23T18:49:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3467 2023-06-23T18:49:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3468 2023-06-23T18:49:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501 2023-06-23T18:49:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501_vm 2023-06-23T18:49:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3502 2023-06-23T18:49:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3503 2023-06-23T18:49:13.313 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3534 2023-06-23T18:49:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541 2023-06-23T18:49:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3541_fsm 2023-06-23T18:49:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541_vm 2023-06-23T18:49:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3542 2023-06-23T18:49:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3574 2023-06-23T18:49:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3575 2023-06-23T18:49:13.314 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576 2023-06-23T18:49:13.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576_vm 2023-06-23T18:49:13.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596 2023-06-23T18:49:13.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596_vm 2023-06-23T18:49:13.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3597 2023-06-23T18:49:13.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598 2023-06-23T18:49:13.315 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598_vm 2023-06-23T18:49:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3599 2023-06-23T18:49:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600 2023-06-23T18:49:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3600_fsm 2023-06-23T18:49:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600_vm 2023-06-23T18:49:13.316 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601 2023-06-23T18:49:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3601_fsm 2023-06-23T18:49:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601_vm 2023-06-23T18:49:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602 2023-06-23T18:49:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3602_fsm 2023-06-23T18:49:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602_vm 2023-06-23T18:49:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3603 2023-06-23T18:49:13.317 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3603_fsm 2023-06-23T18:49:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3603_vm 2023-06-23T18:49:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3604 2023-06-23T18:49:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3605 2023-06-23T18:49:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3606 2023-06-23T18:49:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3607 2023-06-23T18:49:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3608 2023-06-23T18:49:13.318 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 3609 2023-06-23T18:49:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3712 2023-06-23T18:49:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764 2023-06-23T18:49:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3764_fsm 2023-06-23T18:49:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764_vm 2023-06-23T18:49:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3766 2023-06-23T18:49:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3767 2023-06-23T18:49:13.319 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3997 2023-06-23T18:49:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 5002 2023-06-23T18:49:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 548 2023-06-23T18:49:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 549 2023-06-23T18:49:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102 2023-06-23T18:49:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102_vm 2023-06-23T18:49:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104 2023-06-23T18:49:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104_vm 2023-06-23T18:49:13.320 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106 2023-06-23T18:49:13.321 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106_vm 2023-06-23T18:49:13.637 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6110 2023-06-23T18:49:13.638 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6111 2023-06-23T18:49:13.638 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6112 2023-06-23T18:49:13.639 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6113 2023-06-23T18:49:13.639 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6117 2023-06-23T18:49:13.639 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826 2023-06-23T18:49:13.639 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826_vm 2023-06-23T18:49:13.640 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 827 2023-06-23T18:49:13.640 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 828 2023-06-23T18:49:13.640 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 23 18:22 pg_filenode.map 2023-06-23T18:49:13.641 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 23 18:22 PG_VERSION 2023-06-23T18:49:13.641 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.641 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-06-23T18:49:13.641 INFO:teuthology.orchestra.run.smithi002.stdout:total 7723305 2023-06-23T18:49:13.641 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7908662407 Jun 23 18:35 . 2023-06-23T18:49:13.642 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7923409037 Jun 23 18:32 .. 2023-06-23T18:49:13.642 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 112 2023-06-23T18:49:13.642 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 113 2023-06-23T18:49:13.642 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jun 23 18:22 12182 2023-06-23T18:49:13.643 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12182_fsm 2023-06-23T18:49:13.643 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12182_vm 2023-06-23T18:49:13.643 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12184 2023-06-23T18:49:13.644 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12186 2023-06-23T18:49:13.644 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187 2023-06-23T18:49:13.644 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12187_fsm 2023-06-23T18:49:13.644 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12187_vm 2023-06-23T18:49:13.644 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12189 2023-06-23T18:49:13.644 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12191 2023-06-23T18:49:13.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192 2023-06-23T18:49:13.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12192_fsm 2023-06-23T18:49:13.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12192_vm 2023-06-23T18:49:13.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12194 2023-06-23T18:49:13.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12196 2023-06-23T18:49:13.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197 2023-06-23T18:49:13.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12197_fsm 2023-06-23T18:49:13.645 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12197_vm 2023-06-23T18:49:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12199 2023-06-23T18:49:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12201 2023-06-23T18:49:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202 2023-06-23T18:49:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12202_fsm 2023-06-23T18:49:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12202_vm 2023-06-23T18:49:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12204 2023-06-23T18:49:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12206 2023-06-23T18:49:13.646 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207 2023-06-23T18:49:13.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 12207_fsm 2023-06-23T18:49:13.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12207_vm 2023-06-23T18:49:13.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12209 2023-06-23T18:49:13.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12211 2023-06-23T18:49:13.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12212 2023-06-23T18:49:13.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 12214 2023-06-23T18:49:13.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 12216 2023-06-23T18:49:13.647 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jun 23 18:23 1247 2023-06-23T18:49:13.648 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:23 1247_fsm 2023-06-23T18:49:13.648 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:23 1247_vm 2023-06-23T18:49:13.648 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jun 23 18:34 1249 2023-06-23T18:49:13.648 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:23 1249_fsm 2023-06-23T18:49:13.648 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:23 1249_vm 2023-06-23T18:49:13.649 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jun 23 18:22 1255 2023-06-23T18:49:13.649 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1255_fsm 2023-06-23T18:49:13.649 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1255_vm 2023-06-23T18:49:13.650 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jun 23 18:48 1259 2023-06-23T18:49:13.650 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1259_fsm 2023-06-23T18:49:13.650 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:23 1259_vm 2023-06-23T18:49:13.650 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417 2023-06-23T18:49:13.650 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1417_vm 2023-06-23T18:49:13.650 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418 2023-06-23T18:49:13.651 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 1418_vm 2023-06-23T18:49:13.651 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1146880 Jun 23 18:48 16387 2023-06-23T18:49:13.651 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:48 16387_fsm 2023-06-23T18:49:13.651 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:48 16387_vm 2023-06-23T18:49:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 917504 Jun 23 18:48 16393 2023-06-23T18:49:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:48 16393_fsm 2023-06-23T18:49:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:48 16393_vm 2023-06-23T18:49:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:48 16396 2023-06-23T18:49:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:48 16396.1 2023-06-23T18:49:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:48 16396.2 2023-06-23T18:49:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:48 16396.3 2023-06-23T18:49:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:49 16396.4 2023-06-23T18:49:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:49 16396.5 2023-06-23T18:49:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 310468608 Jun 23 18:49 16396.6 2023-06-23T18:49:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1679360 Jun 23 18:49 16396_fsm 2023-06-23T18:49:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Jun 23 18:49 16396_vm 2023-06-23T18:49:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 23 18:49 16397 2023-06-23T18:49:13.653 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Jun 23 18:49 16399 2023-06-23T18:49:13.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:49 16401 2023-06-23T18:49:13.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49373184 Jun 23 18:49 16401.1 2023-06-23T18:49:13.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 20381696 Jun 23 18:49 16403 2023-06-23T18:49:13.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:49 16403_fsm 2023-06-23T18:49:13.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 174 2023-06-23T18:49:13.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 175 2023-06-23T18:49:13.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2187 2023-06-23T18:49:13.654 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224 2023-06-23T18:49:13.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2224_vm 2023-06-23T18:49:13.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328 2023-06-23T18:49:13.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2328_vm 2023-06-23T18:49:13.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336 2023-06-23T18:49:13.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2336_vm 2023-06-23T18:49:13.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2337 2023-06-23T18:49:13.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2600 2023-06-23T18:49:13.655 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2600_fsm 2023-06-23T18:49:13.656 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2600_vm 2023-06-23T18:49:13.656 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601 2023-06-23T18:49:13.656 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2601_fsm 2023-06-23T18:49:13.658 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2601_vm 2023-06-23T18:49:13.658 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2602 2023-06-23T18:49:13.658 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2602_fsm 2023-06-23T18:49:13.658 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2602_vm 2023-06-23T18:49:13.658 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2603 2023-06-23T18:49:13.658 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2603_fsm 2023-06-23T18:49:13.658 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2603_vm 2023-06-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604 2023-06-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2604_vm 2023-06-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2605 2023-06-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2605_fsm 2023-06-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2605_vm 2023-06-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:33 2606 2023-06-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2606_fsm 2023-06-23T18:49:13.659 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:33 2606_vm 2023-06-23T18:49:13.660 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607 2023-06-23T18:49:13.660 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2607_fsm 2023-06-23T18:49:13.660 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2607_vm 2023-06-23T18:49:13.660 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Jun 23 18:33 2608 2023-06-23T18:49:13.660 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:33 2608_fsm 2023-06-23T18:49:13.660 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:33 2608_vm 2023-06-23T18:49:13.661 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jun 23 18:22 2609 2023-06-23T18:49:13.661 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2609_fsm 2023-06-23T18:49:13.661 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2609_vm 2023-06-23T18:49:13.661 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:34 2610 2023-06-23T18:49:13.661 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:33 2610_fsm 2023-06-23T18:49:13.661 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:33 2610_vm 2023-06-23T18:49:13.662 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611 2023-06-23T18:49:13.662 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2611_vm 2023-06-23T18:49:13.662 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612 2023-06-23T18:49:13.662 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2612_fsm 2023-06-23T18:49:13.663 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2612_vm 2023-06-23T18:49:13.663 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613 2023-06-23T18:49:13.663 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2613_vm 2023-06-23T18:49:13.663 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615 2023-06-23T18:49:13.663 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2615_fsm 2023-06-23T18:49:13.664 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2615_vm 2023-06-23T18:49:13.664 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616 2023-06-23T18:49:13.664 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2616_fsm 2023-06-23T18:49:13.664 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2616_vm 2023-06-23T18:49:13.664 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jun 23 18:22 2617 2023-06-23T18:49:13.664 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2617_fsm 2023-06-23T18:49:13.665 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2617_vm 2023-06-23T18:49:13.665 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jun 23 18:22 2618 2023-06-23T18:49:13.665 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2618_fsm 2023-06-23T18:49:13.665 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2618_vm 2023-06-23T18:49:13.665 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 Jun 23 18:48 2619 2023-06-23T18:49:13.665 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:47 2619_fsm 2023-06-23T18:49:13.665 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:33 2619_vm 2023-06-23T18:49:13.665 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620 2023-06-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2620_vm 2023-06-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2650 2023-06-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2651 2023-06-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2652 2023-06-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2653 2023-06-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2654 2023-06-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2655 2023-06-23T18:49:13.666 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2656 2023-06-23T18:49:13.667 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2657 2023-06-23T18:49:13.667 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jun 23 18:33 2658 2023-06-23T18:49:13.667 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:33 2659 2023-06-23T18:49:13.667 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2660 2023-06-23T18:49:13.667 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2661 2023-06-23T18:49:13.667 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:34 2662 2023-06-23T18:49:13.667 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:34 2663 2023-06-23T18:49:13.668 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:33 2664 2023-06-23T18:49:13.668 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:33 2665 2023-06-23T18:49:13.668 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:33 2666 2023-06-23T18:49:13.668 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:33 2667 2023-06-23T18:49:13.668 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2668 2023-06-23T18:49:13.668 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2669 2023-06-23T18:49:13.669 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2670 2023-06-23T18:49:13.669 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 327680 Jun 23 18:33 2673 2023-06-23T18:49:13.669 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jun 23 18:33 2674 2023-06-23T18:49:13.669 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:22 2675 2023-06-23T18:49:13.669 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:33 2678 2023-06-23T18:49:13.669 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:33 2679 2023-06-23T18:49:13.669 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2680 2023-06-23T18:49:13.670 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2681 2023-06-23T18:49:13.670 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2682 2023-06-23T18:49:13.670 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2683 2023-06-23T18:49:13.670 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2684 2023-06-23T18:49:13.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2685 2023-06-23T18:49:13.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2686 2023-06-23T18:49:13.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2687 2023-06-23T18:49:13.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:22 2688 2023-06-23T18:49:13.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jun 23 18:22 2689 2023-06-23T18:49:13.671 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 2690 2023-06-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jun 23 18:22 2691 2023-06-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2692 2023-06-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2693 2023-06-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:48 2696 2023-06-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2699 2023-06-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2701 2023-06-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2702 2023-06-23T18:49:13.672 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:23 2703 2023-06-23T18:49:13.673 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:23 2704 2023-06-23T18:49:13.673 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2753 2023-06-23T18:49:13.673 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2753_fsm 2023-06-23T18:49:13.673 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2753_vm 2023-06-23T18:49:13.673 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2754 2023-06-23T18:49:13.673 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2755 2023-06-23T18:49:13.674 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2756 2023-06-23T18:49:13.674 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 2757 2023-06-23T18:49:13.674 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830 2023-06-23T18:49:13.674 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2830_vm 2023-06-23T18:49:13.674 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2831 2023-06-23T18:49:13.674 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832 2023-06-23T18:49:13.674 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2832_vm 2023-06-23T18:49:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2833 2023-06-23T18:49:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834 2023-06-23T18:49:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2834_vm 2023-06-23T18:49:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2835 2023-06-23T18:49:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836 2023-06-23T18:49:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2836_vm 2023-06-23T18:49:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2837 2023-06-23T18:49:13.675 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jun 23 18:22 2838 2023-06-23T18:49:13.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2838_fsm 2023-06-23T18:49:13.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2838_vm 2023-06-23T18:49:13.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2839 2023-06-23T18:49:13.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840 2023-06-23T18:49:13.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2840_fsm 2023-06-23T18:49:13.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2840_vm 2023-06-23T18:49:13.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2841 2023-06-23T18:49:13.676 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995 2023-06-23T18:49:13.677 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2995_vm 2023-06-23T18:49:13.677 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2996 2023-06-23T18:49:13.677 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079 2023-06-23T18:49:13.677 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3079_fsm 2023-06-23T18:49:13.677 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3079_vm 2023-06-23T18:49:13.677 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3080 2023-06-23T18:49:13.677 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3081 2023-06-23T18:49:13.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3085 2023-06-23T18:49:13.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118 2023-06-23T18:49:13.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3118_vm 2023-06-23T18:49:13.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3119 2023-06-23T18:49:13.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3164 2023-06-23T18:49:13.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256 2023-06-23T18:49:13.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3256_vm 2023-06-23T18:49:13.678 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3257 2023-06-23T18:49:13.679 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3258 2023-06-23T18:49:13.679 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350 2023-06-23T18:49:13.679 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3350_vm 2023-06-23T18:49:13.679 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3351 2023-06-23T18:49:13.679 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3379 2023-06-23T18:49:13.679 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3380 2023-06-23T18:49:13.679 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381 2023-06-23T18:49:13.680 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3381_vm 2023-06-23T18:49:13.680 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3394 2023-06-23T18:49:13.680 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3394_fsm 2023-06-23T18:49:13.680 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3394_vm 2023-06-23T18:49:13.680 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3395 2023-06-23T18:49:13.680 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439 2023-06-23T18:49:13.680 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3439_vm 2023-06-23T18:49:13.681 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3440 2023-06-23T18:49:13.681 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jun 23 18:34 3455 2023-06-23T18:49:13.681 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3456 2023-06-23T18:49:13.681 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3456_fsm 2023-06-23T18:49:13.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3456_vm 2023-06-23T18:49:13.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466 2023-06-23T18:49:13.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3466_vm 2023-06-23T18:49:13.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3467 2023-06-23T18:49:13.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3468 2023-06-23T18:49:13.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501 2023-06-23T18:49:13.682 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3501_vm 2023-06-23T18:49:13.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3502 2023-06-23T18:49:13.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3503 2023-06-23T18:49:13.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3534 2023-06-23T18:49:13.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541 2023-06-23T18:49:13.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3541_fsm 2023-06-23T18:49:13.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3541_vm 2023-06-23T18:49:13.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3542 2023-06-23T18:49:13.683 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3574 2023-06-23T18:49:13.684 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3575 2023-06-23T18:49:13.684 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576 2023-06-23T18:49:13.684 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3576_vm 2023-06-23T18:49:13.684 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596 2023-06-23T18:49:13.684 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3596_vm 2023-06-23T18:49:13.684 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3597 2023-06-23T18:49:13.685 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598 2023-06-23T18:49:13.685 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3598_vm 2023-06-23T18:49:13.685 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3599 2023-06-23T18:49:13.685 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600 2023-06-23T18:49:13.685 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3600_fsm 2023-06-23T18:49:13.685 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3600_vm 2023-06-23T18:49:13.685 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601 2023-06-23T18:49:13.686 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3601_fsm 2023-06-23T18:49:13.686 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3601_vm 2023-06-23T18:49:13.686 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602 2023-06-23T18:49:13.687 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3602_fsm 2023-06-23T18:49:13.687 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3602_vm 2023-06-23T18:49:13.687 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3603 2023-06-23T18:49:13.687 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3603_fsm 2023-06-23T18:49:13.687 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3603_vm 2023-06-23T18:49:13.687 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3604 2023-06-23T18:49:13.687 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3605 2023-06-23T18:49:13.688 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3606 2023-06-23T18:49:13.688 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3607 2023-06-23T18:49:13.688 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3608 2023-06-23T18:49:13.688 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jun 23 18:22 3609 2023-06-23T18:49:13.688 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3712 2023-06-23T18:49:13.688 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764 2023-06-23T18:49:13.688 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 3764_fsm 2023-06-23T18:49:13.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3764_vm 2023-06-23T18:49:13.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3766 2023-06-23T18:49:13.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 3767 2023-06-23T18:49:13.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3997 2023-06-23T18:49:13.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 5002 2023-06-23T18:49:13.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 548 2023-06-23T18:49:13.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 549 2023-06-23T18:49:13.689 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102 2023-06-23T18:49:13.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6102_vm 2023-06-23T18:49:13.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104 2023-06-23T18:49:13.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6104_vm 2023-06-23T18:49:13.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106 2023-06-23T18:49:13.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6106_vm 2023-06-23T18:49:13.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6110 2023-06-23T18:49:13.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6111 2023-06-23T18:49:13.690 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6112 2023-06-23T18:49:13.691 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6113 2023-06-23T18:49:13.691 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6117 2023-06-23T18:49:13.691 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826 2023-06-23T18:49:13.691 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 826_vm 2023-06-23T18:49:13.691 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 827 2023-06-23T18:49:13.692 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 828 2023-06-23T18:49:13.692 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 23 18:22 pg_filenode.map 2023-06-23T18:49:13.692 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Jun 23 18:22 pg_internal.init 2023-06-23T18:49:13.692 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jun 23 18:22 PG_VERSION 2023-06-23T18:49:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.692 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-06-23T18:49:13.693 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.693 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:33 . 2023-06-23T18:49:13.693 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7923409037 Jun 23 18:32 .. 2023-06-23T18:49:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.694 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/global: 2023-06-23T18:49:13.694 INFO:teuthology.orchestra.run.smithi002.stdout:total 575 2023-06-23T18:49:13.694 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 587404 Jun 23 18:22 . 2023-06-23T18:49:13.694 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.694 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1136 2023-06-23T18:49:13.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1136_fsm 2023-06-23T18:49:13.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1136_vm 2023-06-23T18:49:13.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 1137 2023-06-23T18:49:13.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1213 2023-06-23T18:49:13.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1213_fsm 2023-06-23T18:49:13.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1213_vm 2023-06-23T18:49:13.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1214 2023-06-23T18:49:13.695 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1214_fsm 2023-06-23T18:49:13.696 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1214_vm 2023-06-23T18:49:13.696 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 1232 2023-06-23T18:49:13.696 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 1233 2023-06-23T18:49:13.696 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1260 2023-06-23T18:49:13.696 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1260_fsm 2023-06-23T18:49:13.696 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1260_vm 2023-06-23T18:49:13.696 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1261 2023-06-23T18:49:13.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1261_fsm 2023-06-23T18:49:13.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1261_vm 2023-06-23T18:49:13.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1262 2023-06-23T18:49:13.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 1262_fsm 2023-06-23T18:49:13.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 1262_vm 2023-06-23T18:49:13.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2396 2023-06-23T18:49:13.697 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jun 23 18:22 2396_fsm 2023-06-23T18:49:13.698 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2396_vm 2023-06-23T18:49:13.698 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2397 2023-06-23T18:49:13.698 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2671 2023-06-23T18:49:13.698 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2672 2023-06-23T18:49:13.698 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2676 2023-06-23T18:49:13.699 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2677 2023-06-23T18:49:13.699 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2694 2023-06-23T18:49:13.699 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2695 2023-06-23T18:49:13.699 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2697 2023-06-23T18:49:13.700 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 2698 2023-06-23T18:49:13.700 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2846 2023-06-23T18:49:13.700 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2846_vm 2023-06-23T18:49:13.700 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2847 2023-06-23T18:49:13.700 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2964 2023-06-23T18:49:13.724 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2964_vm 2023-06-23T18:49:13.724 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2965 2023-06-23T18:49:13.725 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2966 2023-06-23T18:49:13.725 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 2966_vm 2023-06-23T18:49:13.725 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 2967 2023-06-23T18:49:13.725 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3592 2023-06-23T18:49:13.725 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 3592_vm 2023-06-23T18:49:13.725 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 3593 2023-06-23T18:49:13.725 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 4060 2023-06-23T18:49:13.726 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 4060_vm 2023-06-23T18:49:13.726 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 4061 2023-06-23T18:49:13.726 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6000 2023-06-23T18:49:13.726 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6000_vm 2023-06-23T18:49:13.726 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6001 2023-06-23T18:49:13.726 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6002 2023-06-23T18:49:13.726 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6100 2023-06-23T18:49:13.727 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 6100_vm 2023-06-23T18:49:13.727 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6114 2023-06-23T18:49:13.727 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 6115 2023-06-23T18:49:13.727 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:49 pg_control 2023-06-23T18:49:13.727 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jun 23 18:22 pg_filenode.map 2023-06-23T18:49:13.727 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Jun 23 18:22 pg_internal.init 2023-06-23T18:49:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.728 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/log: 2023-06-23T18:49:13.728 INFO:teuthology.orchestra.run.smithi002.stdout:total 2 2023-06-23T18:49:13.728 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 944 Jun 23 18:22 . 2023-06-23T18:49:13.728 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.728 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1016 Jun 23 18:49 postgresql-Fri.log 2023-06-23T18:49:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.728 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-06-23T18:49:13.729 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.729 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.729 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.729 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-06-23T18:49:13.729 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.729 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.730 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.730 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-06-23T18:49:13.730 INFO:teuthology.orchestra.run.smithi002.stdout:total 3 2023-06-23T18:49:13.730 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:49 . 2023-06-23T18:49:13.730 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.731 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 mappings 2023-06-23T18:49:13.731 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:49 replorigin_checkpoint 2023-06-23T18:49:13.731 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 snapshots 2023-06-23T18:49:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.731 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-06-23T18:49:13.732 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.732 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.732 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:49 .. 2023-06-23T18:49:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.732 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-06-23T18:49:13.732 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.732 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.733 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jun 23 18:49 .. 2023-06-23T18:49:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.733 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-06-23T18:49:13.826 INFO:teuthology.orchestra.run.smithi002.stdout:total 2 2023-06-23T18:49:13.827 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 . 2023-06-23T18:49:13.827 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.827 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 members 2023-06-23T18:49:13.827 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 offsets 2023-06-23T18:49:13.827 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.828 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-06-23T18:49:13.828 INFO:teuthology.orchestra.run.smithi002.stdout:total 9 2023-06-23T18:49:13.828 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 . 2023-06-23T18:49:13.828 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 .. 2023-06-23T18:49:13.828 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 0000 2023-06-23T18:49:13.828 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.828 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-06-23T18:49:13.829 INFO:teuthology.orchestra.run.smithi002.stdout:total 9 2023-06-23T18:49:13.829 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 . 2023-06-23T18:49:13.829 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jun 23 18:22 .. 2023-06-23T18:49:13.829 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:23 0000 2023-06-23T18:49:13.829 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.830 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-06-23T18:49:13.830 INFO:teuthology.orchestra.run.smithi002.stdout:total 9 2023-06-23T18:49:13.830 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 . 2023-06-23T18:49:13.830 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.830 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jun 23 18:22 0000 2023-06-23T18:49:13.830 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.831 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-06-23T18:49:13.831 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.831 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.831 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.832 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-06-23T18:49:13.832 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.832 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.832 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.832 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.833 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-06-23T18:49:13.833 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.833 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.833 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.833 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.833 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-06-23T18:49:13.833 INFO:teuthology.orchestra.run.smithi002.stdout:total 14 2023-06-23T18:49:13.834 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Jun 23 18:49 . 2023-06-23T18:49:13.834 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.834 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Jun 23 18:49 db_0.stat 2023-06-23T18:49:13.834 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Jun 23 18:49 db_12347.stat 2023-06-23T18:49:13.834 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Jun 23 18:49 global.stat 2023-06-23T18:49:13.834 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.834 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-06-23T18:49:13.834 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.835 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:49 . 2023-06-23T18:49:13.835 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.835 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.835 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-06-23T18:49:13.835 INFO:teuthology.orchestra.run.smithi002.stdout:total 177 2023-06-23T18:49:13.835 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:46 . 2023-06-23T18:49:13.835 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.835 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jun 23 18:48 0004 2023-06-23T18:49:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.836 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-06-23T18:49:13.836 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.836 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.836 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.836 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.836 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-06-23T18:49:13.837 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.837 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.837 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.837 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-06-23T18:49:13.837 INFO:teuthology.orchestra.run.smithi002.stdout:total 1048578 2023-06-23T18:49:13.837 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:49 . 2023-06-23T18:49:13.837 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 000000010000000200000072 2023-06-23T18:49:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 000000010000000200000073 2023-06-23T18:49:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 000000010000000200000074 2023-06-23T18:49:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 000000010000000200000075 2023-06-23T18:49:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 000000010000000200000076 2023-06-23T18:49:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 000000010000000200000077 2023-06-23T18:49:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 000000010000000200000078 2023-06-23T18:49:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 000000010000000200000079 2023-06-23T18:49:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 00000001000000020000007A 2023-06-23T18:49:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 00000001000000020000007B 2023-06-23T18:49:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 00000001000000020000007C 2023-06-23T18:49:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 00000001000000020000007D 2023-06-23T18:49:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 00000001000000020000007E 2023-06-23T18:49:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 00000001000000020000007F 2023-06-23T18:49:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 000000010000000200000080 2023-06-23T18:49:13.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 000000010000000200000081 2023-06-23T18:49:13.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 000000010000000200000082 2023-06-23T18:49:13.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 000000010000000200000083 2023-06-23T18:49:13.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:48 000000010000000200000084 2023-06-23T18:49:13.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:49 000000010000000200000085 2023-06-23T18:49:13.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000086 2023-06-23T18:49:13.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000087 2023-06-23T18:49:13.840 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:44 000000010000000200000088 2023-06-23T18:49:13.841 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000089 2023-06-23T18:49:13.841 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:44 00000001000000020000008A 2023-06-23T18:49:13.841 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 00000001000000020000008B 2023-06-23T18:49:13.841 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:44 00000001000000020000008C 2023-06-23T18:49:13.841 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 00000001000000020000008D 2023-06-23T18:49:13.841 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 00000001000000020000008E 2023-06-23T18:49:13.842 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 00000001000000020000008F 2023-06-23T18:49:13.842 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:44 000000010000000200000090 2023-06-23T18:49:13.842 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000091 2023-06-23T18:49:13.842 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000092 2023-06-23T18:49:13.843 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:44 000000010000000200000093 2023-06-23T18:49:13.843 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000094 2023-06-23T18:49:13.844 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000095 2023-06-23T18:49:13.844 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:44 000000010000000200000096 2023-06-23T18:49:13.844 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:44 000000010000000200000097 2023-06-23T18:49:13.844 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000098 2023-06-23T18:49:13.845 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 000000010000000200000099 2023-06-23T18:49:13.845 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 00000001000000020000009A 2023-06-23T18:49:13.845 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 00000001000000020000009B 2023-06-23T18:49:13.845 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 00000001000000020000009C 2023-06-23T18:49:13.845 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 00000001000000020000009D 2023-06-23T18:49:13.845 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 00000001000000020000009E 2023-06-23T18:49:13.845 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 00000001000000020000009F 2023-06-23T18:49:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000A0 2023-06-23T18:49:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 0000000100000002000000A1 2023-06-23T18:49:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 0000000100000002000000A2 2023-06-23T18:49:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000A3 2023-06-23T18:49:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 0000000100000002000000A4 2023-06-23T18:49:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000A5 2023-06-23T18:49:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 0000000100000002000000A6 2023-06-23T18:49:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:45 0000000100000002000000A7 2023-06-23T18:49:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 0000000100000002000000A8 2023-06-23T18:49:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000A9 2023-06-23T18:49:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000AA 2023-06-23T18:49:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000AB 2023-06-23T18:49:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 0000000100000002000000AC 2023-06-23T18:49:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000AD 2023-06-23T18:49:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000AE 2023-06-23T18:49:13.847 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000AF 2023-06-23T18:49:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:46 0000000100000002000000B0 2023-06-23T18:49:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jun 23 18:47 0000000100000002000000B1 2023-06-23T18:49:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 archive_status 2023-06-23T18:49:13.848 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-06-23T18:49:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:total 1 2023-06-23T18:49:13.848 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jun 23 18:22 . 2023-06-23T18:49:13.849 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jun 23 18:49 .. 2023-06-23T18:49:13.849 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:13.849 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-06-23T18:49:13.849 INFO:teuthology.orchestra.run.smithi002.stdout:total 81 2023-06-23T18:49:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:22 . 2023-06-23T18:49:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8998066774 Jun 23 18:49 .. 2023-06-23T18:49:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jun 23 18:48 0000 2023-06-23T18:49:13.850 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-06-23T18:49:13.863 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-06-23T18:49:15.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:14 smithi082 ceph-mon[97889]: pgmap v1112: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 756 op/s 2023-06-23T18:49:15.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:14 smithi017 ceph-mon[97472]: pgmap v1112: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 756 op/s 2023-06-23T18:49:15.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:14 smithi002 ceph-mon[95947]: pgmap v1112: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 6.1 MiB/s wr, 756 op/s 2023-06-23T18:49:17.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:16 smithi082 ceph-mon[97889]: pgmap v1113: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.2 MiB/s wr, 777 op/s 2023-06-23T18:49:17.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:16 smithi017 ceph-mon[97472]: pgmap v1113: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.2 MiB/s wr, 777 op/s 2023-06-23T18:49:17.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:16 smithi002 ceph-mon[95947]: pgmap v1113: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.2 MiB/s wr, 777 op/s 2023-06-23T18:49:19.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:18 smithi082 ceph-mon[97889]: pgmap v1114: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.7 MiB/s wr, 718 op/s 2023-06-23T18:49:19.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:18 smithi017 ceph-mon[97472]: pgmap v1114: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.7 MiB/s wr, 718 op/s 2023-06-23T18:49:19.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:18 smithi002 ceph-mon[95947]: pgmap v1114: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 5.7 MiB/s wr, 718 op/s 2023-06-23T18:49:21.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:20 smithi082 ceph-mon[97889]: pgmap v1115: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 615 op/s 2023-06-23T18:49:21.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:20 smithi017 ceph-mon[97472]: pgmap v1115: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 615 op/s 2023-06-23T18:49:21.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:20 smithi002 ceph-mon[95947]: pgmap v1115: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.9 MiB/s wr, 615 op/s 2023-06-23T18:49:23.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:22 smithi082 ceph-mon[97889]: pgmap v1116: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.4 MiB/s wr, 554 op/s 2023-06-23T18:49:23.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:22 smithi017 ceph-mon[97472]: pgmap v1116: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.4 MiB/s wr, 554 op/s 2023-06-23T18:49:23.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:22 smithi002 ceph-mon[95947]: pgmap v1116: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.4 MiB/s wr, 554 op/s 2023-06-23T18:49:25.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:24 smithi082 ceph-mon[97889]: pgmap v1117: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.9 MiB/s wr, 368 op/s 2023-06-23T18:49:25.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:24 smithi017 ceph-mon[97472]: pgmap v1117: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.9 MiB/s wr, 368 op/s 2023-06-23T18:49:25.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:24 smithi002 ceph-mon[95947]: pgmap v1117: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 2.9 MiB/s wr, 368 op/s 2023-06-23T18:49:27.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:26 smithi082 ceph-mon[97889]: pgmap v1118: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 914 KiB/s wr, 133 op/s 2023-06-23T18:49:27.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:26 smithi017 ceph-mon[97472]: pgmap v1118: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 914 KiB/s wr, 133 op/s 2023-06-23T18:49:27.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:26 smithi002 ceph-mon[95947]: pgmap v1118: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 914 KiB/s wr, 133 op/s 2023-06-23T18:49:29.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:28 smithi082 ceph-mon[97889]: pgmap v1119: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-06-23T18:49:29.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:28 smithi017 ceph-mon[97472]: pgmap v1119: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-06-23T18:49:29.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:28 smithi002 ceph-mon[95947]: pgmap v1119: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 21 KiB/s wr, 15 op/s 2023-06-23T18:49:29.500 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-06-23T18:49:29.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 -- 172.21.15.2:0/2704424688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04200fea80 msgr2=0x7f04200fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 --2- 172.21.15.2:0/2704424688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04200fea80 0x7f04200fee90 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f0410009ca0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 -- 172.21.15.2:0/2704424688 shutdown_connections 2023-06-23T18:49:29.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 --2- 172.21.15.2:0/2704424688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0420100600 0x7f0420106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 --2- 172.21.15.2:0/2704424688 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04200fea80 0x7f04200fee90 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 --2- 172.21.15.2:0/2704424688 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04200ffc70 0x7f04201000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 -- 172.21.15.2:0/2704424688 >> 172.21.15.2:0/2704424688 conn(0x7f04200fa0e0 msgr2=0x7f04200fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:29.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 -- 172.21.15.2:0/2704424688 shutdown_connections 2023-06-23T18:49:29.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.640+0000 7f042503c700 1 -- 172.21.15.2:0/2704424688 wait complete. 2023-06-23T18:49:29.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.641+0000 7f042503c700 1 Processor -- start 2023-06-23T18:49:29.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.641+0000 7f042503c700 1 -- start start 2023-06-23T18:49:29.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.641+0000 7f042503c700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04200fea80 0x7f0420194620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.641+0000 7f042503c700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04200ffc70 0x7f0420194b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.641+0000 7f042503c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0420100600 0x7f042019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.641+0000 7f042503c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042019c1a0 con 0x7f0420100600 2023-06-23T18:49:29.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f042503c700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042019c2e0 con 0x7f04200fea80 2023-06-23T18:49:29.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f042503c700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042019c470 con 0x7f04200ffc70 2023-06-23T18:49:29.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0420100600 0x7f042019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0420100600 0x7f042019bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58538/0 (socket says 172.21.15.2:58538) 2023-06-23T18:49:29.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 -- 172.21.15.2:0/1468019038 learned_addr learned my addr 172.21.15.2:0/1468019038 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:29.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041e59c700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04200ffc70 0x7f0420194b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041ed9d700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04200fea80 0x7f0420194620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 -- 172.21.15.2:0/1468019038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04200ffc70 msgr2=0x7f0420194b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04200ffc70 0x7f0420194b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 -- 172.21.15.2:0/1468019038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04200fea80 msgr2=0x7f0420194620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04200fea80 0x7f0420194620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 -- 172.21.15.2:0/1468019038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04100095f0 con 0x7f0420100600 2023-06-23T18:49:29.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f041f59e700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0420100600 0x7f042019bbd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0418007ee0 tx=0x7f041800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:29.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f0407fff700 1 -- 172.21.15.2:0/1468019038 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418017070 con 0x7f0420100600 2023-06-23T18:49:29.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f042503c700 1 -- 172.21.15.2:0/1468019038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042019c750 con 0x7f0420100600 2023-06-23T18:49:29.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f0407fff700 1 -- 172.21.15.2:0/1468019038 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f041800f040 con 0x7f0420100600 2023-06-23T18:49:29.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f0407fff700 1 -- 172.21.15.2:0/1468019038 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418005570 con 0x7f0420100600 2023-06-23T18:49:29.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.642+0000 7f042503c700 1 -- 172.21.15.2:0/1468019038 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042019cc60 con 0x7f0420100600 2023-06-23T18:49:29.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.645+0000 7f0407fff700 1 -- 172.21.15.2:0/1468019038 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f041800db20 con 0x7f0420100600 2023-06-23T18:49:29.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.645+0000 7f0407fff700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f04080a1150 0x7f04080a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.645+0000 7f041ed9d700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f04080a1150 0x7f04080a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.645+0000 7f0407fff700 1 -- 172.21.15.2:0/1468019038 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f04180cc0c0 con 0x7f0420100600 2023-06-23T18:49:29.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f04311c1b80 1 Processor -- start 2023-06-23T18:49:29.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f04311c1b80 1 -- start start 2023-06-23T18:49:29.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f041ed9d700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f04080a1150 0x7f04080a3600 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f041000b5c0 tx=0x7f0410005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:29.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d41c80 0x5556c9d42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d42a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d43470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f04311c1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9c46cb0 con 0x5556c9d425d0 2023-06-23T18:49:29.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f04311c1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9c99a40 con 0x5556c9d41c80 2023-06-23T18:49:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f04311c1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9c92ed0 con 0x5556c9d43000 2023-06-23T18:49:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f041e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d42a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f041e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d42a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58544/0 (socket says 172.21.15.2:58544) 2023-06-23T18:49:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f041e59c700 1 -- 172.21.15.2:0/2109109127 learned_addr learned my addr 172.21.15.2:0/2109109127 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f041f59e700 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d43470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.646+0000 7f041ed9d700 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d41c80 0x5556c9d42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f041e59c700 1 -- 172.21.15.2:0/2109109127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 msgr2=0x5556c9d43470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:49:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f041e59c700 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d43470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f041e59c700 1 -- 172.21.15.2:0/2109109127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d41c80 msgr2=0x5556c9d42090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:49:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f041e59c700 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d41c80 0x5556c9d42090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f041e59c700 1 -- 172.21.15.2:0/2109109127 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556c9b14630 con 0x5556c9d425d0 2023-06-23T18:49:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f041e59c700 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d42a70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f041400ad70 tx=0x7f041400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411030f017ba3eb6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:29.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f04057fa700 1 -- 172.21.15.2:0/2109109127 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400a870 con 0x5556c9d425d0 2023-06-23T18:49:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f04057fa700 1 -- 172.21.15.2:0/2109109127 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04140044b0 con 0x5556c9d425d0 2023-06-23T18:49:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.647+0000 7f04057fa700 1 -- 172.21.15.2:0/2109109127 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414018440 con 0x5556c9d425d0 2023-06-23T18:49:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- 172.21.15.2:0/2109109127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 msgr2=0x5556c9d42a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d42a70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f041400ad70 tx=0x7f041400f6a0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- 172.21.15.2:0/2109109127 shutdown_connections 2023-06-23T18:49:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d43470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d41c80 0x5556c9d42090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 --2- 172.21.15.2:0/2109109127 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d42a70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- 172.21.15.2:0/2109109127 >> 172.21.15.2:0/2109109127 conn(0x5556c9d373f0 msgr2=0x5556c9d3c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:29.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- 172.21.15.2:0/2109109127 shutdown_connections 2023-06-23T18:49:29.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- 172.21.15.2:0/2109109127 wait complete. 2023-06-23T18:49:29.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- start start 2023-06-23T18:49:29.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 Processor -- start 2023-06-23T18:49:29.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 0x5556c9d4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d4ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d43000 0x5556c9d52ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9c32ac0 con 0x5556c9d43000 2023-06-23T18:49:29.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9c327c0 con 0x5556c9d425d0 2023-06-23T18:49:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.648+0000 7f04311c1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9b16920 con 0x5556c9d41c80 2023-06-23T18:49:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f04311c1b80 10 client.0 _ll_register_callbacks cb 0x5556c9c38260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:49:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d43000 0x5556c9d52ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d43000 0x5556c9d52ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58552/0 (socket says 172.21.15.2:58552) 2023-06-23T18:49:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 -- 172.21.15.2:0/299921239 learned_addr learned my addr 172.21.15.2:0/299921239 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041ed9d700 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 0x5556c9d4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041e59c700 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d4ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 -- 172.21.15.2:0/299921239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 msgr2=0x5556c9d4b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:49:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 0x5556c9d4b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 -- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 msgr2=0x5556c9d4ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:49:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d4ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 -- 172.21.15.2:0/299921239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041400a9d0 con 0x5556c9d43000 2023-06-23T18:49:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041e59c700 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d4ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:49:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f041f59e700 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d43000 0x5556c9d52ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0418099460 tx=0x7f0418004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dab7c7a744663a6 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:29.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f0402ff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:49:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f0402ff5700 1 -- 172.21.15.2:0/299921239 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04180994d0 con 0x5556c9d43000 2023-06-23T18:49:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f0402ff5700 1 -- 172.21.15.2:0/299921239 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f041809d500 con 0x5556c9d43000 2023-06-23T18:49:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f04311c1b80 1 -- 172.21.15.2:0/299921239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556c9d34bb0 con 0x5556c9d43000 2023-06-23T18:49:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f0402ff5700 1 -- 172.21.15.2:0/299921239 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04180a1920 con 0x5556c9d43000 2023-06-23T18:49:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f04311c1b80 10 client.17223 fetch_fsmap learned FSMap version 42 2023-06-23T18:49:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.649+0000 7f04311c1b80 1 -- 172.21.15.2:0/299921239 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5556c9ac0de0 con 0x5556c9d43000 2023-06-23T18:49:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f0402ff5700 1 -- 172.21.15.2:0/299921239 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04180c7020 con 0x5556c9d43000 2023-06-23T18:49:29.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f0402ff5700 1 -- 172.21.15.2:0/299921239 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 42) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0418099730 con 0x5556c9d43000 2023-06-23T18:49:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f04311c1b80 10 client.17223 fetch_fsmap finished waiting for FSMap version 42 2023-06-23T18:49:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f04311c1b80 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:49:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f04311c1b80 20 client.17223 populate_metadata read hostname 'smithi002' 2023-06-23T18:49:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f04311c1b80 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5556c9d573d0 0x5556c9d597e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f04311c1b80 4 client.17223 mds_command: new command op to 24533 tid=0[{"prefix": "get_command_descriptions"}] 2023-06-23T18:49:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f04311c1b80 1 -- 172.21.15.2:0/299921239 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556c9c8beb0 con 0x5556c9d573d0 2023-06-23T18:49:29.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.650+0000 7f041ed9d700 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5556c9d573d0 0x5556c9d597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.651+0000 7f0402ff5700 10 client.17223 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:49:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.651+0000 7f041ed9d700 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5556c9d573d0 0x5556c9d597e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60f6b30045153d0 server_cookie=db137c30cc326a5f in_seq=0 out_seq=0 2023-06-23T18:49:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.651+0000 7f0402ff5700 1 -- 172.21.15.2:0/299921239 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5556c9c8beb0 con 0x5556c9d573d0 2023-06-23T18:49:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.651+0000 7f0402ff5700 10 client.17223 handle_command_reply: tid=0 2023-06-23T18:49:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.651+0000 7f04311c1b80 1 client.17223 shutdown 2023-06-23T18:49:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.651+0000 7f04311c1b80 1 -- 172.21.15.2:0/299921239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d43000 msgr2=0x5556c9d52ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.651+0000 7f04311c1b80 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d43000 0x5556c9d52ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0418099460 tx=0x7f0418004ad0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 1 -- 172.21.15.2:0/299921239 shutdown_connections 2023-06-23T18:49:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5556c9d573d0 0x5556c9d597e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d43000 0x5556c9d52ab0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 0x5556c9d4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 1 --2- 172.21.15.2:0/299921239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d4ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 1 -- 172.21.15.2:0/299921239 >> 172.21.15.2:0/299921239 conn(0x5556c9d373f0 msgr2=0x5556c9d3bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f0402ff5700 0 client.17223 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:49:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 1 -- 172.21.15.2:0/299921239 shutdown_connections 2023-06-23T18:49:29.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 1 -- 172.21.15.2:0/299921239 wait complete. 2023-06-23T18:49:29.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.652+0000 7f04311c1b80 20 client.17223 trim_cache size 0 max 16384 2023-06-23T18:49:29.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f04311c1b80 1 Processor -- start 2023-06-23T18:49:29.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f04311c1b80 1 -- start start 2023-06-23T18:49:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 0x5556c9d45ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d46400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d43000 0x5556c9d4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f04311c1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9d4b0a0 con 0x5556c9d425d0 2023-06-23T18:49:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f04311c1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9d3e010 con 0x5556c9d43000 2023-06-23T18:49:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f04311c1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9d36f70 con 0x5556c9d41c80 2023-06-23T18:49:29.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d46400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d46400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58558/0 (socket says 172.21.15.2:58558) 2023-06-23T18:49:29.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 -- 172.21.15.2:0/1968207254 learned_addr learned my addr 172.21.15.2:0/1968207254 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:29.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041ed9d700 1 --2- 172.21.15.2:0/1968207254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 0x5556c9d45ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 -- 172.21.15.2:0/1968207254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 msgr2=0x5556c9d45ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:49:29.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 --2- 172.21.15.2:0/1968207254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 0x5556c9d45ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 -- 172.21.15.2:0/1968207254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d43000 msgr2=0x5556c9d4b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:49:29.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 --2- 172.21.15.2:0/1968207254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d43000 0x5556c9d4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 -- 172.21.15.2:0/1968207254 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556c9d438f0 con 0x5556c9d425d0 2023-06-23T18:49:29.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.669+0000 7f041e59c700 1 --2- 172.21.15.2:0/1968207254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d46400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0414019000 tx=0x7f0414005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87aed329aed28b3 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:29.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f0405ffb700 1 -- 172.21.15.2:0/1968207254 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041400fed0 con 0x5556c9d425d0 2023-06-23T18:49:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f0405ffb700 1 -- 172.21.15.2:0/1968207254 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04140054d0 con 0x5556c9d425d0 2023-06-23T18:49:29.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f0405ffb700 1 -- 172.21.15.2:0/1968207254 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0414004da0 con 0x5556c9d425d0 2023-06-23T18:49:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 -- 172.21.15.2:0/1968207254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 msgr2=0x5556c9d46400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1968207254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d46400 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0414019000 tx=0x7f0414005f50 comp rx=0 tx=0).stop 2023-06-23T18:49:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 -- 172.21.15.2:0/1968207254 shutdown_connections 2023-06-23T18:49:29.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1968207254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d43000 0x5556c9d4b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1968207254 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d41c80 0x5556c9d45ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1968207254 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d425d0 0x5556c9d46400 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 -- 172.21.15.2:0/1968207254 >> 172.21.15.2:0/1968207254 conn(0x5556c9d373f0 msgr2=0x5556c9d593e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 -- 172.21.15.2:0/1968207254 shutdown_connections 2023-06-23T18:49:29.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.670+0000 7f04311c1b80 1 -- 172.21.15.2:0/1968207254 wait complete. 2023-06-23T18:49:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 1 -- start start 2023-06-23T18:49:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 1 Processor -- start 2023-06-23T18:49:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d41c80 0x5556c9d52910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d51070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9d51c30 con 0x5556c9d41c80 2023-06-23T18:49:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9d51da0 con 0x5556c9d425d0 2023-06-23T18:49:29.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c9dad290 con 0x5556c9d43000 2023-06-23T18:49:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f04311c1b80 10 client.0 _ll_register_callbacks cb 0x5556c9c38260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-06-23T18:49:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d41c80 0x5556c9d52910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d41c80 0x5556c9d52910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58562/0 (socket says 172.21.15.2:58562) 2023-06-23T18:49:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041ed9d700 1 -- 172.21.15.2:0/1214799940 learned_addr learned my addr 172.21.15.2:0/1214799940 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041f59e700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041e59c700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d51070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041ed9d700 1 -- 172.21.15.2:0/1214799940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 msgr2=0x5556c9d515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:49:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041ed9d700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041ed9d700 1 -- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 msgr2=0x5556c9d51070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-06-23T18:49:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041ed9d700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d51070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041ed9d700 1 -- 172.21.15.2:0/1214799940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556c9b14630 con 0x5556c9d41c80 2023-06-23T18:49:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041f59e700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:49:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.671+0000 7f041e59c700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d51070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:49:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f041ed9d700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d41c80 0x5556c9d52910 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f041001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86b27f3a46f397c server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:29.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f0402ff5700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-06-23T18:49:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f0402ff5700 1 -- 172.21.15.2:0/1214799940 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410033be0 con 0x5556c9d41c80 2023-06-23T18:49:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f0402ff5700 1 -- 172.21.15.2:0/1214799940 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0410033d40 con 0x5556c9d41c80 2023-06-23T18:49:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 1 -- 172.21.15.2:0/1214799940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f04140044b0 con 0x5556c9d41c80 2023-06-23T18:49:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f0402ff5700 1 -- 172.21.15.2:0/1214799940 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0410004500 con 0x5556c9d41c80 2023-06-23T18:49:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 10 client.17229 fetch_fsmap learned FSMap version 42 2023-06-23T18:49:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 1 -- 172.21.15.2:0/1214799940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5556c9aa9c60 con 0x5556c9d41c80 2023-06-23T18:49:29.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f0402ff5700 1 -- 172.21.15.2:0/1214799940 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f04140044b0 con 0x5556c9d41c80 2023-06-23T18:49:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f0402ff5700 1 -- 172.21.15.2:0/1214799940 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 42) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0410031040 con 0x5556c9d41c80 2023-06-23T18:49:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 42 2023-06-23T18:49:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 10 client.17229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-06-23T18:49:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 20 client.17229 populate_metadata read hostname 'smithi002' 2023-06-23T18:49:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5556c9daeb90 0x5556c9db0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 4 client.17229 mds_command: new command op to 24533 tid=0[{"prefix": "scrub status"}] 2023-06-23T18:49:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.672+0000 7f04311c1b80 1 -- 172.21.15.2:0/1214799940 --> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556c9c4e050 con 0x5556c9daeb90 2023-06-23T18:49:29.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f041e59c700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5556c9daeb90 0x5556c9db0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f041e59c700 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5556c9daeb90 0x5556c9db0fa0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa46800ccbea24f6 server_cookie=40bd1ef0fd818dab in_seq=0 out_seq=0 2023-06-23T18:49:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f0402ff5700 10 client.17229 ms_handle_connect on v2:172.21.15.17:6838/4283937693 2023-06-23T18:49:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f0402ff5700 1 -- 172.21.15.2:0/1214799940 <== mds.0 v2:172.21.15.17:6838/4283937693 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5556c9c4e050 con 0x5556c9daeb90 2023-06-23T18:49:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f0402ff5700 10 client.17229 handle_command_reply: tid=0 2023-06-23T18:49:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 client.17229 shutdown 2023-06-23T18:49:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 -- 172.21.15.2:0/1214799940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d41c80 msgr2=0x5556c9d52910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d41c80 0x5556c9d52910 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f041001a050 comp rx=0 tx=0).stop 2023-06-23T18:49:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 -- 172.21.15.2:0/1214799940 shutdown_connections 2023-06-23T18:49:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:6838/4283937693,v1:172.21.15.17:6839/4283937693] conn(0x5556c9daeb90 0x5556c9db0fa0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5556c9d43000 0x5556c9d515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f0402ff5700 0 client.17229 ms_handle_reset on v2:172.21.15.17:6838/4283937693 2023-06-23T18:49:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x5556c9d41c80 0x5556c9d52910 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 --2- 172.21.15.2:0/1214799940 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x5556c9d425d0 0x5556c9d51070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 -- 172.21.15.2:0/1214799940 >> 172.21.15.2:0/1214799940 conn(0x5556c9d373f0 msgr2=0x5556c9d45ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:29.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 -- 172.21.15.2:0/1214799940 shutdown_connections 2023-06-23T18:49:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.673+0000 7f04311c1b80 1 -- 172.21.15.2:0/1214799940 wait complete. 2023-06-23T18:49:29.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f04311c1b80 20 client.17229 trim_cache size 0 max 16384 2023-06-23T18:49:29.690 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:49:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: "status": "IDLE (waiting for more scrubs)", 2023-06-23T18:49:29.690 INFO:teuthology.orchestra.run.smithi002.stdout: "scrubs": { 2023-06-23T18:49:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "65c89b01-4c28-4d60-ad58-0436dd63a674": { 2023-06-23T18:49:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "path": "/", 2023-06-23T18:49:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tag": "65c89b01-4c28-4d60-ad58-0436dd63a674", 2023-06-23T18:49:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: "options": "recursive,force" 2023-06-23T18:49:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:29.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:29.691 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:49:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 -- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f04080a1150 msgr2=0x7f04080a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f04080a1150 0x7f04080a3600 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f041000b5c0 tx=0x7f0410005960 comp rx=0 tx=0).stop 2023-06-23T18:49:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 -- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0420100600 msgr2=0x7f042019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0420100600 0x7f042019bbd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f0418007ee0 tx=0x7f041800d3b0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 -- 172.21.15.2:0/1468019038 shutdown_connections 2023-06-23T18:49:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0420100600 0x7f042019bbd0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f04080a1150 0x7f04080a3600 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04200fea80 0x7f0420194620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 --2- 172.21.15.2:0/1468019038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04200ffc70 0x7f0420194b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 -- 172.21.15.2:0/1468019038 >> 172.21.15.2:0/1468019038 conn(0x7f04200fa0e0 msgr2=0x7f04200fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 -- 172.21.15.2:0/1468019038 shutdown_connections 2023-06-23T18:49:29.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.674+0000 7f042503c700 1 -- 172.21.15.2:0/1468019038 wait complete. 2023-06-23T18:49:29.695 INFO:tasks.cephfs.filesystem:scrub status for tag:65c89b01-4c28-4d60-ad58-0436dd63a674 - {'path': '/', 'tag': '65c89b01-4c28-4d60-ad58-0436dd63a674', 'options': 'recursive,force'} 2023-06-23T18:49:29.695 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephfs/filesystem.py", line 1681, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-06-23T18:49:29.695 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-06-23T18:49:29.696 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-06-23T18:49:29.717 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:49:29.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 -- 172.21.15.2:0/3452602053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20680fea60 msgr2=0x7f20680fee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 --2- 172.21.15.2:0/3452602053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20680fea60 0x7f20680fee70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f2058009990 tx=0x7f2058009ca0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 -- 172.21.15.2:0/3452602053 shutdown_connections 2023-06-23T18:49:29.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 --2- 172.21.15.2:0/3452602053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20681005e0 0x7f2068106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 --2- 172.21.15.2:0/3452602053 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20680fea60 0x7f20680fee70 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 --2- 172.21.15.2:0/3452602053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20680ffc50 0x7f20681000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 -- 172.21.15.2:0/3452602053 >> 172.21.15.2:0/3452602053 conn(0x7f20680fa0e0 msgr2=0x7f20680fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:29.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 -- 172.21.15.2:0/3452602053 shutdown_connections 2023-06-23T18:49:29.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.839+0000 7f206db2f700 1 -- 172.21.15.2:0/3452602053 wait complete. 2023-06-23T18:49:29.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.840+0000 7f206db2f700 1 Processor -- start 2023-06-23T18:49:29.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.840+0000 7f206db2f700 1 -- start start 2023-06-23T18:49:29.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f206db2f700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20680fea60 0x7f2068068370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f206db2f700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20680ffc50 0x7f206819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f206db2f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20681005e0 0x7f206819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f206db2f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206819d310 con 0x7f20681005e0 2023-06-23T18:49:29.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f206db2f700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206819d480 con 0x7f20680fea60 2023-06-23T18:49:29.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f206db2f700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206819d640 con 0x7f20680ffc50 2023-06-23T18:49:29.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20681005e0 0x7f206819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2066ffd700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20680ffc50 0x7f206819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20681005e0 0x7f206819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58580/0 (socket says 172.21.15.2:58580) 2023-06-23T18:49:29.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 -- 172.21.15.2:0/1242562032 learned_addr learned my addr 172.21.15.2:0/1242562032 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:29.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f20677fe700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20680fea60 0x7f2068068370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 -- 172.21.15.2:0/1242562032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20680ffc50 msgr2=0x7f206819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20680ffc50 0x7f206819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 -- 172.21.15.2:0/1242562032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20680fea60 msgr2=0x7f2068068370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:29.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20680fea60 0x7f2068068370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:29.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 -- 172.21.15.2:0/1242562032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20580095f0 con 0x7f20681005e0 2023-06-23T18:49:29.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2066ffd700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20680ffc50 0x7f206819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-06-23T18:49:29.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.841+0000 7f2067fff700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20681005e0 0x7f206819cb70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f206000ada0 tx=0x7f206000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:29.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.844+0000 7f2064ff9700 1 -- 172.21.15.2:0/1242562032 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206000f070 con 0x7f20681005e0 2023-06-23T18:49:29.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.844+0000 7f2064ff9700 1 -- 172.21.15.2:0/1242562032 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2060007b30 con 0x7f20681005e0 2023-06-23T18:49:29.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.844+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206819d920 con 0x7f20681005e0 2023-06-23T18:49:29.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.844+0000 7f2064ff9700 1 -- 172.21.15.2:0/1242562032 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2060005280 con 0x7f20681005e0 2023-06-23T18:49:29.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.844+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206819dde0 con 0x7f20681005e0 2023-06-23T18:49:29.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.845+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2068105a40 con 0x7f20681005e0 2023-06-23T18:49:29.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.846+0000 7f2064ff9700 1 -- 172.21.15.2:0/1242562032 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f206000c8d0 con 0x7f20681005e0 2023-06-23T18:49:29.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.846+0000 7f2064ff9700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20500a1150 0x7f20500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:29.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.846+0000 7f2064ff9700 1 -- 172.21.15.2:0/1242562032 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f20600cbe80 con 0x7f20681005e0 2023-06-23T18:49:29.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.847+0000 7f20677fe700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20500a1150 0x7f20500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:29.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.847+0000 7f20677fe700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20500a1150 0x7f20500a3600 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f205800b5c0 tx=0x7f2058005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:29.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:29.851+0000 7f2064ff9700 1 -- 172.21.15.2:0/1242562032 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f2060095660 con 0x7f20681005e0 2023-06-23T18:49:30.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.051+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20680610e0 con 0x7f20681005e0 2023-06-23T18:49:30.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.053+0000 7f2064ff9700 1 -- 172.21.15.2:0/1242562032 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f2060098c80 con 0x7f20681005e0 2023-06-23T18:49:30.058 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:30.058 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:49:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20500a1150 msgr2=0x7f20500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20500a1150 0x7f20500a3600 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f205800b5c0 tx=0x7f2058005960 comp rx=0 tx=0).stop 2023-06-23T18:49:30.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20681005e0 msgr2=0x7f206819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20681005e0 0x7f206819cb70 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f206000ada0 tx=0x7f206000c330 comp rx=0 tx=0).stop 2023-06-23T18:49:30.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 shutdown_connections 2023-06-23T18:49:30.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f20681005e0 0x7f206819cb70 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f20500a1150 0x7f20500a3600 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20680fea60 0x7f2068068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 --2- 172.21.15.2:0/1242562032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20680ffc50 0x7f206819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 >> 172.21.15.2:0/1242562032 conn(0x7f20680fa0e0 msgr2=0x7f20680fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:30.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 shutdown_connections 2023-06-23T18:49:30.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.055+0000 7f206db2f700 1 -- 172.21.15.2:0/1242562032 wait complete. 2023-06-23T18:49:30.073 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:49:30.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:49:30.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:29 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:49:30.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:29 smithi017 ceph-mon[97472]: scrub summary: active paths [/] 2023-06-23T18:49:30.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:29 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:49:30.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:29 smithi017 ceph-mon[97472]: scrub summary: idle+waiting paths [/] 2023-06-23T18:49:30.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 -- 172.21.15.2:0/3282562010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000ffc90 msgr2=0x7fc0001000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 --2- 172.21.15.2:0/3282562010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000ffc90 0x7fc0001000e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8009990 tx=0x7fbfe8009ca0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 -- 172.21.15.2:0/3282562010 shutdown_connections 2023-06-23T18:49:30.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 --2- 172.21.15.2:0/3282562010 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc000100620 0x7fc000106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 --2- 172.21.15.2:0/3282562010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0000feaa0 0x7fc0000feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 --2- 172.21.15.2:0/3282562010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000ffc90 0x7fc0001000e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 -- 172.21.15.2:0/3282562010 >> 172.21.15.2:0/3282562010 conn(0x7fc0000fa0c0 msgr2=0x7fc0000fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 -- 172.21.15.2:0/3282562010 shutdown_connections 2023-06-23T18:49:30.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.198+0000 7fc00533b700 1 -- 172.21.15.2:0/3282562010 wait complete. 2023-06-23T18:49:30.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fc00533b700 1 Processor -- start 2023-06-23T18:49:30.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fc00533b700 1 -- start start 2023-06-23T18:49:30.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fc00533b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000feaa0 0x7fc000194720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fc00533b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0000ffc90 0x7fc000194c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fc00533b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc000100620 0x7fc00019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fc00533b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00019c2a0 con 0x7fc0000feaa0 2023-06-23T18:49:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fc00533b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00019c3e0 con 0x7fc000100620 2023-06-23T18:49:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fc00533b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00019c570 con 0x7fc0000ffc90 2023-06-23T18:49:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fbffeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000feaa0 0x7fc000194720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fbffeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000feaa0 0x7fc000194720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58590/0 (socket says 172.21.15.2:58590) 2023-06-23T18:49:30.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fbffeffd700 1 -- 172.21.15.2:0/1937005841 learned_addr learned my addr 172.21.15.2:0/1937005841 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:30.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fbfff7fe700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc000100620 0x7fc00019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.199+0000 7fbffe7fc700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0000ffc90 0x7fc000194c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbffeffd700 1 -- 172.21.15.2:0/1937005841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0000ffc90 msgr2=0x7fc000194c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbffeffd700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0000ffc90 0x7fc000194c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbffeffd700 1 -- 172.21.15.2:0/1937005841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc000100620 msgr2=0x7fc00019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbffeffd700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc000100620 0x7fc00019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbffeffd700 1 -- 172.21.15.2:0/1937005841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80095f0 con 0x7fc0000feaa0 2023-06-23T18:49:30.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbffeffd700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000feaa0 0x7fc000194720 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e970 tx=0x7fbff000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:30.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbff7fff700 1 -- 172.21.15.2:0/1937005841 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0009790 con 0x7fc0000feaa0 2023-06-23T18:49:30.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbff7fff700 1 -- 172.21.15.2:0/1937005841 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbff000c7e0 con 0x7fc0000feaa0 2023-06-23T18:49:30.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00019c850 con 0x7fc0000feaa0 2023-06-23T18:49:30.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fbff7fff700 1 -- 172.21.15.2:0/1937005841 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0003bd0 con 0x7fc0000feaa0 2023-06-23T18:49:30.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.200+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00019cd60 con 0x7fc0000feaa0 2023-06-23T18:49:30.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.201+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc000105a20 con 0x7fc0000feaa0 2023-06-23T18:49:30.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.202+0000 7fbff7fff700 1 -- 172.21.15.2:0/1937005841 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbff000c950 con 0x7fc0000feaa0 2023-06-23T18:49:30.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.203+0000 7fbff7fff700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfec0a1150 0x7fbfec0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.203+0000 7fbffe7fc700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfec0a1150 0x7fbfec0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.203+0000 7fbff7fff700 1 -- 172.21.15.2:0/1937005841 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbff0014070 con 0x7fc0000feaa0 2023-06-23T18:49:30.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.203+0000 7fbffe7fc700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfec0a1150 0x7fbfec0a3600 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8006010 tx=0x7fbfe800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:30.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.207+0000 7fbff7fff700 1 -- 172.21.15.2:0/1937005841 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fbff0096250 con 0x7fc0000feaa0 2023-06-23T18:49:30.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:29 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:49:30.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:29 smithi002 ceph-mon[95947]: scrub summary: active paths [/] 2023-06-23T18:49:30.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:29 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:49:30.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:29 smithi002 ceph-mon[95947]: scrub summary: idle+waiting paths [/] 2023-06-23T18:49:30.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:29 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:49:30.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:29 smithi082 ceph-mon[97889]: scrub summary: active paths [/] 2023-06-23T18:49:30.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:29 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:49:30.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:29 smithi082 ceph-mon[97889]: scrub summary: idle+waiting paths [/] 2023-06-23T18:49:30.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.405+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0000610e0 con 0x7fc0000feaa0 2023-06-23T18:49:30.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.406+0000 7fbff7fff700 1 -- 172.21.15.2:0/1937005841 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbff0099870 con 0x7fc0000feaa0 2023-06-23T18:49:30.411 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:49:30.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfec0a1150 msgr2=0x7fbfec0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfec0a1150 0x7fbfec0a3600 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8006010 tx=0x7fbfe800b260 comp rx=0 tx=0).stop 2023-06-23T18:49:30.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000feaa0 msgr2=0x7fc000194720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000feaa0 0x7fc000194720 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e970 tx=0x7fbff000ec80 comp rx=0 tx=0).stop 2023-06-23T18:49:30.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 shutdown_connections 2023-06-23T18:49:30.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fbfec0a1150 0x7fbfec0a3600 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc000100620 0x7fc00019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0000feaa0 0x7fc000194720 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 --2- 172.21.15.2:0/1937005841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0000ffc90 0x7fc000194c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 >> 172.21.15.2:0/1937005841 conn(0x7fc0000fa0c0 msgr2=0x7fc0000fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:30.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 shutdown_connections 2023-06-23T18:49:30.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.408+0000 7fc00533b700 1 -- 172.21.15.2:0/1937005841 wait complete. 2023-06-23T18:49:30.426 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:49:30.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/4207753322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 msgr2=0x7fe9f0106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/4207753322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 0x7fe9f0106ec0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800b0a0 tx=0x7fe9e800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/4207753322 shutdown_connections 2023-06-23T18:49:30.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/4207753322 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 0x7fe9f0106ec0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/4207753322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f00fea80 0x7fe9f00fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/4207753322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f00ffc70 0x7fe9f01000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/4207753322 >> 172.21.15.2:0/4207753322 conn(0x7fe9f00fa0e0 msgr2=0x7fe9f00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:30.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/4207753322 shutdown_connections 2023-06-23T18:49:30.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/4207753322 wait complete. 2023-06-23T18:49:30.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 Processor -- start 2023-06-23T18:49:30.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.551+0000 7fe9f7f5b700 1 -- start start 2023-06-23T18:49:30.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f7f5b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f00fea80 0x7fe9f0068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f7f5b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f00ffc70 0x7fe9f019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f7f5b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 0x7fe9f019cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f7f5b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f019d3f0 con 0x7fe9f0100600 2023-06-23T18:49:30.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f7f5b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f019d560 con 0x7fe9f00ffc70 2023-06-23T18:49:30.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f7f5b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f019d720 con 0x7fe9f00fea80 2023-06-23T18:49:30.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 0x7fe9f019cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 0x7fe9f019cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58608/0 (socket says 172.21.15.2:58608) 2023-06-23T18:49:30.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 -- 172.21.15.2:0/2738842352 learned_addr learned my addr 172.21.15.2:0/2738842352 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:30.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f5cf7700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f00fea80 0x7fe9f0068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f54f6700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f00ffc70 0x7fe9f019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 -- 172.21.15.2:0/2738842352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f00fea80 msgr2=0x7fe9f0068310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f00fea80 0x7fe9f0068310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 -- 172.21.15.2:0/2738842352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f00ffc70 msgr2=0x7fe9f019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f00ffc70 0x7fe9f019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 -- 172.21.15.2:0/2738842352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8009d00 con 0x7fe9f0100600 2023-06-23T18:49:30.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f54f6700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f00ffc70 0x7fe9f019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:49:30.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f64f8700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 0x7fe9f019cc50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800bad0 tx=0x7fe9e800bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:30.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9e6ffd700 1 -- 172.21.15.2:0/2738842352 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800e040 con 0x7fe9f0100600 2023-06-23T18:49:30.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f019d9a0 con 0x7fe9f0100600 2023-06-23T18:49:30.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9e6ffd700 1 -- 172.21.15.2:0/2738842352 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9e8004e40 con 0x7fe9f0100600 2023-06-23T18:49:30.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9e6ffd700 1 -- 172.21.15.2:0/2738842352 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80053b0 con 0x7fe9f0100600 2023-06-23T18:49:30.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.552+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f019de20 con 0x7fe9f0100600 2023-06-23T18:49:30.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.553+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f0106010 con 0x7fe9f0100600 2023-06-23T18:49:30.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.555+0000 7fe9e6ffd700 1 -- 172.21.15.2:0/2738842352 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe9e801da90 con 0x7fe9f0100600 2023-06-23T18:49:30.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.555+0000 7fe9e6ffd700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9dc0a1150 0x7fe9dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.555+0000 7fe9f5cf7700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9dc0a1150 0x7fe9dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.555+0000 7fe9e6ffd700 1 -- 172.21.15.2:0/2738842352 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe9e80ce2d0 con 0x7fe9f0100600 2023-06-23T18:49:30.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.556+0000 7fe9f5cf7700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9dc0a1150 0x7fe9dc0a3600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00ffad0 tx=0x7fe9ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:30.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.558+0000 7fe9e6ffd700 1 -- 172.21.15.2:0/2738842352 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fe9e8097bd0 con 0x7fe9f0100600 2023-06-23T18:49:30.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.756+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9f002ca40 con 0x7fe9f0100600 2023-06-23T18:49:30.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.757+0000 7fe9e6ffd700 1 -- 172.21.15.2:0/2738842352 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9e809b1f0 con 0x7fe9f0100600 2023-06-23T18:49:30.762 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:49:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9dc0a1150 msgr2=0x7fe9dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9dc0a1150 0x7fe9dc0a3600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00ffad0 tx=0x7fe9ec009260 comp rx=0 tx=0).stop 2023-06-23T18:49:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 msgr2=0x7fe9f019cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 0x7fe9f019cc50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800bad0 tx=0x7fe9e800bbb0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 shutdown_connections 2023-06-23T18:49:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9f0100600 0x7fe9f019cc50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9f00fea80 0x7fe9f0068310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fe9dc0a1150 0x7fe9dc0a3600 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 --2- 172.21.15.2:0/2738842352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9f00ffc70 0x7fe9f019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 >> 172.21.15.2:0/2738842352 conn(0x7fe9f00fa0e0 msgr2=0x7fe9f0103150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 shutdown_connections 2023-06-23T18:49:30.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.759+0000 7fe9f7f5b700 1 -- 172.21.15.2:0/2738842352 wait complete. 2023-06-23T18:49:30.776 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:49:30.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.902+0000 7f4402888700 1 -- 172.21.15.2:0/2154021025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc101710 msgr2=0x7f43fc107fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.902+0000 7f4402888700 1 --2- 172.21.15.2:0/2154021025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc101710 0x7f43fc107fd0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f43f400b0a0 tx=0x7f43f400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.902+0000 7f4402888700 1 -- 172.21.15.2:0/2154021025 shutdown_connections 2023-06-23T18:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.902+0000 7f4402888700 1 --2- 172.21.15.2:0/2154021025 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc101710 0x7f43fc107fd0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.902+0000 7f4402888700 1 --2- 172.21.15.2:0/2154021025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43fc0ffb90 0x7f43fc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.902+0000 7f4402888700 1 --2- 172.21.15.2:0/2154021025 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43fc100d80 0x7f43fc1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.902+0000 7f4402888700 1 -- 172.21.15.2:0/2154021025 >> 172.21.15.2:0/2154021025 conn(0x7f43fc0fb1b0 msgr2=0x7f43fc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 -- 172.21.15.2:0/2154021025 shutdown_connections 2023-06-23T18:49:30.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 -- 172.21.15.2:0/2154021025 wait complete. 2023-06-23T18:49:30.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 Processor -- start 2023-06-23T18:49:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 -- start start 2023-06-23T18:49:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc0ffb90 0x7f43fc195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43fc100d80 0x7f43fc195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43fc101710 0x7f43fc19ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc19d4a0 con 0x7f43fc0ffb90 2023-06-23T18:49:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc19d5e0 con 0x7f43fc101710 2023-06-23T18:49:30.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.903+0000 7f4402888700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc19d770 con 0x7f43fc100d80 2023-06-23T18:49:30.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc0ffb90 0x7f43fc195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fb7fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43fc100d80 0x7f43fc195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc0ffb90 0x7f43fc195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58618/0 (socket says 172.21.15.2:58618) 2023-06-23T18:49:30.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 -- 172.21.15.2:0/2263335031 learned_addr learned my addr 172.21.15.2:0/2263335031 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:30.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f4400e25700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43fc101710 0x7f43fc19ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 -- 172.21.15.2:0/2263335031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43fc100d80 msgr2=0x7f43fc195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43fc100d80 0x7f43fc195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 -- 172.21.15.2:0/2263335031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43fc101710 msgr2=0x7f43fc19ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:30.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43fc101710 0x7f43fc19ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:30.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 -- 172.21.15.2:0/2263335031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f4009d00 con 0x7f43fc0ffb90 2023-06-23T18:49:30.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43fbfff700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc0ffb90 0x7f43fc195920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00c970 tx=0x7f43ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:30.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43f97fa700 1 -- 172.21.15.2:0/2263335031 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec004220 con 0x7f43fc0ffb90 2023-06-23T18:49:30.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43f97fa700 1 -- 172.21.15.2:0/2263335031 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43ec004380 con 0x7f43fc0ffb90 2023-06-23T18:49:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f43f97fa700 1 -- 172.21.15.2:0/2263335031 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec003ae0 con 0x7f43fc0ffb90 2023-06-23T18:49:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc19da50 con 0x7f43fc0ffb90 2023-06-23T18:49:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.904+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc19df10 con 0x7f43fc0ffb90 2023-06-23T18:49:30.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.905+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc107130 con 0x7f43fc0ffb90 2023-06-23T18:49:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.907+0000 7f43f97fa700 1 -- 172.21.15.2:0/2263335031 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f43ec003c40 con 0x7f43fc0ffb90 2023-06-23T18:49:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.908+0000 7f43f97fa700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43e40a1150 0x7f43e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.908+0000 7f43f97fa700 1 -- 172.21.15.2:0/2263335031 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f43ec0cc7e0 con 0x7f43fc0ffb90 2023-06-23T18:49:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.908+0000 7f43fb7fe700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43e40a1150 0x7f43e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:30.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.908+0000 7f43fb7fe700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43e40a1150 0x7f43e40a3600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005950 tx=0x7f43f000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:30.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:30.911+0000 7f43f97fa700 1 -- 172.21.15.2:0/2263335031 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f43ec095ef0 con 0x7f43fc0ffb90 2023-06-23T18:49:31.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:30 smithi002 ceph-mon[95947]: pgmap v1120: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2023-06-23T18:49:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:30 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1242562032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:49:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:30 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1937005841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:49:31.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:30 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2738842352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:49:31.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.112+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43fc02ca80 con 0x7f43fc0ffb90 2023-06-23T18:49:31.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.112+0000 7f43f97fa700 1 -- 172.21.15.2:0/2263335031 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43ec01d0a0 con 0x7f43fc0ffb90 2023-06-23T18:49:31.117 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:49:31.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43e40a1150 msgr2=0x7f43e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:31.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43e40a1150 0x7f43e40a3600 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005950 tx=0x7f43f000b410 comp rx=0 tx=0).stop 2023-06-23T18:49:31.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc0ffb90 msgr2=0x7f43fc195920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:31.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc0ffb90 0x7f43fc195920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00c970 tx=0x7f43ec00cc80 comp rx=0 tx=0).stop 2023-06-23T18:49:31.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 shutdown_connections 2023-06-23T18:49:31.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f43e40a1150 0x7f43e40a3600 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43fc101710 0x7f43fc19ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f43fc0ffb90 0x7f43fc195920 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 --2- 172.21.15.2:0/2263335031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43fc100d80 0x7f43fc195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 >> 172.21.15.2:0/2263335031 conn(0x7f43fc0fb1b0 msgr2=0x7f43fc104270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:31.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 shutdown_connections 2023-06-23T18:49:31.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.114+0000 7f4402888700 1 -- 172.21.15.2:0/2263335031 wait complete. 2023-06-23T18:49:31.132 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-06-23T18:49:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:30 smithi017 ceph-mon[97472]: pgmap v1120: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2023-06-23T18:49:31.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:30 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1242562032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:49:31.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:30 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1937005841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:49:31.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:30 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2738842352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:49:31.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 -- 172.21.15.2:0/1756503701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380ffc50 msgr2=0x7f8f381000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:31.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/1756503701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380ffc50 0x7f8f381000a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009990 tx=0x7f8f34009ca0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 -- 172.21.15.2:0/1756503701 shutdown_connections 2023-06-23T18:49:31.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/1756503701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f381005e0 0x7f8f38106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/1756503701 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f380fea60 0x7f8f380fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/1756503701 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380ffc50 0x7f8f381000a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 -- 172.21.15.2:0/1756503701 >> 172.21.15.2:0/1756503701 conn(0x7f8f380fa0e0 msgr2=0x7f8f380fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:31.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 -- 172.21.15.2:0/1756503701 shutdown_connections 2023-06-23T18:49:31.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.253+0000 7f8f3fab3700 1 -- 172.21.15.2:0/1756503701 wait complete. 2023-06-23T18:49:31.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3fab3700 1 Processor -- start 2023-06-23T18:49:31.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3fab3700 1 -- start start 2023-06-23T18:49:31.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3fab3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380fea60 0x7f8f381946a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:31.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3fab3700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f380ffc50 0x7f8f38194be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:31.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3fab3700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f381005e0 0x7f8f3819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:31.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3fab3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3819c220 con 0x7f8f380fea60 2023-06-23T18:49:31.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3fab3700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3819c360 con 0x7f8f380ffc50 2023-06-23T18:49:31.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3fab3700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3819c4f0 con 0x7f8f381005e0 2023-06-23T18:49:31.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3d84f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380fea60 0x7f8f381946a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:31.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3d84f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380fea60 0x7f8f381946a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58632/0 (socket says 172.21.15.2:58632) 2023-06-23T18:49:31.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3d84f700 1 -- 172.21.15.2:0/3429087876 learned_addr learned my addr 172.21.15.2:0/3429087876 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:49:31.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3d04e700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f380ffc50 0x7f8f38194be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:31.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.254+0000 7f8f3e050700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f381005e0 0x7f8f3819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:31.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f3d84f700 1 -- 172.21.15.2:0/3429087876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f381005e0 msgr2=0x7f8f3819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:31.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f3d84f700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f381005e0 0x7f8f3819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f3d84f700 1 -- 172.21.15.2:0/3429087876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f380ffc50 msgr2=0x7f8f38194be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:31.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f3d84f700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f380ffc50 0x7f8f38194be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:31.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f3d84f700 1 -- 172.21.15.2:0/3429087876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f340095f0 con 0x7f8f380fea60 2023-06-23T18:49:31.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f3d84f700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380fea60 0x7f8f381946a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00e970 tx=0x7f8f2c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:31.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f2affd700 1 -- 172.21.15.2:0/3429087876 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c009790 con 0x7f8f380fea60 2023-06-23T18:49:31.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f2affd700 1 -- 172.21.15.2:0/3429087876 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f2c00c7e0 con 0x7f8f380fea60 2023-06-23T18:49:31.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f3819c7d0 con 0x7f8f380fea60 2023-06-23T18:49:31.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f2affd700 1 -- 172.21.15.2:0/3429087876 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c003bd0 con 0x7f8f380fea60 2023-06-23T18:49:31.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.255+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f3819cd30 con 0x7f8f380fea60 2023-06-23T18:49:31.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.256+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f3810a0e0 con 0x7f8f380fea60 2023-06-23T18:49:31.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.257+0000 7f8f2affd700 1 -- 172.21.15.2:0/3429087876 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8f2c00c950 con 0x7f8f380fea60 2023-06-23T18:49:31.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.258+0000 7f8f2affd700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8f240a11d0 0x7f8f240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:49:31.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.258+0000 7f8f2affd700 1 -- 172.21.15.2:0/3429087876 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8f2c014070 con 0x7f8f380fea60 2023-06-23T18:49:31.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.258+0000 7f8f3d04e700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8f240a11d0 0x7f8f240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:49:31.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.258+0000 7f8f3d04e700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8f240a11d0 0x7f8f240a3680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b2e0 tx=0x7f8f34004000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:49:31.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.261+0000 7f8f2affd700 1 -- 172.21.15.2:0/3429087876 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f8f2c096300 con 0x7f8f380fea60 2023-06-23T18:49:31.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:30 smithi082 ceph-mon[97889]: pgmap v1120: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 9.7 KiB/s wr, 9 op/s 2023-06-23T18:49:31.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:30 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1242562032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:49:31.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:30 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1937005841' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:49:31.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:30 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2738842352' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:49:31.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.517+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f8f381057e0 con 0x7f8f380fea60 2023-06-23T18:49:31.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.525+0000 7f8f2affd700 1 -- 172.21.15.2:0/3429087876 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) v1 ==== 81+0+733234 (secure 0 0 0) 0x7f8f2c01f340 con 0x7f8f380fea60 2023-06-23T18:49:31.537 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:49:31.538 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:49:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 18, 2023-06-23T18:49:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "active_gid": 14150, 2023-06-23T18:49:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "active_name": "x", 2023-06-23T18:49:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addrs": { 2023-06-23T18:49:31.538 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-06-23T18:49:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-06-23T18:49:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2023-06-23T18:49:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3622114004 2023-06-23T18:49:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2023-06-23T18:49:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2023-06-23T18:49:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3622114004 2023-06-23T18:49:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.540 INFO:teuthology.orchestra.run.smithi002.stdout: "active_addr": "172.21.15.2:6801/3622114004", 2023-06-23T18:49:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "active_change": "2023-06-23T18:14:36.653692+0000", 2023-06-23T18:49:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "active_mgr_features": 4540138322906710015, 2023-06-23T18:49:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "available": true, 2023-06-23T18:49:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "standbys": [ 2023-06-23T18:49:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 24109, 2023-06-23T18:49:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "z", 2023-06-23T18:49:31.541 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540138322906710015, 2023-06-23T18:49:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2023-06-23T18:49:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2023-06-23T18:49:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:31.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.543 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2023-06-23T18:49:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.545 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.546 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.549 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.550 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.551 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2023-06-23T18:49:31.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2023-06-23T18:49:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2023-06-23T18:49:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2023-06-23T18:49:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2023-06-23T18:49:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.555 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2023-06-23T18:49:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2023-06-23T18:49:31.556 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2023-06-23T18:49:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2023-06-23T18:49:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2023-06-23T18:49:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2023-06-23T18:49:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2023-06-23T18:49:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2023-06-23T18:49:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2023-06-23T18:49:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2023-06-23T18:49:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2023-06-23T18:49:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2023-06-23T18:49:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.562 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2023-06-23T18:49:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2023-06-23T18:49:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2023-06-23T18:49:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2023-06-23T18:49:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2023-06-23T18:49:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2023-06-23T18:49:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2023-06-23T18:49:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2023-06-23T18:49:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.567 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2023-06-23T18:49:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2023-06-23T18:49:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2023-06-23T18:49:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2023-06-23T18:49:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2023-06-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2023-06-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2023-06-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2023-06-23T18:49:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2023-06-23T18:49:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-06-23T18:49:31.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2023-06-23T18:49:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2023-06-23T18:49:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-06-23T18:49:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-06-23T18:49:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-06-23T18:49:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-06-23T18:49:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.574 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2023-06-23T18:49:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2023-06-23T18:49:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2023-06-23T18:49:31.575 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2023-06-23T18:49:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-06-23T18:49:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.576 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2023-06-23T18:49:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2023-06-23T18:49:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2023-06-23T18:49:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.577 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-06-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-06-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2023-06-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2023-06-23T18:49:31.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2023-06-23T18:49:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2023-06-23T18:49:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2023-06-23T18:49:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2023-06-23T18:49:31.579 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-06-23T18:49:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2023-06-23T18:49:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2023-06-23T18:49:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.580 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2023-06-23T18:49:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2023-06-23T18:49:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-06-23T18:49:31.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2023-06-23T18:49:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2023-06-23T18:49:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-06-23T18:49:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-06-23T18:49:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-06-23T18:49:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-06-23T18:49:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.583 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.584 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.585 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.586 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.587 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.588 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.589 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.590 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.592 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2023-06-23T18:49:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2023-06-23T18:49:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-06-23T18:49:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.594 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-06-23T18:49:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-06-23T18:49:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.595 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2023-06-23T18:49:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2023-06-23T18:49:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2023-06-23T18:49:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap" 2023-06-23T18:49:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.596 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2023-06-23T18:49:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2023-06-23T18:49:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2023-06-23T18:49:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.597 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-06-23T18:49:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.598 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.599 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-06-23T18:49:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2023-06-23T18:49:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2023-06-23T18:49:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:31.601 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2023-06-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-06-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2023-06-23T18:49:31.602 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2023-06-23T18:49:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.603 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-06-23T18:49:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2023-06-23T18:49:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2023-06-23T18:49:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2023-06-23T18:49:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:31.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-06-23T18:49:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-06-23T18:49:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2023-06-23T18:49:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2023-06-23T18:49:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.607 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2023-06-23T18:49:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-06-23T18:49:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2023-06-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2023-06-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2023-06-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-06-23T18:49:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "alertmanager_web_password": { 2023-06-23T18:49:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alertmanager_web_password", 2023-06-23T18:49:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.611 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Alertmanager web password", 2023-06-23T18:49:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "alertmanager_web_user": { 2023-06-23T18:49:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alertmanager_web_user", 2023-06-23T18:49:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Alertmanager web user", 2023-06-23T18:49:31.615 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.616 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2023-06-23T18:49:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2023-06-23T18:49:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-06-23T18:49:31.619 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-06-23T18:49:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2023-06-23T18:49:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2023-06-23T18:49:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2023-06-23T18:49:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2023-06-23T18:49:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2023-06-23T18:49:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2023-06-23T18:49:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-06-23T18:49:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2023-06-23T18:49:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2023-06-23T18:49:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-06-23T18:49:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2023-06-23T18:49:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2023-06-23T18:49:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-06-23T18:49:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2023-06-23T18:49:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2023-06-23T18:49:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-06-23T18:49:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2023-06-23T18:49:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2023-06-23T18:49:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-06-23T18:49:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2023-06-23T18:49:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2023-06-23T18:49:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2023-06-23T18:49:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2023-06-23T18:49:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2023-06-23T18:49:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2023-06-23T18:49:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-06-23T18:49:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2023-06-23T18:49:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2023-06-23T18:49:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2023-06-23T18:49:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-06-23T18:49:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2023-06-23T18:49:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2023-06-23T18:49:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-06-23T18:49:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2023-06-23T18:49:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2023-06-23T18:49:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2023-06-23T18:49:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-06-23T18:49:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2023-06-23T18:49:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2023-06-23T18:49:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2023-06-23T18:49:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-06-23T18:49:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2023-06-23T18:49:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2023-06-23T18:49:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2023-06-23T18:49:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-06-23T18:49:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2023-06-23T18:49:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2023-06-23T18:49:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2023-06-23T18:49:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-06-23T18:49:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2023-06-23T18:49:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2023-06-23T18:49:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2023-06-23T18:49:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-06-23T18:49:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2023-06-23T18:49:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2023-06-23T18:49:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2023-06-23T18:49:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-06-23T18:49:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2023-06-23T18:49:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2023-06-23T18:49:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-06-23T18:49:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2023-06-23T18:49:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2023-06-23T18:49:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-06-23T18:49:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2023-06-23T18:49:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2023-06-23T18:49:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2023-06-23T18:49:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-06-23T18:49:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2023-06-23T18:49:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2023-06-23T18:49:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2023-06-23T18:49:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2023-06-23T18:49:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2023-06-23T18:49:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2023-06-23T18:49:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-06-23T18:49:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2023-06-23T18:49:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2023-06-23T18:49:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2023-06-23T18:49:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-06-23T18:49:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2023-06-23T18:49:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2023-06-23T18:49:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2023-06-23T18:49:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-06-23T18:49:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2023-06-23T18:49:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2023-06-23T18:49:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2023-06-23T18:49:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2023-06-23T18:49:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2023-06-23T18:49:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2023-06-23T18:49:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2023-06-23T18:49:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2023-06-23T18:49:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2023-06-23T18:49:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2023-06-23T18:49:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2023-06-23T18:49:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2023-06-23T18:49:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2023-06-23T18:49:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2023-06-23T18:49:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2023-06-23T18:49:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-06-23T18:49:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-06-23T18:49:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2023-06-23T18:49:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-06-23T18:49:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-06-23T18:49:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-06-23T18:49:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-06-23T18:49:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2023-06-23T18:49:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-06-23T18:49:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2023-06-23T18:49:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2023-06-23T18:49:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2023-06-23T18:49:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2023-06-23T18:49:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2023-06-23T18:49:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-06-23T18:49:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2023-06-23T18:49:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2023-06-23T18:49:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2023-06-23T18:49:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-06-23T18:49:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-06-23T18:49:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2023-06-23T18:49:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2023-06-23T18:49:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2023-06-23T18:49:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2023-06-23T18:49:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2023-06-23T18:49:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2023-06-23T18:49:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-06-23T18:49:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-06-23T18:49:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_web_password": { 2023-06-23T18:49:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_web_password", 2023-06-23T18:49:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus web password", 2023-06-23T18:49:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_web_user": { 2023-06-23T18:49:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_web_user", 2023-06-23T18:49:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.712 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus web user", 2023-06-23T18:49:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2023-06-23T18:49:31.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2023-06-23T18:49:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.714 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-06-23T18:49:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2023-06-23T18:49:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2023-06-23T18:49:31.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-06-23T18:49:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2023-06-23T18:49:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2023-06-23T18:49:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-06-23T18:49:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2023-06-23T18:49:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2023-06-23T18:49:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-06-23T18:49:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2023-06-23T18:49:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2023-06-23T18:49:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-06-23T18:49:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2023-06-23T18:49:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2023-06-23T18:49:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2023-06-23T18:49:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2023-06-23T18:49:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2023-06-23T18:49:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2023-06-23T18:49:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-06-23T18:49:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2023-06-23T18:49:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2023-06-23T18:49:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-06-23T18:49:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2023-06-23T18:49:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2023-06-23T18:49:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-06-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2023-06-23T18:49:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2023-06-23T18:49:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2023-06-23T18:49:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2023-06-23T18:49:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2023-06-23T18:49:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-06-23T18:49:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2023-06-23T18:49:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2023-06-23T18:49:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-06-23T18:49:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2023-06-23T18:49:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2023-06-23T18:49:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2023-06-23T18:49:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2023-06-23T18:49:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2023-06-23T18:49:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2023-06-23T18:49:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2023-06-23T18:49:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2023-06-23T18:49:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2023-06-23T18:49:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2023-06-23T18:49:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-06-23T18:49:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-06-23T18:49:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2023-06-23T18:49:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2023-06-23T18:49:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-06-23T18:49:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-06-23T18:49:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2023-06-23T18:49:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2023-06-23T18:49:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-06-23T18:49:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-06-23T18:49:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2023-06-23T18:49:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2023-06-23T18:49:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-06-23T18:49:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-06-23T18:49:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-06-23T18:49:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-06-23T18:49:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-06-23T18:49:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-06-23T18:49:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-06-23T18:49:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-06-23T18:49:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2023-06-23T18:49:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-06-23T18:49:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2023-06-23T18:49:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-06-23T18:49:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2023-06-23T18:49:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-06-23T18:49:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-06-23T18:49:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-06-23T18:49:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2023-06-23T18:49:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2023-06-23T18:49:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-06-23T18:49:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-06-23T18:49:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2023-06-23T18:49:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2023-06-23T18:49:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2023-06-23T18:49:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2023-06-23T18:49:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-06-23T18:49:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-06-23T18:49:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-06-23T18:49:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-06-23T18:49:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-06-23T18:49:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-06-23T18:49:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2023-06-23T18:49:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-06-23T18:49:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2023-06-23T18:49:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2023-06-23T18:49:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-06-23T18:49:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-06-23T18:49:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-06-23T18:49:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-06-23T18:49:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-06-23T18:49:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-06-23T18:49:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-06-23T18:49:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-06-23T18:49:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-06-23T18:49:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-06-23T18:49:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-06-23T18:49:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-06-23T18:49:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-06-23T18:49:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-06-23T18:49:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-06-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-06-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2023-06-23T18:49:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2023-06-23T18:49:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-06-23T18:49:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-06-23T18:49:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.807 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.807 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-06-23T18:49:31.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.807 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.807 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.808 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.808 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-06-23T18:49:31.808 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-06-23T18:49:31.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.808 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-06-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-06-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2023-06-23T18:49:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2023-06-23T18:49:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-06-23T18:49:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2023-06-23T18:49:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2023-06-23T18:49:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2023-06-23T18:49:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-06-23T18:49:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-06-23T18:49:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2023-06-23T18:49:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2023-06-23T18:49:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2023-06-23T18:49:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2023-06-23T18:49:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-06-23T18:49:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-06-23T18:49:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-06-23T18:49:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-06-23T18:49:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-06-23T18:49:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-06-23T18:49:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2023-06-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2023-06-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2023-06-23T18:49:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2023-06-23T18:49:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2023-06-23T18:49:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2023-06-23T18:49:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2023-06-23T18:49:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2023-06-23T18:49:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2023-06-23T18:49:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2023-06-23T18:49:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-06-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-06-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2023-06-23T18:49:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2023-06-23T18:49:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2023-06-23T18:49:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2023-06-23T18:49:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2023-06-23T18:49:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-06-23T18:49:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2023-06-23T18:49:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-06-23T18:49:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-06-23T18:49:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2023-06-23T18:49:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2023-06-23T18:49:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2023-06-23T18:49:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-06-23T18:49:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-06-23T18:49:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2023-06-23T18:49:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2023-06-23T18:49:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-06-23T18:49:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-06-23T18:49:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-06-23T18:49:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-06-23T18:49:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-06-23T18:49:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2023-06-23T18:49:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2023-06-23T18:49:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2023-06-23T18:49:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2023-06-23T18:49:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2023-06-23T18:49:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2023-06-23T18:49:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2023-06-23T18:49:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2023-06-23T18:49:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2023-06-23T18:49:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-06-23T18:49:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2023-06-23T18:49:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2023-06-23T18:49:31.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.874 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.874 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2023-06-23T18:49:31.874 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.874 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.874 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2023-06-23T18:49:31.875 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.875 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.875 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2023-06-23T18:49:31.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2023-06-23T18:49:31.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.876 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2023-06-23T18:49:31.876 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2023-06-23T18:49:31.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.877 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.877 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2023-06-23T18:49:31.877 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2023-06-23T18:49:31.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.877 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.877 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-06-23T18:49:31.878 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.878 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.878 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.878 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2023-06-23T18:49:31.878 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.878 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.879 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2023-06-23T18:49:31.879 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2023-06-23T18:49:31.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.879 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:31.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.879 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.880 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-06-23T18:49:31.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.880 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.880 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:31.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:31.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.881 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.881 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:31.881 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.881 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.881 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.882 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2023-06-23T18:49:31.882 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.882 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.882 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2023-06-23T18:49:31.882 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2023-06-23T18:49:31.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.883 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.883 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.883 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2023-06-23T18:49:31.883 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.883 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.883 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-06-23T18:49:31.883 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.884 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.884 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2023-06-23T18:49:31.884 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.885 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.885 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.885 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.885 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.885 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.886 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.886 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.886 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.886 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.886 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.888 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.888 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.888 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.888 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.888 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.889 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.889 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.889 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.889 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.889 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.890 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.890 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.890 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.890 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.890 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.890 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.890 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.891 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.891 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.891 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.891 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.891 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.891 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.891 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.893 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.893 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.893 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.893 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.894 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2023-06-23T18:49:31.894 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2023-06-23T18:49:31.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-06-23T18:49:31.895 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.895 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.895 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.896 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2023-06-23T18:49:31.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2023-06-23T18:49:31.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.896 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.896 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2023-06-23T18:49:31.896 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.896 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.897 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.897 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.897 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.897 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.897 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.897 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:31.897 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:31.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.898 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.898 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:31.898 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.898 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.898 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.898 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.899 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.899 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.899 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.899 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.899 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2023-06-23T18:49:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2023-06-23T18:49:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2023-06-23T18:49:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2023-06-23T18:49:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2023-06-23T18:49:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.900 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2023-06-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-06-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.902 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.902 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2023-06-23T18:49:31.902 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2023-06-23T18:49:31.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.902 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.902 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2023-06-23T18:49:31.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.903 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.903 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-06-23T18:49:31.903 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.903 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.904 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2023-06-23T18:49:31.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2023-06-23T18:49:31.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.904 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.904 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.904 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.904 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.905 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.905 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2023-06-23T18:49:31.905 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.905 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.905 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:31.905 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:31.906 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:31.906 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.906 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.906 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2023-06-23T18:49:31.906 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2023-06-23T18:49:31.906 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.906 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-06-23T18:49:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.908 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.908 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.909 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.909 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.909 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.909 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.909 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.909 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.909 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.909 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.910 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.910 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.910 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.910 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.910 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.911 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.911 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.911 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.911 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.911 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.912 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.912 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.912 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.912 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.912 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.912 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.913 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.913 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.913 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.913 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.913 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.913 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.913 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.913 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.914 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.915 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.915 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.915 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.915 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.915 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.915 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.916 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.916 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.916 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2023-06-23T18:49:31.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2023-06-23T18:49:31.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.916 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.917 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.917 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.917 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2023-06-23T18:49:31.917 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.918 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.918 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2023-06-23T18:49:31.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2023-06-23T18:49:31.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.918 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2023-06-23T18:49:31.919 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.919 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.919 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.919 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2023-06-23T18:49:31.919 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.919 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.919 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.920 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-06-23T18:49:31.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-06-23T18:49:31.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.920 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.920 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2023-06-23T18:49:31.920 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.920 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-06-23T18:49:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2023-06-23T18:49:31.921 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2023-06-23T18:49:31.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.922 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.922 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:31.922 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:31.922 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2023-06-23T18:49:31.922 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.922 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-06-23T18:49:31.923 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.923 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.923 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2023-06-23T18:49:31.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2023-06-23T18:49:31.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.924 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.924 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2023-06-23T18:49:31.924 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.924 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.925 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.925 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2023-06-23T18:49:31.925 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2023-06-23T18:49:31.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.925 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.925 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.925 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2023-06-23T18:49:31.926 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.926 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.926 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.926 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-06-23T18:49:31.926 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.926 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.926 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2023-06-23T18:49:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.927 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.928 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.928 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.928 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.928 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.928 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.929 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.929 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.929 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.929 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.929 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.929 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.929 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.929 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.930 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.931 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.931 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.931 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.931 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.932 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.932 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.932 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.933 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.934 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.934 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.934 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.934 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.934 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.934 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.934 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.935 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.936 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.936 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.936 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.936 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.937 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2023-06-23T18:49:31.937 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.937 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.937 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.937 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.937 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.938 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.938 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.938 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.938 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.938 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.938 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.939 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.939 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.939 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.939 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.939 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.939 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.940 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.940 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.940 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.940 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.940 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.941 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.941 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.941 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.941 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.941 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.943 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.943 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.943 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.943 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.943 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.943 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.943 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.944 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.944 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.944 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.944 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.944 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.945 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.945 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.945 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.945 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.945 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.946 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.946 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.946 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.947 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2023-06-23T18:49:31.947 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.947 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.947 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.947 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2023-06-23T18:49:31.947 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2023-06-23T18:49:31.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.947 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.948 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2023-06-23T18:49:31.948 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.948 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.948 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.948 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2023-06-23T18:49:31.948 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2023-06-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2023-06-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:31.950 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:31.950 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2023-06-23T18:49:31.950 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.950 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.950 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-06-23T18:49:31.950 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.950 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.951 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.951 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.951 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.951 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.951 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.952 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.952 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.952 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.952 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.952 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.952 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.952 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.952 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.953 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.954 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.954 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.954 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.954 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.954 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.954 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.955 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.955 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.955 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.955 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.956 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.956 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.956 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.956 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.956 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.956 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.956 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.957 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.957 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.957 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.957 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.957 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.957 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.957 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.958 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.958 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.958 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.958 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.959 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.959 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.959 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.959 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.959 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.959 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.959 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.960 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2023-06-23T18:49:31.960 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.960 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2023-06-23T18:49:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2023-06-23T18:49:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2023-06-23T18:49:31.961 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.962 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.962 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2023-06-23T18:49:31.962 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-06-23T18:49:31.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.962 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.962 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.963 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.963 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.963 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.963 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.963 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.963 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.964 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.965 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.967 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.968 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.969 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.970 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2023-06-23T18:49:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2023-06-23T18:49:31.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2023-06-23T18:49:31.972 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-06-23T18:49:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2023-06-23T18:49:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2023-06-23T18:49:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.973 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2023-06-23T18:49:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2023-06-23T18:49:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.974 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2023-06-23T18:49:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2023-06-23T18:49:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.975 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2023-06-23T18:49:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2023-06-23T18:49:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2023-06-23T18:49:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2023-06-23T18:49:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.977 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2023-06-23T18:49:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2023-06-23T18:49:31.978 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2023-06-23T18:49:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2023-06-23T18:49:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.979 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2023-06-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-06-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.980 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2023-06-23T18:49:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.981 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.982 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.983 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.984 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.987 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.989 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:31.990 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2023-06-23T18:49:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.992 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.994 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:31.995 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:31.996 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.997 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:31.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:31.999 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2023-06-23T18:49:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.000 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.001 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.002 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.003 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.004 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.006 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.008 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-06-23T18:49:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.010 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2023-06-23T18:49:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2023-06-23T18:49:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.011 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-06-23T18:49:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.012 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.013 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.014 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.015 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.017 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.018 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.019 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.020 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2023-06-23T18:49:32.021 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-06-23T18:49:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.022 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2023-06-23T18:49:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2023-06-23T18:49:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2023-06-23T18:49:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2023-06-23T18:49:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2023-06-23T18:49:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.024 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.026 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.027 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.028 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.030 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.031 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.032 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2023-06-23T18:49:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.035 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.037 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.038 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.039 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.040 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.041 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.042 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2023-06-23T18:49:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.045 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.047 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.049 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.050 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.051 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.052 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "noautoscale": { 2023-06-23T18:49:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "noautoscale", 2023-06-23T18:49:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.053 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "global autoscale flag", 2023-06-23T18:49:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-06-23T18:49:32.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.055 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.056 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2023-06-23T18:49:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2023-06-23T18:49:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-06-23T18:49:32.057 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2023-06-23T18:49:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2023-06-23T18:49:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-06-23T18:49:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2023-06-23T18:49:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.059 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2023-06-23T18:49:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2023-06-23T18:49:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.060 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2023-06-23T18:49:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-06-23T18:49:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-06-23T18:49:32.061 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.062 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.064 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.064 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.064 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.064 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.064 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.064 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.064 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.065 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.065 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.065 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.065 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.065 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.065 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.065 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.066 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.067 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.067 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.067 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.067 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.067 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.067 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.067 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.068 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.069 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.069 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.069 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.069 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.069 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.069 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.069 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.069 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.070 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.070 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.070 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.070 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.070 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.071 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.071 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.071 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.071 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.071 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.071 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.071 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.071 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.072 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.072 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2023-06-23T18:49:32.072 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2023-06-23T18:49:32.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.072 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.073 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2023-06-23T18:49:32.073 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.073 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.073 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.073 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2023-06-23T18:49:32.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.073 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:32.074 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.075 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.075 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.075 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2023-06-23T18:49:32.075 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.075 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.075 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.076 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2023-06-23T18:49:32.076 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.076 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.076 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.076 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2023-06-23T18:49:32.077 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2023-06-23T18:49:32.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.077 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.077 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.077 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.077 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.077 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.078 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.078 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.078 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.078 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2023-06-23T18:49:32.078 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2023-06-23T18:49:32.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.079 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.079 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.079 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.079 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.079 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.079 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-06-23T18:49:32.079 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-06-23T18:49:32.080 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.080 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.080 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.081 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.081 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.081 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.081 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.081 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.081 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.081 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.081 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.082 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.082 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.082 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.082 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.082 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.082 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.082 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.083 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.083 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.083 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.083 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.083 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.085 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.085 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.085 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.085 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.085 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.085 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.086 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.086 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.086 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.086 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.086 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.086 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.086 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.086 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.087 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.087 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.087 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.087 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.087 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.088 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.088 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.088 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.088 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.088 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.088 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.088 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2023-06-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2023-06-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.090 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.091 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.091 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2023-06-23T18:49:32.091 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-06-23T18:49:32.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.091 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.091 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2023-06-23T18:49:32.092 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.092 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.092 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.092 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.092 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.092 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.092 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2023-06-23T18:49:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2023-06-23T18:49:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2023-06-23T18:49:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.093 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.094 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.094 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.094 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.094 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.094 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:32.094 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:32.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.095 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.095 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.095 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-06-23T18:49:32.095 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.095 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.095 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.095 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-06-23T18:49:32.096 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.096 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.096 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.096 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:32.096 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:32.096 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.096 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.097 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2023-06-23T18:49:32.097 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.097 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.097 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.097 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-06-23T18:49:32.097 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.098 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2023-06-23T18:49:32.098 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2023-06-23T18:49:32.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.098 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.099 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2023-06-23T18:49:32.099 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.099 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.099 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.099 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.099 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.099 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.099 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.100 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-06-23T18:49:32.100 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-06-23T18:49:32.100 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.100 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.100 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.100 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2023-06-23T18:49:32.100 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.101 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.101 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.101 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2023-06-23T18:49:32.101 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2023-06-23T18:49:32.101 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.101 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.101 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.102 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.102 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-06-23T18:49:32.102 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-06-23T18:49:32.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.102 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.102 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-06-23T18:49:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-06-23T18:49:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-06-23T18:49:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.103 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.104 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.104 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.104 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2023-06-23T18:49:32.104 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.104 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.105 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.106 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.107 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.107 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.107 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.107 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.107 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.107 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.108 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.108 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.108 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.108 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.108 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.108 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.109 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.110 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.110 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.110 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.110 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.110 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.110 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.110 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.111 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.111 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.111 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.111 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.111 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.112 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.112 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.112 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.113 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.113 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.113 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.113 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.114 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2023-06-23T18:49:32.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2023-06-23T18:49:32.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.114 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.114 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.114 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.114 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2023-06-23T18:49:32.115 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2023-06-23T18:49:32.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.116 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.116 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.116 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.116 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.116 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.116 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.117 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.117 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.117 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.117 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2023-06-23T18:49:32.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2023-06-23T18:49:32.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.117 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.118 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.118 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.118 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.118 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.119 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.119 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2023-06-23T18:49:32.119 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.119 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.120 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.120 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2023-06-23T18:49:32.120 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2023-06-23T18:49:32.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.120 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.120 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.120 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.121 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.121 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.121 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.121 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.122 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-06-23T18:49:32.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-06-23T18:49:32.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.122 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.122 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.122 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.122 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.123 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.124 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.124 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.124 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.124 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.124 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.124 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.124 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.125 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.125 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.125 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.125 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.125 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.125 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.125 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.125 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.126 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.126 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.126 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.126 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.127 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.127 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.127 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.127 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.127 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.127 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.127 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.128 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.128 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.128 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.128 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.129 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.129 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.129 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.129 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.129 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.129 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.130 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.130 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.130 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.130 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.130 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.130 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.130 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.131 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.131 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.131 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.132 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.132 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.132 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.132 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.132 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.132 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.133 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:32.133 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:32.133 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.133 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.133 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.133 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.133 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.133 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.134 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.134 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.134 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.134 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.134 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.134 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:32.134 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:32.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.135 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.135 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.135 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.135 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.135 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.135 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2023-06-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.137 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.138 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.138 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.138 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.138 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.138 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.138 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.138 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.139 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.140 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.141 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.141 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.141 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.141 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.141 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.141 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.142 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.142 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.142 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.142 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.143 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.143 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.143 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.143 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.143 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.143 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.143 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.143 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.144 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.144 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.144 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.144 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.144 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.145 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.145 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.145 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.145 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.145 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.146 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2023-06-23T18:49:32.146 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.147 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.147 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.147 INFO:teuthology.orchestra.run.smithi002.stdout: "drive_group_interval": { 2023-06-23T18:49:32.147 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "drive_group_interval", 2023-06-23T18:49:32.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.147 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.147 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.147 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300.0", 2023-06-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-06-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.148 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.149 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.150 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.150 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.150 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.150 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.150 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.150 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.150 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.151 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.151 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.151 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.151 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.151 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.151 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.152 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.152 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.152 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.152 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.152 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.152 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.153 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.154 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.155 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.155 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.155 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.155 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.155 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.155 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.155 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.156 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2023-06-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2023-06-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2023-06-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-06-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.160 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2023-06-23T18:49:32.160 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.160 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.160 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.160 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.160 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.161 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.161 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.161 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.161 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.161 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.161 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.162 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.164 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.164 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.164 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.164 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.164 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.164 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.164 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.165 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.165 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.165 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.165 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.165 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.165 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.165 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.166 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.166 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.166 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.166 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.166 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.166 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.166 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.166 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.167 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.168 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.169 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.169 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.169 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2023-06-23T18:49:32.169 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2023-06-23T18:49:32.169 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.169 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.170 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.171 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.171 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2023-06-23T18:49:32.171 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2023-06-23T18:49:32.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.171 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.172 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2023-06-23T18:49:32.172 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.172 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.172 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.172 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.172 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.173 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2023-06-23T18:49:32.173 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2023-06-23T18:49:32.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.173 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.173 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.173 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.174 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2023-06-23T18:49:32.175 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2023-06-23T18:49:32.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.175 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.175 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.175 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.175 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.175 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.176 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.176 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.176 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.176 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2023-06-23T18:49:32.176 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2023-06-23T18:49:32.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.177 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.178 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.178 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2023-06-23T18:49:32.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2023-06-23T18:49:32.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.178 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.179 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.179 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.179 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.179 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.180 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.180 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2023-06-23T18:49:32.180 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2023-06-23T18:49:32.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.180 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.181 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.181 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.181 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.181 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.181 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.181 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.181 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.181 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.182 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2023-06-23T18:49:32.182 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2023-06-23T18:49:32.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.182 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.182 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.182 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.183 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.183 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.183 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.183 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.183 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.183 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2023-06-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2023-06-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2023-06-23T18:49:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.185 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.185 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.185 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.185 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2023-06-23T18:49:32.185 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2023-06-23T18:49:32.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.186 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.186 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.186 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.186 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.186 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.186 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.186 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.186 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.187 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.187 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2023-06-23T18:49:32.187 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2023-06-23T18:49:32.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.187 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.188 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.188 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.188 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.189 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.189 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2023-06-23T18:49:32.189 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2023-06-23T18:49:32.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.189 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.189 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.189 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2023-06-23T18:49:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.191 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2023-06-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2023-06-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.192 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2023-06-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2023-06-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2023-06-23T18:49:32.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.194 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.194 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.194 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.194 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.194 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.194 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2023-06-23T18:49:32.194 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.195 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.195 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.195 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.195 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.195 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.196 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.198 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.199 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.200 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.200 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.200 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.200 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.200 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.201 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.202 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.202 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.202 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.202 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.202 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.202 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.203 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.203 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.203 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.203 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.203 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.203 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.204 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.204 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2023-06-23T18:49:32.204 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.206 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.206 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.206 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.206 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.206 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.206 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.206 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.206 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.207 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.207 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.207 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.207 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.207 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.207 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.208 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.209 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.210 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.211 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.211 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.211 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.211 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.211 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.211 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.211 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.211 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.212 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.212 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.212 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.213 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.213 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.213 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.213 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.213 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.213 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.214 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.214 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2023-06-23T18:49:32.214 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.214 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.215 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.215 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.215 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.215 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.215 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.215 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.216 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.217 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.217 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.217 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.217 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.218 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.218 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.218 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.218 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.218 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.218 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.219 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.219 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.219 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.219 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.220 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.220 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.220 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.220 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.220 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.220 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.220 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.220 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.221 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.221 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.221 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.221 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.221 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.221 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.221 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.222 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.222 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.222 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.222 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.222 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.222 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.223 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.223 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.223 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.223 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.223 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.223 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.224 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2023-06-23T18:49:32.224 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.224 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.224 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.225 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2023-06-23T18:49:32.225 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2023-06-23T18:49:32.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.225 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.225 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-06-23T18:49:32.225 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.226 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.226 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.226 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.226 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.226 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:32.226 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:32.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.227 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.227 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2023-06-23T18:49:32.227 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.227 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.227 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.227 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.228 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.228 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.228 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.228 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.228 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.228 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.229 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.229 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.229 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.229 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.229 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.229 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.229 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.230 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.230 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.230 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.230 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.230 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.230 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.230 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.231 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.231 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.231 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.231 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.232 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.232 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.232 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.232 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.232 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.232 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.232 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.233 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.233 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.233 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.233 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.233 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.233 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.234 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.235 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.235 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.235 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.235 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.235 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.236 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.237 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.237 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.237 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2023-06-23T18:49:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2023-06-23T18:49:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2023-06-23T18:49:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.239 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.239 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.239 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.239 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.239 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2023-06-23T18:49:32.239 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.239 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.240 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2023-06-23T18:49:32.240 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2023-06-23T18:49:32.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.240 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.241 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.241 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.241 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.241 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-06-23T18:49:32.241 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.241 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.241 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.242 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2023-06-23T18:49:32.242 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2023-06-23T18:49:32.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.242 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.242 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.242 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.242 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.243 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.243 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-06-23T18:49:32.243 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.243 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.243 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2023-06-23T18:49:32.243 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2023-06-23T18:49:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.244 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-06-23T18:49:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2023-06-23T18:49:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2023-06-23T18:49:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2023-06-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.247 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.247 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2023-06-23T18:49:32.247 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2023-06-23T18:49:32.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.248 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.248 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.248 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.248 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.248 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.249 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2023-06-23T18:49:32.249 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2023-06-23T18:49:32.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.249 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.250 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.251 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.251 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2023-06-23T18:49:32.251 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2023-06-23T18:49:32.251 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.251 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.251 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-06-23T18:49:32.252 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.252 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.252 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.252 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.252 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.253 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-06-23T18:49:32.253 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-06-23T18:49:32.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.253 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.254 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.255 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:32.255 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:32.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.255 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.255 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2023-06-23T18:49:32.256 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2023-06-23T18:49:32.256 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.256 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.256 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.256 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.256 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.256 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.257 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2023-06-23T18:49:32.257 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2023-06-23T18:49:32.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.257 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.257 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2023-06-23T18:49:32.257 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.257 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.258 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.258 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.258 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.258 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2023-06-23T18:49:32.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2023-06-23T18:49:32.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.259 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.259 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.259 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.259 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.260 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.260 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.260 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.260 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.260 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2023-06-23T18:49:32.260 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2023-06-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.261 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.262 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.262 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.262 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.262 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.262 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.262 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.263 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.263 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.263 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.263 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.263 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.264 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.264 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.264 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.264 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.264 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.264 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.265 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.265 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.265 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.266 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.266 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.266 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.266 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.266 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.266 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.267 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.267 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.267 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.267 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.267 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.267 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.268 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.268 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.268 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.268 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.268 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.268 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.268 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.269 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.269 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.269 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.269 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.270 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.270 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.270 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.270 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.270 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.270 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.270 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.271 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.271 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2023-06-23T18:49:32.271 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2023-06-23T18:49:32.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.271 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.272 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.273 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2023-06-23T18:49:32.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2023-06-23T18:49:32.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.273 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.273 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.273 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.274 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.274 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.274 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2023-06-23T18:49:32.275 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2023-06-23T18:49:32.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.275 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.275 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-06-23T18:49:32.275 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.275 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.276 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.276 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.276 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.276 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.276 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2023-06-23T18:49:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.278 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.278 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.278 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.278 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.278 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.279 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.279 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.279 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.279 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.279 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.279 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.279 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.280 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.280 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.280 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.280 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.280 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.280 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.280 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.281 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.281 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.281 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.281 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.281 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.281 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.283 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.283 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.283 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.283 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.283 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.283 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.283 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.283 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.285 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.285 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.285 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.285 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.285 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.286 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.286 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.286 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.287 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2023-06-23T18:49:32.288 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.288 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.288 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.288 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.288 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.289 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.289 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.289 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.289 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.289 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.289 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.291 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.291 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.291 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.291 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.291 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.292 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.292 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.292 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.292 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.292 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.292 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.293 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.293 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.293 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.294 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.294 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.294 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.294 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.294 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.294 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.294 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.294 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.295 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.295 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.295 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.295 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.295 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.296 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.296 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.296 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.296 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.296 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2023-06-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2023-06-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2023-06-23T18:49:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2023-06-23T18:49:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.299 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2023-06-23T18:49:32.299 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2023-06-23T18:49:32.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.299 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.299 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:32.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.300 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.300 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.300 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-06-23T18:49:32.300 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.300 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.300 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.300 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2023-06-23T18:49:32.301 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.301 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.301 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.301 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2023-06-23T18:49:32.301 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2023-06-23T18:49:32.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:32.302 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.302 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2023-06-23T18:49:32.302 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.302 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.302 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.302 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.303 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.303 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2023-06-23T18:49:32.303 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2023-06-23T18:49:32.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.304 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.304 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.304 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.304 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.305 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:32.305 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:32.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.305 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.306 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:32.306 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.306 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.306 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.306 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.307 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.307 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.307 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.308 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.308 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.308 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.308 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.308 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.308 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.308 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.309 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.310 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.310 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.310 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.310 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.310 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.310 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.311 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.311 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.311 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.311 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.311 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.312 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.312 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.312 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.312 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.312 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.312 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.312 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.312 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.313 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.313 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.313 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.313 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.313 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.313 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.313 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.314 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.314 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.314 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.316 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2023-06-23T18:49:32.316 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2023-06-23T18:49:32.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.316 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.316 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.316 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.317 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2023-06-23T18:49:32.318 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2023-06-23T18:49:32.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2023-06-23T18:49:32.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.318 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.318 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.319 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2023-06-23T18:49:32.319 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2023-06-23T18:49:32.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.320 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-06-23T18:49:32.320 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.320 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.320 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.320 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.321 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.321 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:32.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.322 INFO:teuthology.orchestra.run.smithi002.stdout: "gid": 24113, 2023-06-23T18:49:32.322 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "y", 2023-06-23T18:49:32.322 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_features": 4540138322906710015, 2023-06-23T18:49:32.322 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2023-06-23T18:49:32.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2023-06-23T18:49:32.323 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.323 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.323 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.323 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:32.324 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:32.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.324 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.324 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:32.324 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.325 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2023-06-23T18:49:32.325 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.325 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.325 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.326 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.326 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.326 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.326 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.326 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.326 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.327 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.327 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.327 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.327 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.327 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.327 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.327 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.328 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.328 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.329 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.329 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.330 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.330 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.330 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.331 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.331 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.331 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.331 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.331 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.331 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.332 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.332 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.332 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.332 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.332 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.332 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.333 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.333 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.333 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.333 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.334 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.334 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.335 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.335 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.335 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.335 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.336 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.336 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2023-06-23T18:49:32.336 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2023-06-23T18:49:32.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.336 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2023-06-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.337 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.338 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2023-06-23T18:49:32.338 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2023-06-23T18:49:32.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.338 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.338 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2023-06-23T18:49:32.338 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.339 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.339 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.339 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2023-06-23T18:49:32.339 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.339 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.339 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.340 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2023-06-23T18:49:32.340 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2023-06-23T18:49:32.340 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.340 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.340 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2023-06-23T18:49:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.341 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.342 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2023-06-23T18:49:32.342 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2023-06-23T18:49:32.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.342 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.343 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.343 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.343 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2023-06-23T18:49:32.344 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.344 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.344 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2023-06-23T18:49:32.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2023-06-23T18:49:32.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.345 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2023-06-23T18:49:32.345 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.345 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.346 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2023-06-23T18:49:32.346 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.346 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.346 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2023-06-23T18:49:32.347 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2023-06-23T18:49:32.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.347 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.347 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.347 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.347 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.348 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2023-06-23T18:49:32.348 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.348 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.348 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.348 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2023-06-23T18:49:32.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2023-06-23T18:49:32.349 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.349 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.349 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.349 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.349 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.349 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.349 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.349 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2023-06-23T18:49:32.350 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.350 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.350 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2023-06-23T18:49:32.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2023-06-23T18:49:32.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.351 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.351 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.351 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.352 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.352 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2023-06-23T18:49:32.352 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.352 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.353 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.353 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.353 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2023-06-23T18:49:32.353 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.354 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.354 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.354 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2023-06-23T18:49:32.354 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2023-06-23T18:49:32.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.355 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.355 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.355 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.356 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.356 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.356 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2023-06-23T18:49:32.356 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.356 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.357 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2023-06-23T18:49:32.357 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2023-06-23T18:49:32.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.357 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.357 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2023-06-23T18:49:32.358 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.358 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.358 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.358 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2023-06-23T18:49:32.358 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-06-23T18:49:32.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.359 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2023-06-23T18:49:32.359 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2023-06-23T18:49:32.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:32.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:32.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-06-23T18:49:32.360 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-06-23T18:49:32.360 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.360 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-06-23T18:49:32.360 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-06-23T18:49:32.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.360 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.361 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2023-06-23T18:49:32.361 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2023-06-23T18:49:32.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:32.361 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.361 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2023-06-23T18:49:32.362 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:32.362 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2023-06-23T18:49:32.362 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.362 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-06-23T18:49:32.362 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.362 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.363 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2023-06-23T18:49:32.363 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2023-06-23T18:49:32.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.363 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.363 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2023-06-23T18:49:32.364 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.364 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.364 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.364 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-06-23T18:49:32.364 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-06-23T18:49:32.364 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.365 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.365 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2023-06-23T18:49:32.365 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2023-06-23T18:49:32.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.365 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.365 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2023-06-23T18:49:32.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2023-06-23T18:49:32.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2023-06-23T18:49:32.366 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.366 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2023-06-23T18:49:32.366 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-06-23T18:49:32.366 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.366 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.367 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2023-06-23T18:49:32.367 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2023-06-23T18:49:32.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.367 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.367 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2023-06-23T18:49:32.367 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.367 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.368 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2023-06-23T18:49:32.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-06-23T18:49:32.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.368 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.368 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2023-06-23T18:49:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2023-06-23T18:49:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-06-23T18:49:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-06-23T18:49:32.369 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.370 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-06-23T18:49:32.370 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-06-23T18:49:32.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.370 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.370 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.370 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.371 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.371 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.371 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.371 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.371 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.372 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.372 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.372 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.372 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.372 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.372 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.372 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.373 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.374 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.374 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.374 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.374 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.375 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.375 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.375 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.375 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.375 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.375 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.376 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.376 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.376 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.376 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.376 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.377 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.377 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.377 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.377 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.377 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.377 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.377 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.379 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.379 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.380 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2023-06-23T18:49:32.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2023-06-23T18:49:32.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.380 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.380 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:32.381 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.381 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.381 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.381 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-06-23T18:49:32.381 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.381 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.382 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-06-23T18:49:32.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-06-23T18:49:32.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.382 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.383 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2023-06-23T18:49:32.383 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.383 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.383 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.383 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2023-06-23T18:49:32.384 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2023-06-23T18:49:32.384 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap" 2023-06-23T18:49:32.384 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2023-06-23T18:49:32.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.385 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2023-06-23T18:49:32.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2023-06-23T18:49:32.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.386 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.386 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.386 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.386 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-06-23T18:49:32.386 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.387 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.387 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:32.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:32.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.388 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.388 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.388 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:32.388 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.389 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-06-23T18:49:32.389 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.389 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.389 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2023-06-23T18:49:32.390 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2023-06-23T18:49:32.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.390 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.390 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:32.390 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:32.390 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.391 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.391 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2023-06-23T18:49:32.391 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-06-23T18:49:32.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.392 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2023-06-23T18:49:32.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2023-06-23T18:49:32.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:32.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.394 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.394 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.394 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.394 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.395 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-06-23T18:49:32.395 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.395 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2023-06-23T18:49:32.397 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.397 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.397 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.398 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2023-06-23T18:49:32.398 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2023-06-23T18:49:32.398 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.399 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.399 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-06-23T18:49:32.399 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.400 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.400 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.400 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-06-23T18:49:32.400 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.401 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.402 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2023-06-23T18:49:32.402 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2023-06-23T18:49:32.402 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.403 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.403 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2023-06-23T18:49:32.403 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.403 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.404 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.404 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-06-23T18:49:32.404 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.405 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2023-06-23T18:49:32.405 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2023-06-23T18:49:32.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.406 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2023-06-23T18:49:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-06-23T18:49:32.408 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.408 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.409 INFO:teuthology.orchestra.run.smithi002.stdout: "alertmanager_web_password": { 2023-06-23T18:49:32.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alertmanager_web_password", 2023-06-23T18:49:32.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.409 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.411 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.411 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Alertmanager web password", 2023-06-23T18:49:32.411 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.412 INFO:teuthology.orchestra.run.smithi002.stdout: "alertmanager_web_user": { 2023-06-23T18:49:32.412 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alertmanager_web_user", 2023-06-23T18:49:32.412 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.412 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.412 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:32.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.413 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.413 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.413 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Alertmanager web user", 2023-06-23T18:49:32.413 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.413 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.413 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.413 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2023-06-23T18:49:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2023-06-23T18:49:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.414 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-06-23T18:49:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-06-23T18:49:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2023-06-23T18:49:32.415 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2023-06-23T18:49:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.416 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2023-06-23T18:49:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2023-06-23T18:49:32.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2023-06-23T18:49:32.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.418 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.418 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2023-06-23T18:49:32.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.418 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.418 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.419 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-06-23T18:49:32.419 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.419 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.419 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2023-06-23T18:49:32.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2023-06-23T18:49:32.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.420 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.420 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.420 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.420 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.420 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.420 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-06-23T18:49:32.420 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2023-06-23T18:49:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2023-06-23T18:49:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.422 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.422 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.422 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.422 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-06-23T18:49:32.422 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.422 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.423 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2023-06-23T18:49:32.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2023-06-23T18:49:32.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.423 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-06-23T18:49:32.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.424 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.424 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2023-06-23T18:49:32.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2023-06-23T18:49:32.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.425 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.425 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-06-23T18:49:32.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.425 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.425 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.426 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:32.426 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.426 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.426 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2023-06-23T18:49:32.426 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2023-06-23T18:49:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2023-06-23T18:49:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2023-06-23T18:49:32.428 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.428 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.428 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2023-06-23T18:49:32.428 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2023-06-23T18:49:32.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.428 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-06-23T18:49:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2023-06-23T18:49:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.430 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.430 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2023-06-23T18:49:32.430 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2023-06-23T18:49:32.430 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-06-23T18:49:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.432 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2023-06-23T18:49:32.432 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2023-06-23T18:49:32.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.432 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.432 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-06-23T18:49:32.432 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.432 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2023-06-23T18:49:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2023-06-23T18:49:32.433 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2023-06-23T18:49:32.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.434 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.434 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.434 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-06-23T18:49:32.434 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2023-06-23T18:49:32.435 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.435 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.435 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2023-06-23T18:49:32.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2023-06-23T18:49:32.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-06-23T18:49:32.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.436 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.436 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.436 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2023-06-23T18:49:32.436 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.437 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2023-06-23T18:49:32.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2023-06-23T18:49:32.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.437 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-06-23T18:49:32.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2023-06-23T18:49:32.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.438 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.438 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2023-06-23T18:49:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2023-06-23T18:49:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.439 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-06-23T18:49:32.440 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.440 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.440 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.440 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2023-06-23T18:49:32.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.440 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.441 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2023-06-23T18:49:32.441 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2023-06-23T18:49:32.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-06-23T18:49:32.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.441 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.442 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2023-06-23T18:49:32.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.442 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2023-06-23T18:49:32.442 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2023-06-23T18:49:32.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.443 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.443 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-06-23T18:49:32.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.443 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.443 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2023-06-23T18:49:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2023-06-23T18:49:32.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.445 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.445 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-06-23T18:49:32.445 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.445 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.445 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.445 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:32.445 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.446 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.446 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2023-06-23T18:49:32.446 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2023-06-23T18:49:32.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.446 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.447 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-06-23T18:49:32.447 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.447 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.447 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.447 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2023-06-23T18:49:32.447 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.448 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2023-06-23T18:49:32.448 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2023-06-23T18:49:32.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.448 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-06-23T18:49:32.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.449 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.449 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2023-06-23T18:49:32.449 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.449 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2023-06-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2023-06-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.450 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2023-06-23T18:49:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2023-06-23T18:49:32.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2023-06-23T18:49:32.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.452 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.452 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:32.452 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-06-23T18:49:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2023-06-23T18:49:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2023-06-23T18:49:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2023-06-23T18:49:32.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.454 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.454 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-06-23T18:49:32.454 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2023-06-23T18:49:32.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2023-06-23T18:49:32.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.455 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.455 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2023-06-23T18:49:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-06-23T18:49:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.457 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2023-06-23T18:49:32.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2023-06-23T18:49:32.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.457 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.457 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2023-06-23T18:49:32.457 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.457 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.458 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2023-06-23T18:49:32.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.458 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2023-06-23T18:49:32.458 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2023-06-23T18:49:32.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.459 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.459 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.459 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2023-06-23T18:49:32.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.460 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2023-06-23T18:49:32.460 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2023-06-23T18:49:32.460 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.460 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:32.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.461 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.461 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.461 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2023-06-23T18:49:32.461 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2023-06-23T18:49:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2023-06-23T18:49:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.463 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:32.463 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.463 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.463 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.463 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2023-06-23T18:49:32.463 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.463 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.463 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.464 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.464 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.464 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.464 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.465 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.466 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.466 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.466 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.466 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.466 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.466 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2023-06-23T18:49:32.467 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2023-06-23T18:49:32.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.467 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.467 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.467 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.467 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.467 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.468 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-06-23T18:49:32.468 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.468 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.468 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.469 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.469 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.469 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.469 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.469 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-06-23T18:49:32.469 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.470 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.470 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.471 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.471 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.471 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.471 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.473 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.473 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.473 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.473 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.474 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.474 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.474 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.474 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2023-06-23T18:49:32.474 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-06-23T18:49:32.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.475 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.475 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.475 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.475 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.475 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-06-23T18:49:32.476 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.476 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.476 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-06-23T18:49:32.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-06-23T18:49:32.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.477 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.477 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.477 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2023-06-23T18:49:32.477 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.477 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.477 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-06-23T18:49:32.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.478 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.478 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.478 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2023-06-23T18:49:32.478 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2023-06-23T18:49:32.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.478 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.479 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.479 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.479 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.479 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2023-06-23T18:49:32.479 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.479 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.480 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2023-06-23T18:49:32.480 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2023-06-23T18:49:32.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.480 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.480 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.480 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.480 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-06-23T18:49:32.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.481 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2023-06-23T18:49:32.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2023-06-23T18:49:32.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.482 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.482 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.482 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.482 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.482 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.482 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.483 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2023-06-23T18:49:32.483 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.483 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.483 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-06-23T18:49:32.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-06-23T18:49:32.484 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.484 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.484 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.484 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2023-06-23T18:49:32.484 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.484 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.484 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.484 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2023-06-23T18:49:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2023-06-23T18:49:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2023-06-23T18:49:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.485 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2023-06-23T18:49:32.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2023-06-23T18:49:32.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.486 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.486 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.486 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-06-23T18:49:32.486 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.486 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.486 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.487 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-06-23T18:49:32.487 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.487 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.487 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_web_password": { 2023-06-23T18:49:32.487 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_web_password", 2023-06-23T18:49:32.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.488 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.488 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:32.488 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.488 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.488 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus web password", 2023-06-23T18:49:32.489 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.489 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.489 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_web_user": { 2023-06-23T18:49:32.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_web_user", 2023-06-23T18:49:32.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.490 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:32.490 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.490 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.490 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.490 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus web user", 2023-06-23T18:49:32.490 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.491 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.491 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2023-06-23T18:49:32.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2023-06-23T18:49:32.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.491 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.492 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.492 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.492 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.492 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-06-23T18:49:32.492 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.492 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2023-06-23T18:49:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2023-06-23T18:49:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-06-23T18:49:32.494 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.494 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.494 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2023-06-23T18:49:32.494 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2023-06-23T18:49:32.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.495 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.495 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.495 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.495 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.495 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.495 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.495 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-06-23T18:49:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2023-06-23T18:49:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2023-06-23T18:49:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.496 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-06-23T18:49:32.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2023-06-23T18:49:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2023-06-23T18:49:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-06-23T18:49:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.500 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2023-06-23T18:49:32.500 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2023-06-23T18:49:32.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.500 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.500 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2023-06-23T18:49:32.500 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.500 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.501 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.501 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2023-06-23T18:49:32.501 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.501 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.501 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2023-06-23T18:49:32.501 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2023-06-23T18:49:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-06-23T18:49:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2023-06-23T18:49:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2023-06-23T18:49:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.504 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.504 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.504 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.504 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.504 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-06-23T18:49:32.504 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.505 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2023-06-23T18:49:32.505 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2023-06-23T18:49:32.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.506 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.506 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-06-23T18:49:32.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.507 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2023-06-23T18:49:32.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2023-06-23T18:49:32.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.507 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.508 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.508 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2023-06-23T18:49:32.508 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.508 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2023-06-23T18:49:32.509 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2023-06-23T18:49:32.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.509 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.510 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-06-23T18:49:32.510 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.510 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.510 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2023-06-23T18:49:32.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2023-06-23T18:49:32.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.511 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.511 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.511 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.511 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-06-23T18:49:32.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2023-06-23T18:49:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.514 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.514 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.514 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.514 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.514 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.514 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.515 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.515 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.515 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.515 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.516 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.516 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.516 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.516 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.516 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.516 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.517 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.517 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.517 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.517 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.518 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.518 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.518 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.518 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.518 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.518 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.519 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.520 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.521 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.522 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.522 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2023-06-23T18:49:32.522 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2023-06-23T18:49:32.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.522 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.522 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2023-06-23T18:49:32.523 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.523 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.523 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.523 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2023-06-23T18:49:32.523 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.524 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2023-06-23T18:49:32.524 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2023-06-23T18:49:32.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.524 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.524 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2023-06-23T18:49:32.524 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.524 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.525 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2023-06-23T18:49:32.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.525 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2023-06-23T18:49:32.526 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.526 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.526 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.526 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-06-23T18:49:32.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-06-23T18:49:32.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.527 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.527 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.527 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.527 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.528 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.528 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.528 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.528 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2023-06-23T18:49:32.528 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2023-06-23T18:49:32.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.529 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.529 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.529 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.529 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-06-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-06-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.530 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.531 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2023-06-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2023-06-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.532 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.533 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.533 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.533 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.533 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.533 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.533 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-06-23T18:49:32.534 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-06-23T18:49:32.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.534 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.534 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.534 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.534 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.535 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.535 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.535 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.535 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2023-06-23T18:49:32.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2023-06-23T18:49:32.535 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.536 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.537 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.537 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.537 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-06-23T18:49:32.537 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-06-23T18:49:32.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.537 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.538 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.538 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.538 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.538 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.538 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-06-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-06-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.539 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.540 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.540 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.540 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.540 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.540 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-06-23T18:49:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-06-23T18:49:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.542 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.542 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.542 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.542 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-06-23T18:49:32.542 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-06-23T18:49:32.543 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.543 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.543 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.543 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.543 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.543 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.543 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.543 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.544 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.544 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.544 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.544 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2023-06-23T18:49:32.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-06-23T18:49:32.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.545 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.545 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.545 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.545 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2023-06-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-06-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.546 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.547 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.547 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.547 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.547 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.548 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2023-06-23T18:49:32.548 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-06-23T18:49:32.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.548 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.548 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.548 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-06-23T18:49:32.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-06-23T18:49:32.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.550 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.550 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.550 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.550 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.550 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.550 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.551 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.551 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2023-06-23T18:49:32.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2023-06-23T18:49:32.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:32.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.552 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-06-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-06-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.553 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.554 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.554 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.554 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.555 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.555 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2023-06-23T18:49:32.555 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2023-06-23T18:49:32.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.555 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.555 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.556 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.556 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.556 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.556 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.556 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.556 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.556 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.557 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2023-06-23T18:49:32.557 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2023-06-23T18:49:32.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.557 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.557 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:32.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.558 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.558 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.558 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-06-23T18:49:32.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-06-23T18:49:32.559 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.559 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.560 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-06-23T18:49:32.560 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-06-23T18:49:32.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.560 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.561 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.561 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.561 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.561 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.561 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.561 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.562 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.562 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-06-23T18:49:32.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-06-23T18:49:32.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.563 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.563 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.563 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.563 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.563 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.564 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2023-06-23T18:49:32.564 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-06-23T18:49:32.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.564 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.564 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.564 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.565 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.565 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.565 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.565 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.565 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2023-06-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2023-06-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.566 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.567 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.567 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.567 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.567 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-06-23T18:49:32.567 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-06-23T18:49:32.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.568 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.568 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.568 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.568 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.568 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.569 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-06-23T18:49:32.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-06-23T18:49:32.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.569 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.570 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-06-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-06-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.571 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.572 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.572 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.572 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.572 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.572 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.572 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.573 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-06-23T18:49:32.573 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-06-23T18:49:32.573 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.573 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.574 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.574 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.574 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.574 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-06-23T18:49:32.574 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-06-23T18:49:32.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.575 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.575 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.575 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.575 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.575 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.576 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.576 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.576 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.576 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.576 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-06-23T18:49:32.576 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-06-23T18:49:32.576 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.577 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.577 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.577 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.577 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.577 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.578 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.578 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.578 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-06-23T18:49:32.578 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-06-23T18:49:32.578 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.578 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.578 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.579 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.580 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-06-23T18:49:32.580 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-06-23T18:49:32.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.580 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.580 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.581 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.581 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.581 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.581 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.581 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.581 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.582 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2023-06-23T18:49:32.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2023-06-23T18:49:32.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.582 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.582 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.582 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.582 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.582 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.583 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.583 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.583 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.583 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.583 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-06-23T18:49:32.583 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-06-23T18:49:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-06-23T18:49:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.584 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-06-23T18:49:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-06-23T18:49:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.585 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-06-23T18:49:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-06-23T18:49:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2023-06-23T18:49:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2023-06-23T18:49:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-06-23T18:49:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2023-06-23T18:49:32.589 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.590 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2023-06-23T18:49:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2023-06-23T18:49:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.591 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-06-23T18:49:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-06-23T18:49:32.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2023-06-23T18:49:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2023-06-23T18:49:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.595 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2023-06-23T18:49:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2023-06-23T18:49:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.597 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-06-23T18:49:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-06-23T18:49:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.598 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-06-23T18:49:32.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-06-23T18:49:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.600 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-06-23T18:49:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-06-23T18:49:32.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.602 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2023-06-23T18:49:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2023-06-23T18:49:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2023-06-23T18:49:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2023-06-23T18:49:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.605 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.606 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2023-06-23T18:49:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2023-06-23T18:49:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.607 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2023-06-23T18:49:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2023-06-23T18:49:32.608 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2023-06-23T18:49:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2023-06-23T18:49:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.609 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-06-23T18:49:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-06-23T18:49:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.610 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.611 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.612 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.613 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.614 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.615 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.617 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.619 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2023-06-23T18:49:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2023-06-23T18:49:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.621 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2023-06-23T18:49:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2023-06-23T18:49:32.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2023-06-23T18:49:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.623 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:32.624 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-06-23T18:49:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2023-06-23T18:49:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-06-23T18:49:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-06-23T18:49:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.628 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2023-06-23T18:49:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2023-06-23T18:49:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2023-06-23T18:49:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.630 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-06-23T18:49:32.631 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-06-23T18:49:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2023-06-23T18:49:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.632 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2023-06-23T18:49:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.633 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-06-23T18:49:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-06-23T18:49:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-06-23T18:49:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-06-23T18:49:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-06-23T18:49:32.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2023-06-23T18:49:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2023-06-23T18:49:32.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.636 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2023-06-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2023-06-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2023-06-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2023-06-23T18:49:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.641 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.643 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.644 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.645 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.646 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.647 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2023-06-23T18:49:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2023-06-23T18:49:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2023-06-23T18:49:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.649 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-06-23T18:49:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2023-06-23T18:49:32.650 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2023-06-23T18:49:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2023-06-23T18:49:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2023-06-23T18:49:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2023-06-23T18:49:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2023-06-23T18:49:32.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2023-06-23T18:49:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2023-06-23T18:49:32.653 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2023-06-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2023-06-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-06-23T18:49:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2023-06-23T18:49:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2023-06-23T18:49:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2023-06-23T18:49:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-06-23T18:49:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2023-06-23T18:49:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2023-06-23T18:49:32.659 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2023-06-23T18:49:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2023-06-23T18:49:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.660 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-06-23T18:49:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.661 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2023-06-23T18:49:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.663 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.665 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.666 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.667 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.668 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.669 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.671 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.671 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2023-06-23T18:49:32.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2023-06-23T18:49:32.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.671 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-06-23T18:49:32.672 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.672 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.672 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.672 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.672 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.673 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2023-06-23T18:49:32.673 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2023-06-23T18:49:32.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.673 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2023-06-23T18:49:32.673 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.674 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:32.675 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:32.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.675 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:32.675 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.675 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.675 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.676 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.676 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.676 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2023-06-23T18:49:32.677 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2023-06-23T18:49:32.677 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2023-06-23T18:49:32.677 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.677 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2023-06-23T18:49:32.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2023-06-23T18:49:32.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.677 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.678 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2023-06-23T18:49:32.678 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.678 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.678 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.678 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-06-23T18:49:32.678 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2023-06-23T18:49:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2023-06-23T18:49:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.679 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2023-06-23T18:49:32.680 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.680 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.680 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.680 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-06-23T18:49:32.680 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.680 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.681 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2023-06-23T18:49:32.681 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2023-06-23T18:49:32.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.681 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.682 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.682 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2023-06-23T18:49:32.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.682 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:32.683 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:32.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2023-06-23T18:49:32.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2023-06-23T18:49:32.683 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-06-23T18:49:32.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.684 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.684 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.684 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.685 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.685 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.685 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.685 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.685 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.685 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.685 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.687 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.690 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.691 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.692 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2023-06-23T18:49:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2023-06-23T18:49:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.693 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2023-06-23T18:49:32.694 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2023-06-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2023-06-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2023-06-23T18:49:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2023-06-23T18:49:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.696 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-06-23T18:49:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-06-23T18:49:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2023-06-23T18:49:32.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-06-23T18:49:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.698 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2023-06-23T18:49:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2023-06-23T18:49:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2023-06-23T18:49:32.699 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-06-23T18:49:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2023-06-23T18:49:32.700 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2023-06-23T18:49:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.701 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2023-06-23T18:49:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2023-06-23T18:49:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2023-06-23T18:49:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2023-06-23T18:49:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-06-23T18:49:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2023-06-23T18:49:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.704 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.705 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.706 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.709 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.710 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2023-06-23T18:49:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.714 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.715 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.719 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.720 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.721 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2023-06-23T18:49:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2023-06-23T18:49:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2023-06-23T18:49:32.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2023-06-23T18:49:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2023-06-23T18:49:32.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2023-06-23T18:49:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2023-06-23T18:49:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2023-06-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-06-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.728 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.729 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.730 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.731 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.733 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.734 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.735 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2023-06-23T18:49:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.737 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2023-06-23T18:49:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2023-06-23T18:49:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2023-06-23T18:49:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.738 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2023-06-23T18:49:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-06-23T18:49:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.740 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.741 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.743 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.744 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.745 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.746 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.747 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.748 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2023-06-23T18:49:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2023-06-23T18:49:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.749 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2023-06-23T18:49:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-06-23T18:49:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2023-06-23T18:49:32.750 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2023-06-23T18:49:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2023-06-23T18:49:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.751 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2023-06-23T18:49:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2023-06-23T18:49:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2023-06-23T18:49:32.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2023-06-23T18:49:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2023-06-23T18:49:32.753 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2023-06-23T18:49:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2023-06-23T18:49:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2023-06-23T18:49:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2023-06-23T18:49:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2023-06-23T18:49:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.756 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2023-06-23T18:49:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2023-06-23T18:49:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-06-23T18:49:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2023-06-23T18:49:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.758 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2023-06-23T18:49:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2023-06-23T18:49:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-06-23T18:49:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2023-06-23T18:49:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2023-06-23T18:49:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-06-23T18:49:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2023-06-23T18:49:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-06-23T18:49:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2023-06-23T18:49:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2023-06-23T18:49:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2023-06-23T18:49:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2023-06-23T18:49:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2023-06-23T18:49:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2023-06-23T18:49:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2023-06-23T18:49:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "noautoscale": { 2023-06-23T18:49:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "noautoscale", 2023-06-23T18:49:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "global autoscale flag", 2023-06-23T18:49:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-06-23T18:49:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2023-06-23T18:49:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2023-06-23T18:49:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-06-23T18:49:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2023-06-23T18:49:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2023-06-23T18:49:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-06-23T18:49:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2023-06-23T18:49:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2023-06-23T18:49:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2023-06-23T18:49:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2023-06-23T18:49:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-06-23T18:49:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-06-23T18:49:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2023-06-23T18:49:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2023-06-23T18:49:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2023-06-23T18:49:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2023-06-23T18:49:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2023-06-23T18:49:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2023-06-23T18:49:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2023-06-23T18:49:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2023-06-23T18:49:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2023-06-23T18:49:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2023-06-23T18:49:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-06-23T18:49:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-06-23T18:49:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2023-06-23T18:49:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2023-06-23T18:49:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2023-06-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-06-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2023-06-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.868 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.869 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.869 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.869 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.869 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.869 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2023-06-23T18:49:32.869 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2023-06-23T18:49:32.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.870 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.870 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2023-06-23T18:49:32.870 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.870 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.870 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.870 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.871 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.871 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.871 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.871 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:32.871 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:32.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.871 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.872 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-06-23T18:49:32.872 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.872 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.872 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.872 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-06-23T18:49:32.872 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.873 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.873 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:32.873 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:32.873 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.873 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.874 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2023-06-23T18:49:32.874 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.874 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.874 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.874 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-06-23T18:49:32.874 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.874 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.875 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2023-06-23T18:49:32.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2023-06-23T18:49:32.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.875 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.875 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.875 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2023-06-23T18:49:32.875 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.876 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.876 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.876 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.876 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.876 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.876 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-06-23T18:49:32.877 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-06-23T18:49:32.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.877 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.877 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2023-06-23T18:49:32.877 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.877 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.877 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2023-06-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2023-06-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.879 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-06-23T18:49:32.879 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-06-23T18:49:32.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.879 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.879 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-06-23T18:49:32.879 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-06-23T18:49:32.879 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-06-23T18:49:32.880 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.880 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.880 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.880 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.880 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.880 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.880 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2023-06-23T18:49:32.881 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.881 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.881 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.881 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.881 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.882 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.882 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.882 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.882 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.882 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.882 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.882 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.883 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.883 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.883 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.883 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.883 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.883 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.883 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.884 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.884 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.884 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.884 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.884 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.885 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.885 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.885 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.885 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.885 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.885 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.885 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.886 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.887 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.888 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.888 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.888 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.888 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.888 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.889 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.889 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.889 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.889 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.889 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.889 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.890 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.890 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.890 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.890 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2023-06-23T18:49:32.890 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2023-06-23T18:49:32.890 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.890 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.891 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:32.891 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.891 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.891 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.891 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.891 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.892 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.892 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2023-06-23T18:49:32.892 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2023-06-23T18:49:32.892 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.892 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.892 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.893 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.893 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.893 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.893 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.893 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.893 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.893 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.894 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2023-06-23T18:49:32.894 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2023-06-23T18:49:32.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.894 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.894 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.894 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.895 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.895 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.895 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.895 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.895 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2023-06-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2023-06-23T18:49:32.896 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2023-06-23T18:49:32.897 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.897 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.897 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.897 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.897 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.897 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.897 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.897 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-06-23T18:49:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-06-23T18:49:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.898 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.899 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.900 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.901 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.901 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.901 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.901 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.901 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.901 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.901 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.901 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.902 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.902 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.902 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.902 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.902 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.903 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.903 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.903 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.903 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.903 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.903 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.903 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.903 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.904 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.904 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.904 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.904 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.904 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.904 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.904 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.905 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.905 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.905 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.905 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.905 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.905 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.906 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.907 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.907 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.907 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.907 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.908 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.908 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.908 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.908 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.908 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.908 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.908 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.909 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:32.909 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:32.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.909 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.909 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.909 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.910 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.910 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.910 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.910 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.910 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.910 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:32.911 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:32.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.911 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.911 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.911 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.911 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.911 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.912 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.912 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.912 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.913 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2023-06-23T18:49:32.913 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.913 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.913 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.913 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.913 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.913 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.914 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.914 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.914 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.914 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.914 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.914 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.914 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.915 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.916 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.916 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.916 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.916 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.917 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.917 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.917 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.917 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.917 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.917 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.917 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.918 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.918 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.918 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.918 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.918 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.919 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.919 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.919 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.919 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.919 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.919 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.919 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.919 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.920 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.920 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.920 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.920 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.920 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.920 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.921 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.921 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.921 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.921 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.921 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.921 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.922 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.922 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.922 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2023-06-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi002.stdout: "drive_group_interval": { 2023-06-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "drive_group_interval", 2023-06-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.923 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.924 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300.0", 2023-06-23T18:49:32.924 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.924 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.924 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.925 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-06-23T18:49:32.925 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.925 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.925 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.925 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.926 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.926 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.926 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.926 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.926 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.926 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.927 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.927 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.927 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.927 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.927 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.927 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.927 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.927 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.928 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.928 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.928 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.928 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.928 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.929 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.929 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.929 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.929 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.929 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.929 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.931 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.931 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.931 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.931 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.931 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.931 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.931 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.931 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.932 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.932 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.932 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.932 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.932 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.932 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.933 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.934 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.934 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.934 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.934 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.934 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2023-06-23T18:49:32.934 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2023-06-23T18:49:32.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.935 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.935 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2023-06-23T18:49:32.935 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.935 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.935 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.935 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-06-23T18:49:32.936 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.936 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.936 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.937 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2023-06-23T18:49:32.937 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.937 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.937 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.937 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.937 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.937 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.938 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.939 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.939 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.939 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.939 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.939 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.939 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.939 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.940 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.940 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.940 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.940 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.940 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.942 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.942 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.942 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.942 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.942 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.942 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.943 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.943 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.943 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.943 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.943 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.943 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.943 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.943 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.944 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.944 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.944 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.944 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.944 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.945 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.945 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.945 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.945 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.945 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.945 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.946 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.946 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.946 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.946 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2023-06-23T18:49:32.946 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2023-06-23T18:49:32.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.946 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.947 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.947 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.947 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.947 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.947 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.947 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.948 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.948 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2023-06-23T18:49:32.948 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2023-06-23T18:49:32.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.948 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.949 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2023-06-23T18:49:32.949 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.949 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.949 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.949 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.949 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.949 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.950 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2023-06-23T18:49:32.950 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2023-06-23T18:49:32.950 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.950 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.950 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.950 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.950 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.951 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.951 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.951 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.951 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.951 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.951 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2023-06-23T18:49:32.952 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2023-06-23T18:49:32.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.952 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.952 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.952 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.952 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.952 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.953 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.953 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.953 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.953 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2023-06-23T18:49:32.953 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2023-06-23T18:49:32.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:32.954 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.954 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.954 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.954 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.954 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.954 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.955 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.955 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.955 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.955 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.955 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2023-06-23T18:49:32.955 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2023-06-23T18:49:32.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.956 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.956 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.956 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.956 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.956 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.956 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.957 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.957 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.957 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2023-06-23T18:49:32.957 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2023-06-23T18:49:32.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.957 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.958 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.958 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.958 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.958 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.958 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.958 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.959 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.959 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2023-06-23T18:49:32.959 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2023-06-23T18:49:32.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.959 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.960 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:32.960 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.960 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.960 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.960 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.960 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.960 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.961 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.961 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2023-06-23T18:49:32.961 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2023-06-23T18:49:32.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:32.961 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.962 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.962 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:32.962 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2023-06-23T18:49:32.962 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.962 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.962 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.963 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.963 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2023-06-23T18:49:32.963 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2023-06-23T18:49:32.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.963 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.964 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.964 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.964 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.964 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.964 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.964 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.964 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.964 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.965 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2023-06-23T18:49:32.965 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2023-06-23T18:49:32.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.965 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.965 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.966 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.966 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.966 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.966 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.966 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.966 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.967 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2023-06-23T18:49:32.967 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2023-06-23T18:49:32.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.967 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:32.967 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.967 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.967 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.968 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.968 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.968 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.968 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.969 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2023-06-23T18:49:32.969 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.969 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.969 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.969 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2023-06-23T18:49:32.969 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2023-06-23T18:49:32.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.970 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.970 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.970 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.970 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.970 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.970 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2023-06-23T18:49:32.971 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.971 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.971 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2023-06-23T18:49:32.971 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2023-06-23T18:49:32.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.971 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.972 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.972 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.972 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.972 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.972 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2023-06-23T18:49:32.972 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.973 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.973 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.973 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.973 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.973 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.973 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.974 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.974 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.974 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.974 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.974 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.974 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.974 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.974 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.975 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.975 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.975 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.975 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.975 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.975 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.975 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.976 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.976 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.976 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.976 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.976 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.978 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.978 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.978 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.978 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.978 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.978 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.979 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.979 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.979 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.979 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.979 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.979 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.979 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.980 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.980 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.980 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.980 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.980 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.981 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.981 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.981 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.981 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.981 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.981 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.981 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.982 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.982 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.982 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2023-06-23T18:49:32.982 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.982 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.983 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.983 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.983 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.983 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.983 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.983 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.983 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.984 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.984 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.984 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.984 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.984 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.984 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.984 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.984 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.985 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.985 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.985 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.985 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.985 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.986 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.986 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.986 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.986 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.986 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.986 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.986 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.987 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.987 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.987 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.987 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.987 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.987 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.988 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.988 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.988 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.988 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.988 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.988 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.989 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.990 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.990 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:32.990 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:32.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.990 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.991 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.991 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.991 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.991 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.991 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.991 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.991 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.992 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:32.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:32.992 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2023-06-23T18:49:32.992 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:32.992 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:32.993 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:32.993 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:32.993 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:32.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.993 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.993 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:32.994 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.994 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.994 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.994 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.994 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.994 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.994 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.994 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.995 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.995 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.995 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.995 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.995 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.995 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.995 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:32.996 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:32.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:32.996 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.996 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:32.996 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.996 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.996 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:32.997 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.997 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:32.997 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:32.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:32.997 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:32.997 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:32.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:32.998 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:32.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:32.998 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:32.998 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:32.998 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:32.998 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:32.998 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:32.998 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:32.999 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:32.999 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:32.999 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:32.999 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:32.999 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:32.999 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:32.999 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:32.999 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.000 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.000 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.000 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.001 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.001 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.001 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.001 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.001 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.002 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.002 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.002 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.003 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2023-06-23T18:49:33.003 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.003 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.003 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.003 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2023-06-23T18:49:33.003 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2023-06-23T18:49:33.003 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.003 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.004 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-06-23T18:49:33.004 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.004 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.004 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.004 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.004 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.004 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.006 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2023-06-23T18:49:33.006 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.006 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.006 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.006 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.006 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.006 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.007 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.007 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.007 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.007 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.007 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.008 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.009 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.009 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.009 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.009 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.009 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.009 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.010 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.010 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.010 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.010 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.010 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.010 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.011 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.011 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.011 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.011 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.011 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.012 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.012 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.012 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.012 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.012 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.012 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.012 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.013 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.013 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.013 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.013 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.013 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.013 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.013 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.014 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.014 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.014 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.014 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.014 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.015 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.015 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.015 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.015 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.015 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.016 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2023-06-23T18:49:33.016 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.016 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.016 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.016 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2023-06-23T18:49:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2023-06-23T18:49:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.018 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2023-06-23T18:49:33.018 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.018 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.018 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2023-06-23T18:49:33.018 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2023-06-23T18:49:33.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.019 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.019 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.019 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.019 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.019 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.019 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-06-23T18:49:33.020 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.020 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.020 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2023-06-23T18:49:33.020 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2023-06-23T18:49:33.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.020 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.021 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.021 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.021 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.021 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.021 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.021 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-06-23T18:49:33.021 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.022 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.022 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2023-06-23T18:49:33.022 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2023-06-23T18:49:33.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.022 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.022 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.023 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.023 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.023 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.023 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-06-23T18:49:33.023 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.023 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.023 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.024 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2023-06-23T18:49:33.024 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2023-06-23T18:49:33.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.024 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.024 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.024 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.025 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.025 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.025 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2023-06-23T18:49:33.025 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.025 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.025 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2023-06-23T18:49:33.026 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2023-06-23T18:49:33.026 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.026 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.026 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.026 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.026 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.026 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.026 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.027 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.027 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.027 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.027 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2023-06-23T18:49:33.027 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2023-06-23T18:49:33.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.028 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.028 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.028 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.028 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.028 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.028 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.028 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.029 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.029 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2023-06-23T18:49:33.029 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2023-06-23T18:49:33.029 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.029 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.029 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-06-23T18:49:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.030 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-06-23T18:49:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-06-23T18:49:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.031 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.032 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.032 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.032 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.032 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.032 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.032 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.033 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:33.033 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:33.033 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.033 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.033 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.033 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2023-06-23T18:49:33.033 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2023-06-23T18:49:33.033 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.034 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.034 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.034 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.034 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.034 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2023-06-23T18:49:33.034 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2023-06-23T18:49:33.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.035 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.035 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2023-06-23T18:49:33.035 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.035 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.035 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.035 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.036 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.036 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.036 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2023-06-23T18:49:33.036 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2023-06-23T18:49:33.036 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.036 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.037 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.037 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.037 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.037 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.037 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.037 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.038 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.038 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2023-06-23T18:49:33.038 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2023-06-23T18:49:33.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.038 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.038 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.039 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.039 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.039 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.039 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.039 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.039 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.040 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.041 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.041 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.041 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.041 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.041 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.041 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.041 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.041 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.042 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.042 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.042 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.042 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.042 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.042 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.043 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.043 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.043 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.043 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.043 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.044 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.044 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.044 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.044 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.044 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.045 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.045 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.045 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.045 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.045 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.045 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.045 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.047 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.048 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.048 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.048 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.048 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.048 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.048 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.049 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2023-06-23T18:49:33.049 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2023-06-23T18:49:33.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.049 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.049 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.049 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.049 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.050 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.050 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.050 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.050 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.050 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2023-06-23T18:49:33.050 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2023-06-23T18:49:33.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.051 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.051 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.051 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.051 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.051 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.051 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.052 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.052 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.052 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2023-06-23T18:49:33.052 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2023-06-23T18:49:33.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.053 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.053 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-06-23T18:49:33.053 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.053 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.053 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.053 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.053 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.054 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.054 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2023-06-23T18:49:33.054 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.055 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.055 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.055 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.055 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.055 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.055 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.056 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.056 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.056 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.056 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.056 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.056 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.056 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.056 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.057 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.057 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.057 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.057 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.057 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.057 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.057 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.058 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.058 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.058 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.058 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.058 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.058 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.059 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.059 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.059 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.059 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.059 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.060 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.060 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.060 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.060 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.060 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.060 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.061 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.061 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.061 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.061 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.061 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.061 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.061 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.061 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.062 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.062 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.062 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.062 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.062 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.063 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.063 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.063 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.063 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.063 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.063 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.063 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.064 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2023-06-23T18:49:33.064 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.064 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.064 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.065 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.065 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.065 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.065 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.065 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.065 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.066 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.066 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.066 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.066 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.066 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.066 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.066 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.067 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.067 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.067 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.067 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.067 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.067 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.067 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.068 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.069 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.069 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.069 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.069 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.069 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.069 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.070 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.070 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.070 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.070 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.070 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.070 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.070 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.071 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.071 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.071 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.071 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.071 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.071 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.071 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.071 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.072 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.072 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.072 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.072 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.072 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.072 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.072 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.073 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.073 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.073 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.073 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.073 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.073 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2023-06-23T18:49:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2023-06-23T18:49:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2023-06-23T18:49:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.074 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2023-06-23T18:49:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2023-06-23T18:49:33.075 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2023-06-23T18:49:33.076 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.076 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.076 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.076 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:33.076 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.076 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.076 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.076 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-06-23T18:49:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.078 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2023-06-23T18:49:33.078 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.078 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.078 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.078 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2023-06-23T18:49:33.078 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2023-06-23T18:49:33.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:33.078 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.079 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2023-06-23T18:49:33.079 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.079 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.079 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.079 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.079 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.080 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.080 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2023-06-23T18:49:33.080 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2023-06-23T18:49:33.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.080 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.081 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.081 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.081 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.081 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.081 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.081 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.082 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.082 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.082 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.082 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:33.082 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:33.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.083 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.083 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:33.083 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.083 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.084 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.085 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.085 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.085 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.085 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.085 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.085 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.085 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.086 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.086 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.086 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.086 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.086 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.086 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.086 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.086 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.087 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.087 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.087 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.087 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.087 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.087 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.088 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.088 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.088 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.088 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.088 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.088 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.089 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.089 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.089 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.089 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.089 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.089 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.089 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.090 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.090 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.090 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.090 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.090 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.090 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.090 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.090 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.091 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.091 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.091 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.091 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.091 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.091 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.092 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.093 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.093 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.093 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.093 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.093 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2023-06-23T18:49:33.093 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2023-06-23T18:49:33.094 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.094 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.094 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.094 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.094 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.094 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.094 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.095 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.095 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.095 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.095 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2023-06-23T18:49:33.095 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2023-06-23T18:49:33.095 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.095 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.096 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.096 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2023-06-23T18:49:33.096 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.096 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.096 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.096 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.096 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.096 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.097 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.097 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2023-06-23T18:49:33.097 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2023-06-23T18:49:33.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.097 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.097 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-06-23T18:49:33.098 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.098 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.098 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.098 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.098 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.098 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.098 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.099 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:33.099 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.099 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.099 INFO:teuthology.orchestra.run.smithi002.stdout: "modules": [ 2023-06-23T18:49:33.099 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2023-06-23T18:49:33.100 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard", 2023-06-23T18:49:33.100 INFO:teuthology.orchestra.run.smithi002.stdout: "iostat", 2023-06-23T18:49:33.100 INFO:teuthology.orchestra.run.smithi002.stdout: "nfs", 2023-06-23T18:49:33.100 INFO:teuthology.orchestra.run.smithi002.stdout: "restful" 2023-06-23T18:49:33.100 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.100 INFO:teuthology.orchestra.run.smithi002.stdout: "available_modules": [ 2023-06-23T18:49:33.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.100 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alerts", 2023-06-23T18:49:33.101 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.101 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.101 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.101 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:33.101 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:33.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.101 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.102 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:33.102 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.102 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.102 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.102 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How frequently to reexamine health status", 2023-06-23T18:49:33.102 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.102 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.102 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.103 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.103 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.103 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.103 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.103 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.104 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.104 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.104 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.104 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.104 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.104 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.104 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.105 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.105 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.105 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.105 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.105 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.105 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.107 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.107 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.107 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.107 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.107 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.107 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.108 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.109 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.109 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.109 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.109 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.109 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.109 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.109 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.110 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.110 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.110 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.110 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.110 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.111 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.111 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.111 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.111 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.111 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.111 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.112 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_destination": { 2023-06-23T18:49:33.112 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_destination", 2023-06-23T18:49:33.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.112 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.112 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.112 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.113 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.113 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.113 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email address to send alerts to", 2023-06-23T18:49:33.113 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.113 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.113 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_from_name": { 2023-06-23T18:49:33.114 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_from_name", 2023-06-23T18:49:33.114 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.114 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.114 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.114 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "Ceph", 2023-06-23T18:49:33.114 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.114 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.114 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.115 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Email From: name", 2023-06-23T18:49:33.115 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.115 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.115 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_host": { 2023-06-23T18:49:33.115 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_host", 2023-06-23T18:49:33.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP server", 2023-06-23T18:49:33.116 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.117 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.117 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.117 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_password": { 2023-06-23T18:49:33.117 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_password", 2023-06-23T18:49:33.117 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.117 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.117 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.118 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.118 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.118 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.118 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.118 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Password to authenticate with", 2023-06-23T18:49:33.118 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.118 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.119 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_port": { 2023-06-23T18:49:33.119 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_port", 2023-06-23T18:49:33.119 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.119 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.119 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.119 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "465", 2023-06-23T18:49:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP port", 2023-06-23T18:49:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_sender": { 2023-06-23T18:49:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_sender", 2023-06-23T18:49:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.122 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.122 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SMTP envelope sender", 2023-06-23T18:49:33.122 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.122 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.122 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_ssl": { 2023-06-23T18:49:33.122 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_ssl", 2023-06-23T18:49:33.123 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.123 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.123 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.123 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.123 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.123 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.123 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.123 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use SSL to connect to SMTP server", 2023-06-23T18:49:33.124 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.124 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.124 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.124 INFO:teuthology.orchestra.run.smithi002.stdout: "smtp_user": { 2023-06-23T18:49:33.124 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "smtp_user", 2023-06-23T18:49:33.124 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "User to authenticate as", 2023-06-23T18:49:33.125 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "balancer", 2023-06-23T18:49:33.126 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.127 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.127 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.127 INFO:teuthology.orchestra.run.smithi002.stdout: "active": { 2023-06-23T18:49:33.127 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "active", 2023-06-23T18:49:33.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.127 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.127 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.128 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.128 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.128 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.128 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically balance PGs across cluster", 2023-06-23T18:49:33.128 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.128 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.128 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.129 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_time": { 2023-06-23T18:49:33.129 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_time", 2023-06-23T18:49:33.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.129 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.129 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0000", 2023-06-23T18:49:33.129 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.129 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.130 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.130 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "beginning time of day to automatically balance", 2023-06-23T18:49:33.130 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-06-23T18:49:33.130 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.130 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.130 INFO:teuthology.orchestra.run.smithi002.stdout: "begin_weekday": { 2023-06-23T18:49:33.130 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "begin_weekday", 2023-06-23T18:49:33.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:33.131 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.131 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:33.131 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-06-23T18:49:33.131 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-06-23T18:49:33.131 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.132 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-06-23T18:49:33.132 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-06-23T18:49:33.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.132 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.132 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_max_iterations": { 2023-06-23T18:49:33.132 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_max_iterations", 2023-06-23T18:49:33.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:33.133 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.133 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.133 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "25", 2023-06-23T18:49:33.133 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:33.133 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "250", 2023-06-23T18:49:33.133 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.133 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-06-23T18:49:33.133 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.134 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.134 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.134 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_metrics": { 2023-06-23T18:49:33.134 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_metrics", 2023-06-23T18:49:33.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.134 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "pgs,objects,bytes", 2023-06-23T18:49:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-06-23T18:49:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-06-23T18:49:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.135 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.136 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_compat_step": { 2023-06-23T18:49:33.136 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crush_compat_step", 2023-06-23T18:49:33.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:33.136 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.136 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.5", 2023-06-23T18:49:33.137 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0.001", 2023-06-23T18:49:33.137 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "0.999", 2023-06-23T18:49:33.137 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.137 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "aggressiveness of optimization", 2023-06-23T18:49:33.137 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-06-23T18:49:33.137 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.137 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.138 INFO:teuthology.orchestra.run.smithi002.stdout: "end_time": { 2023-06-23T18:49:33.138 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_time", 2023-06-23T18:49:33.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.138 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.138 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2359", 2023-06-23T18:49:33.138 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.138 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.139 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.139 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ending time of day to automatically balance", 2023-06-23T18:49:33.139 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-06-23T18:49:33.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.139 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.139 INFO:teuthology.orchestra.run.smithi002.stdout: "end_weekday": { 2023-06-23T18:49:33.139 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "end_weekday", 2023-06-23T18:49:33.140 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:33.140 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.140 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.140 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:33.140 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "0", 2023-06-23T18:49:33.140 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "6", 2023-06-23T18:49:33.140 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.140 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-06-23T18:49:33.141 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-06-23T18:49:33.141 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.141 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.141 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.141 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.141 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.141 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.142 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.142 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.142 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.142 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.142 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.142 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.142 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.144 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.144 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.144 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.144 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.144 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.144 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.145 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.145 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.145 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.145 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.145 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.145 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.146 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.146 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.146 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.146 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.146 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.146 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.147 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.148 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.149 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.149 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.149 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.149 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.149 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.149 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.150 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.150 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.150 INFO:teuthology.orchestra.run.smithi002.stdout: "min_score": { 2023-06-23T18:49:33.150 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_score", 2023-06-23T18:49:33.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:33.150 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-06-23T18:49:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.151 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-06-23T18:49:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-06-23T18:49:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "upmap", 2023-06-23T18:49:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-compat", 2023-06-23T18:49:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "none", 2023-06-23T18:49:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap" 2023-06-23T18:49:33.153 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Balancer mode", 2023-06-23T18:49:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_ids": { 2023-06-23T18:49:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_ids", 2023-06-23T18:49:33.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-06-23T18:49:33.155 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-06-23T18:49:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.157 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_deviation": { 2023-06-23T18:49:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_deviation", 2023-06-23T18:49:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:33.158 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "deviation below which no optimization is attempted", 2023-06-23T18:49:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-06-23T18:49:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.159 INFO:teuthology.orchestra.run.smithi002.stdout: "upmap_max_optimizations": { 2023-06-23T18:49:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "upmap_max_optimizations", 2023-06-23T18:49:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.160 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-06-23T18:49:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cephadm", 2023-06-23T18:49:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_down_multiplier": { 2023-06-23T18:49:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_down_multiplier", 2023-06-23T18:49:33.162 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-06-23T18:49:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-06-23T18:49:33.163 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_refresh_rate": { 2023-06-23T18:49:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_refresh_rate", 2023-06-23T18:49:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "20", 2023-06-23T18:49:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-06-23T18:49:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.165 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "agent_starting_port": { 2023-06-23T18:49:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "agent_starting_port", 2023-06-23T18:49:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4721", 2023-06-23T18:49:33.166 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-06-23T18:49:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.167 INFO:teuthology.orchestra.run.smithi002.stdout: "alertmanager_web_password": { 2023-06-23T18:49:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alertmanager_web_password", 2023-06-23T18:49:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.168 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Alertmanager web password", 2023-06-23T18:49:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "alertmanager_web_user": { 2023-06-23T18:49:33.169 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "alertmanager_web_user", 2023-06-23T18:49:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Alertmanager web user", 2023-06-23T18:49:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_ptrace": { 2023-06-23T18:49:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_ptrace", 2023-06-23T18:49:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.171 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-06-23T18:49:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-06-23T18:49:33.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_interval": { 2023-06-23T18:49:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_interval", 2023-06-23T18:49:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.173 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to autotune daemon memory", 2023-06-23T18:49:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "autotune_memory_target_ratio": { 2023-06-23T18:49:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "autotune_memory_target_ratio", 2023-06-23T18:49:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0.7", 2023-06-23T18:49:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.175 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-06-23T18:49:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.176 INFO:teuthology.orchestra.run.smithi002.stdout: "cgroups_split": { 2023-06-23T18:49:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cgroups_split", 2023-06-23T18:49:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.177 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-06-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "config_checks_enabled": { 2023-06-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_checks_enabled", 2023-06-23T18:49:33.178 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-06-23T18:49:33.179 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "config_dashboard": { 2023-06-23T18:49:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_dashboard", 2023-06-23T18:49:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-06-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.181 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_alertmanager": { 2023-06-23T18:49:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_alertmanager", 2023-06-23T18:49:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-06-23T18:49:33.182 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.183 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_base": { 2023-06-23T18:49:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_base", 2023-06-23T18:49:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph", 2023-06-23T18:49:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.184 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Container image name, without the tag", 2023-06-23T18:49:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_elasticsearch": { 2023-06-23T18:49:33.185 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_elasticsearch", 2023-06-23T18:49:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-06-23T18:49:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.186 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "elasticsearch container image", 2023-06-23T18:49:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_grafana": { 2023-06-23T18:49:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_grafana", 2023-06-23T18:49:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.187 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-06-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_haproxy": { 2023-06-23T18:49:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_haproxy", 2023-06-23T18:49:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.189 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-06-23T18:49:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "HAproxy container image", 2023-06-23T18:49:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.190 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_agent": { 2023-06-23T18:49:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_agent", 2023-06-23T18:49:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-06-23T18:49:33.191 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger agent container image", 2023-06-23T18:49:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.192 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_collector": { 2023-06-23T18:49:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_collector", 2023-06-23T18:49:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-06-23T18:49:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.193 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger collector container image", 2023-06-23T18:49:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_jaeger_query": { 2023-06-23T18:49:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_jaeger_query", 2023-06-23T18:49:33.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-06-23T18:49:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Jaeger query container image", 2023-06-23T18:49:33.195 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_keepalived": { 2023-06-23T18:49:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_keepalived", 2023-06-23T18:49:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-06-23T18:49:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Keepalived container image", 2023-06-23T18:49:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.197 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_loki": { 2023-06-23T18:49:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_loki", 2023-06-23T18:49:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-06-23T18:49:33.198 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Loki container image", 2023-06-23T18:49:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_node_exporter": { 2023-06-23T18:49:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_node_exporter", 2023-06-23T18:49:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-06-23T18:49:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.200 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.201 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_prometheus": { 2023-06-23T18:49:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_prometheus", 2023-06-23T18:49:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-06-23T18:49:33.202 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus container image", 2023-06-23T18:49:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.203 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_promtail": { 2023-06-23T18:49:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_promtail", 2023-06-23T18:49:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-06-23T18:49:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.204 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Promtail container image", 2023-06-23T18:49:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image_snmp_gateway": { 2023-06-23T18:49:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_image_snmp_gateway", 2023-06-23T18:49:33.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-06-23T18:49:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "SNMP Gateway container image", 2023-06-23T18:49:33.206 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "container_init": { 2023-06-23T18:49:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "container_init", 2023-06-23T18:49:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Run podman/docker with `--init`", 2023-06-23T18:49:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.208 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "daemon_cache_timeout": { 2023-06-23T18:49:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "daemon_cache_timeout", 2023-06-23T18:49:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:33.209 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-06-23T18:49:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: "default_cephadm_command_timeout": { 2023-06-23T18:49:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_cephadm_command_timeout", 2023-06-23T18:49:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "900", 2023-06-23T18:49:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.211 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-06-23T18:49:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "default_registry": { 2023-06-23T18:49:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "default_registry", 2023-06-23T18:49:33.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "docker.io", 2023-06-23T18:49:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-06-23T18:49:33.213 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "device_cache_timeout": { 2023-06-23T18:49:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_cache_timeout", 2023-06-23T18:49:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.214 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1800", 2023-06-23T18:49:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache device inventory", 2023-06-23T18:49:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.215 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "device_enhanced_scan": { 2023-06-23T18:49:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_enhanced_scan", 2023-06-23T18:49:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.216 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use libstoragemgmt during device scans", 2023-06-23T18:49:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "facts_cache_timeout": { 2023-06-23T18:49:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "facts_cache_timeout", 2023-06-23T18:49:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.218 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "seconds to cache host facts data", 2023-06-23T18:49:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "host_check_interval": { 2023-06-23T18:49:33.219 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "host_check_interval", 2023-06-23T18:49:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.220 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to perform a host check", 2023-06-23T18:49:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.221 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.222 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "log_refresh_metadata": { 2023-06-23T18:49:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_refresh_metadata", 2023-06-23T18:49:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.224 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-06-23T18:49:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.226 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-06-23T18:49:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.227 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.228 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.229 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.230 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf": { 2023-06-23T18:49:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-06-23T18:49:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.232 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-06-23T18:49:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-06-23T18:49:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-06-23T18:49:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "*", 2023-06-23T18:49:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.234 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-06-23T18:49:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "max_count_per_host": { 2023-06-23T18:49:33.235 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_count_per_host", 2023-06-23T18:49:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.236 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of daemons per service per host", 2023-06-23T18:49:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd_draining_count": { 2023-06-23T18:49:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_osd_draining_count", 2023-06-23T18:49:33.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-06-23T18:49:33.238 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "migration_current": { 2023-06-23T18:49:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "migration_current", 2023-06-23T18:49:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "internal - do not modify", 2023-06-23T18:49:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.240 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "mode": { 2023-06-23T18:49:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mode", 2023-06-23T18:49:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "root", 2023-06-23T18:49:33.241 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm-package", 2023-06-23T18:49:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "root" 2023-06-23T18:49:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "mode for remote execution of cephadm", 2023-06-23T18:49:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_alerts_path": { 2023-06-23T18:49:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_alerts_path", 2023-06-23T18:49:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.243 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-06-23T18:49:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-06-23T18:49:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_web_password": { 2023-06-23T18:49:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_web_password", 2023-06-23T18:49:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.245 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus web password", 2023-06-23T18:49:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "prometheus_web_user": { 2023-06-23T18:49:33.246 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus_web_user", 2023-06-23T18:49:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.247 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Prometheus web user", 2023-06-23T18:49:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_insecure": { 2023-06-23T18:49:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_insecure", 2023-06-23T18:49:33.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-06-23T18:49:33.249 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_password": { 2023-06-23T18:49:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_password", 2023-06-23T18:49:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-06-23T18:49:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.251 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_url": { 2023-06-23T18:49:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_url", 2023-06-23T18:49:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.252 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-06-23T18:49:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "registry_username": { 2023-06-23T18:49:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "registry_username", 2023-06-23T18:49:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.254 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-06-23T18:49:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "secure_monitoring_stack": { 2023-06-23T18:49:33.255 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "secure_monitoring_stack", 2023-06-23T18:49:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.256 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-06-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "service_discovery_port": { 2023-06-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "service_discovery_port", 2023-06-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.257 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8765", 2023-06-23T18:49:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "cephadm service discovery port", 2023-06-23T18:49:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.258 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "ssh_config_file": { 2023-06-23T18:49:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssh_config_file", 2023-06-23T18:49:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.259 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-06-23T18:49:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "use_agent": { 2023-06-23T18:49:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_agent", 2023-06-23T18:49:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.261 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-06-23T18:49:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "use_repo_digest": { 2023-06-23T18:49:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "use_repo_digest", 2023-06-23T18:49:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-06-23T18:49:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_failed_host_check": { 2023-06-23T18:49:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_failed_host_check", 2023-06-23T18:49:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if the host check fails", 2023-06-23T18:49:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_daemons": { 2023-06-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_daemons", 2023-06-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-06-23T18:49:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_on_stray_hosts": { 2023-06-23T18:49:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_on_stray_hosts", 2023-06-23T18:49:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-06-23T18:49:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crash", 2023-06-23T18:49:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "retain_interval": { 2023-06-23T18:49:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retain_interval", 2023-06-23T18:49:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "31536000", 2023-06-23T18:49:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain crashes before pruning them", 2023-06-23T18:49:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_recent_interval": { 2023-06-23T18:49:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_recent_interval", 2023-06-23T18:49:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1209600", 2023-06-23T18:49:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "time interval in which to warn about recent crashes", 2023-06-23T18:49:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dashboard", 2023-06-23T18:49:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-06-23T18:49:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-06-23T18:49:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_HOST": { 2023-06-23T18:49:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_HOST", 2023-06-23T18:49:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-06-23T18:49:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-06-23T18:49:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_ENABLED": { 2023-06-23T18:49:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_ENABLED", 2023-06-23T18:49:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-06-23T18:49:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-06-23T18:49:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "ENABLE_BROWSABLE_API": { 2023-06-23T18:49:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ENABLE_BROWSABLE_API", 2023-06-23T18:49:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-06-23T18:49:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-06-23T18:49:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-06-23T18:49:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-06-23T18:49:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-06-23T18:49:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-06-23T18:49:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-06-23T18:49:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-06-23T18:49:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_NFS": { 2023-06-23T18:49:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-06-23T18:49:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RBD": { 2023-06-23T18:49:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-06-23T18:49:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "FEATURE_TOGGLE_RGW": { 2023-06-23T18:49:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-06-23T18:49:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-06-23T18:49:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-06-23T18:49:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_PASSWORD": { 2023-06-23T18:49:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_PASSWORD", 2023-06-23T18:49:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-06-23T18:49:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-06-23T18:49:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_URL": { 2023-06-23T18:49:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_URL", 2023-06-23T18:49:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_API_USERNAME": { 2023-06-23T18:49:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_API_USERNAME", 2023-06-23T18:49:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-06-23T18:49:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-06-23T18:49:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-06-23T18:49:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-06-23T18:49:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-06-23T18:49:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-06-23T18:49:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "ISSUE_TRACKER_API_KEY": { 2023-06-23T18:49:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-06-23T18:49:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_HOST": { 2023-06-23T18:49:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_HOST", 2023-06-23T18:49:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-06-23T18:49:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-06-23T18:49:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-06-23T18:49:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-06-23T18:49:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-06-23T18:49:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-06-23T18:49:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-06-23T18:49:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-06-23T18:49:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-06-23T18:49:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-06-23T18:49:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-06-23T18:49:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-06-23T18:49:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-06-23T18:49:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-06-23T18:49:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-06-23T18:49:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-06-23T18:49:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_ENABLED": { 2023-06-23T18:49:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_ENABLED", 2023-06-23T18:49:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-06-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-06-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-06-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-06-23T18:49:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-06-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-06-23T18:49:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-06-23T18:49:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8", 2023-06-23T18:49:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "REST_REQUESTS_TIMEOUT": { 2023-06-23T18:49:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-06-23T18:49:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "45", 2023-06-23T18:49:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ACCESS_KEY": { 2023-06-23T18:49:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ACCESS_KEY", 2023-06-23T18:49:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-06-23T18:49:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-06-23T18:49:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "admin", 2023-06-23T18:49:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SECRET_KEY": { 2023-06-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SECRET_KEY", 2023-06-23T18:49:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "RGW_API_SSL_VERIFY": { 2023-06-23T18:49:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "RGW_API_SSL_VERIFY", 2023-06-23T18:49:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-06-23T18:49:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-06-23T18:49:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-06-23T18:49:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-06-23T18:49:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-06-23T18:49:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-06-23T18:49:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "cross_origin_url": { 2023-06-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cross_origin_url", 2023-06-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "crt_file": { 2023-06-23T18:49:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "crt_file", 2023-06-23T18:49:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "debug": { 2023-06-23T18:49:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "debug", 2023-06-23T18:49:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Enable/disable debug options", 2023-06-23T18:49:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "jwt_token_ttl": { 2023-06-23T18:49:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "jwt_token_ttl", 2023-06-23T18:49:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "28800", 2023-06-23T18:49:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-06-23T18:49:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-06-23T18:49:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "motd": { 2023-06-23T18:49:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "motd", 2023-06-23T18:49:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "The message of the day", 2023-06-23T18:49:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect_resolve_ip_addr": { 2023-06-23T18:49:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "redirect_resolve_ip_addr", 2023-06-23T18:49:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-06-23T18:49:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8080", 2023-06-23T18:49:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-06-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-06-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl_server_port": { 2023-06-23T18:49:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl_server_port", 2023-06-23T18:49:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8443", 2023-06-23T18:49:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-06-23T18:49:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-06-23T18:49:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "redirect", 2023-06-23T18:49:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.389 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "redirect" 2023-06-23T18:49:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-06-23T18:49:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-06-23T18:49:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-06-23T18:49:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-06-23T18:49:33.391 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-06-23T18:49:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.392 INFO:teuthology.orchestra.run.smithi002.stdout: "url_prefix": { 2023-06-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url_prefix", 2023-06-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.393 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2023-06-23T18:49:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_monitoring": { 2023-06-23T18:49:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_monitoring", 2023-06-23T18:49:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.395 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "monitor device health metrics", 2023-06-23T18:49:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.397 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.398 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.399 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.400 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.402 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.403 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.404 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "mark_out_threshold": { 2023-06-23T18:49:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mark_out_threshold", 2023-06-23T18:49:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "2419200", 2023-06-23T18:49:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.406 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-06-23T18:49:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": { 2023-06-23T18:49:33.407 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pool_name", 2023-06-23T18:49:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "device_health_metrics", 2023-06-23T18:49:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.408 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name of pool in which to store device health metrics", 2023-06-23T18:49:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "retention_period": { 2023-06-23T18:49:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "retention_period", 2023-06-23T18:49:33.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15552000", 2023-06-23T18:49:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long to retain device health metrics", 2023-06-23T18:49:33.410 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_frequency": { 2023-06-23T18:49:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_frequency", 2023-06-23T18:49:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-06-23T18:49:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to scrape device health metrics", 2023-06-23T18:49:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.412 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "self_heal": { 2023-06-23T18:49:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "self_heal", 2023-06-23T18:49:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.413 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-06-23T18:49:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.414 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.415 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how frequently to wake up and check device health", 2023-06-23T18:49:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "warn_threshold": { 2023-06-23T18:49:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "warn_threshold", 2023-06-23T18:49:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7257600", 2023-06-23T18:49:33.417 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-06-23T18:49:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "diskprediction_local", 2023-06-23T18:49:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.419 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.420 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.421 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.423 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.424 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.426 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.427 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.428 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "predict_interval": { 2023-06-23T18:49:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predict_interval", 2023-06-23T18:49:33.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "86400", 2023-06-23T18:49:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.430 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "predictor_model": { 2023-06-23T18:49:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "predictor_model", 2023-06-23T18:49:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "prophetstor", 2023-06-23T18:49:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.432 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "600", 2023-06-23T18:49:33.433 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "influx", 2023-06-23T18:49:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": false, 2023-06-23T18:49:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "influxdb python module not found", 2023-06-23T18:49:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "batch_size": { 2023-06-23T18:49:33.435 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "batch_size", 2023-06-23T18:49:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5000", 2023-06-23T18:49:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.436 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-06-23T18:49:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "database": { 2023-06-23T18:49:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "database", 2023-06-23T18:49:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.437 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "ceph", 2023-06-23T18:49:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-06-23T18:49:33.438 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "hostname": { 2023-06-23T18:49:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "hostname", 2023-06-23T18:49:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server hostname", 2023-06-23T18:49:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.440 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "30", 2023-06-23T18:49:33.441 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "5", 2023-06-23T18:49:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-06-23T18:49:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.442 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.443 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.444 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.445 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.446 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.448 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.450 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "password": { 2023-06-23T18:49:33.451 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "password", 2023-06-23T18:49:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.452 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "password of InfluxDB server user", 2023-06-23T18:49:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "port": { 2023-06-23T18:49:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "port", 2023-06-23T18:49:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.453 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "8086", 2023-06-23T18:49:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.454 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "InfluxDB server port", 2023-06-23T18:49:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "ssl": { 2023-06-23T18:49:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ssl", 2023-06-23T18:49:33.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "false", 2023-06-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-06-23T18:49:33.456 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "threads": { 2023-06-23T18:49:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threads", 2023-06-23T18:49:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "32", 2023-06-23T18:49:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-06-23T18:49:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.458 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "username": { 2023-06-23T18:49:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "username", 2023-06-23T18:49:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.459 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "username of InfluxDB server user", 2023-06-23T18:49:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.460 INFO:teuthology.orchestra.run.smithi002.stdout: "verify_ssl": { 2023-06-23T18:49:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "verify_ssl", 2023-06-23T18:49:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "true", 2023-06-23T18:49:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.461 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-06-23T18:49:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "insights", 2023-06-23T18:49:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.463 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.464 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.465 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.466 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.467 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.468 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.469 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.471 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "iostat", 2023-06-23T18:49:33.472 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.473 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.474 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.475 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.476 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.477 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.478 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.479 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.480 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.481 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "k8sevents", 2023-06-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.482 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "ceph_event_retention_days": { 2023-06-23T18:49:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "ceph_event_retention_days", 2023-06-23T18:49:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "7", 2023-06-23T18:49:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.483 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Days to hold ceph event information within local cache", 2023-06-23T18:49:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.484 INFO:teuthology.orchestra.run.smithi002.stdout: "config_check_secs": { 2023-06-23T18:49:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "config_check_secs", 2023-06-23T18:49:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "10", 2023-06-23T18:49:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.485 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-06-23T18:49:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.487 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.489 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.491 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.492 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.494 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "localpool", 2023-06-23T18:49:33.495 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "failure_domain": { 2023-06-23T18:49:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "failure_domain", 2023-06-23T18:49:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "host", 2023-06-23T18:49:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "failure domain for any created local pool", 2023-06-23T18:49:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-06-23T18:49:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.497 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.498 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.499 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.500 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.501 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.503 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.503 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.503 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.503 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.503 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.503 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.503 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.503 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.505 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": { 2023-06-23T18:49:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "min_size", 2023-06-23T18:49:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.507 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default min_size for any created local pool", 2023-06-23T18:49:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-06-23T18:49:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "num_rep": { 2023-06-23T18:49:33.508 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "num_rep", 2023-06-23T18:49:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3", 2023-06-23T18:49:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.509 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default replica count for any created local pool", 2023-06-23T18:49:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": { 2023-06-23T18:49:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_num", 2023-06-23T18:49:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.510 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "128", 2023-06-23T18:49:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "default pg_num for any created local pool", 2023-06-23T18:49:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "prefix": { 2023-06-23T18:49:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prefix", 2023-06-23T18:49:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.512 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "name prefix for any created local pool", 2023-06-23T18:49:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.513 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "subtree": { 2023-06-23T18:49:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "subtree", 2023-06-23T18:49:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "rack", 2023-06-23T18:49:33.514 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "CRUSH level for which to create a local pool", 2023-06-23T18:49:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-06-23T18:49:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mds_autoscaler", 2023-06-23T18:49:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.516 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.517 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.518 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.520 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.521 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.522 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.523 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.524 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirroring", 2023-06-23T18:49:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.526 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.527 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.528 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.529 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.530 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.531 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.532 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.534 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "nfs", 2023-06-23T18:49:33.535 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.536 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.538 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.539 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.540 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.541 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.542 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.543 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.544 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-06-23T18:49:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "fail_fs": { 2023-06-23T18:49:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "fail_fs", 2023-06-23T18:49:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.547 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-06-23T18:49:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.549 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.550 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.552 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.553 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.554 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.555 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.556 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.557 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator": { 2023-06-23T18:49:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "orchestrator", 2023-06-23T18:49:33.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.559 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm", 2023-06-23T18:49:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "rook", 2023-06-23T18:49:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "test_orchestrator" 2023-06-23T18:49:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Orchestrator backend", 2023-06-23T18:49:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_perf_query", 2023-06-23T18:49:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.561 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.562 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.564 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.565 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.567 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.568 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.570 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.571 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "osd_support", 2023-06-23T18:49:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.572 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.573 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.574 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.575 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.577 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.578 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.579 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.580 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "pg_autoscaler", 2023-06-23T18:49:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.582 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.583 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.584 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.585 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.587 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.588 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.589 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.590 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "noautoscale": { 2023-06-23T18:49:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "noautoscale", 2023-06-23T18:49:33.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.592 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "global autoscale flag", 2023-06-23T18:49:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-06-23T18:49:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.593 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.594 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "threshold": { 2023-06-23T18:49:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "threshold", 2023-06-23T18:49:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "3.0", 2023-06-23T18:49:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1.0", 2023-06-23T18:49:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "scaling threshold", 2023-06-23T18:49:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-06-23T18:49:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.596 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "progress", 2023-06-23T18:49:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.597 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_pg_recovery_event": { 2023-06-23T18:49:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_pg_recovery_event", 2023-06-23T18:49:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.598 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow the module to show pg recovery progress", 2023-06-23T18:49:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-06-23T18:49:33.599 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-06-23T18:49:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.600 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.601 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.602 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.603 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.604 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.605 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.606 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.607 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.608 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.609 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "max_completed_events": { 2023-06-23T18:49:33.610 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_completed_events", 2023-06-23T18:49:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "50", 2023-06-23T18:49:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.611 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "number of past completed events to remember", 2023-06-23T18:49:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "sleep_interval": { 2023-06-23T18:49:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "sleep_interval", 2023-06-23T18:49:33.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "5", 2023-06-23T18:49:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "how long the module is going to sleep", 2023-06-23T18:49:33.613 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "prometheus", 2023-06-23T18:49:33.614 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "cache": { 2023-06-23T18:49:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "cache", 2023-06-23T18:49:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.615 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "exclude_perf_counters": { 2023-06-23T18:49:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "exclude_perf_counters", 2023-06-23T18:49:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.617 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-06-23T18:49:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-06-23T18:49:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.619 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.620 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.621 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.622 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.623 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.624 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.625 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.626 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.627 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools": { 2023-06-23T18:49:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools", 2023-06-23T18:49:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.629 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_stats_pools_refresh_interval": { 2023-06-23T18:49:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-06-23T18:49:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300", 2023-06-23T18:49:33.630 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.631 INFO:teuthology.orchestra.run.smithi002.stdout: "scrape_interval": { 2023-06-23T18:49:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "scrape_interval", 2023-06-23T18:49:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15.0", 2023-06-23T18:49:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.632 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:33.633 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "::", 2023-06-23T18:49:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.634 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-06-23T18:49:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:33.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "9283", 2023-06-23T18:49:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.636 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-06-23T18:49:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "stale_cache_strategy": { 2023-06-23T18:49:33.637 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stale_cache_strategy", 2023-06-23T18:49:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "log", 2023-06-23T18:49:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.638 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_behaviour": { 2023-06-23T18:49:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_behaviour", 2023-06-23T18:49:33.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "default", 2023-06-23T18:49:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.640 INFO:teuthology.orchestra.run.smithi002.stdout: "default", 2023-06-23T18:49:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "error" 2023-06-23T18:49:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.641 INFO:teuthology.orchestra.run.smithi002.stdout: "standby_error_status_code": { 2023-06-23T18:49:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "standby_error_status_code", 2023-06-23T18:49:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "500", 2023-06-23T18:49:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "400", 2023-06-23T18:49:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "599", 2023-06-23T18:49:33.642 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2023-06-23T18:49:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.644 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.645 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.646 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.648 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.649 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.650 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.651 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.652 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_snap_create": { 2023-06-23T18:49:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_snap_create", 2023-06-23T18:49:33.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10", 2023-06-23T18:49:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.654 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "mirror_snapshot_schedule": { 2023-06-23T18:49:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mirror_snapshot_schedule", 2023-06-23T18:49:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.655 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "trash_purge_schedule": { 2023-06-23T18:49:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "trash_purge_schedule", 2023-06-23T18:49:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "restful", 2023-06-23T18:49:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.659 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_auth": { 2023-06-23T18:49:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enable_auth", 2023-06-23T18:49:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.660 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.661 INFO:teuthology.orchestra.run.smithi002.stdout: "key_file": { 2023-06-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "key_file", 2023-06-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.662 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.663 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.663 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.663 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.663 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.663 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.663 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.664 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.664 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.664 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.664 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.664 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.664 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.665 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.665 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.665 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.665 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.665 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.665 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.665 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.665 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.666 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.666 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.666 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.666 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.666 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.667 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.667 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.667 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.667 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.667 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.667 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.667 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.668 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.668 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.668 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.668 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.668 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.668 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.668 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.669 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.669 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.669 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.669 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.669 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.670 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.670 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.670 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.670 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.670 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.670 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.670 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.671 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.671 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.671 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.671 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.671 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.672 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.672 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.672 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.672 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.672 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.672 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.672 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.673 INFO:teuthology.orchestra.run.smithi002.stdout: "server_addr": { 2023-06-23T18:49:33.673 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_addr", 2023-06-23T18:49:33.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.673 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.674 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.674 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.674 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.674 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.674 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.674 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.674 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.675 INFO:teuthology.orchestra.run.smithi002.stdout: "server_port": { 2023-06-23T18:49:33.675 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "server_port", 2023-06-23T18:49:33.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.675 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.675 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.675 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.675 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rgw", 2023-06-23T18:49:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.678 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.678 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.678 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.678 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.678 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.678 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.679 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.679 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.679 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.679 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.679 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.679 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.679 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.679 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.680 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.680 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.680 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.680 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.680 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.681 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.681 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.681 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.681 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.681 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.681 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.682 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.682 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.682 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.682 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.682 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.682 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.683 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.683 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.683 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.683 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.683 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.683 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.684 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.684 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.684 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.684 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.684 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.684 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.684 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.684 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.686 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.686 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.686 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.686 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.686 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.686 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.686 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.687 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.687 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rook", 2023-06-23T18:49:33.687 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.687 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.688 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.688 INFO:teuthology.orchestra.run.smithi002.stdout: "drive_group_interval": { 2023-06-23T18:49:33.688 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "drive_group_interval", 2023-06-23T18:49:33.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:33.688 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.688 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "300.0", 2023-06-23T18:49:33.688 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-06-23T18:49:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.689 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.690 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.690 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.690 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.690 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.690 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.690 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.691 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.691 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.691 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.691 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.691 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.691 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.691 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.691 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.692 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.692 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.692 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.692 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.694 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.694 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.694 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.694 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.694 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.695 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.695 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.695 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.695 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.695 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.695 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.695 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.696 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.696 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.696 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.696 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.696 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.696 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.696 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.697 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.697 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.697 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.697 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.697 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.698 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.698 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.698 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.698 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.699 INFO:teuthology.orchestra.run.smithi002.stdout: "storage_class": { 2023-06-23T18:49:33.699 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "storage_class", 2023-06-23T18:49:33.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.699 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.699 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "local", 2023-06-23T18:49:33.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.699 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.700 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.700 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-06-23T18:49:33.700 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.700 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "selftest", 2023-06-23T18:49:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.702 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.702 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.702 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.702 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.702 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.703 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.703 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.703 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.703 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.703 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.703 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.703 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.704 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.704 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.704 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.704 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.705 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.705 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.705 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.705 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.706 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.706 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.706 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.706 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.706 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.706 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.706 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.707 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.708 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.708 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.708 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.708 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.708 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.708 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.708 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.708 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.709 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.709 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.709 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.709 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.710 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.710 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.710 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.710 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.710 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.710 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.710 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.711 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.711 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.711 INFO:teuthology.orchestra.run.smithi002.stdout: "roption1": { 2023-06-23T18:49:33.711 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption1", 2023-06-23T18:49:33.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.711 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.712 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.712 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.712 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.712 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.712 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.712 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.712 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.713 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.713 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.713 INFO:teuthology.orchestra.run.smithi002.stdout: "roption2": { 2023-06-23T18:49:33.713 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "roption2", 2023-06-23T18:49:33.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.713 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.714 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "xyz", 2023-06-23T18:49:33.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.714 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.714 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.714 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.714 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.714 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.715 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption1": { 2023-06-23T18:49:33.715 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption1", 2023-06-23T18:49:33.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.715 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.715 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.716 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.716 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.716 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.716 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.716 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.717 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption2": { 2023-06-23T18:49:33.717 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption2", 2023-06-23T18:49:33.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.717 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.717 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.717 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.718 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption3": { 2023-06-23T18:49:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption3", 2023-06-23T18:49:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "float", 2023-06-23T18:49:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.719 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.720 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.720 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.720 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.720 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption4": { 2023-06-23T18:49:33.720 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption4", 2023-06-23T18:49:33.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.721 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.721 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.721 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.721 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.721 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.721 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.722 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.722 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.722 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption5": { 2023-06-23T18:49:33.722 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption5", 2023-06-23T18:49:33.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.722 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption6": { 2023-06-23T18:49:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption6", 2023-06-23T18:49:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.725 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.726 INFO:teuthology.orchestra.run.smithi002.stdout: "rwoption7": { 2023-06-23T18:49:33.726 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rwoption7", 2023-06-23T18:49:33.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.726 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.726 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.726 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.726 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "1", 2023-06-23T18:49:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "42", 2023-06-23T18:49:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.727 INFO:teuthology.orchestra.run.smithi002.stdout: "testkey": { 2023-06-23T18:49:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testkey", 2023-06-23T18:49:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.728 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.729 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.729 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.729 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.729 INFO:teuthology.orchestra.run.smithi002.stdout: "testlkey": { 2023-06-23T18:49:33.729 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testlkey", 2023-06-23T18:49:33.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.730 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.730 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.730 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.730 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.730 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.730 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.730 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.731 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.731 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.731 INFO:teuthology.orchestra.run.smithi002.stdout: "testnewline": { 2023-06-23T18:49:33.731 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "testnewline", 2023-06-23T18:49:33.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.731 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.731 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.732 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.732 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.732 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.732 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.732 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.732 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.732 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.733 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snap_schedule", 2023-06-23T18:49:33.733 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.733 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.734 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.734 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_m_granularity": { 2023-06-23T18:49:33.734 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "allow_m_granularity", 2023-06-23T18:49:33.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.734 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.734 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.734 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.734 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.735 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.735 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.735 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "allow minute scheduled snapshots", 2023-06-23T18:49:33.735 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.735 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.735 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.735 INFO:teuthology.orchestra.run.smithi002.stdout: "dump_on_update": { 2023-06-23T18:49:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "dump_on_update", 2023-06-23T18:49:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.736 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "dump database to debug log on update", 2023-06-23T18:49:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.738 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.739 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.739 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.739 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.739 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.739 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.739 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.739 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.740 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.740 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.740 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.740 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.740 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.740 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.741 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.741 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.741 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.741 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.741 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.741 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.743 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.745 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "stats", 2023-06-23T18:49:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.747 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.748 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.749 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.750 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.751 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.753 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.754 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.755 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.756 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "status", 2023-06-23T18:49:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.759 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.760 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.763 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.764 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.766 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telegraf", 2023-06-23T18:49:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.768 INFO:teuthology.orchestra.run.smithi002.stdout: "address": { 2023-06-23T18:49:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "address", 2023-06-23T18:49:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-06-23T18:49:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.769 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:33.770 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "15", 2023-06-23T18:49:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.771 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.773 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.774 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.775 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.776 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.778 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.780 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "telemetry", 2023-06-23T18:49:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_basic": { 2023-06-23T18:49:33.782 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_basic", 2023-06-23T18:49:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.783 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share basic cluster information (size, version)", 2023-06-23T18:49:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_crash": { 2023-06-23T18:49:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_crash", 2023-06-23T18:49:33.784 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-06-23T18:49:33.785 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_device": { 2023-06-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_device", 2023-06-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "True", 2023-06-23T18:49:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-06-23T18:49:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.787 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_ident": { 2023-06-23T18:49:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_ident", 2023-06-23T18:49:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.788 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-06-23T18:49:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.789 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "channel_perf": { 2023-06-23T18:49:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "channel_perf", 2023-06-23T18:49:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.790 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Share various performance metrics of a cluster", 2023-06-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.791 INFO:teuthology.orchestra.run.smithi002.stdout: "contact": { 2023-06-23T18:49:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "contact", 2023-06-23T18:49:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.792 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "description": { 2023-06-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "description", 2023-06-23T18:49:33.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.794 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "device_url": { 2023-06-23T18:49:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "device_url", 2023-06-23T18:49:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.795 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-06-23T18:49:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "enabled": { 2023-06-23T18:49:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "enabled", 2023-06-23T18:49:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.797 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.798 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "24", 2023-06-23T18:49:33.799 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "8", 2023-06-23T18:49:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.800 INFO:teuthology.orchestra.run.smithi002.stdout: "last_opt_revision": { 2023-06-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "last_opt_revision", 2023-06-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "1", 2023-06-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.801 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard": { 2023-06-23T18:49:33.802 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard", 2023-06-23T18:49:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.804 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "leaderboard_description": { 2023-06-23T18:49:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "leaderboard_description", 2023-06-23T18:49:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.805 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.812 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.813 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.814 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.815 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "organization": { 2023-06-23T18:49:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "organization", 2023-06-23T18:49:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "proxy": { 2023-06-23T18:49:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "proxy", 2023-06-23T18:49:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "url": { 2023-06-23T18:49:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "url", 2023-06-23T18:49:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-06-23T18:49:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "test_orchestrator", 2023-06-23T18:49:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2023-06-23T18:49:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "max_concurrent_clones": { 2023-06-23T18:49:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "max_concurrent_clones", 2023-06-23T18:49:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "4", 2023-06-23T18:49:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Number of asynchronous cloner threads", 2023-06-23T18:49:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "snapshot_clone_delay": { 2023-06-23T18:49:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "snapshot_clone_delay", 2023-06-23T18:49:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "0", 2023-06-23T18:49:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-06-23T18:49:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix", 2023-06-23T18:49:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "can_run": true, 2023-06-23T18:49:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "error_string": "", 2023-06-23T18:49:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "module_options": { 2023-06-23T18:49:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "discovery_interval": { 2023-06-23T18:49:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "discovery_interval", 2023-06-23T18:49:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "uint", 2023-06-23T18:49:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "100", 2023-06-23T18:49:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "identifier": { 2023-06-23T18:49:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "identifier", 2023-06-23T18:49:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "interval": { 2023-06-23T18:49:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "interval", 2023-06-23T18:49:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "secs", 2023-06-23T18:49:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "60", 2023-06-23T18:49:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "log_level": { 2023-06-23T18:49:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_level", 2023-06-23T18:49:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster": { 2023-06-23T18:49:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster", 2023-06-23T18:49:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_cluster_level": { 2023-06-23T18:49:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_cluster_level", 2023-06-23T18:49:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "info", 2023-06-23T18:49:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [ 2023-06-23T18:49:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "", 2023-06-23T18:49:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "critical", 2023-06-23T18:49:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "debug", 2023-06-23T18:49:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "error", 2023-06-23T18:49:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "info", 2023-06-23T18:49:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "warning" 2023-06-23T18:49:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": { 2023-06-23T18:49:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "log_to_file", 2023-06-23T18:49:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "bool", 2023-06-23T18:49:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2023-06-23T18:49:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "False", 2023-06-23T18:49:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_host": { 2023-06-23T18:49:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_host", 2023-06-23T18:49:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "", 2023-06-23T18:49:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_port": { 2023-06-23T18:49:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_port", 2023-06-23T18:49:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "int", 2023-06-23T18:49:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "10051", 2023-06-23T18:49:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "zabbix_sender": { 2023-06-23T18:49:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "zabbix_sender", 2023-06-23T18:49:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "str", 2023-06-23T18:49:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "level": "advanced", 2023-06-23T18:49:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 0, 2023-06-23T18:49:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-06-23T18:49:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "min": "", 2023-06-23T18:49:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "max": "", 2023-06-23T18:49:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "enum_allowed": [], 2023-06-23T18:49:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "desc": "", 2023-06-23T18:49:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "long_desc": "", 2023-06-23T18:49:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "tags": [], 2023-06-23T18:49:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "see_also": [] 2023-06-23T18:49:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "services": { 2023-06-23T18:49:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "dashboard": "https://172.21.15.2:8443/" 2023-06-23T18:49:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "always_on_modules": { 2023-06-23T18:49:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "octopus": [ 2023-06-23T18:49:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2023-06-23T18:49:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2023-06-23T18:49:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2023-06-23T18:49:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2023-06-23T18:49:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2023-06-23T18:49:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2023-06-23T18:49:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2023-06-23T18:49:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2023-06-23T18:49:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2023-06-23T18:49:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2023-06-23T18:49:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "pacific": [ 2023-06-23T18:49:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2023-06-23T18:49:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2023-06-23T18:49:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2023-06-23T18:49:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2023-06-23T18:49:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2023-06-23T18:49:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2023-06-23T18:49:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2023-06-23T18:49:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2023-06-23T18:49:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2023-06-23T18:49:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2023-06-23T18:49:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "quincy": [ 2023-06-23T18:49:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2023-06-23T18:49:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2023-06-23T18:49:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2023-06-23T18:49:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2023-06-23T18:49:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2023-06-23T18:49:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2023-06-23T18:49:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2023-06-23T18:49:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2023-06-23T18:49:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2023-06-23T18:49:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2023-06-23T18:49:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2023-06-23T18:49:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "reef": [ 2023-06-23T18:49:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "balancer", 2023-06-23T18:49:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "crash", 2023-06-23T18:49:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "devicehealth", 2023-06-23T18:49:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "orchestrator", 2023-06-23T18:49:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscaler", 2023-06-23T18:49:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "progress", 2023-06-23T18:49:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_support", 2023-06-23T18:49:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "status", 2023-06-23T18:49:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "telemetry", 2023-06-23T18:49:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "volumes" 2023-06-23T18:49:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "last_failure_osd_epoch": 3, 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "active_clients": [ 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "devicehealth", 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-06-23T18:49:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 787254849 2023-06-23T18:49:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "libcephsqlite", 2023-06-23T18:49:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-06-23T18:49:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-06-23T18:49:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-06-23T18:49:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3906344914 2023-06-23T18:49:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "rbd_support", 2023-06-23T18:49:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-06-23T18:49:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-06-23T18:49:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-06-23T18:49:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 366730668 2023-06-23T18:49:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "volumes", 2023-06-23T18:49:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2023-06-23T18:49:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-06-23T18:49:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2023-06-23T18:49:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:0", 2023-06-23T18:49:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1228448929 2023-06-23T18:49:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-06-23T18:49:33.879 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:49:33.880 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:31 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2263335031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:49:33.881 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:31 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3429087876' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-06-23T18:49:33.881 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:32 smithi082 ceph-mon[97889]: pgmap v1121: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2023-06-23T18:49:33.881 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:31 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2263335031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:49:33.881 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:31 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3429087876' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-06-23T18:49:33.881 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:32 smithi017 ceph-mon[97472]: pgmap v1121: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2023-06-23T18:49:33.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8f240a11d0 msgr2=0x7f8f240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:33.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8f240a11d0 0x7f8f240a3680 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400b2e0 tx=0x7f8f34004000 comp rx=0 tx=0).stop 2023-06-23T18:49:33.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380fea60 msgr2=0x7f8f381946a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:49:33.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380fea60 0x7f8f381946a0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00e970 tx=0x7f8f2c00ec80 comp rx=0 tx=0).stop 2023-06-23T18:49:33.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 shutdown_connections 2023-06-23T18:49:33.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f381005e0 0x7f8f3819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:33.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f380fea60 0x7f8f381946a0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:33.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f8f240a11d0 0x7f8f240a3680 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:33.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 --2- 172.21.15.2:0/3429087876 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f380ffc50 0x7f8f38194be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:49:33.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 >> 172.21.15.2:0/3429087876 conn(0x7f8f380fa0e0 msgr2=0x7f8f380fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:49:33.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 shutdown_connections 2023-06-23T18:49:33.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:49:31.531+0000 7f8f3fab3700 1 -- 172.21.15.2:0/3429087876 wait complete. 2023-06-23T18:49:33.890 DEBUG:tasks.check_counter:Getting stats from a 2023-06-23T18:49:33.891 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.a perf dump 2023-06-23T18:49:33.891 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:31 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2263335031' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:49:33.892 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:31 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3429087876' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-06-23T18:49:33.892 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:32 smithi002 ceph-mon[95947]: pgmap v1121: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 7.8 KiB/s wr, 7 op/s 2023-06-23T18:49:34.481 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:49:35.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:34 smithi017 ceph-mon[97472]: pgmap v1122: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-06-23T18:49:35.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:34 smithi017 ceph-mon[97472]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:49:35.192 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:34 smithi002 ceph-mon[95947]: pgmap v1122: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-06-23T18:49:35.193 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:34 smithi002 ceph-mon[95947]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:49:35.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:34 smithi082 ceph-mon[97889]: pgmap v1122: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-06-23T18:49:35.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:34 smithi082 ceph-mon[97889]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2023-06-23T18:49:36.282 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:49:36.282 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 463, 2023-06-23T18:49:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 460, 2023-06-23T18:49:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 541664, 2023-06-23T18:49:36.283 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 264160, 2023-06-23T18:49:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2023-06-23T18:49:36.284 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.059400362, 2023-06-23T18:49:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.023504887, 2023-06-23T18:49:36.285 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.019794440, 2023-06-23T18:49:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.011406595, 2023-06-23T18:49:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 459, 2023-06-23T18:49:36.286 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.022796593, 2023-06-23T18:49:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000049665 2023-06-23T18:49:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:36.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 541664, 2023-06-23T18:49:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 264160 2023-06-23T18:49:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2023-06-23T18:49:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 352, 2023-06-23T18:49:36.288 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2023-06-23T18:49:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 213344, 2023-06-23T18:49:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2023-06-23T18:49:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.021226227, 2023-06-23T18:49:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.016889713, 2023-06-23T18:49:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001615727, 2023-06-23T18:49:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:36.289 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 350, 2023-06-23T18:49:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.025659330, 2023-06-23T18:49:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000073312 2023-06-23T18:49:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:36.290 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2023-06-23T18:49:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 213344 2023-06-23T18:49:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 4, 2023-06-23T18:49:36.291 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 4, 2023-06-23T18:49:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 2960, 2023-06-23T18:49:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 448, 2023-06-23T18:49:36.292 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2023-06-23T18:49:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2023-06-23T18:49:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.000862989, 2023-06-23T18:49:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000055451, 2023-06-23T18:49:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000789095, 2023-06-23T18:49:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:36.293 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 3, 2023-06-23T18:49:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000900590, 2023-06-23T18:49:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000300196 2023-06-23T18:49:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:36.294 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-06-23T18:49:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 448 2023-06-23T18:49:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.295 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2023-06-23T18:49:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2023-06-23T18:49:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2023-06-23T18:49:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2023-06-23T18:49:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:36.296 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:36.297 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:36.298 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:36.299 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2023-06-23T18:49:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:36.300 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148704, 2023-06-23T18:49:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 29, 2023-06-23T18:49:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2023-06-23T18:49:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2023-06-23T18:49:36.301 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2023-06-23T18:49:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2023-06-23T18:49:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:36.302 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2023-06-23T18:49:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2023-06-23T18:49:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2023-06-23T18:49:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2023-06-23T18:49:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:36.303 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2023-06-23T18:49:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-06-23T18:49:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-06-23T18:49:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-06-23T18:49:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 465, 2023-06-23T18:49:36.304 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 476147, 2023-06-23T18:49:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 465, 2023-06-23T18:49:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-06-23T18:49:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-06-23T18:49:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 465, 2023-06-23T18:49:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 476147, 2023-06-23T18:49:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-06-23T18:49:36.305 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:36.306 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:36.306 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:49:36.647 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-06-23T18:49:36.647 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-06-23T18:49:36.648 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-06-23T18:49:36.649 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-06-23T18:49:36.649 DEBUG:tasks.check_counter:Getting stats from d 2023-06-23T18:49:36.650 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.d perf dump 2023-06-23T18:49:37.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:36 smithi002 ceph-mon[95947]: pgmap v1123: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2023-06-23T18:49:37.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:36 smithi017 ceph-mon[97472]: pgmap v1123: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2023-06-23T18:49:37.216 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:49:37.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:36 smithi082 ceph-mon[97889]: pgmap v1123: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 8 op/s 2023-06-23T18:49:38.470 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:49:38.470 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:38.470 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2023-06-23T18:49:38.470 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2023-06-23T18:49:38.471 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2023-06-23T18:49:38.471 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2023-06-23T18:49:38.471 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:38.471 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2023-06-23T18:49:38.471 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001638981, 2023-06-23T18:49:38.471 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2023-06-23T18:49:38.471 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000363738, 2023-06-23T18:49:38.472 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:38.472 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:38.472 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:38.472 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:38.472 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:38.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.473 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:38.473 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:38.473 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:38.473 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:38.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.474 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2023-06-23T18:49:38.474 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2023-06-23T18:49:38.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.474 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:38.474 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 472, 2023-06-23T18:49:38.474 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 467, 2023-06-23T18:49:38.474 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 545984, 2023-06-23T18:49:38.475 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 267568, 2023-06-23T18:49:38.475 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2023-06-23T18:49:38.475 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2023-06-23T18:49:38.475 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.060033480, 2023-06-23T18:49:38.475 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.024124549, 2023-06-23T18:49:38.475 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.020850504, 2023-06-23T18:49:38.475 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.011617686, 2023-06-23T18:49:38.475 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:38.476 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 467, 2023-06-23T18:49:38.476 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.019812196, 2023-06-23T18:49:38.476 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000042424 2023-06-23T18:49:38.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.477 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:38.477 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:38.477 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:38.477 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:38.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 545984, 2023-06-23T18:49:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 267568 2023-06-23T18:49:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2023-06-23T18:49:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 356, 2023-06-23T18:49:38.478 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2023-06-23T18:49:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 214048, 2023-06-23T18:49:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2023-06-23T18:49:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.022176537, 2023-06-23T18:49:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.017775516, 2023-06-23T18:49:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001668220, 2023-06-23T18:49:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:38.479 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 354, 2023-06-23T18:49:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.018676008, 2023-06-23T18:49:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000052757 2023-06-23T18:49:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:38.480 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2023-06-23T18:49:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 214048 2023-06-23T18:49:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2023-06-23T18:49:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2023-06-23T18:49:38.481 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2023-06-23T18:49:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2023-06-23T18:49:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:38.482 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:38.483 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:38.484 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:38.485 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:38.486 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:38.487 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:38.488 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2023-06-23T18:49:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:38.489 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148744, 2023-06-23T18:49:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2023-06-23T18:49:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2023-06-23T18:49:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2023-06-23T18:49:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:38.490 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2023-06-23T18:49:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2023-06-23T18:49:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:38.491 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2023-06-23T18:49:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2023-06-23T18:49:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2023-06-23T18:49:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2023-06-23T18:49:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2023-06-23T18:49:38.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-06-23T18:49:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-06-23T18:49:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-06-23T18:49:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 470, 2023-06-23T18:49:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 476797, 2023-06-23T18:49:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:38.493 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 470, 2023-06-23T18:49:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-06-23T18:49:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-06-23T18:49:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 470, 2023-06-23T18:49:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 476797, 2023-06-23T18:49:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-06-23T18:49:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:38.494 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:38.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:38.495 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:49:38.985 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-06-23T18:49:38.986 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-06-23T18:49:38.986 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-06-23T18:49:38.986 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-06-23T18:49:38.986 DEBUG:tasks.check_counter:Getting stats from g 2023-06-23T18:49:38.987 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.g perf dump 2023-06-23T18:49:39.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:38 smithi017 ceph-mon[97472]: pgmap v1124: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2023-06-23T18:49:39.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:38 smithi002 ceph-mon[95947]: pgmap v1124: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2023-06-23T18:49:39.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:38 smithi082 ceph-mon[97889]: pgmap v1124: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 7 op/s 2023-06-23T18:49:39.548 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:49:40.884 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:40 smithi002 ceph-mon[95947]: pgmap v1125: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-06-23T18:49:40.884 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:40 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:49:40.884 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:40 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:49:40.885 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:40 smithi002 ceph-mon[95947]: scrub summary: idle 2023-06-23T18:49:40.930 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:49:40.930 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:40.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2023-06-23T18:49:40.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2023-06-23T18:49:40.931 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2023-06-23T18:49:40.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2023-06-23T18:49:40.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:40.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2023-06-23T18:49:40.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001616794, 2023-06-23T18:49:40.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2023-06-23T18:49:40.932 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000394445, 2023-06-23T18:49:40.933 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:40.933 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:40.933 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:40.933 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:40.933 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:40.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.933 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2023-06-23T18:49:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2023-06-23T18:49:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.934 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:40.935 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 469, 2023-06-23T18:49:40.935 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 464, 2023-06-23T18:49:40.935 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 545168, 2023-06-23T18:49:40.935 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 265792, 2023-06-23T18:49:40.935 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2023-06-23T18:49:40.935 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2023-06-23T18:49:40.935 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.059138628, 2023-06-23T18:49:40.935 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.023717867, 2023-06-23T18:49:40.936 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.020614162, 2023-06-23T18:49:40.936 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.011356617, 2023-06-23T18:49:40.936 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:40.936 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 464, 2023-06-23T18:49:40.936 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.020381241, 2023-06-23T18:49:40.936 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000043925 2023-06-23T18:49:40.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.937 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:40.937 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:40.937 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:40.937 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:40.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.937 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 545168, 2023-06-23T18:49:40.937 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 265792 2023-06-23T18:49:40.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.938 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:40.938 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2023-06-23T18:49:40.938 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 353, 2023-06-23T18:49:40.938 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2023-06-23T18:49:40.938 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 213520, 2023-06-23T18:49:40.938 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:40.938 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2023-06-23T18:49:40.938 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.021789294, 2023-06-23T18:49:40.939 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.017341520, 2023-06-23T18:49:40.939 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001618826, 2023-06-23T18:49:40.939 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:40.939 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:40.940 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 351, 2023-06-23T18:49:40.940 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.025313441, 2023-06-23T18:49:40.940 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000072118 2023-06-23T18:49:40.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.941 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:40.941 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:40.941 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:40.942 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:40.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.942 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2023-06-23T18:49:40.943 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 213520 2023-06-23T18:49:40.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.943 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2023-06-23T18:49:40.943 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2023-06-23T18:49:40.944 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2023-06-23T18:49:40.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.944 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2023-06-23T18:49:40.944 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:40.945 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:40.945 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:40.945 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:40.945 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:40.945 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:40.945 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:40.945 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:40.945 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:40.946 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:40.946 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:40.946 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:40.946 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:40.946 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:40.946 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:40.946 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:40.946 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:40.947 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:40.948 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:40.948 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:40.948 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:40.948 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:40.948 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:40.948 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:40.948 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2023-06-23T18:49:40.949 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:40.949 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:40.949 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:40.949 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:40.949 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148744, 2023-06-23T18:49:40.949 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 30, 2023-06-23T18:49:40.949 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:40.950 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:40.950 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2023-06-23T18:49:40.950 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2023-06-23T18:49:40.950 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:40.950 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:40.950 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:40.950 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:40.950 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2023-06-23T18:49:40.951 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2023-06-23T18:49:40.951 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:40.951 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:40.951 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:40.951 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2023-06-23T18:49:40.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2023-06-23T18:49:40.951 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2023-06-23T18:49:40.952 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:40.952 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2023-06-23T18:49:40.952 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:40.952 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2023-06-23T18:49:40.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:40.952 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:40.952 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-06-23T18:49:40.953 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-06-23T18:49:40.953 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-06-23T18:49:40.953 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 467, 2023-06-23T18:49:40.953 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 476407, 2023-06-23T18:49:40.953 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:40.953 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 467, 2023-06-23T18:49:40.953 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-06-23T18:49:40.954 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-06-23T18:49:40.954 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 467, 2023-06-23T18:49:40.954 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 476407, 2023-06-23T18:49:40.954 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-06-23T18:49:40.954 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:40.954 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:40.955 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:40.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:40.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:40.955 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:49:41.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:40 smithi017 ceph-mon[97472]: pgmap v1125: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-06-23T18:49:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:40 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:49:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:40 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:49:41.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:40 smithi017 ceph-mon[97472]: scrub summary: idle 2023-06-23T18:49:41.318 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-06-23T18:49:41.318 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-06-23T18:49:41.318 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-06-23T18:49:41.319 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-06-23T18:49:41.319 DEBUG:tasks.check_counter:Getting stats from j 2023-06-23T18:49:41.320 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.j perf dump 2023-06-23T18:49:41.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:40 smithi082 ceph-mon[97889]: pgmap v1125: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 6.7 KiB/s wr, 8 op/s 2023-06-23T18:49:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:40 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:49:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:40 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:49:41.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:40 smithi082 ceph-mon[97889]: scrub summary: idle 2023-06-23T18:49:41.885 INFO:teuthology.orchestra.run.smithi002.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.a/config 2023-06-23T18:49:42.156 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:41 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:49:42.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:41 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:49:42.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:41 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-06-23T18:49:43.152 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:42 smithi017 ceph-mon[97472]: pgmap v1126: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2023-06-23T18:49:43.277 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:42 smithi002 ceph-mon[95947]: pgmap v1126: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2023-06-23T18:49:43.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:42 smithi082 ceph-mon[97889]: pgmap v1126: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 2.9 KiB/s wr, 6 op/s 2023-06-23T18:49:43.524 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-06-23T18:49:43.524 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 0, 2023-06-23T18:49:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 2, 2023-06-23T18:49:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 0, 2023-06-23T18:49:43.525 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 0, 2023-06-23T18:49:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 0, 2023-06-23T18:49:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.001431556, 2023-06-23T18:49:43.526 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.000000000, 2023-06-23T18:49:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.000318428, 2023-06-23T18:49:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:43.527 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:43.528 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:43.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 0, 2023-06-23T18:49:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 0 2023-06-23T18:49:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 2, 2023-06-23T18:49:43.530 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 357, 2023-06-23T18:49:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 320, 2023-06-23T18:49:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 214224, 2023-06-23T18:49:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 2, 2023-06-23T18:49:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.022735220, 2023-06-23T18:49:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.017972854, 2023-06-23T18:49:43.531 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.001926440, 2023-06-23T18:49:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 355, 2023-06-23T18:49:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.018541905, 2023-06-23T18:49:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000052230 2023-06-23T18:49:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.532 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 320, 2023-06-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 214224 2023-06-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.533 INFO:teuthology.orchestra.run.smithi002.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_messages": 474, 2023-06-23T18:49:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages": 469, 2023-06-23T18:49:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_bytes": 546528, 2023-06-23T18:49:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_bytes": 268752, 2023-06-23T18:49:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_created_connections": 2, 2023-06-23T18:49:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_active_connections": 1, 2023-06-23T18:49:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_total_time": 0.062001870, 2023-06-23T18:49:43.534 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_send_time": 0.025682157, 2023-06-23T18:49:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_recv_time": 0.021967608, 2023-06-23T18:49:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_running_fast_dispatch_time": 0.011448272, 2023-06-23T18:49:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 469, 2023-06-23T18:49:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.026988643, 2023-06-23T18:49:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000057545 2023-06-23T18:49:43.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_recv_encrypted_bytes": 546528, 2023-06-23T18:49:43.536 INFO:teuthology.orchestra.run.smithi002.stdout: "msgr_send_encrypted_bytes": 268752 2023-06-23T18:49:43.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.537 INFO:teuthology.orchestra.run.smithi002.stdout: "cct": { 2023-06-23T18:49:43.537 INFO:teuthology.orchestra.run.smithi002.stdout: "total_workers": 0, 2023-06-23T18:49:43.537 INFO:teuthology.orchestra.run.smithi002.stdout: "unhealthy_workers": 0 2023-06-23T18:49:43.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.537 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool": { 2023-06-23T18:49:43.537 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:43.538 INFO:teuthology.orchestra.run.smithi002.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:43.538 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:43.538 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:43.538 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:43.539 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:43.539 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:43.539 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:43.539 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:43.539 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:43.539 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:43.539 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:43.539 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:43.540 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:43.540 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:43.540 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:43.540 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:43.540 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:43.540 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:43.540 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:43.541 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:43.541 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:43.541 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:43.541 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:43.541 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:43.541 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:43.541 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:43.542 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:43.542 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:43.542 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:43.542 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:43.542 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_items": 0, 2023-06-23T18:49:43.542 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:43.542 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:43.543 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:43.543 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:43.543 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_bytes": 148704, 2023-06-23T18:49:43.543 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_anon_items": 29, 2023-06-23T18:49:43.543 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:43.543 INFO:teuthology.orchestra.run.smithi002.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:43.543 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bytes": 0, 2023-06-23T18:49:43.543 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_items": 0, 2023-06-23T18:49:43.544 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:43.544 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:43.544 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:43.544 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:43.545 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_bytes": 0, 2023-06-23T18:49:43.545 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_items": 0, 2023-06-23T18:49:43.545 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:43.545 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:43.545 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:43.545 INFO:teuthology.orchestra.run.smithi002.stdout: "pgmap_items": 0, 2023-06-23T18:49:43.545 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_bytes": 928, 2023-06-23T18:49:43.545 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_co_items": 2, 2023-06-23T18:49:43.546 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:43.546 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_1_items": 0, 2023-06-23T18:49:43.546 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:43.546 INFO:teuthology.orchestra.run.smithi002.stdout: "unittest_2_items": 0 2023-06-23T18:49:43.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-06-23T18:49:43.546 INFO:teuthology.orchestra.run.smithi002.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:43.546 INFO:teuthology.orchestra.run.smithi002.stdout: "val": 0, 2023-06-23T18:49:43.547 INFO:teuthology.orchestra.run.smithi002.stdout: "max": 104857600, 2023-06-23T18:49:43.547 INFO:teuthology.orchestra.run.smithi002.stdout: "get_started": 0, 2023-06-23T18:49:43.547 INFO:teuthology.orchestra.run.smithi002.stdout: "get": 472, 2023-06-23T18:49:43.547 INFO:teuthology.orchestra.run.smithi002.stdout: "get_sum": 477057, 2023-06-23T18:49:43.547 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:43.547 INFO:teuthology.orchestra.run.smithi002.stdout: "get_or_fail_success": 472, 2023-06-23T18:49:43.547 INFO:teuthology.orchestra.run.smithi002.stdout: "take": 0, 2023-06-23T18:49:43.547 INFO:teuthology.orchestra.run.smithi002.stdout: "take_sum": 0, 2023-06-23T18:49:43.548 INFO:teuthology.orchestra.run.smithi002.stdout: "put": 472, 2023-06-23T18:49:43.548 INFO:teuthology.orchestra.run.smithi002.stdout: "put_sum": 477057, 2023-06-23T18:49:43.548 INFO:teuthology.orchestra.run.smithi002.stdout: "wait": { 2023-06-23T18:49:43.548 INFO:teuthology.orchestra.run.smithi002.stdout: "avgcount": 0, 2023-06-23T18:49:43.548 INFO:teuthology.orchestra.run.smithi002.stdout: "sum": 0.000000000, 2023-06-23T18:49:43.548 INFO:teuthology.orchestra.run.smithi002.stdout: "avgtime": 0.000000000 2023-06-23T18:49:43.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:43.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-06-23T18:49:43.549 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-06-23T18:49:44.068 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-06-23T18:49:44.068 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-06-23T18:49:44.069 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-06-23T18:49:44.069 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-06-23T18:49:44.069 DEBUG:tasks.check_counter:Getting stats from b 2023-06-23T18:49:44.069 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.b perf dump 2023-06-23T18:49:44.922 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:49:45.154 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:44 smithi017 ceph-mon[97472]: pgmap v1127: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-06-23T18:49:45.304 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:44 smithi002 ceph-mon[95947]: pgmap v1127: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-06-23T18:49:45.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:44 smithi082 ceph-mon[97889]: pgmap v1127: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2023-06-23T18:49:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:46 smithi017 ceph-mon[97472]: pgmap v1128: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-06-23T18:49:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:46 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:46 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:46 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:46 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:46 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:49:47.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:46 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:49:47.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:46 smithi017 ceph-mon[97472]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.203 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-06-23T18:49:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 31369, 2023-06-23T18:49:47.203 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 27874, 2023-06-23T18:49:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 7689331, 2023-06-23T18:49:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 20813405, 2023-06-23T18:49:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 179, 2023-06-23T18:49:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 7, 2023-06-23T18:49:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 1.644111117, 2023-06-23T18:49:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.570047093, 2023-06-23T18:49:47.204 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 1.026463823, 2023-06-23T18:49:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.172813565, 2023-06-23T18:49:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 27874, 2023-06-23T18:49:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 1.414221519, 2023-06-23T18:49:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000050736 2023-06-23T18:49:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:47.205 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 51698, 2023-06-23T18:49:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.007010862, 2023-06-23T18:49:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000135 2023-06-23T18:49:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 0, 2023-06-23T18:49:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 0 2023-06-23T18:49:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:47.206 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 755063, 2023-06-23T18:49:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 1135850, 2023-06-23T18:49:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 240371037, 2023-06-23T18:49:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 464368211, 2023-06-23T18:49:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 11, 2023-06-23T18:49:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 5, 2023-06-23T18:49:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 43.567221131, 2023-06-23T18:49:47.207 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 17.967407321, 2023-06-23T18:49:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 27.934047416, 2023-06-23T18:49:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.244759054, 2023-06-23T18:49:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 1135849, 2023-06-23T18:49:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 40.186390351, 2023-06-23T18:49:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000035380 2023-06-23T18:49:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.208 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 1826393, 2023-06-23T18:49:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.305365516, 2023-06-23T18:49:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000167 2023-06-23T18:49:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 55120, 2023-06-23T18:49:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 823520 2023-06-23T18:49:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.209 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 6721606, 2023-06-23T18:49:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 6595696, 2023-06-23T18:49:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 1182617349, 2023-06-23T18:49:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 1165796121, 2023-06-23T18:49:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 12, 2023-06-23T18:49:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 7, 2023-06-23T18:49:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 200.526003555, 2023-06-23T18:49:47.210 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 67.857418385, 2023-06-23T18:49:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 153.639037006, 2023-06-23T18:49:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.123402312, 2023-06-23T18:49:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 6595693, 2023-06-23T18:49:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 95.970583734, 2023-06-23T18:49:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000014550 2023-06-23T18:49:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.211 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 13239031, 2023-06-23T18:49:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 3.637919069, 2023-06-23T18:49:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000274 2023-06-23T18:49:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 657760, 2023-06-23T18:49:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 286384 2023-06-23T18:49:47.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2023-06-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 1, 2023-06-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2023-06-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-MDSRank": { 2023-06-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2023-06-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2023-06-23T18:49:47.213 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 10789, 2023-06-23T18:49:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 9.112998242, 2023-06-23T18:49:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000844656 2023-06-23T18:49:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-PurgeQueue": { 2023-06-23T18:49:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2023-06-23T18:49:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2023-06-23T18:49:47.214 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 1212, 2023-06-23T18:49:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.084226730, 2023-06-23T18:49:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000069494 2023-06-23T18:49:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": { 2023-06-23T18:49:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "request": 249943, 2023-06-23T18:49:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "reply": 244156, 2023-06-23T18:49:47.215 INFO:teuthology.orchestra.run.smithi017.stdout: "reply_latency": { 2023-06-23T18:49:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 244156, 2023-06-23T18:49:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 360.988983991, 2023-06-23T18:49:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.001478517 2023-06-23T18:49:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_reply": 0, 2023-06-23T18:49:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "forward": 5787, 2023-06-23T18:49:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_complete": 0, 2023-06-23T18:49:47.216 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_keys": 0, 2023-06-23T18:49:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_commit": 39, 2023-06-23T18:49:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_split": 3, 2023-06-23T18:49:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_merge": 0, 2023-06-23T18:49:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes": 228, 2023-06-23T18:49:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_top": 4, 2023-06-23T18:49:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_bottom": 197, 2023-06-23T18:49:47.217 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pin_tail": 27, 2023-06-23T18:49:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pinned": 222, 2023-06-23T18:49:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_expired": 1061, 2023-06-23T18:49:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_with_caps": 178, 2023-06-23T18:49:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": 178, 2023-06-23T18:49:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "subtrees": 10, 2023-06-23T18:49:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse": 489131, 2023-06-23T18:49:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_hit": 481792, 2023-06-23T18:49:47.218 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_forward": 5728, 2023-06-23T18:49:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_discover": 11, 2023-06-23T18:49:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_dir_fetch": 0, 2023-06-23T18:49:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_remote_ino": 0, 2023-06-23T18:49:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_lock": 71, 2023-06-23T18:49:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "load_cent": 1174, 2023-06-23T18:49:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "q": 0, 2023-06-23T18:49:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "exported": 23, 2023-06-23T18:49:47.219 INFO:teuthology.orchestra.run.smithi017.stdout: "exported_inodes": 1040, 2023-06-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "imported": 12, 2023-06-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "imported_inodes": 197, 2023-06-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_dir_fetch": 0, 2023-06-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_backtrace_fetch": 2328, 2023-06-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_peer_discover": 0, 2023-06-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rfiles": 1045, 2023-06-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rbytes": 8998067092, 2023-06-23T18:49:47.220 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rsnaps": 0, 2023-06-23T18:49:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_fetch": 2328, 2023-06-23T18:49:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_set_tag": 0, 2023-06-23T18:49:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_repaired": 0, 2023-06-23T18:49:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_inotable_repaired": 0, 2023-06-23T18:49:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_inodes": 292, 2023-06-23T18:49:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_base_inodes": 11, 2023-06-23T18:49:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dirfrag_rstats": 281, 2023-06-23T18:49:47.221 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_file_inodes": 2036, 2023-06-23T18:49:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_inode_file_caps": 66, 2023-06-23T18:49:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_revoke": 472102, 2023-06-23T18:49:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_grant": 404887, 2023-06-23T18:49:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_trunc": 0, 2023-06-23T18:49:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-06-23T18:49:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flush_ack": 0, 2023-06-23T18:49:47.222 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps": 499843, 2023-06-23T18:49:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps_dirty": 2290, 2023-06-23T18:49:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_cap_release": 103, 2023-06-23T18:49:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "process_request_cap_release": 2629 2023-06-23T18:49:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache": { 2023-06-23T18:49:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update": 88, 2023-06-23T18:49:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update_receipt": 17, 2023-06-23T18:49:47.223 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_try_discover": 9, 2023-06-23T18:49:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_send_discover": 19, 2023-06-23T18:49:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_handle_discover": 457, 2023-06-23T18:49:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays": 0, 2023-06-23T18:49:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_delayed": 0, 2023-06-23T18:49:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_enqueuing": 0, 2023-06-23T18:49:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_created": 328, 2023-06-23T18:49:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_enqueued": 322, 2023-06-23T18:49:47.224 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_reintegrated": 24, 2023-06-23T18:49:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_migrated": 0, 2023-06-23T18:49:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_processing": 0, 2023-06-23T18:49:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_enqueued": 0, 2023-06-23T18:49:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_prioritized": 0, 2023-06-23T18:49:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_started": 0, 2023-06-23T18:49:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_completed": 0, 2023-06-23T18:49:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_enqueue_scrub": 11, 2023-06-23T18:49:47.225 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_exportdir": 23, 2023-06-23T18:49:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_flush": 0, 2023-06-23T18:49:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragmentdir": 3, 2023-06-23T18:49:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragstats": 0, 2023-06-23T18:49:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_inodestats": 0 2023-06-23T18:49:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log": { 2023-06-23T18:49:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "evadd": 10592, 2023-06-23T18:49:47.226 INFO:teuthology.orchestra.run.smithi017.stdout: "evex": 0, 2023-06-23T18:49:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "evtrm": 0, 2023-06-23T18:49:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "ev": 10592, 2023-06-23T18:49:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "evexg": 0, 2023-06-23T18:49:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "evexd": 0, 2023-06-23T18:49:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "segadd": 12, 2023-06-23T18:49:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "segex": 0, 2023-06-23T18:49:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "segtrm": 0, 2023-06-23T18:49:47.227 INFO:teuthology.orchestra.run.smithi017.stdout: "seg": 12, 2023-06-23T18:49:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "segexg": 0, 2023-06-23T18:49:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "segexd": 0, 2023-06-23T18:49:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "expos": 4194304, 2023-06-23T18:49:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "wrpos": 27364537, 2023-06-23T18:49:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "rdpos": 0, 2023-06-23T18:49:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "jlat": { 2023-06-23T18:49:47.228 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 5181, 2023-06-23T18:49:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 68.029414633, 2023-06-23T18:49:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.013130556 2023-06-23T18:49:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "replayed": 0 2023-06-23T18:49:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_mem": { 2023-06-23T18:49:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 206, 2023-06-23T18:49:47.229 INFO:teuthology.orchestra.run.smithi017.stdout: "ino+": 1560, 2023-06-23T18:49:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "ino-": 1354, 2023-06-23T18:49:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "dir": 71, 2023-06-23T18:49:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "dir+": 96, 2023-06-23T18:49:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "dir-": 25, 2023-06-23T18:49:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "dn": 228, 2023-06-23T18:49:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "dn+": 1612, 2023-06-23T18:49:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "dn-": 1384, 2023-06-23T18:49:47.230 INFO:teuthology.orchestra.run.smithi017.stdout: "cap": 178, 2023-06-23T18:49:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "cap+": 1537, 2023-06-23T18:49:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "cap-": 1359, 2023-06-23T18:49:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "rss": 68732, 2023-06-23T18:49:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "heap": 207156 2023-06-23T18:49:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_server": { 2023-06-23T18:49:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_client_request": 422038, 2023-06-23T18:49:47.231 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_server_request": 4653, 2023-06-23T18:49:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_request": 249943, 2023-06-23T18:49:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_session": 98, 2023-06-23T18:49:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_peer_request": 9521, 2023-06-23T18:49:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "req_create_latency": { 2023-06-23T18:49:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 1298, 2023-06-23T18:49:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 9.063747216, 2023-06-23T18:49:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.006982856 2023-06-23T18:49:47.232 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getattr_latency": { 2023-06-23T18:49:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 232744, 2023-06-23T18:49:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 289.529589987, 2023-06-23T18:49:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.001243983 2023-06-23T18:49:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getfilelock_latency": { 2023-06-23T18:49:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.233 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "req_link_latency": { 2023-06-23T18:49:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 6, 2023-06-23T18:49:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.019157434, 2023-06-23T18:49:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.003192905 2023-06-23T18:49:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.234 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookup_latency": { 2023-06-23T18:49:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 6151, 2023-06-23T18:49:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 55.063151794, 2023-06-23T18:49:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.008951902 2023-06-23T18:49:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookuphash_latency": { 2023-06-23T18:49:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.235 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupino_latency": { 2023-06-23T18:49:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.236 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupname_latency": { 2023-06-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupparent_latency": { 2023-06-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.237 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupsnap_latency": { 2023-06-23T18:49:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lssnap_latency": { 2023-06-23T18:49:47.238 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mkdir_latency": { 2023-06-23T18:49:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 34, 2023-06-23T18:49:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.062248117, 2023-06-23T18:49:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.001830826 2023-06-23T18:49:47.239 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mknod_latency": { 2023-06-23T18:49:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mksnap_latency": { 2023-06-23T18:49:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.240 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "req_open_latency": { 2023-06-23T18:49:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3479, 2023-06-23T18:49:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 4.921936779, 2023-06-23T18:49:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.001414756 2023-06-23T18:49:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.241 INFO:teuthology.orchestra.run.smithi017.stdout: "req_readdir_latency": { 2023-06-23T18:49:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 71, 2023-06-23T18:49:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.312203683, 2023-06-23T18:49:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.004397234 2023-06-23T18:49:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rename_latency": { 2023-06-23T18:49:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 335, 2023-06-23T18:49:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 1.910450794, 2023-06-23T18:49:47.242 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.005702838 2023-06-23T18:49:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "req_renamesnap_latency": { 2023-06-23T18:49:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.243 INFO:teuthology.orchestra.run.smithi017.stdout: "req_snapdiff_latency": { 2023-06-23T18:49:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmdir_latency": { 2023-06-23T18:49:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.244 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmsnap_latency": { 2023-06-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmxattr_latency": { 2023-06-23T18:49:47.245 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setattr_latency": { 2023-06-23T18:49:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 1, 2023-06-23T18:49:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.001435361, 2023-06-23T18:49:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.001435361 2023-06-23T18:49:47.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setdirlayout_latency": { 2023-06-23T18:49:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setfilelock_latency": { 2023-06-23T18:49:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.247 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setlayout_latency": { 2023-06-23T18:49:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.248 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setxattr_latency": { 2023-06-23T18:49:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 6, 2023-06-23T18:49:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.019206696, 2023-06-23T18:49:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.003201116 2023-06-23T18:49:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "req_symlink_latency": { 2023-06-23T18:49:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.249 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "req_unlink_latency": { 2023-06-23T18:49:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 31, 2023-06-23T18:49:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.085856130, 2023-06-23T18:49:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.002769552 2023-06-23T18:49:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.250 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_revoke_eviction": 0, 2023-06-23T18:49:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_acquisition_throttle": 0, 2023-06-23T18:49:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getvxattr_latency": { 2023-06-23T18:49:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sessions": { 2023-06-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "session_count": 1, 2023-06-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "session_add": 2, 2023-06-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "session_remove": 1, 2023-06-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_open": 1, 2023-06-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_stale": 0, 2023-06-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "total_load": 16140, 2023-06-23T18:49:47.252 INFO:teuthology.orchestra.run.smithi017.stdout: "average_load": 16140, 2023-06-23T18:49:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "avg_session_uptime": 1658 2023-06-23T18:49:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2023-06-23T18:49:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:47.253 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:47.254 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:47.255 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:47.256 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2023-06-23T18:49:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:47.257 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 299999, 2023-06-23T18:49:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 122, 2023-06-23T18:49:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2023-06-23T18:49:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2023-06-23T18:49:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:47.258 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 7624, 2023-06-23T18:49:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 160, 2023-06-23T18:49:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:47.259 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2023-06-23T18:49:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 771036, 2023-06-23T18:49:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 5436, 2023-06-23T18:49:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2023-06-23T18:49:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2023-06-23T18:49:47.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter": { 2023-06-23T18:49:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "op_active": 0, 2023-06-23T18:49:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "op_laggy": 0, 2023-06-23T18:49:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send": 9155, 2023-06-23T18:49:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send_bytes": 23632387, 2023-06-23T18:49:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "op_resend": 30, 2023-06-23T18:49:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "op_reply": 9155, 2023-06-23T18:49:47.261 INFO:teuthology.orchestra.run.smithi017.stdout: "op_latency": { 2023-06-23T18:49:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 9155, 2023-06-23T18:49:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 33.494674559, 2023-06-23T18:49:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.003658620 2023-06-23T18:49:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "op_inflight": 0, 2023-06-23T18:49:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "oplen_avg": { 2023-06-23T18:49:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 9155, 2023-06-23T18:49:47.262 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 9407 2023-06-23T18:49:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "op": 9155, 2023-06-23T18:49:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "op_r": 2518, 2023-06-23T18:49:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "op_w": 6637, 2023-06-23T18:49:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "op_rmw": 0, 2023-06-23T18:49:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "op_pg": 0, 2023-06-23T18:49:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_stat": 0, 2023-06-23T18:49:47.263 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_create": 11, 2023-06-23T18:49:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_read": 179, 2023-06-23T18:49:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_write": 5349, 2023-06-23T18:49:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writefull": 230, 2023-06-23T18:49:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writesame": 0, 2023-06-23T18:49:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_append": 0, 2023-06-23T18:49:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_zero": 0, 2023-06-23T18:49:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_truncate": 3, 2023-06-23T18:49:47.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_delete": 839, 2023-06-23T18:49:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_mapext": 0, 2023-06-23T18:49:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_sparse_read": 0, 2023-06-23T18:49:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_clonerange": 0, 2023-06-23T18:49:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_getxattr": 2339, 2023-06-23T18:49:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_setxattr": 22, 2023-06-23T18:49:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_cmpxattr": 0, 2023-06-23T18:49:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_rmxattr": 0, 2023-06-23T18:49:47.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_resetxattrs": 0, 2023-06-23T18:49:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_call": 0, 2023-06-23T18:49:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_watch": 0, 2023-06-23T18:49:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_notify": 0, 2023-06-23T18:49:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_src_cmpxattr": 0, 2023-06-23T18:49:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls": 0, 2023-06-23T18:49:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls_filter": 0, 2023-06-23T18:49:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_other": 0, 2023-06-23T18:49:47.266 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_active": 0, 2023-06-23T18:49:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_send": 0, 2023-06-23T18:49:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_resend": 0, 2023-06-23T18:49:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_ping": 0, 2023-06-23T18:49:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_active": 0, 2023-06-23T18:49:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_send": 0, 2023-06-23T18:49:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_resend": 0, 2023-06-23T18:49:47.267 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_active": 0, 2023-06-23T18:49:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_send": 0, 2023-06-23T18:49:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_resend": 0, 2023-06-23T18:49:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_active": 0, 2023-06-23T18:49:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_send": 0, 2023-06-23T18:49:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_resend": 0, 2023-06-23T18:49:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "command_active": 0, 2023-06-23T18:49:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "command_send": 0, 2023-06-23T18:49:47.268 INFO:teuthology.orchestra.run.smithi017.stdout: "command_resend": 0, 2023-06-23T18:49:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "map_epoch": 84, 2023-06-23T18:49:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "map_full": 0, 2023-06-23T18:49:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "map_inc": 3, 2023-06-23T18:49:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_sessions": 12, 2023-06-23T18:49:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_open": 12, 2023-06-23T18:49:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_close": 0, 2023-06-23T18:49:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_laggy": 0, 2023-06-23T18:49:47.269 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_wr": 331, 2023-06-23T18:49:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_rd": 0, 2023-06-23T18:49:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_del": 104 2023-06-23T18:49:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "oft": { 2023-06-23T18:49:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_objs": 1, 2023-06-23T18:49:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_kv_pairs": 0, 2023-06-23T18:49:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_updates": 0, 2023-06-23T18:49:47.270 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_removes": 0 2023-06-23T18:49:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "osd-slow-ops": { 2023-06-23T18:49:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_ops_count": 254677 2023-06-23T18:49:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "purge_queue": { 2023-06-23T18:49:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops": 0, 2023-06-23T18:49:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops_high_water": 10, 2023-06-23T18:49:47.271 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing": 0, 2023-06-23T18:49:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_high_water": 10, 2023-06-23T18:49:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executed": 322, 2023-06-23T18:49:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_item_in_journal": 0 2023-06-23T18:49:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:47.272 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2023-06-23T18:49:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 7508034, 2023-06-23T18:49:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 754917940, 2023-06-23T18:49:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 7508034, 2023-06-23T18:49:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:47.273 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 7508034, 2023-06-23T18:49:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 754917940, 2023-06-23T18:49:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.274 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_bytes": { 2023-06-23T18:49:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2023-06-23T18:49:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:47.275 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 9155, 2023-06-23T18:49:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 23664909, 2023-06-23T18:49:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 9155, 2023-06-23T18:49:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 23664909, 2023-06-23T18:49:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_ops": { 2023-06-23T18:49:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 1024, 2023-06-23T18:49:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:47.277 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 9155, 2023-06-23T18:49:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 9155, 2023-06-23T18:49:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 9155, 2023-06-23T18:49:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 9155, 2023-06-23T18:49:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:47.278 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle": { 2023-06-23T18:49:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2023-06-23T18:49:47.279 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 322, 2023-06-23T18:49:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 32522, 2023-06-23T18:49:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 322, 2023-06-23T18:49:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:47.280 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 168, 2023-06-23T18:49:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 32522, 2023-06-23T18:49:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:47.281 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle-0x55b3237bdcc0": { 2023-06-23T18:49:47.282 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:47.282 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2023-06-23T18:49:47.282 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:47.282 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 10592, 2023-06-23T18:49:47.282 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 23170233, 2023-06-23T18:49:47.282 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:47.282 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 10592, 2023-06-23T18:49:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 5181, 2023-06-23T18:49:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 23170233, 2023-06-23T18:49:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:47.283 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:47.284 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:47.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:47.284 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-06-23T18:49:47.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:46 smithi002 ceph-mon[95947]: pgmap v1128: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-06-23T18:49:47.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:46 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:46 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:46 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:46 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:46 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:49:47.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:46 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:49:47.302 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:46 smithi002 ceph-mon[95947]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:46 smithi082 ceph-mon[97889]: pgmap v1128: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2023-06-23T18:49:47.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:46 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:46 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:46 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:46 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:46 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-06-23T18:49:47.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:46 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-06-23T18:49:47.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:46 smithi082 ceph-mon[97889]: from='mgr.14150 172.21.15.2:0/378385680' entity='mgr.x' 2023-06-23T18:49:47.586 INFO:tasks.check_counter:Daemon mds.b mds.exported=23 2023-06-23T18:49:47.586 INFO:tasks.check_counter:Daemon mds.b mds.imported=12 2023-06-23T18:49:47.586 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=88 2023-06-23T18:49:47.586 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=17 2023-06-23T18:49:47.587 DEBUG:tasks.check_counter:Getting stats from e 2023-06-23T18:49:47.587 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.e perf dump 2023-06-23T18:49:48.150 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:49:49.057 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:48 smithi017 ceph-mon[97472]: pgmap v1129: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-06-23T18:49:49.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:48 smithi002 ceph-mon[95947]: pgmap v1129: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-06-23T18:49:49.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:48 smithi082 ceph-mon[97889]: pgmap v1129: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-06-23T18:49:49.619 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-06-23T18:49:49.619 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:49.620 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 1500, 2023-06-23T18:49:49.620 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 1502, 2023-06-23T18:49:49.620 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 12968871, 2023-06-23T18:49:49.620 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 466564, 2023-06-23T18:49:49.620 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 8, 2023-06-23T18:49:49.620 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2023-06-23T18:49:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.125548398, 2023-06-23T18:49:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.038901500, 2023-06-23T18:49:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.036561341, 2023-06-23T18:49:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.038714304, 2023-06-23T18:49:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 1500, 2023-06-23T18:49:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.055572451, 2023-06-23T18:49:49.621 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000037048 2023-06-23T18:49:49.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.622 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:49.622 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.622 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.622 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.622 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.622 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 0, 2023-06-23T18:49:49.622 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 0 2023-06-23T18:49:49.623 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.623 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:49.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 1892, 2023-06-23T18:49:49.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 2247, 2023-06-23T18:49:49.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 19071227, 2023-06-23T18:49:49.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 1275178, 2023-06-23T18:49:49.623 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 6, 2023-06-23T18:49:49.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 5, 2023-06-23T18:49:49.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.168614699, 2023-06-23T18:49:49.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.045767886, 2023-06-23T18:49:49.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.049257316, 2023-06-23T18:49:49.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.058662121, 2023-06-23T18:49:49.624 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:49.624 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 2245, 2023-06-23T18:49:49.625 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.096316885, 2023-06-23T18:49:49.625 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000042902 2023-06-23T18:49:49.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.625 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:49.625 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.625 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.625 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.625 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 160, 2023-06-23T18:49:49.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 687072 2023-06-23T18:49:49.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.626 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:49.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 3301, 2023-06-23T18:49:49.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 3275, 2023-06-23T18:49:49.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 18270550, 2023-06-23T18:49:49.626 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 1140425, 2023-06-23T18:49:49.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 5, 2023-06-23T18:49:49.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 4, 2023-06-23T18:49:49.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.294145162, 2023-06-23T18:49:49.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.082500792, 2023-06-23T18:49:49.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.090700148, 2023-06-23T18:49:49.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.098100787, 2023-06-23T18:49:49.627 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:49.627 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3275, 2023-06-23T18:49:49.628 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.144711163, 2023-06-23T18:49:49.628 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000044186 2023-06-23T18:49:49.628 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.628 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:49.628 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.628 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.629 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.629 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 642304, 2023-06-23T18:49:49.629 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 269936 2023-06-23T18:49:49.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.629 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2023-06-23T18:49:49.629 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 1, 2023-06-23T18:49:49.630 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2023-06-23T18:49:49.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.630 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-MDSRank": { 2023-06-23T18:49:49.630 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2023-06-23T18:49:49.630 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2023-06-23T18:49:49.630 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 10553, 2023-06-23T18:49:49.630 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.257575262, 2023-06-23T18:49:49.630 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000024407 2023-06-23T18:49:49.631 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.631 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-PurgeQueue": { 2023-06-23T18:49:49.631 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2023-06-23T18:49:49.631 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2023-06-23T18:49:49.631 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.631 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.631 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.632 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": { 2023-06-23T18:49:49.632 INFO:teuthology.orchestra.run.smithi017.stdout: "request": 0, 2023-06-23T18:49:49.632 INFO:teuthology.orchestra.run.smithi017.stdout: "reply": 0, 2023-06-23T18:49:49.632 INFO:teuthology.orchestra.run.smithi017.stdout: "reply_latency": { 2023-06-23T18:49:49.632 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.632 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.633 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.633 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_reply": 0, 2023-06-23T18:49:49.633 INFO:teuthology.orchestra.run.smithi017.stdout: "forward": 0, 2023-06-23T18:49:49.633 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_complete": 0, 2023-06-23T18:49:49.633 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_keys": 0, 2023-06-23T18:49:49.633 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_commit": 0, 2023-06-23T18:49:49.634 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_split": 0, 2023-06-23T18:49:49.634 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_merge": 0, 2023-06-23T18:49:49.634 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes": 731, 2023-06-23T18:49:49.634 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_top": 0, 2023-06-23T18:49:49.634 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_bottom": 0, 2023-06-23T18:49:49.635 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pin_tail": 731, 2023-06-23T18:49:49.635 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pinned": 731, 2023-06-23T18:49:49.635 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_expired": 3, 2023-06-23T18:49:49.635 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_with_caps": 0, 2023-06-23T18:49:49.635 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": 0, 2023-06-23T18:49:49.635 INFO:teuthology.orchestra.run.smithi017.stdout: "subtrees": 8, 2023-06-23T18:49:49.635 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse": 0, 2023-06-23T18:49:49.636 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_hit": 0, 2023-06-23T18:49:49.636 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_forward": 0, 2023-06-23T18:49:49.636 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_discover": 0, 2023-06-23T18:49:49.636 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_dir_fetch": 0, 2023-06-23T18:49:49.636 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_remote_ino": 0, 2023-06-23T18:49:49.636 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_lock": 0, 2023-06-23T18:49:49.636 INFO:teuthology.orchestra.run.smithi017.stdout: "load_cent": 0, 2023-06-23T18:49:49.637 INFO:teuthology.orchestra.run.smithi017.stdout: "q": 0, 2023-06-23T18:49:49.637 INFO:teuthology.orchestra.run.smithi017.stdout: "exported": 0, 2023-06-23T18:49:49.637 INFO:teuthology.orchestra.run.smithi017.stdout: "exported_inodes": 0, 2023-06-23T18:49:49.637 INFO:teuthology.orchestra.run.smithi017.stdout: "imported": 0, 2023-06-23T18:49:49.637 INFO:teuthology.orchestra.run.smithi017.stdout: "imported_inodes": 0, 2023-06-23T18:49:49.637 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_dir_fetch": 0, 2023-06-23T18:49:49.637 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_backtrace_fetch": 0, 2023-06-23T18:49:49.637 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_peer_discover": 0, 2023-06-23T18:49:49.638 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rfiles": 0, 2023-06-23T18:49:49.638 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rbytes": 0, 2023-06-23T18:49:49.638 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rsnaps": 0, 2023-06-23T18:49:49.638 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_fetch": 0, 2023-06-23T18:49:49.638 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_set_tag": 0, 2023-06-23T18:49:49.638 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_repaired": 0, 2023-06-23T18:49:49.638 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_inotable_repaired": 0, 2023-06-23T18:49:49.639 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_inodes": 0, 2023-06-23T18:49:49.639 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_base_inodes": 0, 2023-06-23T18:49:49.639 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dirfrag_rstats": 0, 2023-06-23T18:49:49.639 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_file_inodes": 0, 2023-06-23T18:49:49.639 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_inode_file_caps": 0, 2023-06-23T18:49:49.639 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_revoke": 0, 2023-06-23T18:49:49.639 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_grant": 0, 2023-06-23T18:49:49.639 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_trunc": 0, 2023-06-23T18:49:49.640 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-06-23T18:49:49.640 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flush_ack": 0, 2023-06-23T18:49:49.640 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps": 0, 2023-06-23T18:49:49.640 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps_dirty": 0, 2023-06-23T18:49:49.640 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_cap_release": 0, 2023-06-23T18:49:49.640 INFO:teuthology.orchestra.run.smithi017.stdout: "process_request_cap_release": 0 2023-06-23T18:49:49.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.641 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache": { 2023-06-23T18:49:49.641 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update": 0, 2023-06-23T18:49:49.641 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update_receipt": 0, 2023-06-23T18:49:49.641 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_try_discover": 0, 2023-06-23T18:49:49.641 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_send_discover": 0, 2023-06-23T18:49:49.641 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_handle_discover": 0, 2023-06-23T18:49:49.641 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays": 0, 2023-06-23T18:49:49.641 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_delayed": 0, 2023-06-23T18:49:49.642 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_enqueuing": 0, 2023-06-23T18:49:49.642 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_created": 0, 2023-06-23T18:49:49.642 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_enqueued": 0, 2023-06-23T18:49:49.642 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_reintegrated": 0, 2023-06-23T18:49:49.642 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_migrated": 0, 2023-06-23T18:49:49.642 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_processing": 0, 2023-06-23T18:49:49.642 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_enqueued": 0, 2023-06-23T18:49:49.642 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_prioritized": 0, 2023-06-23T18:49:49.643 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_started": 0, 2023-06-23T18:49:49.643 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_completed": 0, 2023-06-23T18:49:49.643 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_enqueue_scrub": 0, 2023-06-23T18:49:49.643 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_exportdir": 0, 2023-06-23T18:49:49.643 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_flush": 0, 2023-06-23T18:49:49.643 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragmentdir": 0, 2023-06-23T18:49:49.643 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragstats": 0, 2023-06-23T18:49:49.643 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_inodestats": 0 2023-06-23T18:49:49.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.644 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log": { 2023-06-23T18:49:49.644 INFO:teuthology.orchestra.run.smithi017.stdout: "evadd": 0, 2023-06-23T18:49:49.645 INFO:teuthology.orchestra.run.smithi017.stdout: "evex": 0, 2023-06-23T18:49:49.645 INFO:teuthology.orchestra.run.smithi017.stdout: "evtrm": 0, 2023-06-23T18:49:49.645 INFO:teuthology.orchestra.run.smithi017.stdout: "ev": 0, 2023-06-23T18:49:49.645 INFO:teuthology.orchestra.run.smithi017.stdout: "evexg": 0, 2023-06-23T18:49:49.646 INFO:teuthology.orchestra.run.smithi017.stdout: "evexd": 0, 2023-06-23T18:49:49.646 INFO:teuthology.orchestra.run.smithi017.stdout: "segadd": 0, 2023-06-23T18:49:49.646 INFO:teuthology.orchestra.run.smithi017.stdout: "segex": 0, 2023-06-23T18:49:49.646 INFO:teuthology.orchestra.run.smithi017.stdout: "segtrm": 0, 2023-06-23T18:49:49.647 INFO:teuthology.orchestra.run.smithi017.stdout: "seg": 33, 2023-06-23T18:49:49.647 INFO:teuthology.orchestra.run.smithi017.stdout: "segexg": 0, 2023-06-23T18:49:49.647 INFO:teuthology.orchestra.run.smithi017.stdout: "segexd": 0, 2023-06-23T18:49:49.647 INFO:teuthology.orchestra.run.smithi017.stdout: "expos": 4194304, 2023-06-23T18:49:49.648 INFO:teuthology.orchestra.run.smithi017.stdout: "wrpos": 0, 2023-06-23T18:49:49.648 INFO:teuthology.orchestra.run.smithi017.stdout: "rdpos": 52153830, 2023-06-23T18:49:49.648 INFO:teuthology.orchestra.run.smithi017.stdout: "jlat": { 2023-06-23T18:49:49.648 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.649 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.649 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.650 INFO:teuthology.orchestra.run.smithi017.stdout: "replayed": 24094 2023-06-23T18:49:49.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.650 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_mem": { 2023-06-23T18:49:49.650 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 718, 2023-06-23T18:49:49.650 INFO:teuthology.orchestra.run.smithi017.stdout: "ino+": 1115, 2023-06-23T18:49:49.650 INFO:teuthology.orchestra.run.smithi017.stdout: "ino-": 397, 2023-06-23T18:49:49.651 INFO:teuthology.orchestra.run.smithi017.stdout: "dir": 35, 2023-06-23T18:49:49.651 INFO:teuthology.orchestra.run.smithi017.stdout: "dir+": 43, 2023-06-23T18:49:49.651 INFO:teuthology.orchestra.run.smithi017.stdout: "dir-": 8, 2023-06-23T18:49:49.651 INFO:teuthology.orchestra.run.smithi017.stdout: "dn": 731, 2023-06-23T18:49:49.651 INFO:teuthology.orchestra.run.smithi017.stdout: "dn+": 1132, 2023-06-23T18:49:49.651 INFO:teuthology.orchestra.run.smithi017.stdout: "dn-": 401, 2023-06-23T18:49:49.651 INFO:teuthology.orchestra.run.smithi017.stdout: "cap": 0, 2023-06-23T18:49:49.652 INFO:teuthology.orchestra.run.smithi017.stdout: "cap+": 0, 2023-06-23T18:49:49.652 INFO:teuthology.orchestra.run.smithi017.stdout: "cap-": 0, 2023-06-23T18:49:49.652 INFO:teuthology.orchestra.run.smithi017.stdout: "rss": 53604, 2023-06-23T18:49:49.652 INFO:teuthology.orchestra.run.smithi017.stdout: "heap": 207156 2023-06-23T18:49:49.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.652 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_server": { 2023-06-23T18:49:49.652 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_client_request": 0, 2023-06-23T18:49:49.653 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_server_request": 0, 2023-06-23T18:49:49.653 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_request": 0, 2023-06-23T18:49:49.653 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_session": 0, 2023-06-23T18:49:49.653 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_peer_request": 0, 2023-06-23T18:49:49.653 INFO:teuthology.orchestra.run.smithi017.stdout: "req_create_latency": { 2023-06-23T18:49:49.653 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.653 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.653 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.654 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getattr_latency": { 2023-06-23T18:49:49.654 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.654 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.654 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.654 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getfilelock_latency": { 2023-06-23T18:49:49.655 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.655 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.655 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.655 INFO:teuthology.orchestra.run.smithi017.stdout: "req_link_latency": { 2023-06-23T18:49:49.655 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.655 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.655 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.656 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookup_latency": { 2023-06-23T18:49:49.656 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.656 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.656 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.656 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookuphash_latency": { 2023-06-23T18:49:49.657 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.657 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.657 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.657 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupino_latency": { 2023-06-23T18:49:49.658 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.658 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.658 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.659 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupname_latency": { 2023-06-23T18:49:49.659 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.659 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.660 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.660 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupparent_latency": { 2023-06-23T18:49:49.661 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.661 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.661 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.662 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupsnap_latency": { 2023-06-23T18:49:49.662 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.663 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.663 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.664 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lssnap_latency": { 2023-06-23T18:49:49.664 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.665 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.665 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.665 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mkdir_latency": { 2023-06-23T18:49:49.666 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.666 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.666 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.667 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mknod_latency": { 2023-06-23T18:49:49.667 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.667 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.668 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.668 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mksnap_latency": { 2023-06-23T18:49:49.668 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "req_open_latency": { 2023-06-23T18:49:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.669 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: "req_readdir_latency": { 2023-06-23T18:49:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.670 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rename_latency": { 2023-06-23T18:49:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "req_renamesnap_latency": { 2023-06-23T18:49:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.671 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: "req_snapdiff_latency": { 2023-06-23T18:49:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmdir_latency": { 2023-06-23T18:49:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmsnap_latency": { 2023-06-23T18:49:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.673 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmxattr_latency": { 2023-06-23T18:49:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.674 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setattr_latency": { 2023-06-23T18:49:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setdirlayout_latency": { 2023-06-23T18:49:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.675 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setfilelock_latency": { 2023-06-23T18:49:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.676 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setlayout_latency": { 2023-06-23T18:49:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setxattr_latency": { 2023-06-23T18:49:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.677 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "req_symlink_latency": { 2023-06-23T18:49:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "req_unlink_latency": { 2023-06-23T18:49:49.678 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_revoke_eviction": 0, 2023-06-23T18:49:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_acquisition_throttle": 0, 2023-06-23T18:49:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getvxattr_latency": { 2023-06-23T18:49:49.679 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sessions": { 2023-06-23T18:49:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: "session_count": 1, 2023-06-23T18:49:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: "session_add": 2, 2023-06-23T18:49:49.680 INFO:teuthology.orchestra.run.smithi017.stdout: "session_remove": 1, 2023-06-23T18:49:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_open": 1, 2023-06-23T18:49:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_stale": 0, 2023-06-23T18:49:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "total_load": 0, 2023-06-23T18:49:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "average_load": 0, 2023-06-23T18:49:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "avg_session_uptime": 2994 2023-06-23T18:49:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2023-06-23T18:49:49.681 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:49.682 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:49.683 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:49.684 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:49.685 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2023-06-23T18:49:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 224072, 2023-06-23T18:49:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 73, 2023-06-23T18:49:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:49.686 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2023-06-23T18:49:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2023-06-23T18:49:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:49.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 7624, 2023-06-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 160, 2023-06-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2023-06-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 2006880, 2023-06-23T18:49:49.688 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 6060, 2023-06-23T18:49:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2023-06-23T18:49:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2023-06-23T18:49:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter": { 2023-06-23T18:49:49.689 INFO:teuthology.orchestra.run.smithi017.stdout: "op_active": 0, 2023-06-23T18:49:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "op_laggy": 0, 2023-06-23T18:49:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send": 6190, 2023-06-23T18:49:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send_bytes": 21, 2023-06-23T18:49:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "op_resend": 3, 2023-06-23T18:49:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "op_reply": 6190, 2023-06-23T18:49:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "op_latency": { 2023-06-23T18:49:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 6190, 2023-06-23T18:49:49.690 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 3.684965273, 2023-06-23T18:49:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000595309 2023-06-23T18:49:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "op_inflight": 0, 2023-06-23T18:49:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "oplen_avg": { 2023-06-23T18:49:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 6190, 2023-06-23T18:49:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 6191 2023-06-23T18:49:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.691 INFO:teuthology.orchestra.run.smithi017.stdout: "op": 6190, 2023-06-23T18:49:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "op_r": 6190, 2023-06-23T18:49:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "op_w": 0, 2023-06-23T18:49:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "op_rmw": 0, 2023-06-23T18:49:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "op_pg": 0, 2023-06-23T18:49:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_stat": 3390, 2023-06-23T18:49:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_create": 0, 2023-06-23T18:49:49.692 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_read": 2798, 2023-06-23T18:49:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_write": 0, 2023-06-23T18:49:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writefull": 0, 2023-06-23T18:49:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writesame": 0, 2023-06-23T18:49:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_append": 0, 2023-06-23T18:49:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_zero": 0, 2023-06-23T18:49:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_truncate": 0, 2023-06-23T18:49:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_delete": 0, 2023-06-23T18:49:49.693 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_mapext": 0, 2023-06-23T18:49:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_sparse_read": 0, 2023-06-23T18:49:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_clonerange": 0, 2023-06-23T18:49:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_getxattr": 1, 2023-06-23T18:49:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_setxattr": 0, 2023-06-23T18:49:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_cmpxattr": 0, 2023-06-23T18:49:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_rmxattr": 0, 2023-06-23T18:49:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_resetxattrs": 0, 2023-06-23T18:49:49.694 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_call": 0, 2023-06-23T18:49:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_watch": 0, 2023-06-23T18:49:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_notify": 0, 2023-06-23T18:49:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_src_cmpxattr": 0, 2023-06-23T18:49:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls": 0, 2023-06-23T18:49:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls_filter": 0, 2023-06-23T18:49:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_other": 0, 2023-06-23T18:49:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_active": 0, 2023-06-23T18:49:49.695 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_send": 0, 2023-06-23T18:49:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_resend": 0, 2023-06-23T18:49:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_ping": 0, 2023-06-23T18:49:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_active": 0, 2023-06-23T18:49:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_send": 0, 2023-06-23T18:49:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_resend": 0, 2023-06-23T18:49:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_active": 0, 2023-06-23T18:49:49.696 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_send": 0, 2023-06-23T18:49:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_resend": 0, 2023-06-23T18:49:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_active": 0, 2023-06-23T18:49:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_send": 0, 2023-06-23T18:49:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_resend": 0, 2023-06-23T18:49:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "command_active": 0, 2023-06-23T18:49:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "command_send": 0, 2023-06-23T18:49:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "command_resend": 0, 2023-06-23T18:49:49.697 INFO:teuthology.orchestra.run.smithi017.stdout: "map_epoch": 84, 2023-06-23T18:49:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "map_full": 0, 2023-06-23T18:49:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "map_inc": 2, 2023-06-23T18:49:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_sessions": 9, 2023-06-23T18:49:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_open": 10, 2023-06-23T18:49:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_close": 1, 2023-06-23T18:49:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_laggy": 0, 2023-06-23T18:49:49.698 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_wr": 0, 2023-06-23T18:49:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_rd": 2, 2023-06-23T18:49:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_del": 0 2023-06-23T18:49:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "oft": { 2023-06-23T18:49:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_objs": 0, 2023-06-23T18:49:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_kv_pairs": 0, 2023-06-23T18:49:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_updates": 0, 2023-06-23T18:49:49.699 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_removes": 0 2023-06-23T18:49:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "osd-slow-ops": { 2023-06-23T18:49:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_ops_count": 0 2023-06-23T18:49:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "purge_queue": { 2023-06-23T18:49:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops": 0, 2023-06-23T18:49:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops_high_water": 0, 2023-06-23T18:49:49.700 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing": 0, 2023-06-23T18:49:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_high_water": 0, 2023-06-23T18:49:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executed": 0, 2023-06-23T18:49:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_item_in_journal": 0 2023-06-23T18:49:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2023-06-23T18:49:49.701 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 6689, 2023-06-23T18:49:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 49680045, 2023-06-23T18:49:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 6689, 2023-06-23T18:49:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 6689, 2023-06-23T18:49:49.702 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 49680045, 2023-06-23T18:49:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_bytes": { 2023-06-23T18:49:49.703 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2023-06-23T18:49:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 6190, 2023-06-23T18:49:49.704 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 47959592, 2023-06-23T18:49:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 6190, 2023-06-23T18:49:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 47959592, 2023-06-23T18:49:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.705 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_ops": { 2023-06-23T18:49:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 1024, 2023-06-23T18:49:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:49.706 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 6190, 2023-06-23T18:49:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 6190, 2023-06-23T18:49:49.707 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 6190, 2023-06-23T18:49:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 6190, 2023-06-23T18:49:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.708 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle": { 2023-06-23T18:49:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2023-06-23T18:49:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:49.709 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 0, 2023-06-23T18:49:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 0, 2023-06-23T18:49:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.710 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle-0x55ba992b4000": { 2023-06-23T18:49:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2023-06-23T18:49:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:49.711 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 0, 2023-06-23T18:49:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 0, 2023-06-23T18:49:49.712 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:49.713 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-06-23T18:49:50.102 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-06-23T18:49:50.102 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-06-23T18:49:50.102 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-06-23T18:49:50.102 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-06-23T18:49:50.103 DEBUG:tasks.check_counter:Getting stats from h 2023-06-23T18:49:50.103 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.h perf dump 2023-06-23T18:49:50.664 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:49:51.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:50 smithi017 ceph-mon[97472]: pgmap v1130: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-06-23T18:49:51.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:50 smithi002 ceph-mon[95947]: pgmap v1130: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-06-23T18:49:51.350 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:50 smithi082 ceph-mon[97889]: pgmap v1130: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2023-06-23T18:49:52.057 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-06-23T18:49:52.057 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:52.057 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 3075, 2023-06-23T18:49:52.058 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 3074, 2023-06-23T18:49:52.058 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 9315084, 2023-06-23T18:49:52.058 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 955228, 2023-06-23T18:49:52.058 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 6, 2023-06-23T18:49:52.058 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 2, 2023-06-23T18:49:52.058 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.262041130, 2023-06-23T18:49:52.059 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.084069151, 2023-06-23T18:49:52.059 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.062314495, 2023-06-23T18:49:52.059 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.092595892, 2023-06-23T18:49:52.059 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:52.059 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3073, 2023-06-23T18:49:52.059 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.115795177, 2023-06-23T18:49:52.059 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000037681 2023-06-23T18:49:52.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.060 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:52.060 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.060 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.060 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.060 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-06-23T18:49:52.060 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 448 2023-06-23T18:49:52.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.061 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:52.061 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 2341, 2023-06-23T18:49:52.061 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 2317, 2023-06-23T18:49:52.061 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 9495682, 2023-06-23T18:49:52.061 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 842295, 2023-06-23T18:49:52.061 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:52.061 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 3, 2023-06-23T18:49:52.062 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.190591371, 2023-06-23T18:49:52.062 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.053831024, 2023-06-23T18:49:52.062 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.062436393, 2023-06-23T18:49:52.062 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.060678410, 2023-06-23T18:49:52.062 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:52.062 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 2316, 2023-06-23T18:49:52.063 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.073526981, 2023-06-23T18:49:52.063 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000031747 2023-06-23T18:49:52.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.063 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:52.063 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.063 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.063 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.064 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 639232, 2023-06-23T18:49:52.064 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 269056 2023-06-23T18:49:52.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.064 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:52.064 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 989, 2023-06-23T18:49:52.064 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 1344, 2023-06-23T18:49:52.064 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 6640834, 2023-06-23T18:49:52.065 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 1080430, 2023-06-23T18:49:52.065 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 6, 2023-06-23T18:49:52.065 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 4, 2023-06-23T18:49:52.065 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.093534776, 2023-06-23T18:49:52.065 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.030633567, 2023-06-23T18:49:52.065 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.023092366, 2023-06-23T18:49:52.065 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.030012180, 2023-06-23T18:49:52.065 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:52.066 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 1342, 2023-06-23T18:49:52.066 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.067727114, 2023-06-23T18:49:52.066 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000050467 2023-06-23T18:49:52.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.066 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:52.066 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.066 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.067 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.067 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 320, 2023-06-23T18:49:52.067 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 773472 2023-06-23T18:49:52.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.067 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2023-06-23T18:49:52.067 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 1, 2023-06-23T18:49:52.068 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2023-06-23T18:49:52.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.068 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-MDSRank": { 2023-06-23T18:49:52.068 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2023-06-23T18:49:52.068 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2023-06-23T18:49:52.068 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 9974, 2023-06-23T18:49:52.068 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.245903677, 2023-06-23T18:49:52.069 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000024654 2023-06-23T18:49:52.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.069 INFO:teuthology.orchestra.run.smithi017.stdout: "finisher-PurgeQueue": { 2023-06-23T18:49:52.069 INFO:teuthology.orchestra.run.smithi017.stdout: "queue_len": 0, 2023-06-23T18:49:52.070 INFO:teuthology.orchestra.run.smithi017.stdout: "complete_latency": { 2023-06-23T18:49:52.070 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.070 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.070 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.070 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": { 2023-06-23T18:49:52.071 INFO:teuthology.orchestra.run.smithi017.stdout: "request": 0, 2023-06-23T18:49:52.071 INFO:teuthology.orchestra.run.smithi017.stdout: "reply": 0, 2023-06-23T18:49:52.071 INFO:teuthology.orchestra.run.smithi017.stdout: "reply_latency": { 2023-06-23T18:49:52.071 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.071 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.071 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.072 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.072 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_reply": 0, 2023-06-23T18:49:52.072 INFO:teuthology.orchestra.run.smithi017.stdout: "forward": 0, 2023-06-23T18:49:52.072 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_complete": 0, 2023-06-23T18:49:52.073 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_fetch_keys": 0, 2023-06-23T18:49:52.073 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_commit": 0, 2023-06-23T18:49:52.073 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_split": 0, 2023-06-23T18:49:52.074 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_merge": 0, 2023-06-23T18:49:52.074 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes": 540, 2023-06-23T18:49:52.074 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_top": 0, 2023-06-23T18:49:52.074 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_bottom": 0, 2023-06-23T18:49:52.075 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pin_tail": 540, 2023-06-23T18:49:52.075 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_pinned": 540, 2023-06-23T18:49:52.075 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_expired": 0, 2023-06-23T18:49:52.075 INFO:teuthology.orchestra.run.smithi017.stdout: "inodes_with_caps": 0, 2023-06-23T18:49:52.076 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": 0, 2023-06-23T18:49:52.076 INFO:teuthology.orchestra.run.smithi017.stdout: "subtrees": 6, 2023-06-23T18:49:52.076 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse": 0, 2023-06-23T18:49:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_hit": 0, 2023-06-23T18:49:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_forward": 0, 2023-06-23T18:49:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_discover": 0, 2023-06-23T18:49:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_dir_fetch": 0, 2023-06-23T18:49:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_remote_ino": 0, 2023-06-23T18:49:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "traverse_lock": 0, 2023-06-23T18:49:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "load_cent": 0, 2023-06-23T18:49:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "q": 0, 2023-06-23T18:49:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "exported": 0, 2023-06-23T18:49:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "exported_inodes": 0, 2023-06-23T18:49:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "imported": 0, 2023-06-23T18:49:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "imported_inodes": 0, 2023-06-23T18:49:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_dir_fetch": 0, 2023-06-23T18:49:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_backtrace_fetch": 0, 2023-06-23T18:49:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "openino_peer_discover": 0, 2023-06-23T18:49:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rfiles": 1045, 2023-06-23T18:49:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rbytes": 8998067092, 2023-06-23T18:49:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "root_rsnaps": 0, 2023-06-23T18:49:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_fetch": 0, 2023-06-23T18:49:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_set_tag": 0, 2023-06-23T18:49:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_backtrace_repaired": 0, 2023-06-23T18:49:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_inotable_repaired": 0, 2023-06-23T18:49:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_inodes": 0, 2023-06-23T18:49:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dir_base_inodes": 0, 2023-06-23T18:49:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_dirfrag_rstats": 0, 2023-06-23T18:49:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "scrub_file_inodes": 0, 2023-06-23T18:49:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_inode_file_caps": 0, 2023-06-23T18:49:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_revoke": 0, 2023-06-23T18:49:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_grant": 0, 2023-06-23T18:49:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_trunc": 0, 2023-06-23T18:49:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-06-23T18:49:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_cap_op_flush_ack": 0, 2023-06-23T18:49:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps": 0, 2023-06-23T18:49:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_caps_dirty": 0, 2023-06-23T18:49:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_cap_release": 0, 2023-06-23T18:49:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "process_request_cap_release": 0 2023-06-23T18:49:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_cache": { 2023-06-23T18:49:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update": 0, 2023-06-23T18:49:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_update_receipt": 0, 2023-06-23T18:49:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_try_discover": 0, 2023-06-23T18:49:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_send_discover": 0, 2023-06-23T18:49:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "dir_handle_discover": 0, 2023-06-23T18:49:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays": 0, 2023-06-23T18:49:52.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_delayed": 0, 2023-06-23T18:49:52.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num_strays_enqueuing": 0, 2023-06-23T18:49:52.086 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_created": 0, 2023-06-23T18:49:52.086 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_enqueued": 0, 2023-06-23T18:49:52.086 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_reintegrated": 0, 2023-06-23T18:49:52.086 INFO:teuthology.orchestra.run.smithi017.stdout: "strays_migrated": 0, 2023-06-23T18:49:52.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_processing": 0, 2023-06-23T18:49:52.087 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_enqueued": 0, 2023-06-23T18:49:52.087 INFO:teuthology.orchestra.run.smithi017.stdout: "num_recovering_prioritized": 0, 2023-06-23T18:49:52.087 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_started": 0, 2023-06-23T18:49:52.087 INFO:teuthology.orchestra.run.smithi017.stdout: "recovery_completed": 0, 2023-06-23T18:49:52.087 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_enqueue_scrub": 0, 2023-06-23T18:49:52.087 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_exportdir": 0, 2023-06-23T18:49:52.087 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_flush": 0, 2023-06-23T18:49:52.087 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragmentdir": 0, 2023-06-23T18:49:52.088 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_fragstats": 0, 2023-06-23T18:49:52.088 INFO:teuthology.orchestra.run.smithi017.stdout: "ireq_inodestats": 0 2023-06-23T18:49:52.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.088 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_log": { 2023-06-23T18:49:52.088 INFO:teuthology.orchestra.run.smithi017.stdout: "evadd": 0, 2023-06-23T18:49:52.088 INFO:teuthology.orchestra.run.smithi017.stdout: "evex": 0, 2023-06-23T18:49:52.088 INFO:teuthology.orchestra.run.smithi017.stdout: "evtrm": 0, 2023-06-23T18:49:52.088 INFO:teuthology.orchestra.run.smithi017.stdout: "ev": 0, 2023-06-23T18:49:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "evexg": 0, 2023-06-23T18:49:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "evexd": 0, 2023-06-23T18:49:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "segadd": 0, 2023-06-23T18:49:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "segex": 0, 2023-06-23T18:49:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "segtrm": 0, 2023-06-23T18:49:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "seg": 12, 2023-06-23T18:49:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "segexg": 0, 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "segexd": 0, 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "expos": 4194304, 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "wrpos": 0, 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "rdpos": 27360230, 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "jlat": { 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "replayed": 10592 2023-06-23T18:49:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_mem": { 2023-06-23T18:49:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "ino": 195, 2023-06-23T18:49:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "ino+": 1536, 2023-06-23T18:49:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "ino-": 1341, 2023-06-23T18:49:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "dir": 42, 2023-06-23T18:49:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "dir+": 76, 2023-06-23T18:49:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "dir-": 34, 2023-06-23T18:49:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "dn": 540, 2023-06-23T18:49:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "dn+": 1577, 2023-06-23T18:49:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "dn-": 1037, 2023-06-23T18:49:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "cap": 0, 2023-06-23T18:49:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "cap+": 0, 2023-06-23T18:49:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "cap-": 0, 2023-06-23T18:49:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "rss": 49220, 2023-06-23T18:49:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "heap": 207156 2023-06-23T18:49:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_server": { 2023-06-23T18:49:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_client_request": 0, 2023-06-23T18:49:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "dispatch_server_request": 0, 2023-06-23T18:49:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_request": 0, 2023-06-23T18:49:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_client_session": 0, 2023-06-23T18:49:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: "handle_peer_request": 0, 2023-06-23T18:49:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: "req_create_latency": { 2023-06-23T18:49:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.095 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getattr_latency": { 2023-06-23T18:49:52.095 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.095 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.095 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.095 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getfilelock_latency": { 2023-06-23T18:49:52.095 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.095 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.096 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.096 INFO:teuthology.orchestra.run.smithi017.stdout: "req_link_latency": { 2023-06-23T18:49:52.096 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.096 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.096 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.096 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookup_latency": { 2023-06-23T18:49:52.097 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.097 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.097 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.097 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookuphash_latency": { 2023-06-23T18:49:52.097 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.097 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.098 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.098 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupino_latency": { 2023-06-23T18:49:52.098 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.098 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.098 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.099 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupname_latency": { 2023-06-23T18:49:52.099 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.099 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.099 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.099 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupparent_latency": { 2023-06-23T18:49:52.099 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.099 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.100 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.100 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.100 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lookupsnap_latency": { 2023-06-23T18:49:52.100 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.100 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.100 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.101 INFO:teuthology.orchestra.run.smithi017.stdout: "req_lssnap_latency": { 2023-06-23T18:49:52.101 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.101 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.101 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.101 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.101 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mkdir_latency": { 2023-06-23T18:49:52.101 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.102 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.102 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.102 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mknod_latency": { 2023-06-23T18:49:52.102 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.102 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.102 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.103 INFO:teuthology.orchestra.run.smithi017.stdout: "req_mksnap_latency": { 2023-06-23T18:49:52.103 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.103 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.103 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.103 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.103 INFO:teuthology.orchestra.run.smithi017.stdout: "req_open_latency": { 2023-06-23T18:49:52.103 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.104 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.104 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.104 INFO:teuthology.orchestra.run.smithi017.stdout: "req_readdir_latency": { 2023-06-23T18:49:52.104 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.104 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.104 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.104 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.105 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rename_latency": { 2023-06-23T18:49:52.105 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.105 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.105 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.105 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.105 INFO:teuthology.orchestra.run.smithi017.stdout: "req_renamesnap_latency": { 2023-06-23T18:49:52.105 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.106 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.106 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.106 INFO:teuthology.orchestra.run.smithi017.stdout: "req_snapdiff_latency": { 2023-06-23T18:49:52.106 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.106 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.106 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.106 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.107 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmdir_latency": { 2023-06-23T18:49:52.107 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.107 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.107 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.107 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmsnap_latency": { 2023-06-23T18:49:52.107 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.107 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.108 INFO:teuthology.orchestra.run.smithi017.stdout: "req_rmxattr_latency": { 2023-06-23T18:49:52.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.108 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.108 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.108 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.109 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setattr_latency": { 2023-06-23T18:49:52.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.109 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.109 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.109 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setdirlayout_latency": { 2023-06-23T18:49:52.109 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.109 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.110 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setfilelock_latency": { 2023-06-23T18:49:52.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.110 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.110 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.110 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setlayout_latency": { 2023-06-23T18:49:52.111 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.111 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.111 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.111 INFO:teuthology.orchestra.run.smithi017.stdout: "req_setxattr_latency": { 2023-06-23T18:49:52.111 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.111 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.112 INFO:teuthology.orchestra.run.smithi017.stdout: "req_symlink_latency": { 2023-06-23T18:49:52.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.112 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.112 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.113 INFO:teuthology.orchestra.run.smithi017.stdout: "req_unlink_latency": { 2023-06-23T18:49:52.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.113 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.113 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.113 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_revoke_eviction": 0, 2023-06-23T18:49:52.114 INFO:teuthology.orchestra.run.smithi017.stdout: "cap_acquisition_throttle": 0, 2023-06-23T18:49:52.114 INFO:teuthology.orchestra.run.smithi017.stdout: "req_getvxattr_latency": { 2023-06-23T18:49:52.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.114 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.114 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.114 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.114 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_sessions": { 2023-06-23T18:49:52.115 INFO:teuthology.orchestra.run.smithi017.stdout: "session_count": 1, 2023-06-23T18:49:52.115 INFO:teuthology.orchestra.run.smithi017.stdout: "session_add": 2, 2023-06-23T18:49:52.115 INFO:teuthology.orchestra.run.smithi017.stdout: "session_remove": 1, 2023-06-23T18:49:52.115 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_open": 1, 2023-06-23T18:49:52.115 INFO:teuthology.orchestra.run.smithi017.stdout: "sessions_stale": 0, 2023-06-23T18:49:52.115 INFO:teuthology.orchestra.run.smithi017.stdout: "total_load": 0, 2023-06-23T18:49:52.115 INFO:teuthology.orchestra.run.smithi017.stdout: "average_load": 0, 2023-06-23T18:49:52.115 INFO:teuthology.orchestra.run.smithi017.stdout: "avg_session_uptime": 2996 2023-06-23T18:49:52.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.116 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2023-06-23T18:49:52.116 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:52.116 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:52.116 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:52.116 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:52.116 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:52.116 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:52.117 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:52.117 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:52.117 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:52.117 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:52.117 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:52.117 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:52.117 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:52.117 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:52.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:52.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:52.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:52.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:52.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:52.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:52.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:52.118 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:52.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:52.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:52.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:52.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:52.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:52.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:52.119 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:52.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:52.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:52.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2023-06-23T18:49:52.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:52.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:52.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:52.120 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:52.121 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 199295, 2023-06-23T18:49:52.121 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 58, 2023-06-23T18:49:52.121 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:52.121 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:52.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2023-06-23T18:49:52.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2023-06-23T18:49:52.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:52.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:52.122 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:52.122 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:52.122 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 7624, 2023-06-23T18:49:52.122 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 160, 2023-06-23T18:49:52.122 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:52.122 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:52.122 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:52.122 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2023-06-23T18:49:52.123 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 833081, 2023-06-23T18:49:52.123 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 3476, 2023-06-23T18:49:52.123 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:52.123 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2023-06-23T18:49:52.123 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:52.123 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2023-06-23T18:49:52.123 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.124 INFO:teuthology.orchestra.run.smithi017.stdout: "objecter": { 2023-06-23T18:49:52.124 INFO:teuthology.orchestra.run.smithi017.stdout: "op_active": 0, 2023-06-23T18:49:52.124 INFO:teuthology.orchestra.run.smithi017.stdout: "op_laggy": 0, 2023-06-23T18:49:52.124 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send": 5900, 2023-06-23T18:49:52.124 INFO:teuthology.orchestra.run.smithi017.stdout: "op_send_bytes": 26, 2023-06-23T18:49:52.124 INFO:teuthology.orchestra.run.smithi017.stdout: "op_resend": 4, 2023-06-23T18:49:52.124 INFO:teuthology.orchestra.run.smithi017.stdout: "op_reply": 5900, 2023-06-23T18:49:52.124 INFO:teuthology.orchestra.run.smithi017.stdout: "op_latency": { 2023-06-23T18:49:52.125 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 5900, 2023-06-23T18:49:52.125 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 3.841963089, 2023-06-23T18:49:52.125 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000651180 2023-06-23T18:49:52.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.125 INFO:teuthology.orchestra.run.smithi017.stdout: "op_inflight": 0, 2023-06-23T18:49:52.125 INFO:teuthology.orchestra.run.smithi017.stdout: "oplen_avg": { 2023-06-23T18:49:52.125 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 5900, 2023-06-23T18:49:52.126 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 5901 2023-06-23T18:49:52.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.126 INFO:teuthology.orchestra.run.smithi017.stdout: "op": 5900, 2023-06-23T18:49:52.126 INFO:teuthology.orchestra.run.smithi017.stdout: "op_r": 5900, 2023-06-23T18:49:52.126 INFO:teuthology.orchestra.run.smithi017.stdout: "op_w": 0, 2023-06-23T18:49:52.126 INFO:teuthology.orchestra.run.smithi017.stdout: "op_rmw": 0, 2023-06-23T18:49:52.126 INFO:teuthology.orchestra.run.smithi017.stdout: "op_pg": 0, 2023-06-23T18:49:52.126 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_stat": 3466, 2023-06-23T18:49:52.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_create": 0, 2023-06-23T18:49:52.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_read": 2431, 2023-06-23T18:49:52.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_write": 0, 2023-06-23T18:49:52.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writefull": 0, 2023-06-23T18:49:52.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_writesame": 0, 2023-06-23T18:49:52.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_append": 0, 2023-06-23T18:49:52.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_zero": 0, 2023-06-23T18:49:52.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_truncate": 0, 2023-06-23T18:49:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_delete": 0, 2023-06-23T18:49:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_mapext": 0, 2023-06-23T18:49:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_sparse_read": 0, 2023-06-23T18:49:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_clonerange": 0, 2023-06-23T18:49:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_getxattr": 2, 2023-06-23T18:49:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_setxattr": 0, 2023-06-23T18:49:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_cmpxattr": 0, 2023-06-23T18:49:52.128 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_rmxattr": 0, 2023-06-23T18:49:52.129 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_resetxattrs": 0, 2023-06-23T18:49:52.129 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_call": 0, 2023-06-23T18:49:52.129 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_watch": 0, 2023-06-23T18:49:52.129 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_notify": 0, 2023-06-23T18:49:52.129 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_src_cmpxattr": 0, 2023-06-23T18:49:52.129 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls": 0, 2023-06-23T18:49:52.129 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_pgls_filter": 0, 2023-06-23T18:49:52.129 INFO:teuthology.orchestra.run.smithi017.stdout: "osdop_other": 0, 2023-06-23T18:49:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_active": 0, 2023-06-23T18:49:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_send": 0, 2023-06-23T18:49:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_resend": 0, 2023-06-23T18:49:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "linger_ping": 0, 2023-06-23T18:49:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_active": 0, 2023-06-23T18:49:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_send": 0, 2023-06-23T18:49:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "poolop_resend": 0, 2023-06-23T18:49:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_active": 0, 2023-06-23T18:49:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_send": 0, 2023-06-23T18:49:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "poolstat_resend": 0, 2023-06-23T18:49:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_active": 0, 2023-06-23T18:49:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_send": 0, 2023-06-23T18:49:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "statfs_resend": 0, 2023-06-23T18:49:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "command_active": 0, 2023-06-23T18:49:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "command_send": 0, 2023-06-23T18:49:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "command_resend": 0, 2023-06-23T18:49:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: "map_epoch": 84, 2023-06-23T18:49:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: "map_full": 0, 2023-06-23T18:49:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: "map_inc": 2, 2023-06-23T18:49:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_sessions": 6, 2023-06-23T18:49:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_open": 6, 2023-06-23T18:49:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_session_close": 0, 2023-06-23T18:49:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_laggy": 0, 2023-06-23T18:49:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_wr": 0, 2023-06-23T18:49:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_rd": 2, 2023-06-23T18:49:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_del": 0 2023-06-23T18:49:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "oft": { 2023-06-23T18:49:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_objs": 0, 2023-06-23T18:49:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_kv_pairs": 0, 2023-06-23T18:49:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_updates": 0, 2023-06-23T18:49:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "omap_total_removes": 0 2023-06-23T18:49:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "osd-slow-ops": { 2023-06-23T18:49:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "slow_ops_count": 0 2023-06-23T18:49:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "purge_queue": { 2023-06-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops": 0, 2023-06-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_ops_high_water": 0, 2023-06-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing": 0, 2023-06-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executing_high_water": 0, 2023-06-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_executed": 0, 2023-06-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pq_item_in_journal": 0 2023-06-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2023-06-23T18:49:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 6401, 2023-06-23T18:49:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 24846880, 2023-06-23T18:49:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 6401, 2023-06-23T18:49:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 6401, 2023-06-23T18:49:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 24846880, 2023-06-23T18:49:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_bytes": { 2023-06-23T18:49:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2023-06-23T18:49:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 5900, 2023-06-23T18:49:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 23170923, 2023-06-23T18:49:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 5900, 2023-06-23T18:49:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 23170923, 2023-06-23T18:49:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.143 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.143 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.144 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-objecter_ops": { 2023-06-23T18:49:52.144 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:52.144 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 1024, 2023-06-23T18:49:52.144 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 5900, 2023-06-23T18:49:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 5900, 2023-06-23T18:49:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 5900, 2023-06-23T18:49:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 5900, 2023-06-23T18:49:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.148 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle": { 2023-06-23T18:49:52.148 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:52.149 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2023-06-23T18:49:52.149 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:52.149 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:52.149 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 0, 2023-06-23T18:49:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 0, 2023-06-23T18:49:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-write_buf_throttle-0x55b9d964e000": { 2023-06-23T18:49:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 3758096384, 2023-06-23T18:49:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 0, 2023-06-23T18:49:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 0, 2023-06-23T18:49:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 0, 2023-06-23T18:49:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 0, 2023-06-23T18:49:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:52.157 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.157 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:52.157 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-06-23T18:49:52.523 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-06-23T18:49:52.523 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-06-23T18:49:52.524 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-06-23T18:49:52.524 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-06-23T18:49:52.524 DEBUG:tasks.check_counter:Getting stats from k 2023-06-23T18:49:52.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.k perf dump 2023-06-23T18:49:53.085 INFO:teuthology.orchestra.run.smithi017.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.b/config 2023-06-23T18:49:53.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:52 smithi017 ceph-mon[97472]: pgmap v1131: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:53.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:52 smithi002 ceph-mon[95947]: pgmap v1131: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:53.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:52 smithi082 ceph-mon[97889]: pgmap v1131: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:54.456 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2023-06-23T18:49:54.456 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:54.456 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 473, 2023-06-23T18:49:54.456 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 470, 2023-06-23T18:49:54.456 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 544384, 2023-06-23T18:49:54.457 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 270080, 2023-06-23T18:49:54.457 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:54.457 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 0, 2023-06-23T18:49:54.457 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.062211409, 2023-06-23T18:49:54.457 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.023209048, 2023-06-23T18:49:54.457 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.021828095, 2023-06-23T18:49:54.457 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.012435521, 2023-06-23T18:49:54.458 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:54.458 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 469, 2023-06-23T18:49:54.458 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.030584075, 2023-06-23T18:49:54.458 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000065211 2023-06-23T18:49:54.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.458 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:54.458 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:54.459 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:54.459 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:54.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.459 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 544384, 2023-06-23T18:49:54.459 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 270080 2023-06-23T18:49:54.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.459 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:54.460 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 2, 2023-06-23T18:49:54.460 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 360, 2023-06-23T18:49:54.460 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 320, 2023-06-23T18:49:54.460 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 214752, 2023-06-23T18:49:54.460 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 4, 2023-06-23T18:49:54.460 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 2, 2023-06-23T18:49:54.460 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.017391217, 2023-06-23T18:49:54.461 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.012399108, 2023-06-23T18:49:54.461 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.001663571, 2023-06-23T18:49:54.461 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:54.461 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:54.461 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 358, 2023-06-23T18:49:54.461 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.016976762, 2023-06-23T18:49:54.462 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000047421 2023-06-23T18:49:54.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.462 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:54.462 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:54.462 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:54.462 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:54.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.463 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 320, 2023-06-23T18:49:54.463 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 214752 2023-06-23T18:49:54.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.463 INFO:teuthology.orchestra.run.smithi017.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:54.464 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_messages": 4, 2023-06-23T18:49:54.464 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages": 4, 2023-06-23T18:49:54.465 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_bytes": 2960, 2023-06-23T18:49:54.465 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_bytes": 448, 2023-06-23T18:49:54.465 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_created_connections": 2, 2023-06-23T18:49:54.465 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_active_connections": 1, 2023-06-23T18:49:54.465 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_total_time": 0.000739713, 2023-06-23T18:49:54.465 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_send_time": 0.000052398, 2023-06-23T18:49:54.466 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_recv_time": 0.000674907, 2023-06-23T18:49:54.466 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:49:54.466 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:54.466 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 3, 2023-06-23T18:49:54.466 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000871360, 2023-06-23T18:49:54.466 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000290453 2023-06-23T18:49:54.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.467 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:54.467 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:54.467 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:54.467 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:54.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.467 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-06-23T18:49:54.467 INFO:teuthology.orchestra.run.smithi017.stdout: "msgr_send_encrypted_bytes": 448 2023-06-23T18:49:54.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.468 INFO:teuthology.orchestra.run.smithi017.stdout: "cct": { 2023-06-23T18:49:54.468 INFO:teuthology.orchestra.run.smithi017.stdout: "total_workers": 0, 2023-06-23T18:49:54.468 INFO:teuthology.orchestra.run.smithi017.stdout: "unhealthy_workers": 0 2023-06-23T18:49:54.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.468 INFO:teuthology.orchestra.run.smithi017.stdout: "mempool": { 2023-06-23T18:49:54.468 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:54.468 INFO:teuthology.orchestra.run.smithi017.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:54.469 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:54.469 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:54.469 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:54.469 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:54.469 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:54.469 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:54.469 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:54.470 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:54.470 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:54.470 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:54.470 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:54.470 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:54.470 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:54.470 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:54.471 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:54.471 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:54.471 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:54.471 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:54.471 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:54.471 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:54.471 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:54.471 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:54.472 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:54.472 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:54.472 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:54.472 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:54.472 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:54.472 INFO:teuthology.orchestra.run.smithi017.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:54.472 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:54.473 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_items": 0, 2023-06-23T18:49:54.473 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:54.473 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:54.473 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:54.473 INFO:teuthology.orchestra.run.smithi017.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:54.473 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_bytes": 148721, 2023-06-23T18:49:54.473 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_anon_items": 30, 2023-06-23T18:49:54.474 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:54.474 INFO:teuthology.orchestra.run.smithi017.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:54.474 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_bytes": 0, 2023-06-23T18:49:54.474 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_items": 0, 2023-06-23T18:49:54.474 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:54.474 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:54.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:54.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:54.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_bytes": 0, 2023-06-23T18:49:54.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_items": 0, 2023-06-23T18:49:54.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:54.475 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:54.475 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:54.475 INFO:teuthology.orchestra.run.smithi017.stdout: "pgmap_items": 0, 2023-06-23T18:49:54.476 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_bytes": 928, 2023-06-23T18:49:54.476 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_co_items": 2, 2023-06-23T18:49:54.476 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:54.476 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_1_items": 0, 2023-06-23T18:49:54.476 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:54.476 INFO:teuthology.orchestra.run.smithi017.stdout: "unittest_2_items": 0 2023-06-23T18:49:54.476 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2023-06-23T18:49:54.477 INFO:teuthology.orchestra.run.smithi017.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:54.477 INFO:teuthology.orchestra.run.smithi017.stdout: "val": 0, 2023-06-23T18:49:54.477 INFO:teuthology.orchestra.run.smithi017.stdout: "max": 104857600, 2023-06-23T18:49:54.477 INFO:teuthology.orchestra.run.smithi017.stdout: "get_started": 0, 2023-06-23T18:49:54.477 INFO:teuthology.orchestra.run.smithi017.stdout: "get": 475, 2023-06-23T18:49:54.477 INFO:teuthology.orchestra.run.smithi017.stdout: "get_sum": 477447, 2023-06-23T18:49:54.477 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:54.478 INFO:teuthology.orchestra.run.smithi017.stdout: "get_or_fail_success": 475, 2023-06-23T18:49:54.478 INFO:teuthology.orchestra.run.smithi017.stdout: "take": 0, 2023-06-23T18:49:54.478 INFO:teuthology.orchestra.run.smithi017.stdout: "take_sum": 0, 2023-06-23T18:49:54.478 INFO:teuthology.orchestra.run.smithi017.stdout: "put": 475, 2023-06-23T18:49:54.478 INFO:teuthology.orchestra.run.smithi017.stdout: "put_sum": 477447, 2023-06-23T18:49:54.478 INFO:teuthology.orchestra.run.smithi017.stdout: "wait": { 2023-06-23T18:49:54.478 INFO:teuthology.orchestra.run.smithi017.stdout: "avgcount": 0, 2023-06-23T18:49:54.479 INFO:teuthology.orchestra.run.smithi017.stdout: "sum": 0.000000000, 2023-06-23T18:49:54.479 INFO:teuthology.orchestra.run.smithi017.stdout: "avgtime": 0.000000000 2023-06-23T18:49:54.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:54.479 INFO:teuthology.orchestra.run.smithi017.stdout: } 2023-06-23T18:49:54.479 INFO:teuthology.orchestra.run.smithi017.stdout:} 2023-06-23T18:49:54.947 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-06-23T18:49:54.947 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-06-23T18:49:54.947 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-06-23T18:49:54.948 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-06-23T18:49:54.948 DEBUG:tasks.check_counter:Getting stats from c 2023-06-23T18:49:54.948 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.c perf dump 2023-06-23T18:49:55.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:54 smithi017 ceph-mon[97472]: pgmap v1132: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-06-23T18:49:55.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:54 smithi002 ceph-mon[95947]: pgmap v1132: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-06-23T18:49:55.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:54 smithi082 ceph-mon[97889]: pgmap v1132: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2023-06-23T18:49:55.585 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:49:57.073 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-06-23T18:49:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 6605228, 2023-06-23T18:49:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 6731118, 2023-06-23T18:49:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 1163649452, 2023-06-23T18:49:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 1206281375, 2023-06-23T18:49:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 25, 2023-06-23T18:49:57.074 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 3, 2023-06-23T18:49:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 156.701681891, 2023-06-23T18:49:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 65.527089098, 2023-06-23T18:49:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 101.209776641, 2023-06-23T18:49:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.305246481, 2023-06-23T18:49:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6731119, 2023-06-23T18:49:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 81.279928053, 2023-06-23T18:49:57.075 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000012075 2023-06-23T18:49:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10854223, 2023-06-23T18:49:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.509319875, 2023-06-23T18:49:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000139 2023-06-23T18:49:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.076 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-06-23T18:49:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2023-06-23T18:49:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 969750, 2023-06-23T18:49:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1396030, 2023-06-23T18:49:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 296777200, 2023-06-23T18:49:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 612299043, 2023-06-23T18:49:57.077 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 12, 2023-06-23T18:49:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 7, 2023-06-23T18:49:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 46.371449521, 2023-06-23T18:49:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 19.563028967, 2023-06-23T18:49:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 30.018864584, 2023-06-23T18:49:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.368943334, 2023-06-23T18:49:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1396027, 2023-06-23T18:49:57.078 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 55.075364915, 2023-06-23T18:49:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000039451 2023-06-23T18:49:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2255287, 2023-06-23T18:49:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.329817151, 2023-06-23T18:49:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000146 2023-06-23T18:49:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.079 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 650096, 2023-06-23T18:49:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 284032 2023-06-23T18:49:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 3349788, 2023-06-23T18:49:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 3350346, 2023-06-23T18:49:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 530428384, 2023-06-23T18:49:57.080 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 566890564, 2023-06-23T18:49:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 15, 2023-06-23T18:49:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 6, 2023-06-23T18:49:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 139.106356181, 2023-06-23T18:49:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 61.794987453, 2023-06-23T18:49:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 97.103135526, 2023-06-23T18:49:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.179123883, 2023-06-23T18:49:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:57.081 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3350344, 2023-06-23T18:49:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 56.135444601, 2023-06-23T18:49:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000016755 2023-06-23T18:49:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6685341, 2023-06-23T18:49:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.752049183, 2023-06-23T18:49:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000112 2023-06-23T18:49:57.082 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 1232, 2023-06-23T18:49:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 789168 2023-06-23T18:49:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2023-06-23T18:49:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2023-06-23T18:49:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2023-06-23T18:49:57.083 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2023-06-23T18:49:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2023-06-23T18:49:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2023-06-23T18:49:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 51411, 2023-06-23T18:49:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 19.313336418, 2023-06-23T18:49:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000375665 2023-06-23T18:49:57.084 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2023-06-23T18:49:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2023-06-23T18:49:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2023-06-23T18:49:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 268, 2023-06-23T18:49:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.007541721, 2023-06-23T18:49:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000028140 2023-06-23T18:49:57.085 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2023-06-23T18:49:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 436717, 2023-06-23T18:49:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 435492, 2023-06-23T18:49:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2023-06-23T18:49:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 435492, 2023-06-23T18:49:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 424.287165949, 2023-06-23T18:49:57.086 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000974270 2023-06-23T18:49:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2023-06-23T18:49:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 1225, 2023-06-23T18:49:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2023-06-23T18:49:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2023-06-23T18:49:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 11, 2023-06-23T18:49:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2023-06-23T18:49:57.087 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2023-06-23T18:49:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 811, 2023-06-23T18:49:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 44, 2023-06-23T18:49:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 744, 2023-06-23T18:49:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 23, 2023-06-23T18:49:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 749, 2023-06-23T18:49:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 372, 2023-06-23T18:49:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 722, 2023-06-23T18:49:57.088 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 722, 2023-06-23T18:49:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 8, 2023-06-23T18:49:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 872358, 2023-06-23T18:49:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 870841, 2023-06-23T18:49:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 1218, 2023-06-23T18:49:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 56, 2023-06-23T18:49:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2023-06-23T18:49:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2023-06-23T18:49:57.089 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2023-06-23T18:49:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 258, 2023-06-23T18:49:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2023-06-23T18:49:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 12, 2023-06-23T18:49:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 398, 2023-06-23T18:49:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 19, 2023-06-23T18:49:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 1100, 2023-06-23T18:49:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2023-06-23T18:49:57.090 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 7323, 2023-06-23T18:49:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2023-06-23T18:49:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 1046, 2023-06-23T18:49:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 8926501484, 2023-06-23T18:49:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2023-06-23T18:49:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 7323, 2023-06-23T18:49:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2023-06-23T18:49:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2023-06-23T18:49:57.091 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2023-06-23T18:49:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 53, 2023-06-23T18:49:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2023-06-23T18:49:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 53, 2023-06-23T18:49:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 7270, 2023-06-23T18:49:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 6, 2023-06-23T18:49:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 519713, 2023-06-23T18:49:57.092 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 402487, 2023-06-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2023-06-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-06-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2023-06-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 520712, 2023-06-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 23326, 2023-06-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 5, 2023-06-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 34 2023-06-23T18:49:57.093 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2023-06-23T18:49:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 28, 2023-06-23T18:49:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 689, 2023-06-23T18:49:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 357, 2023-06-23T18:49:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 690, 2023-06-23T18:49:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 31, 2023-06-23T18:49:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2023-06-23T18:49:57.094 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2023-06-23T18:49:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2023-06-23T18:49:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 5, 2023-06-23T18:49:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 5, 2023-06-23T18:49:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2023-06-23T18:49:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2023-06-23T18:49:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2023-06-23T18:49:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2023-06-23T18:49:57.095 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2023-06-23T18:49:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2023-06-23T18:49:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2023-06-23T18:49:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2023-06-23T18:49:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 13, 2023-06-23T18:49:57.096 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2023-06-23T18:49:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2023-06-23T18:49:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2023-06-23T18:49:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2023-06-23T18:49:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2023-06-23T18:49:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 24094, 2023-06-23T18:49:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2023-06-23T18:49:57.097 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2023-06-23T18:49:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 24094, 2023-06-23T18:49:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2023-06-23T18:49:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2023-06-23T18:49:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 33, 2023-06-23T18:49:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2023-06-23T18:49:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2023-06-23T18:49:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 33, 2023-06-23T18:49:57.098 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2023-06-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2023-06-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2023-06-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 52153891, 2023-06-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2023-06-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2023-06-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 23364, 2023-06-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 51.971963436, 2023-06-23T18:49:57.099 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002224446 2023-06-23T18:49:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2023-06-23T18:49:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2023-06-23T18:49:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 804, 2023-06-23T18:49:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 1153, 2023-06-23T18:49:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 349, 2023-06-23T18:49:57.100 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 53, 2023-06-23T18:49:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 373, 2023-06-23T18:49:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 320, 2023-06-23T18:49:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 811, 2023-06-23T18:49:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 1189, 2023-06-23T18:49:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 378, 2023-06-23T18:49:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 722, 2023-06-23T18:49:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 1115, 2023-06-23T18:49:57.101 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 393, 2023-06-23T18:49:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 68032, 2023-06-23T18:49:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 198964 2023-06-23T18:49:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2023-06-23T18:49:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 672493, 2023-06-23T18:49:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 116, 2023-06-23T18:49:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 436717, 2023-06-23T18:49:57.102 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 36, 2023-06-23T18:49:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 752, 2023-06-23T18:49:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2023-06-23T18:49:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 17, 2023-06-23T18:49:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.033553504, 2023-06-23T18:49:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001973735 2023-06-23T18:49:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.103 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2023-06-23T18:49:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 434214, 2023-06-23T18:49:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 420.786892188, 2023-06-23T18:49:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000969077 2023-06-23T18:49:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2023-06-23T18:49:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.104 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2023-06-23T18:49:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.105 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2023-06-23T18:49:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 999, 2023-06-23T18:49:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 3.318481373, 2023-06-23T18:49:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003321803 2023-06-23T18:49:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2023-06-23T18:49:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.106 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2023-06-23T18:49:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2023-06-23T18:49:57.107 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2023-06-23T18:49:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.108 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2023-06-23T18:49:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.109 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2023-06-23T18:49:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2023-06-23T18:49:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1, 2023-06-23T18:49:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.001930368, 2023-06-23T18:49:57.110 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001930368 2023-06-23T18:49:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2023-06-23T18:49:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2023-06-23T18:49:57.111 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2023-06-23T18:49:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 237, 2023-06-23T18:49:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.116465504, 2023-06-23T18:49:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000491415 2023-06-23T18:49:57.112 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2023-06-23T18:49:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 19, 2023-06-23T18:49:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.014915128, 2023-06-23T18:49:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000785006 2023-06-23T18:49:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2023-06-23T18:49:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3, 2023-06-23T18:49:57.113 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.011245326, 2023-06-23T18:49:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003748442 2023-06-23T18:49:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2023-06-23T18:49:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.114 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2023-06-23T18:49:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2023-06-23T18:49:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.115 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2023-06-23T18:49:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.116 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2023-06-23T18:49:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2023-06-23T18:49:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.117 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2023-06-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2023-06-23T18:49:57.118 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2023-06-23T18:49:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.119 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2023-06-23T18:49:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2023-06-23T18:49:57.120 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2023-06-23T18:49:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2, 2023-06-23T18:49:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.003682558, 2023-06-23T18:49:57.121 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001841279 2023-06-23T18:49:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2023-06-23T18:49:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2023-06-23T18:49:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2023-06-23T18:49:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.122 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2023-06-23T18:49:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2023-06-23T18:49:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 2, 2023-06-23T18:49:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 1, 2023-06-23T18:49:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2023-06-23T18:49:57.123 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2023-06-23T18:49:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 29425, 2023-06-23T18:49:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 29425, 2023-06-23T18:49:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 1666 2023-06-23T18:49:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2023-06-23T18:49:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:57.124 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:57.125 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:57.126 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:57.127 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2023-06-23T18:49:57.128 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 282066, 2023-06-23T18:49:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 109, 2023-06-23T18:49:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:57.129 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2023-06-23T18:49:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2023-06-23T18:49:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 7624, 2023-06-23T18:49:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 160, 2023-06-23T18:49:57.130 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2023-06-23T18:49:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 2579464, 2023-06-23T18:49:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 14444, 2023-06-23T18:49:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2023-06-23T18:49:57.131 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2023-06-23T18:49:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2023-06-23T18:49:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2023-06-23T18:49:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2023-06-23T18:49:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 31217, 2023-06-23T18:49:57.132 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 48045581, 2023-06-23T18:49:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 33, 2023-06-23T18:49:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 31217, 2023-06-23T18:49:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2023-06-23T18:49:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 31217, 2023-06-23T18:49:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 55.863970584, 2023-06-23T18:49:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001789536 2023-06-23T18:49:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.133 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2023-06-23T18:49:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2023-06-23T18:49:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 31217, 2023-06-23T18:49:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 31350 2023-06-23T18:49:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 31217, 2023-06-23T18:49:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 7328, 2023-06-23T18:49:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 23889, 2023-06-23T18:49:57.134 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2023-06-23T18:49:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2023-06-23T18:49:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2023-06-23T18:49:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 10, 2023-06-23T18:49:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 5, 2023-06-23T18:49:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 23369, 2023-06-23T18:49:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 112, 2023-06-23T18:49:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2023-06-23T18:49:57.135 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2023-06-23T18:49:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2023-06-23T18:49:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2023-06-23T18:49:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 266, 2023-06-23T18:49:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2023-06-23T18:49:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2023-06-23T18:49:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2023-06-23T18:49:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 7323, 2023-06-23T18:49:57.136 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 20, 2023-06-23T18:49:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2023-06-23T18:49:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2023-06-23T18:49:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2023-06-23T18:49:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2023-06-23T18:49:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2023-06-23T18:49:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2023-06-23T18:49:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2023-06-23T18:49:57.137 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2023-06-23T18:49:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2023-06-23T18:49:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2023-06-23T18:49:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2023-06-23T18:49:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2023-06-23T18:49:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2023-06-23T18:49:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2023-06-23T18:49:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2023-06-23T18:49:57.138 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2023-06-23T18:49:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2023-06-23T18:49:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2023-06-23T18:49:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2023-06-23T18:49:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2023-06-23T18:49:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2023-06-23T18:49:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2023-06-23T18:49:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2023-06-23T18:49:57.139 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2023-06-23T18:49:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2023-06-23T18:49:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2023-06-23T18:49:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 84, 2023-06-23T18:49:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2023-06-23T18:49:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 2, 2023-06-23T18:49:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2023-06-23T18:49:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2023-06-23T18:49:57.140 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2023-06-23T18:49:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2023-06-23T18:49:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 199, 2023-06-23T18:49:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2023-06-23T18:49:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 46 2023-06-23T18:49:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2023-06-23T18:49:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2023-06-23T18:49:57.141 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2023-06-23T18:49:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2023-06-23T18:49:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2023-06-23T18:49:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2023-06-23T18:49:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 436838 2023-06-23T18:49:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.142 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2023-06-23T18:49:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2023-06-23T18:49:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 239, 2023-06-23T18:49:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2023-06-23T18:49:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 1, 2023-06-23T18:49:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 5, 2023-06-23T18:49:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2023-06-23T18:49:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.143 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-06-23T18:49:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 10924762, 2023-06-23T18:49:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 1007595297, 2023-06-23T18:49:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 10924762, 2023-06-23T18:49:57.144 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:49:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:49:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 10924762, 2023-06-23T18:49:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 1007595297, 2023-06-23T18:49:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.145 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2023-06-23T18:49:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-06-23T18:49:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:57.146 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2023-06-23T18:49:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2023-06-23T18:49:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 31217, 2023-06-23T18:49:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 48046086, 2023-06-23T18:49:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 31217, 2023-06-23T18:49:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 48046086, 2023-06-23T18:49:57.147 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2023-06-23T18:49:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:57.148 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2023-06-23T18:49:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2023-06-23T18:49:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2023-06-23T18:49:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 31217, 2023-06-23T18:49:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 31217, 2023-06-23T18:49:57.149 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 31217, 2023-06-23T18:49:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 31217, 2023-06-23T18:49:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.150 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2023-06-23T18:49:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2023-06-23T18:49:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 5, 2023-06-23T18:49:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 505, 2023-06-23T18:49:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 5, 2023-06-23T18:49:57.151 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:49:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:49:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 5, 2023-06-23T18:49:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 505, 2023-06-23T18:49:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.152 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x561a3be97cc0": { 2023-06-23T18:49:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2023-06-23T18:49:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 24094, 2023-06-23T18:49:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 47959587, 2023-06-23T18:49:57.153 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 24094, 2023-06-23T18:49:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:49:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:49:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 23364, 2023-06-23T18:49:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 47959587, 2023-06-23T18:49:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:57.154 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.155 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:57.155 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-06-23T18:49:57.156 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:56 smithi082 ceph-mon[97889]: pgmap v1133: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:57.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:56 smithi017 ceph-mon[97472]: pgmap v1133: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:57.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:56 smithi002 ceph-mon[95947]: pgmap v1133: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:57.534 INFO:tasks.check_counter:Daemon mds.c mds.exported=12 2023-06-23T18:49:57.535 INFO:tasks.check_counter:Daemon mds.c mds.imported=19 2023-06-23T18:49:57.535 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=28 2023-06-23T18:49:57.535 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=689 2023-06-23T18:49:57.535 DEBUG:tasks.check_counter:Getting stats from f 2023-06-23T18:49:57.536 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.f perf dump 2023-06-23T18:49:58.115 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:49:59.184 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:49:58 smithi082 ceph-mon[97889]: pgmap v1134: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:59.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:49:58 smithi017 ceph-mon[97472]: pgmap v1134: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:59.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:49:58 smithi002 ceph-mon[95947]: pgmap v1134: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 5 op/s 2023-06-23T18:49:59.347 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-06-23T18:49:59.348 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:49:59.348 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2750, 2023-06-23T18:49:59.348 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2726, 2023-06-23T18:49:59.348 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 9744856, 2023-06-23T18:49:59.348 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 970422, 2023-06-23T18:49:59.349 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2023-06-23T18:49:59.349 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2023-06-23T18:49:59.349 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.235489140, 2023-06-23T18:49:59.349 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.066068245, 2023-06-23T18:49:59.349 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.069865301, 2023-06-23T18:49:59.350 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.080982746, 2023-06-23T18:49:59.350 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:59.350 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2725, 2023-06-23T18:49:59.350 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.122513880, 2023-06-23T18:49:59.350 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000044959 2023-06-23T18:49:59.351 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.351 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:59.351 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.351 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.351 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.352 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 639888, 2023-06-23T18:49:59.352 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 270672 2023-06-23T18:49:59.352 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.352 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:49:59.353 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 1078, 2023-06-23T18:49:59.353 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 1435, 2023-06-23T18:49:59.353 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 16727283, 2023-06-23T18:49:59.353 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 1025537, 2023-06-23T18:49:59.353 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2023-06-23T18:49:59.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2023-06-23T18:49:59.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.096087664, 2023-06-23T18:49:59.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.027765593, 2023-06-23T18:49:59.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.032725449, 2023-06-23T18:49:59.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.027396710, 2023-06-23T18:49:59.354 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:59.355 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 1433, 2023-06-23T18:49:59.355 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.079912193, 2023-06-23T18:49:59.355 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000055765 2023-06-23T18:49:59.355 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.355 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:59.356 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.356 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.356 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.356 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.357 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2023-06-23T18:49:59.357 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 690528 2023-06-23T18:49:59.357 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.357 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:49:59.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2650, 2023-06-23T18:49:59.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2650, 2023-06-23T18:49:59.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 21649487, 2023-06-23T18:49:59.358 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 823361, 2023-06-23T18:49:59.359 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 6, 2023-06-23T18:49:59.359 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 4, 2023-06-23T18:49:59.359 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.171567983, 2023-06-23T18:49:59.359 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.047451988, 2023-06-23T18:49:59.360 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.049771420, 2023-06-23T18:49:59.360 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.062977180, 2023-06-23T18:49:59.360 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:49:59.360 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2649, 2023-06-23T18:49:59.360 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.079154034, 2023-06-23T18:49:59.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000029880 2023-06-23T18:49:59.361 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.361 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:49:59.361 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.362 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.362 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.362 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.362 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-06-23T18:49:59.362 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 448 2023-06-23T18:49:59.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.363 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2023-06-23T18:49:59.363 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2023-06-23T18:49:59.363 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2023-06-23T18:49:59.363 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.364 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2023-06-23T18:49:59.364 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2023-06-23T18:49:59.364 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2023-06-23T18:49:59.364 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 10186, 2023-06-23T18:49:59.365 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.229933432, 2023-06-23T18:49:59.365 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000022573 2023-06-23T18:49:59.365 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.365 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.365 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2023-06-23T18:49:59.366 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2023-06-23T18:49:59.366 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2023-06-23T18:49:59.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.366 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.366 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.367 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.367 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.367 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2023-06-23T18:49:59.367 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 0, 2023-06-23T18:49:59.368 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 0, 2023-06-23T18:49:59.368 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2023-06-23T18:49:59.368 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.368 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.368 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.369 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.369 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2023-06-23T18:49:59.369 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 0, 2023-06-23T18:49:59.369 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2023-06-23T18:49:59.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2023-06-23T18:49:59.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 0, 2023-06-23T18:49:59.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2023-06-23T18:49:59.370 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2023-06-23T18:49:59.370 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 1300, 2023-06-23T18:49:59.371 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 0, 2023-06-23T18:49:59.371 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 0, 2023-06-23T18:49:59.371 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 1300, 2023-06-23T18:49:59.371 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 1300, 2023-06-23T18:49:59.372 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 4, 2023-06-23T18:49:59.372 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 0, 2023-06-23T18:49:59.372 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 0, 2023-06-23T18:49:59.372 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 6, 2023-06-23T18:49:59.372 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 0, 2023-06-23T18:49:59.372 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 0, 2023-06-23T18:49:59.372 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 0, 2023-06-23T18:49:59.373 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 0, 2023-06-23T18:49:59.373 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2023-06-23T18:49:59.373 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2023-06-23T18:49:59.373 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 0, 2023-06-23T18:49:59.373 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 0, 2023-06-23T18:49:59.373 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2023-06-23T18:49:59.374 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 0, 2023-06-23T18:49:59.374 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 0, 2023-06-23T18:49:59.374 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 0, 2023-06-23T18:49:59.374 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 0, 2023-06-23T18:49:59.374 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2023-06-23T18:49:59.374 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 0, 2023-06-23T18:49:59.374 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2023-06-23T18:49:59.375 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 0, 2023-06-23T18:49:59.375 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 0, 2023-06-23T18:49:59.375 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2023-06-23T18:49:59.375 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 0, 2023-06-23T18:49:59.375 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2023-06-23T18:49:59.375 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2023-06-23T18:49:59.375 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2023-06-23T18:49:59.376 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 0, 2023-06-23T18:49:59.376 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2023-06-23T18:49:59.376 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 0, 2023-06-23T18:49:59.376 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 0, 2023-06-23T18:49:59.376 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 0, 2023-06-23T18:49:59.377 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 0, 2023-06-23T18:49:59.377 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 0, 2023-06-23T18:49:59.377 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2023-06-23T18:49:59.377 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-06-23T18:49:59.378 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2023-06-23T18:49:59.378 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 0, 2023-06-23T18:49:59.378 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 0, 2023-06-23T18:49:59.379 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 0, 2023-06-23T18:49:59.379 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 0 2023-06-23T18:49:59.379 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.379 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2023-06-23T18:49:59.380 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 0, 2023-06-23T18:49:59.380 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 0, 2023-06-23T18:49:59.380 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 0, 2023-06-23T18:49:59.380 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 0, 2023-06-23T18:49:59.381 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 0, 2023-06-23T18:49:59.381 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2023-06-23T18:49:59.381 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2023-06-23T18:49:59.381 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2023-06-23T18:49:59.382 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 0, 2023-06-23T18:49:59.382 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 0, 2023-06-23T18:49:59.382 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 0, 2023-06-23T18:49:59.382 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2023-06-23T18:49:59.382 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2023-06-23T18:49:59.383 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2023-06-23T18:49:59.383 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2023-06-23T18:49:59.383 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2023-06-23T18:49:59.383 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2023-06-23T18:49:59.383 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2023-06-23T18:49:59.383 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 0, 2023-06-23T18:49:59.384 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2023-06-23T18:49:59.384 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2023-06-23T18:49:59.384 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2023-06-23T18:49:59.384 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2023-06-23T18:49:59.384 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.384 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2023-06-23T18:49:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 0, 2023-06-23T18:49:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2023-06-23T18:49:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2023-06-23T18:49:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 0, 2023-06-23T18:49:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2023-06-23T18:49:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2023-06-23T18:49:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 0, 2023-06-23T18:49:59.385 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2023-06-23T18:49:59.386 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2023-06-23T18:49:59.386 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 41, 2023-06-23T18:49:59.386 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2023-06-23T18:49:59.386 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2023-06-23T18:49:59.386 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2023-06-23T18:49:59.386 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 0, 2023-06-23T18:49:59.386 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 49999573, 2023-06-23T18:49:59.387 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2023-06-23T18:49:59.387 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.387 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.387 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.387 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 35151 2023-06-23T18:49:59.387 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.387 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2023-06-23T18:49:59.388 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 211, 2023-06-23T18:49:59.388 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 510, 2023-06-23T18:49:59.388 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 299, 2023-06-23T18:49:59.388 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 23, 2023-06-23T18:49:59.388 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 41, 2023-06-23T18:49:59.388 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 18, 2023-06-23T18:49:59.388 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 1300, 2023-06-23T18:49:59.389 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 1607, 2023-06-23T18:49:59.389 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 307, 2023-06-23T18:49:59.389 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 0, 2023-06-23T18:49:59.389 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 0, 2023-06-23T18:49:59.389 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 0, 2023-06-23T18:49:59.389 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 51264, 2023-06-23T18:49:59.389 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 207156 2023-06-23T18:49:59.389 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.390 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2023-06-23T18:49:59.390 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 0, 2023-06-23T18:49:59.390 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 0, 2023-06-23T18:49:59.390 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 0, 2023-06-23T18:49:59.390 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 0, 2023-06-23T18:49:59.390 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 0, 2023-06-23T18:49:59.390 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2023-06-23T18:49:59.390 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.391 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.391 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.391 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2023-06-23T18:49:59.391 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.391 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.391 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.391 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.392 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2023-06-23T18:49:59.392 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.392 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.392 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.392 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.392 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2023-06-23T18:49:59.392 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.393 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.393 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.393 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2023-06-23T18:49:59.393 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.393 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.393 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.393 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.394 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2023-06-23T18:49:59.394 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.394 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.394 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.394 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.394 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2023-06-23T18:49:59.394 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.395 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.395 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.395 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2023-06-23T18:49:59.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.395 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.395 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.396 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2023-06-23T18:49:59.396 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.396 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.396 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.396 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.396 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2023-06-23T18:49:59.397 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.397 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.397 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.397 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.398 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2023-06-23T18:49:59.398 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.398 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.398 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.399 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.399 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2023-06-23T18:49:59.399 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.399 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.400 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.400 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.400 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2023-06-23T18:49:59.401 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.401 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.401 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.401 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.402 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2023-06-23T18:49:59.402 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.402 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.403 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.403 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.403 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2023-06-23T18:49:59.403 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.403 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.403 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.404 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.404 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2023-06-23T18:49:59.404 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.404 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.405 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.405 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.405 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2023-06-23T18:49:59.405 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.406 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.406 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.406 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.406 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2023-06-23T18:49:59.407 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.407 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.407 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.407 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.408 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2023-06-23T18:49:59.408 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.408 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.412 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.412 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.412 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2023-06-23T18:49:59.412 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.413 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.413 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.413 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.413 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2023-06-23T18:49:59.413 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.413 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.413 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.414 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.414 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2023-06-23T18:49:59.414 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.414 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.414 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.414 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.414 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2023-06-23T18:49:59.414 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.415 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.415 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.415 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.415 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2023-06-23T18:49:59.415 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.415 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.415 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.415 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.416 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2023-06-23T18:49:59.416 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.416 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.416 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.416 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.416 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2023-06-23T18:49:59.416 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.417 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.417 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.417 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.417 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2023-06-23T18:49:59.417 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.417 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.417 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2023-06-23T18:49:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2023-06-23T18:49:59.418 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2023-06-23T18:49:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2023-06-23T18:49:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2023-06-23T18:49:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2023-06-23T18:49:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2023-06-23T18:49:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 1, 2023-06-23T18:49:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2023-06-23T18:49:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2023-06-23T18:49:59.423 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2023-06-23T18:49:59.423 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 0, 2023-06-23T18:49:59.423 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 0, 2023-06-23T18:49:59.423 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 2984 2023-06-23T18:49:59.423 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.423 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2023-06-23T18:49:59.423 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:49:59.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2023-06-23T18:49:59.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:49:59.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:49:59.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:49:59.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:49:59.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:49:59.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:49:59.424 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2023-06-23T18:49:59.425 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:49:59.426 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2023-06-23T18:49:59.426 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:49:59.426 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:49:59.426 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:49:59.426 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:49:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:49:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:49:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:49:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2023-06-23T18:49:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:49:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:49:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:49:59.427 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2023-06-23T18:49:59.428 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2023-06-23T18:49:59.428 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2023-06-23T18:49:59.428 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:49:59.428 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:49:59.428 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:49:59.428 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:49:59.428 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 216759, 2023-06-23T18:49:59.428 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 70, 2023-06-23T18:49:59.429 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:49:59.429 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2023-06-23T18:49:59.429 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2023-06-23T18:49:59.429 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2023-06-23T18:49:59.429 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:49:59.429 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2023-06-23T18:49:59.429 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:49:59.429 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2023-06-23T18:49:59.430 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 7624, 2023-06-23T18:49:59.430 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 160, 2023-06-23T18:49:59.430 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:49:59.430 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:49:59.430 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2023-06-23T18:49:59.430 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2023-06-23T18:49:59.430 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 1396903, 2023-06-23T18:49:59.430 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 30239, 2023-06-23T18:49:59.431 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2023-06-23T18:49:59.431 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2023-06-23T18:49:59.431 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2023-06-23T18:49:59.431 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2023-06-23T18:49:59.431 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.431 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2023-06-23T18:49:59.431 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2023-06-23T18:49:59.431 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2023-06-23T18:49:59.432 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 5973, 2023-06-23T18:49:59.432 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 21, 2023-06-23T18:49:59.432 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 3, 2023-06-23T18:49:59.432 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 5973, 2023-06-23T18:49:59.432 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2023-06-23T18:49:59.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5973, 2023-06-23T18:49:59.432 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2.929945561, 2023-06-23T18:49:59.432 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000490531 2023-06-23T18:49:59.433 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.433 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2023-06-23T18:49:59.433 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2023-06-23T18:49:59.433 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5973, 2023-06-23T18:49:59.433 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 5974 2023-06-23T18:49:59.433 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.433 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 5973, 2023-06-23T18:49:59.434 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 5973, 2023-06-23T18:49:59.434 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 0, 2023-06-23T18:49:59.434 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2023-06-23T18:49:59.434 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2023-06-23T18:49:59.434 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 3412, 2023-06-23T18:49:59.434 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 0, 2023-06-23T18:49:59.434 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 2559, 2023-06-23T18:49:59.434 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 0, 2023-06-23T18:49:59.435 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 0, 2023-06-23T18:49:59.435 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2023-06-23T18:49:59.435 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2023-06-23T18:49:59.435 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2023-06-23T18:49:59.435 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2023-06-23T18:49:59.435 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 0, 2023-06-23T18:49:59.435 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2023-06-23T18:49:59.435 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2023-06-23T18:49:59.436 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2023-06-23T18:49:59.436 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1, 2023-06-23T18:49:59.436 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 0, 2023-06-23T18:49:59.436 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2023-06-23T18:49:59.436 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2023-06-23T18:49:59.436 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2023-06-23T18:49:59.436 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2023-06-23T18:49:59.436 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2023-06-23T18:49:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2023-06-23T18:49:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2023-06-23T18:49:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2023-06-23T18:49:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2023-06-23T18:49:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2023-06-23T18:49:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2023-06-23T18:49:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2023-06-23T18:49:59.437 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2023-06-23T18:49:59.438 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2023-06-23T18:49:59.438 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2023-06-23T18:49:59.438 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2023-06-23T18:49:59.438 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2023-06-23T18:49:59.438 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2023-06-23T18:49:59.438 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2023-06-23T18:49:59.438 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2023-06-23T18:49:59.438 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2023-06-23T18:49:59.439 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2023-06-23T18:49:59.439 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2023-06-23T18:49:59.439 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2023-06-23T18:49:59.439 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2023-06-23T18:49:59.439 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2023-06-23T18:49:59.439 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 84, 2023-06-23T18:49:59.439 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2023-06-23T18:49:59.439 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 2, 2023-06-23T18:49:59.440 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 8, 2023-06-23T18:49:59.440 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 9, 2023-06-23T18:49:59.440 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 1, 2023-06-23T18:49:59.440 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2023-06-23T18:49:59.440 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 0, 2023-06-23T18:49:59.440 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 2, 2023-06-23T18:49:59.440 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 0 2023-06-23T18:49:59.440 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.441 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2023-06-23T18:49:59.441 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 0, 2023-06-23T18:49:59.441 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2023-06-23T18:49:59.441 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2023-06-23T18:49:59.441 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2023-06-23T18:49:59.441 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2023-06-23T18:49:59.441 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 0 2023-06-23T18:49:59.442 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.442 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2023-06-23T18:49:59.442 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2023-06-23T18:49:59.442 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 0, 2023-06-23T18:49:59.442 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2023-06-23T18:49:59.442 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 0, 2023-06-23T18:49:59.442 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 0, 2023-06-23T18:49:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2023-06-23T18:49:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:49:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-06-23T18:49:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 6474, 2023-06-23T18:49:59.443 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 47510269, 2023-06-23T18:49:59.444 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:59.444 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 6474, 2023-06-23T18:49:59.444 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:49:59.444 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:49:59.444 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 6474, 2023-06-23T18:49:59.444 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 47510269, 2023-06-23T18:49:59.444 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:59.444 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.445 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.445 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.445 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.445 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.445 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2023-06-23T18:49:59.445 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:59.445 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-06-23T18:49:59.445 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:59.446 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2023-06-23T18:49:59.446 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2023-06-23T18:49:59.446 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:59.446 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:59.446 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 5973, 2023-06-23T18:49:59.446 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 45821954, 2023-06-23T18:49:59.446 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 5973, 2023-06-23T18:49:59.446 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 45821954, 2023-06-23T18:49:59.447 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:59.447 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.447 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.447 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.447 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.447 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.448 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2023-06-23T18:49:59.448 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:59.448 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2023-06-23T18:49:59.448 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:59.448 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2023-06-23T18:49:59.448 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2023-06-23T18:49:59.448 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:59.449 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:59.449 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 5973, 2023-06-23T18:49:59.449 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 5973, 2023-06-23T18:49:59.449 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 5973, 2023-06-23T18:49:59.449 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 5973, 2023-06-23T18:49:59.449 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:59.449 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.449 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.450 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.450 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.450 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2023-06-23T18:49:59.450 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:59.450 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2023-06-23T18:49:59.450 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:59.450 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2023-06-23T18:49:59.451 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2023-06-23T18:49:59.451 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:59.451 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:59.451 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:49:59.451 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:49:59.451 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2023-06-23T18:49:59.451 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2023-06-23T18:49:59.451 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:59.452 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.452 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.452 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.452 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:49:59.452 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x5650bd854000": { 2023-06-23T18:49:59.452 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:49:59.453 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2023-06-23T18:49:59.453 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:49:59.453 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2023-06-23T18:49:59.453 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2023-06-23T18:49:59.453 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:49:59.453 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2023-06-23T18:49:59.453 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:49:59.453 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:49:59.454 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 0, 2023-06-23T18:49:59.454 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 0, 2023-06-23T18:49:59.454 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:49:59.454 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:49:59.454 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:49:59.454 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:49:59.454 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.454 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:49:59.455 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-06-23T18:49:59.763 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-06-23T18:49:59.763 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-06-23T18:49:59.763 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-06-23T18:49:59.763 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-06-23T18:49:59.764 DEBUG:tasks.check_counter:Getting stats from i 2023-06-23T18:49:59.764 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.i perf dump 2023-06-23T18:50:00.321 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:50:01.142 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:00 smithi082 ceph-mon[97889]: pgmap v1135: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-06-23T18:50:01.142 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:00 smithi082 ceph-mon[97889]: overall HEALTH_OK 2023-06-23T18:50:01.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:00 smithi017 ceph-mon[97472]: pgmap v1135: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-06-23T18:50:01.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:00 smithi017 ceph-mon[97472]: overall HEALTH_OK 2023-06-23T18:50:01.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:00 smithi002 ceph-mon[95947]: pgmap v1135: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-06-23T18:50:01.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:00 smithi002 ceph-mon[95947]: overall HEALTH_OK 2023-06-23T18:50:01.628 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-06-23T18:50:01.629 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:50:01.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 29476, 2023-06-23T18:50:01.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 32977, 2023-06-23T18:50:01.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 7746970, 2023-06-23T18:50:01.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 33720180, 2023-06-23T18:50:01.629 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 27, 2023-06-23T18:50:01.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 7, 2023-06-23T18:50:01.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 1.597859888, 2023-06-23T18:50:01.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.607724493, 2023-06-23T18:50:01.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.913110395, 2023-06-23T18:50:01.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.188082423, 2023-06-23T18:50:01.630 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:50:01.630 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 32975, 2023-06-23T18:50:01.631 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.883615408, 2023-06-23T18:50:01.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000026796 2023-06-23T18:50:01.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.631 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:50:01.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 51479, 2023-06-23T18:50:01.631 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.008562304, 2023-06-23T18:50:01.631 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000166 2023-06-23T18:50:01.631 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2023-06-23T18:50:01.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2023-06-23T18:50:01.632 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.632 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:50:01.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 3343995, 2023-06-23T18:50:01.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 3344148, 2023-06-23T18:50:01.632 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 555704532, 2023-06-23T18:50:01.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 537674732, 2023-06-23T18:50:01.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 11, 2023-06-23T18:50:01.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2023-06-23T18:50:01.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 146.950486952, 2023-06-23T18:50:01.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 62.716819774, 2023-06-23T18:50:01.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 106.501454963, 2023-06-23T18:50:01.633 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.077258787, 2023-06-23T18:50:01.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:50:01.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 3344146, 2023-06-23T18:50:01.634 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 41.560660554, 2023-06-23T18:50:01.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000012427 2023-06-23T18:50:01.634 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.634 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:50:01.634 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 6685356, 2023-06-23T18:50:01.634 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.883474132, 2023-06-23T18:50:01.635 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000132 2023-06-23T18:50:01.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 160, 2023-06-23T18:50:01.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 705392 2023-06-23T18:50:01.635 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.635 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:50:01.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 50544, 2023-06-23T18:50:01.635 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 53386, 2023-06-23T18:50:01.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 16658479, 2023-06-23T18:50:01.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 39375628, 2023-06-23T18:50:01.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 12, 2023-06-23T18:50:01.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 5, 2023-06-23T18:50:01.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 2.508977564, 2023-06-23T18:50:01.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.999927040, 2023-06-23T18:50:01.636 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 1.763010917, 2023-06-23T18:50:01.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.045428161, 2023-06-23T18:50:01.637 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:50:01.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 53386, 2023-06-23T18:50:01.637 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 4.364601364, 2023-06-23T18:50:01.637 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000081755 2023-06-23T18:50:01.637 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:50:01.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 96866, 2023-06-23T18:50:01.638 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.016706872, 2023-06-23T18:50:01.638 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000172 2023-06-23T18:50:01.638 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 655968, 2023-06-23T18:50:01.638 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 287232 2023-06-23T18:50:01.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.639 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2023-06-23T18:50:01.639 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 1, 2023-06-23T18:50:01.639 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2023-06-23T18:50:01.639 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.639 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-MDSRank": { 2023-06-23T18:50:01.639 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2023-06-23T18:50:01.639 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2023-06-23T18:50:01.640 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 12480, 2023-06-23T18:50:01.640 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 9.155565142, 2023-06-23T18:50:01.640 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000733619 2023-06-23T18:50:01.640 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.641 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.641 INFO:teuthology.orchestra.run.smithi082.stdout: "finisher-PurgeQueue": { 2023-06-23T18:50:01.641 INFO:teuthology.orchestra.run.smithi082.stdout: "queue_len": 0, 2023-06-23T18:50:01.641 INFO:teuthology.orchestra.run.smithi082.stdout: "complete_latency": { 2023-06-23T18:50:01.641 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 27, 2023-06-23T18:50:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000599695, 2023-06-23T18:50:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000022210 2023-06-23T18:50:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": { 2023-06-23T18:50:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: "request": 19994, 2023-06-23T18:50:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: "reply": 19159, 2023-06-23T18:50:01.642 INFO:teuthology.orchestra.run.smithi082.stdout: "reply_latency": { 2023-06-23T18:50:01.643 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 19159, 2023-06-23T18:50:01.643 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 48.882145860, 2023-06-23T18:50:01.643 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002551393 2023-06-23T18:50:01.643 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.643 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_reply": 0, 2023-06-23T18:50:01.643 INFO:teuthology.orchestra.run.smithi082.stdout: "forward": 835, 2023-06-23T18:50:01.643 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_complete": 0, 2023-06-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_fetch_keys": 0, 2023-06-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_commit": 11, 2023-06-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_split": 0, 2023-06-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_merge": 0, 2023-06-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes": 1309, 2023-06-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_top": 3, 2023-06-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_bottom": 729, 2023-06-23T18:50:01.644 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pin_tail": 577, 2023-06-23T18:50:01.645 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_pinned": 1304, 2023-06-23T18:50:01.645 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_expired": 298, 2023-06-23T18:50:01.645 INFO:teuthology.orchestra.run.smithi082.stdout: "inodes_with_caps": 201, 2023-06-23T18:50:01.645 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": 201, 2023-06-23T18:50:01.645 INFO:teuthology.orchestra.run.smithi082.stdout: "subtrees": 9, 2023-06-23T18:50:01.645 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse": 79060, 2023-06-23T18:50:01.645 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_hit": 65944, 2023-06-23T18:50:01.645 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_forward": 835, 2023-06-23T18:50:01.646 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_discover": 24, 2023-06-23T18:50:01.646 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_dir_fetch": 0, 2023-06-23T18:50:01.646 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_remote_ino": 0, 2023-06-23T18:50:01.646 INFO:teuthology.orchestra.run.smithi082.stdout: "traverse_lock": 5022, 2023-06-23T18:50:01.646 INFO:teuthology.orchestra.run.smithi082.stdout: "load_cent": 278, 2023-06-23T18:50:01.646 INFO:teuthology.orchestra.run.smithi082.stdout: "q": 0, 2023-06-23T18:50:01.646 INFO:teuthology.orchestra.run.smithi082.stdout: "exported": 12, 2023-06-23T18:50:01.646 INFO:teuthology.orchestra.run.smithi082.stdout: "exported_inodes": 292, 2023-06-23T18:50:01.647 INFO:teuthology.orchestra.run.smithi082.stdout: "imported": 16, 2023-06-23T18:50:01.647 INFO:teuthology.orchestra.run.smithi082.stdout: "imported_inodes": 433, 2023-06-23T18:50:01.647 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_dir_fetch": 0, 2023-06-23T18:50:01.647 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_backtrace_fetch": 1034, 2023-06-23T18:50:01.647 INFO:teuthology.orchestra.run.smithi082.stdout: "openino_peer_discover": 0, 2023-06-23T18:50:01.647 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rfiles": 1046, 2023-06-23T18:50:01.647 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rbytes": 8926501484, 2023-06-23T18:50:01.648 INFO:teuthology.orchestra.run.smithi082.stdout: "root_rsnaps": 0, 2023-06-23T18:50:01.648 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_fetch": 1034, 2023-06-23T18:50:01.648 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_set_tag": 0, 2023-06-23T18:50:01.648 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_backtrace_repaired": 0, 2023-06-23T18:50:01.648 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_inotable_repaired": 0, 2023-06-23T18:50:01.648 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_inodes": 18, 2023-06-23T18:50:01.648 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dir_base_inodes": 0, 2023-06-23T18:50:01.649 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_dirfrag_rstats": 18, 2023-06-23T18:50:01.649 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_file_inodes": 1016, 2023-06-23T18:50:01.649 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_inode_file_caps": 32, 2023-06-23T18:50:01.649 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_revoke": 2533, 2023-06-23T18:50:01.649 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_grant": 2316, 2023-06-23T18:50:01.649 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_trunc": 0, 2023-06-23T18:50:01.649 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-06-23T18:50:01.649 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_cap_op_flush_ack": 0, 2023-06-23T18:50:01.650 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps": 30805, 2023-06-23T18:50:01.650 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_caps_dirty": 26679, 2023-06-23T18:50:01.650 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_cap_release": 2, 2023-06-23T18:50:01.650 INFO:teuthology.orchestra.run.smithi082.stdout: "process_request_cap_release": 2050 2023-06-23T18:50:01.651 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.651 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_cache": { 2023-06-23T18:50:01.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update": 286, 2023-06-23T18:50:01.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_update_receipt": 147, 2023-06-23T18:50:01.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_try_discover": 85, 2023-06-23T18:50:01.651 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_send_discover": 91, 2023-06-23T18:50:01.652 INFO:teuthology.orchestra.run.smithi082.stdout: "dir_handle_discover": 312, 2023-06-23T18:50:01.652 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays": 0, 2023-06-23T18:50:01.652 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_delayed": 0, 2023-06-23T18:50:01.652 INFO:teuthology.orchestra.run.smithi082.stdout: "num_strays_enqueuing": 0, 2023-06-23T18:50:01.652 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_created": 685, 2023-06-23T18:50:01.652 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_enqueued": 2, 2023-06-23T18:50:01.652 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_reintegrated": 2705, 2023-06-23T18:50:01.653 INFO:teuthology.orchestra.run.smithi082.stdout: "strays_migrated": 0, 2023-06-23T18:50:01.653 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_processing": 0, 2023-06-23T18:50:01.653 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_enqueued": 0, 2023-06-23T18:50:01.653 INFO:teuthology.orchestra.run.smithi082.stdout: "num_recovering_prioritized": 0, 2023-06-23T18:50:01.653 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_started": 0, 2023-06-23T18:50:01.653 INFO:teuthology.orchestra.run.smithi082.stdout: "recovery_completed": 0, 2023-06-23T18:50:01.653 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_enqueue_scrub": 0, 2023-06-23T18:50:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_exportdir": 12, 2023-06-23T18:50:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_flush": 0, 2023-06-23T18:50:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragmentdir": 0, 2023-06-23T18:50:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_fragstats": 0, 2023-06-23T18:50:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "ireq_inodestats": 0 2023-06-23T18:50:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_log": { 2023-06-23T18:50:01.654 INFO:teuthology.orchestra.run.smithi082.stdout: "evadd": 35151, 2023-06-23T18:50:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "evex": 0, 2023-06-23T18:50:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "evtrm": 0, 2023-06-23T18:50:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "ev": 35151, 2023-06-23T18:50:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "evexg": 0, 2023-06-23T18:50:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "evexd": 0, 2023-06-23T18:50:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "segadd": 41, 2023-06-23T18:50:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "segex": 0, 2023-06-23T18:50:01.655 INFO:teuthology.orchestra.run.smithi082.stdout: "segtrm": 0, 2023-06-23T18:50:01.656 INFO:teuthology.orchestra.run.smithi082.stdout: "seg": 41, 2023-06-23T18:50:01.656 INFO:teuthology.orchestra.run.smithi082.stdout: "segexg": 0, 2023-06-23T18:50:01.656 INFO:teuthology.orchestra.run.smithi082.stdout: "segexd": 0, 2023-06-23T18:50:01.656 INFO:teuthology.orchestra.run.smithi082.stdout: "expos": 4194304, 2023-06-23T18:50:01.656 INFO:teuthology.orchestra.run.smithi082.stdout: "wrpos": 50006999, 2023-06-23T18:50:01.656 INFO:teuthology.orchestra.run.smithi082.stdout: "rdpos": 0, 2023-06-23T18:50:01.656 INFO:teuthology.orchestra.run.smithi082.stdout: "jlat": { 2023-06-23T18:50:01.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 5918, 2023-06-23T18:50:01.657 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 16.090563354, 2023-06-23T18:50:01.657 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002718919 2023-06-23T18:50:01.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.657 INFO:teuthology.orchestra.run.smithi082.stdout: "replayed": 0 2023-06-23T18:50:01.657 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.657 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_mem": { 2023-06-23T18:50:01.657 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 222, 2023-06-23T18:50:01.658 INFO:teuthology.orchestra.run.smithi082.stdout: "ino+": 516, 2023-06-23T18:50:01.658 INFO:teuthology.orchestra.run.smithi082.stdout: "ino-": 294, 2023-06-23T18:50:01.658 INFO:teuthology.orchestra.run.smithi082.stdout: "dir": 53, 2023-06-23T18:50:01.658 INFO:teuthology.orchestra.run.smithi082.stdout: "dir+": 82, 2023-06-23T18:50:01.658 INFO:teuthology.orchestra.run.smithi082.stdout: "dir-": 29, 2023-06-23T18:50:01.658 INFO:teuthology.orchestra.run.smithi082.stdout: "dn": 1309, 2023-06-23T18:50:01.658 INFO:teuthology.orchestra.run.smithi082.stdout: "dn+": 1615, 2023-06-23T18:50:01.659 INFO:teuthology.orchestra.run.smithi082.stdout: "dn-": 306, 2023-06-23T18:50:01.659 INFO:teuthology.orchestra.run.smithi082.stdout: "cap": 201, 2023-06-23T18:50:01.659 INFO:teuthology.orchestra.run.smithi082.stdout: "cap+": 489, 2023-06-23T18:50:01.659 INFO:teuthology.orchestra.run.smithi082.stdout: "cap-": 288, 2023-06-23T18:50:01.659 INFO:teuthology.orchestra.run.smithi082.stdout: "rss": 62856, 2023-06-23T18:50:01.659 INFO:teuthology.orchestra.run.smithi082.stdout: "heap": 198964 2023-06-23T18:50:01.659 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.660 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_server": { 2023-06-23T18:50:01.660 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_client_request": 47515, 2023-06-23T18:50:01.660 INFO:teuthology.orchestra.run.smithi082.stdout: "dispatch_server_request": 137, 2023-06-23T18:50:01.660 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_request": 19994, 2023-06-23T18:50:01.660 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_client_session": 31, 2023-06-23T18:50:01.660 INFO:teuthology.orchestra.run.smithi082.stdout: "handle_peer_request": 4391, 2023-06-23T18:50:01.660 INFO:teuthology.orchestra.run.smithi082.stdout: "req_create_latency": { 2023-06-23T18:50:01.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 59, 2023-06-23T18:50:01.661 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.132865550, 2023-06-23T18:50:01.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002251958 2023-06-23T18:50:01.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.661 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getattr_latency": { 2023-06-23T18:50:01.661 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 24, 2023-06-23T18:50:01.662 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.025175200, 2023-06-23T18:50:01.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001048966 2023-06-23T18:50:01.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.662 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getfilelock_latency": { 2023-06-23T18:50:01.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.662 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.662 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.663 INFO:teuthology.orchestra.run.smithi082.stdout: "req_link_latency": { 2023-06-23T18:50:01.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 683, 2023-06-23T18:50:01.663 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2.419954266, 2023-06-23T18:50:01.663 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003543124 2023-06-23T18:50:01.663 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.664 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookup_latency": { 2023-06-23T18:50:01.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 12497, 2023-06-23T18:50:01.664 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 6.465579777, 2023-06-23T18:50:01.664 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000517370 2023-06-23T18:50:01.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.665 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookuphash_latency": { 2023-06-23T18:50:01.665 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.665 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.665 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.665 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.666 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupino_latency": { 2023-06-23T18:50:01.666 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.666 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.666 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.666 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.666 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupname_latency": { 2023-06-23T18:50:01.666 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.667 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.667 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.667 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupparent_latency": { 2023-06-23T18:50:01.667 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.667 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.668 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.668 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lookupsnap_latency": { 2023-06-23T18:50:01.668 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.668 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.668 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.668 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.669 INFO:teuthology.orchestra.run.smithi082.stdout: "req_lssnap_latency": { 2023-06-23T18:50:01.669 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.669 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.669 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.669 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.669 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mkdir_latency": { 2023-06-23T18:50:01.669 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.669 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.670 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.670 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.670 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mknod_latency": { 2023-06-23T18:50:01.670 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.670 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.670 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.670 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.670 INFO:teuthology.orchestra.run.smithi082.stdout: "req_mksnap_latency": { 2023-06-23T18:50:01.671 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.671 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.671 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.671 INFO:teuthology.orchestra.run.smithi082.stdout: "req_open_latency": { 2023-06-23T18:50:01.671 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2444, 2023-06-23T18:50:01.671 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 1.588607712, 2023-06-23T18:50:01.672 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000650003 2023-06-23T18:50:01.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.672 INFO:teuthology.orchestra.run.smithi082.stdout: "req_readdir_latency": { 2023-06-23T18:50:01.672 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 62, 2023-06-23T18:50:01.672 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.062455317, 2023-06-23T18:50:01.672 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.001007343 2023-06-23T18:50:01.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.673 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rename_latency": { 2023-06-23T18:50:01.673 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 2705, 2023-06-23T18:50:01.673 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 35.942803464, 2023-06-23T18:50:01.673 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.013287542 2023-06-23T18:50:01.673 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.673 INFO:teuthology.orchestra.run.smithi082.stdout: "req_renamesnap_latency": { 2023-06-23T18:50:01.673 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.674 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.674 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.674 INFO:teuthology.orchestra.run.smithi082.stdout: "req_snapdiff_latency": { 2023-06-23T18:50:01.674 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.674 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.674 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.675 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmdir_latency": { 2023-06-23T18:50:01.675 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.675 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.675 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.675 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmsnap_latency": { 2023-06-23T18:50:01.675 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.676 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.676 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.676 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.676 INFO:teuthology.orchestra.run.smithi082.stdout: "req_rmxattr_latency": { 2023-06-23T18:50:01.676 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.676 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.676 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.677 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setattr_latency": { 2023-06-23T18:50:01.677 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.677 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.677 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.677 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setdirlayout_latency": { 2023-06-23T18:50:01.678 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.678 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.678 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.678 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.678 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setfilelock_latency": { 2023-06-23T18:50:01.678 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.678 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.678 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.679 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setlayout_latency": { 2023-06-23T18:50:01.679 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.679 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.679 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.679 INFO:teuthology.orchestra.run.smithi082.stdout: "req_setxattr_latency": { 2023-06-23T18:50:01.679 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.680 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.680 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.680 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.680 INFO:teuthology.orchestra.run.smithi082.stdout: "req_symlink_latency": { 2023-06-23T18:50:01.680 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.680 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.680 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.680 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.681 INFO:teuthology.orchestra.run.smithi082.stdout: "req_unlink_latency": { 2023-06-23T18:50:01.681 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 685, 2023-06-23T18:50:01.681 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 2.244704574, 2023-06-23T18:50:01.681 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.003276940 2023-06-23T18:50:01.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.681 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_revoke_eviction": 0, 2023-06-23T18:50:01.681 INFO:teuthology.orchestra.run.smithi082.stdout: "cap_acquisition_throttle": 0, 2023-06-23T18:50:01.681 INFO:teuthology.orchestra.run.smithi082.stdout: "req_getvxattr_latency": { 2023-06-23T18:50:01.682 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.682 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.682 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.682 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.682 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.682 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_sessions": { 2023-06-23T18:50:01.682 INFO:teuthology.orchestra.run.smithi082.stdout: "session_count": 1, 2023-06-23T18:50:01.682 INFO:teuthology.orchestra.run.smithi082.stdout: "session_add": 1, 2023-06-23T18:50:01.683 INFO:teuthology.orchestra.run.smithi082.stdout: "session_remove": 0, 2023-06-23T18:50:01.683 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_open": 1, 2023-06-23T18:50:01.683 INFO:teuthology.orchestra.run.smithi082.stdout: "sessions_stale": 0, 2023-06-23T18:50:01.683 INFO:teuthology.orchestra.run.smithi082.stdout: "total_load": 1434, 2023-06-23T18:50:01.683 INFO:teuthology.orchestra.run.smithi082.stdout: "average_load": 1434, 2023-06-23T18:50:01.683 INFO:teuthology.orchestra.run.smithi082.stdout: "avg_session_uptime": 1671 2023-06-23T18:50:01.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.684 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2023-06-23T18:50:01.684 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:50:01.684 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2023-06-23T18:50:01.684 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:50:01.684 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:50:01.685 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:50:01.685 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:50:01.685 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:50:01.685 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:50:01.685 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:50:01.685 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:50:01.685 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:50:01.686 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:50:01.686 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:50:01.686 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:50:01.686 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:50:01.686 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2023-06-23T18:50:01.686 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:50:01.686 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2023-06-23T18:50:01.686 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:50:01.687 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:50:01.687 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:50:01.687 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:50:01.687 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:50:01.687 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:50:01.687 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:50:01.687 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2023-06-23T18:50:01.688 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:50:01.688 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:50:01.688 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:50:01.688 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2023-06-23T18:50:01.688 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2023-06-23T18:50:01.688 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2023-06-23T18:50:01.688 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:50:01.688 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:50:01.689 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:50:01.689 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:50:01.689 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 295595, 2023-06-23T18:50:01.689 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 106, 2023-06-23T18:50:01.689 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:50:01.689 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2023-06-23T18:50:01.690 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2023-06-23T18:50:01.690 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2023-06-23T18:50:01.690 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:50:01.690 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2023-06-23T18:50:01.690 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:50:01.690 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2023-06-23T18:50:01.690 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 7624, 2023-06-23T18:50:01.690 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 160, 2023-06-23T18:50:01.691 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:50:01.691 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:50:01.691 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2023-06-23T18:50:01.691 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2023-06-23T18:50:01.691 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 1613775, 2023-06-23T18:50:01.691 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 39728, 2023-06-23T18:50:01.691 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2023-06-23T18:50:01.691 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2023-06-23T18:50:01.692 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2023-06-23T18:50:01.692 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2023-06-23T18:50:01.692 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.692 INFO:teuthology.orchestra.run.smithi082.stdout: "objecter": { 2023-06-23T18:50:01.692 INFO:teuthology.orchestra.run.smithi082.stdout: "op_active": 0, 2023-06-23T18:50:01.692 INFO:teuthology.orchestra.run.smithi082.stdout: "op_laggy": 0, 2023-06-23T18:50:01.692 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send": 7412, 2023-06-23T18:50:01.693 INFO:teuthology.orchestra.run.smithi082.stdout: "op_send_bytes": 45936217, 2023-06-23T18:50:01.693 INFO:teuthology.orchestra.run.smithi082.stdout: "op_resend": 28, 2023-06-23T18:50:01.693 INFO:teuthology.orchestra.run.smithi082.stdout: "op_reply": 7412, 2023-06-23T18:50:01.693 INFO:teuthology.orchestra.run.smithi082.stdout: "op_latency": { 2023-06-23T18:50:01.693 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 7412, 2023-06-23T18:50:01.693 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 17.658672118, 2023-06-23T18:50:01.693 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.002382443 2023-06-23T18:50:01.693 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.694 INFO:teuthology.orchestra.run.smithi082.stdout: "op_inflight": 0, 2023-06-23T18:50:01.694 INFO:teuthology.orchestra.run.smithi082.stdout: "oplen_avg": { 2023-06-23T18:50:01.694 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 7412, 2023-06-23T18:50:01.694 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 7969 2023-06-23T18:50:01.694 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.694 INFO:teuthology.orchestra.run.smithi082.stdout: "op": 7412, 2023-06-23T18:50:01.694 INFO:teuthology.orchestra.run.smithi082.stdout: "op_r": 1036, 2023-06-23T18:50:01.694 INFO:teuthology.orchestra.run.smithi082.stdout: "op_w": 6376, 2023-06-23T18:50:01.695 INFO:teuthology.orchestra.run.smithi082.stdout: "op_rmw": 0, 2023-06-23T18:50:01.695 INFO:teuthology.orchestra.run.smithi082.stdout: "op_pg": 0, 2023-06-23T18:50:01.695 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_stat": 0, 2023-06-23T18:50:01.695 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_create": 10, 2023-06-23T18:50:01.695 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_read": 2, 2023-06-23T18:50:01.696 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_write": 5920, 2023-06-23T18:50:01.696 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writefull": 110, 2023-06-23T18:50:01.696 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_writesame": 0, 2023-06-23T18:50:01.696 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_append": 0, 2023-06-23T18:50:01.696 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_zero": 0, 2023-06-23T18:50:01.696 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_truncate": 0, 2023-06-23T18:50:01.696 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_delete": 27, 2023-06-23T18:50:01.696 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_mapext": 0, 2023-06-23T18:50:01.697 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_sparse_read": 0, 2023-06-23T18:50:01.697 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_clonerange": 0, 2023-06-23T18:50:01.697 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_getxattr": 1034, 2023-06-23T18:50:01.697 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_setxattr": 20, 2023-06-23T18:50:01.697 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_cmpxattr": 0, 2023-06-23T18:50:01.697 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_rmxattr": 0, 2023-06-23T18:50:01.697 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_resetxattrs": 0, 2023-06-23T18:50:01.698 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_call": 0, 2023-06-23T18:50:01.698 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_watch": 0, 2023-06-23T18:50:01.698 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_notify": 0, 2023-06-23T18:50:01.698 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_src_cmpxattr": 0, 2023-06-23T18:50:01.698 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls": 0, 2023-06-23T18:50:01.698 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_pgls_filter": 0, 2023-06-23T18:50:01.698 INFO:teuthology.orchestra.run.smithi082.stdout: "osdop_other": 0, 2023-06-23T18:50:01.698 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_active": 0, 2023-06-23T18:50:01.699 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_send": 0, 2023-06-23T18:50:01.699 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_resend": 0, 2023-06-23T18:50:01.699 INFO:teuthology.orchestra.run.smithi082.stdout: "linger_ping": 0, 2023-06-23T18:50:01.699 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_active": 0, 2023-06-23T18:50:01.699 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_send": 0, 2023-06-23T18:50:01.699 INFO:teuthology.orchestra.run.smithi082.stdout: "poolop_resend": 0, 2023-06-23T18:50:01.699 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_active": 0, 2023-06-23T18:50:01.699 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_send": 0, 2023-06-23T18:50:01.700 INFO:teuthology.orchestra.run.smithi082.stdout: "poolstat_resend": 0, 2023-06-23T18:50:01.700 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_active": 0, 2023-06-23T18:50:01.700 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_send": 0, 2023-06-23T18:50:01.700 INFO:teuthology.orchestra.run.smithi082.stdout: "statfs_resend": 0, 2023-06-23T18:50:01.700 INFO:teuthology.orchestra.run.smithi082.stdout: "command_active": 0, 2023-06-23T18:50:01.700 INFO:teuthology.orchestra.run.smithi082.stdout: "command_send": 0, 2023-06-23T18:50:01.700 INFO:teuthology.orchestra.run.smithi082.stdout: "command_resend": 0, 2023-06-23T18:50:01.700 INFO:teuthology.orchestra.run.smithi082.stdout: "map_epoch": 84, 2023-06-23T18:50:01.701 INFO:teuthology.orchestra.run.smithi082.stdout: "map_full": 0, 2023-06-23T18:50:01.701 INFO:teuthology.orchestra.run.smithi082.stdout: "map_inc": 2, 2023-06-23T18:50:01.701 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_sessions": 12, 2023-06-23T18:50:01.701 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_open": 12, 2023-06-23T18:50:01.701 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_session_close": 0, 2023-06-23T18:50:01.701 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_laggy": 0, 2023-06-23T18:50:01.701 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_wr": 571, 2023-06-23T18:50:01.701 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_rd": 0, 2023-06-23T18:50:01.702 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_del": 275 2023-06-23T18:50:01.702 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.702 INFO:teuthology.orchestra.run.smithi082.stdout: "oft": { 2023-06-23T18:50:01.702 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_objs": 1, 2023-06-23T18:50:01.702 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_kv_pairs": 0, 2023-06-23T18:50:01.702 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_updates": 0, 2023-06-23T18:50:01.702 INFO:teuthology.orchestra.run.smithi082.stdout: "omap_total_removes": 0 2023-06-23T18:50:01.702 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.703 INFO:teuthology.orchestra.run.smithi082.stdout: "osd-slow-ops": { 2023-06-23T18:50:01.703 INFO:teuthology.orchestra.run.smithi082.stdout: "slow_ops_count": 20143 2023-06-23T18:50:01.703 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.703 INFO:teuthology.orchestra.run.smithi082.stdout: "purge_queue": { 2023-06-23T18:50:01.703 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops": 0, 2023-06-23T18:50:01.703 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_ops_high_water": 4, 2023-06-23T18:50:01.703 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing": 0, 2023-06-23T18:50:01.703 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executing_high_water": 1, 2023-06-23T18:50:01.704 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_executed": 2, 2023-06-23T18:50:01.704 INFO:teuthology.orchestra.run.smithi082.stdout: "pq_item_in_journal": 0 2023-06-23T18:50:01.704 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.704 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:50:01.704 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:50:01.704 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-06-23T18:50:01.704 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:50:01.705 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 3424011, 2023-06-23T18:50:01.705 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 271918150, 2023-06-23T18:50:01.705 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:50:01.705 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 3424011, 2023-06-23T18:50:01.705 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:50:01.705 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:50:01.705 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 3424011, 2023-06-23T18:50:01.706 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 271918150, 2023-06-23T18:50:01.706 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:50:01.706 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.706 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.707 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.707 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.707 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.707 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_bytes": { 2023-06-23T18:50:01.707 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:50:01.707 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-06-23T18:50:01.707 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:50:01.708 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2023-06-23T18:50:01.708 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2023-06-23T18:50:01.708 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:50:01.708 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2023-06-23T18:50:01.708 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 7412, 2023-06-23T18:50:01.708 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 45936419, 2023-06-23T18:50:01.708 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 7412, 2023-06-23T18:50:01.709 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 45936419, 2023-06-23T18:50:01.709 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:50:01.709 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.709 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.709 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.709 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.709 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.709 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-objecter_ops": { 2023-06-23T18:50:01.710 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:50:01.710 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 1024, 2023-06-23T18:50:01.710 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:50:01.710 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 0, 2023-06-23T18:50:01.710 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 0, 2023-06-23T18:50:01.710 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:50:01.710 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 0, 2023-06-23T18:50:01.710 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 7412, 2023-06-23T18:50:01.711 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 7412, 2023-06-23T18:50:01.711 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 7412, 2023-06-23T18:50:01.711 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 7412, 2023-06-23T18:50:01.711 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:50:01.711 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.711 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.711 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.712 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.712 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.712 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle": { 2023-06-23T18:50:01.712 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:50:01.712 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2023-06-23T18:50:01.712 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:50:01.712 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 2, 2023-06-23T18:50:01.713 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 202, 2023-06-23T18:50:01.713 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:50:01.713 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 2, 2023-06-23T18:50:01.713 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:50:01.713 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:50:01.713 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 2, 2023-06-23T18:50:01.713 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 202, 2023-06-23T18:50:01.713 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:50:01.714 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.714 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.714 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.714 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.714 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:01.714 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-write_buf_throttle-0x55a46a44fcc0": { 2023-06-23T18:50:01.714 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:50:01.714 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 3758096384, 2023-06-23T18:50:01.715 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:50:01.715 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 35151, 2023-06-23T18:50:01.715 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 45812695, 2023-06-23T18:50:01.715 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:50:01.715 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 35151, 2023-06-23T18:50:01.715 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:50:01.715 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:50:01.715 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 5918, 2023-06-23T18:50:01.716 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 45812695, 2023-06-23T18:50:01.716 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:50:01.716 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:01.716 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:01.716 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:01.716 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.716 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:01.717 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-06-23T18:50:02.070 INFO:tasks.check_counter:Daemon mds.i mds.exported=12 2023-06-23T18:50:02.070 INFO:tasks.check_counter:Daemon mds.i mds.imported=16 2023-06-23T18:50:02.070 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=286 2023-06-23T18:50:02.070 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=147 2023-06-23T18:50:02.071 DEBUG:tasks.check_counter:Getting stats from l 2023-06-23T18:50:02.071 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8627af3e0adcb765a3c249fcc209cba9f4873e1b shell --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c -- ceph daemon mds.l perf dump 2023-06-23T18:50:02.624 INFO:teuthology.orchestra.run.smithi082.stderr:Inferring config /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/mon.c/config 2023-06-23T18:50:03.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:02 smithi082 ceph-mon[97889]: pgmap v1136: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:50:03.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:02 smithi017 ceph-mon[97472]: pgmap v1136: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:50:03.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:02 smithi002 ceph-mon[95947]: pgmap v1136: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:50:03.863 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2023-06-23T18:50:03.863 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-0": { 2023-06-23T18:50:03.864 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 0, 2023-06-23T18:50:03.864 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 2, 2023-06-23T18:50:03.864 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 0, 2023-06-23T18:50:03.864 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 0, 2023-06-23T18:50:03.864 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2023-06-23T18:50:03.864 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 0, 2023-06-23T18:50:03.864 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.001027434, 2023-06-23T18:50:03.865 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.000000000, 2023-06-23T18:50:03.865 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.000172985, 2023-06-23T18:50:03.865 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:50:03.865 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:50:03.865 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:03.865 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:03.865 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:03.866 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.866 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:50:03.866 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:03.866 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:03.866 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:03.866 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.866 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 0, 2023-06-23T18:50:03.866 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 0 2023-06-23T18:50:03.867 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.867 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-1": { 2023-06-23T18:50:03.867 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 480, 2023-06-23T18:50:03.867 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 475, 2023-06-23T18:50:03.867 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 548160, 2023-06-23T18:50:03.867 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 272304, 2023-06-23T18:50:03.867 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 2, 2023-06-23T18:50:03.868 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 1, 2023-06-23T18:50:03.868 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.056431597, 2023-06-23T18:50:03.868 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.021624246, 2023-06-23T18:50:03.868 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.020151000, 2023-06-23T18:50:03.868 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.011375899, 2023-06-23T18:50:03.868 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:50:03.868 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 475, 2023-06-23T18:50:03.868 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.017283076, 2023-06-23T18:50:03.869 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000036385 2023-06-23T18:50:03.869 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.869 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:50:03.869 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:03.869 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:03.869 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:03.870 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.870 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 548160, 2023-06-23T18:50:03.870 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 272304 2023-06-23T18:50:03.870 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.870 INFO:teuthology.orchestra.run.smithi082.stdout: "AsyncMessenger::Worker-2": { 2023-06-23T18:50:03.870 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_messages": 2, 2023-06-23T18:50:03.870 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages": 362, 2023-06-23T18:50:03.870 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_bytes": 320, 2023-06-23T18:50:03.871 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_bytes": 215104, 2023-06-23T18:50:03.871 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_created_connections": 4, 2023-06-23T18:50:03.871 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_active_connections": 2, 2023-06-23T18:50:03.871 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_total_time": 0.014737559, 2023-06-23T18:50:03.871 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_send_time": 0.010953712, 2023-06-23T18:50:03.871 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_recv_time": 0.001159750, 2023-06-23T18:50:03.872 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-06-23T18:50:03.872 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_messages_queue_lat": { 2023-06-23T18:50:03.872 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 360, 2023-06-23T18:50:03.872 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.018262659, 2023-06-23T18:50:03.872 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000050729 2023-06-23T18:50:03.872 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.872 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_handle_ack_lat": { 2023-06-23T18:50:03.872 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:03.873 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:03.873 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:03.873 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.873 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_recv_encrypted_bytes": 320, 2023-06-23T18:50:03.873 INFO:teuthology.orchestra.run.smithi082.stdout: "msgr_send_encrypted_bytes": 215104 2023-06-23T18:50:03.873 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi082.stdout: "cct": { 2023-06-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi082.stdout: "total_workers": 0, 2023-06-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi082.stdout: "unhealthy_workers": 0 2023-06-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi082.stdout: "mempool": { 2023-06-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_bytes": 0, 2023-06-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi082.stdout: "bloom_filter_items": 0, 2023-06-23T18:50:03.874 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_bytes": 0, 2023-06-23T18:50:03.875 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_alloc_items": 0, 2023-06-23T18:50:03.875 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_bytes": 0, 2023-06-23T18:50:03.875 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_data_items": 0, 2023-06-23T18:50:03.875 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_bytes": 0, 2023-06-23T18:50:03.875 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_onode_items": 0, 2023-06-23T18:50:03.875 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_bytes": 0, 2023-06-23T18:50:03.875 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_meta_items": 0, 2023-06-23T18:50:03.875 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_bytes": 0, 2023-06-23T18:50:03.876 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_other_items": 0, 2023-06-23T18:50:03.876 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_bytes": 0, 2023-06-23T18:50:03.876 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_cache_buffer_items": 0, 2023-06-23T18:50:03.876 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_bytes": 0, 2023-06-23T18:50:03.876 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_extent_items": 0, 2023-06-23T18:50:03.876 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_bytes": 0, 2023-06-23T18:50:03.876 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_blob_items": 0, 2023-06-23T18:50:03.876 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_bytes": 0, 2023-06-23T18:50:03.877 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_shared_blob_items": 0, 2023-06-23T18:50:03.877 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_bytes": 0, 2023-06-23T18:50:03.877 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_inline_bl_items": 0, 2023-06-23T18:50:03.877 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_bytes": 0, 2023-06-23T18:50:03.877 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_fsck_items": 0, 2023-06-23T18:50:03.877 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_bytes": 0, 2023-06-23T18:50:03.877 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_txc_items": 0, 2023-06-23T18:50:03.878 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_bytes": 0, 2023-06-23T18:50:03.878 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_deferred_items": 0, 2023-06-23T18:50:03.878 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_bytes": 0, 2023-06-23T18:50:03.878 INFO:teuthology.orchestra.run.smithi082.stdout: "bluestore_writing_items": 0, 2023-06-23T18:50:03.878 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_bytes": 0, 2023-06-23T18:50:03.878 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_items": 0, 2023-06-23T18:50:03.878 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_bytes": 0, 2023-06-23T18:50:03.879 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_reader_items": 0, 2023-06-23T18:50:03.879 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_bytes": 0, 2023-06-23T18:50:03.879 INFO:teuthology.orchestra.run.smithi082.stdout: "bluefs_file_writer_items": 0, 2023-06-23T18:50:03.879 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_bytes": 148722, 2023-06-23T18:50:03.879 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_anon_items": 30, 2023-06-23T18:50:03.879 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_bytes": 0, 2023-06-23T18:50:03.879 INFO:teuthology.orchestra.run.smithi082.stdout: "buffer_meta_items": 0, 2023-06-23T18:50:03.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_bytes": 0, 2023-06-23T18:50:03.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_items": 0, 2023-06-23T18:50:03.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_bytes": 0, 2023-06-23T18:50:03.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_mapbl_items": 0, 2023-06-23T18:50:03.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_bytes": 0, 2023-06-23T18:50:03.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_pglog_items": 0, 2023-06-23T18:50:03.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_bytes": 0, 2023-06-23T18:50:03.880 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_items": 0, 2023-06-23T18:50:03.881 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_bytes": 0, 2023-06-23T18:50:03.881 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_mapping_items": 0, 2023-06-23T18:50:03.881 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_bytes": 0, 2023-06-23T18:50:03.881 INFO:teuthology.orchestra.run.smithi082.stdout: "pgmap_items": 0, 2023-06-23T18:50:03.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_bytes": 928, 2023-06-23T18:50:03.881 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_co_items": 2, 2023-06-23T18:50:03.881 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_bytes": 0, 2023-06-23T18:50:03.882 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_1_items": 0, 2023-06-23T18:50:03.882 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_bytes": 0, 2023-06-23T18:50:03.882 INFO:teuthology.orchestra.run.smithi082.stdout: "unittest_2_items": 0 2023-06-23T18:50:03.882 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2023-06-23T18:50:03.882 INFO:teuthology.orchestra.run.smithi082.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-06-23T18:50:03.882 INFO:teuthology.orchestra.run.smithi082.stdout: "val": 0, 2023-06-23T18:50:03.882 INFO:teuthology.orchestra.run.smithi082.stdout: "max": 104857600, 2023-06-23T18:50:03.883 INFO:teuthology.orchestra.run.smithi082.stdout: "get_started": 0, 2023-06-23T18:50:03.883 INFO:teuthology.orchestra.run.smithi082.stdout: "get": 478, 2023-06-23T18:50:03.883 INFO:teuthology.orchestra.run.smithi082.stdout: "get_sum": 477837, 2023-06-23T18:50:03.883 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_fail": 0, 2023-06-23T18:50:03.883 INFO:teuthology.orchestra.run.smithi082.stdout: "get_or_fail_success": 478, 2023-06-23T18:50:03.883 INFO:teuthology.orchestra.run.smithi082.stdout: "take": 0, 2023-06-23T18:50:03.883 INFO:teuthology.orchestra.run.smithi082.stdout: "take_sum": 0, 2023-06-23T18:50:03.883 INFO:teuthology.orchestra.run.smithi082.stdout: "put": 478, 2023-06-23T18:50:03.884 INFO:teuthology.orchestra.run.smithi082.stdout: "put_sum": 477837, 2023-06-23T18:50:03.884 INFO:teuthology.orchestra.run.smithi082.stdout: "wait": { 2023-06-23T18:50:03.884 INFO:teuthology.orchestra.run.smithi082.stdout: "avgcount": 0, 2023-06-23T18:50:03.884 INFO:teuthology.orchestra.run.smithi082.stdout: "sum": 0.000000000, 2023-06-23T18:50:03.884 INFO:teuthology.orchestra.run.smithi082.stdout: "avgtime": 0.000000000 2023-06-23T18:50:03.884 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:03.884 INFO:teuthology.orchestra.run.smithi082.stdout: } 2023-06-23T18:50:03.885 INFO:teuthology.orchestra.run.smithi082.stdout:} 2023-06-23T18:50:04.272 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-06-23T18:50:04.272 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-06-23T18:50:04.272 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-06-23T18:50:04.272 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-06-23T18:50:04.273 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-06-23T18:50:04.283 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-06-23T18:50:04.284 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:50:04.284 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:50:04.323 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:50:04.324 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:50:04.384 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:50:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 -- 172.21.15.2:0/2053085710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00ffc50 msgr2=0x7f16f01000a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/2053085710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00ffc50 0x7f16f01000a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009990 tx=0x7f16dc009ca0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 -- 172.21.15.2:0/2053085710 shutdown_connections 2023-06-23T18:50:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/2053085710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f01005e0 0x7f16f0106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/2053085710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16f00fea60 0x7f16f00fee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/2053085710 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f00ffc50 0x7f16f01000a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 -- 172.21.15.2:0/2053085710 >> 172.21.15.2:0/2053085710 conn(0x7f16f00fa0e0 msgr2=0x7f16f00fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:04.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 -- 172.21.15.2:0/2053085710 shutdown_connections 2023-06-23T18:50:04.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 -- 172.21.15.2:0/2053085710 wait complete. 2023-06-23T18:50:04.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 Processor -- start 2023-06-23T18:50:04.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.561+0000 7f16f6b6b700 1 -- start start 2023-06-23T18:50:04.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f6b6b700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16f00fea60 0x7f16f0194720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:04.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f6b6b700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f00ffc50 0x7f16f0194c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:04.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f6b6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f01005e0 0x7f16f019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f6b6b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f019c2a0 con 0x7f16f01005e0 2023-06-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f6b6b700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f019c3e0 con 0x7f16f00ffc50 2023-06-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f6b6b700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f019c570 con 0x7f16f00fea60 2023-06-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f01005e0 0x7f16f019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f01005e0 0x7f16f019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58650/0 (socket says 172.21.15.2:58650) 2023-06-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 -- 172.21.15.2:0/1201366002 learned_addr learned my addr 172.21.15.2:0/1201366002 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f4907700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16f00fea60 0x7f16f0194720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16effff700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f00ffc50 0x7f16f0194c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 -- 172.21.15.2:0/1201366002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16f00fea60 msgr2=0x7f16f0194720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16f00fea60 0x7f16f0194720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 -- 172.21.15.2:0/1201366002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f00ffc50 msgr2=0x7f16f0194c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f00ffc50 0x7f16f0194c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 -- 172.21.15.2:0/1201366002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16dc0095f0 con 0x7f16f01005e0 2023-06-23T18:50:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.562+0000 7f16f5108700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f01005e0 0x7f16f019bcd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f16e800ef30 tx=0x7f16e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.563+0000 7f16edffb700 1 -- 172.21.15.2:0/1201366002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8018070 con 0x7f16f01005e0 2023-06-23T18:50:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.563+0000 7f16edffb700 1 -- 172.21.15.2:0/1201366002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16e800f040 con 0x7f16f01005e0 2023-06-23T18:50:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.563+0000 7f16edffb700 1 -- 172.21.15.2:0/1201366002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800cc70 con 0x7f16f01005e0 2023-06-23T18:50:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.563+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f019c7f0 con 0x7f16f01005e0 2023-06-23T18:50:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.563+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f019ccb0 con 0x7f16f01005e0 2023-06-23T18:50:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.564+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f010a130 con 0x7f16f01005e0 2023-06-23T18:50:04.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.565+0000 7f16edffb700 1 -- 172.21.15.2:0/1201366002 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f16e8007500 con 0x7f16f01005e0 2023-06-23T18:50:04.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.565+0000 7f16edffb700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f16e00a11d0 0x7f16e00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:04.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.566+0000 7f16f4907700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f16e00a11d0 0x7f16e00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:04.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.566+0000 7f16edffb700 1 -- 172.21.15.2:0/1201366002 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f16e80cc420 con 0x7f16f01005e0 2023-06-23T18:50:04.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.566+0000 7f16f4907700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f16e00a11d0 0x7f16e00a3680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f16f00ffab0 tx=0x7f16e400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:04.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.570+0000 7f16edffb700 1 -- 172.21.15.2:0/1201366002 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f16e8095c00 con 0x7f16f01005e0 2023-06-23T18:50:04.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.781+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16f019d2f0 con 0x7f16f01005e0 2023-06-23T18:50:04.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.783+0000 7f16edffb700 1 -- 172.21.15.2:0/1201366002 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f16e80b7e60 con 0x7f16f01005e0 2023-06-23T18:50:04.787 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:50:04.787 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:50:04.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f16e00a11d0 msgr2=0x7f16e00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:04.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f16e00a11d0 0x7f16e00a3680 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f16f00ffab0 tx=0x7f16e400b410 comp rx=0 tx=0).stop 2023-06-23T18:50:04.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f01005e0 msgr2=0x7f16f019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:04.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f01005e0 0x7f16f019bcd0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f16e800ef30 tx=0x7f16e800c3a0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 shutdown_connections 2023-06-23T18:50:04.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16f01005e0 0x7f16f019bcd0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16f00fea60 0x7f16f0194720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f16e00a11d0 0x7f16e00a3680 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 --2- 172.21.15.2:0/1201366002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16f00ffc50 0x7f16f0194c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.785+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 >> 172.21.15.2:0/1201366002 conn(0x7f16f00fa0e0 msgr2=0x7f16f00fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:04.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.786+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 shutdown_connections 2023-06-23T18:50:04.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.786+0000 7f16f6b6b700 1 -- 172.21.15.2:0/1201366002 wait complete. 2023-06-23T18:50:04.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:50:04.803 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:50:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 -- 172.21.15.2:0/2889157947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80ffc70 msgr2=0x7f76a81000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:04.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 --2- 172.21.15.2:0/2889157947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80ffc70 0x7f76a81000c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009990 tx=0x7f76a4009ca0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 -- 172.21.15.2:0/2889157947 shutdown_connections 2023-06-23T18:50:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 --2- 172.21.15.2:0/2889157947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a8100600 0x7f76a8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 --2- 172.21.15.2:0/2889157947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a80fea80 0x7f76a80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 --2- 172.21.15.2:0/2889157947 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80ffc70 0x7f76a81000c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 -- 172.21.15.2:0/2889157947 >> 172.21.15.2:0/2889157947 conn(0x7f76a80fa0e0 msgr2=0x7f76a80fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 -- 172.21.15.2:0/2889157947 shutdown_connections 2023-06-23T18:50:04.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.931+0000 7f76b01ca700 1 -- 172.21.15.2:0/2889157947 wait complete. 2023-06-23T18:50:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76b01ca700 1 Processor -- start 2023-06-23T18:50:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76b01ca700 1 -- start start 2023-06-23T18:50:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76b01ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80fea80 0x7f76a8068330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76b01ca700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a80ffc70 0x7f76a819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76b01ca700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a8100600 0x7f76a819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76b01ca700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819d320 con 0x7f76a80fea80 2023-06-23T18:50:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76b01ca700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819d490 con 0x7f76a8100600 2023-06-23T18:50:04.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76b01ca700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a819d650 con 0x7f76a80ffc70 2023-06-23T18:50:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80fea80 0x7f76a8068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80fea80 0x7f76a8068330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58658/0 (socket says 172.21.15.2:58658) 2023-06-23T18:50:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 -- 172.21.15.2:0/4088948646 learned_addr learned my addr 172.21.15.2:0/4088948646 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76ae767700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a8100600 0x7f76a819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76ad765700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a80ffc70 0x7f76a819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 -- 172.21.15.2:0/4088948646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a80ffc70 msgr2=0x7f76a819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:04.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a80ffc70 0x7f76a819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 -- 172.21.15.2:0/4088948646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a8100600 msgr2=0x7f76a819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a8100600 0x7f76a819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 -- 172.21.15.2:0/4088948646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a40095f0 con 0x7f76a80fea80 2023-06-23T18:50:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76ad765700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a80ffc70 0x7f76a819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:50:04.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.932+0000 7f76adf66700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80fea80 0x7f76a8068330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f769800e970 tx=0x7f769800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:04.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.933+0000 7f769effd700 1 -- 172.21.15.2:0/4088948646 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698009790 con 0x7f76a80fea80 2023-06-23T18:50:04.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.933+0000 7f769effd700 1 -- 172.21.15.2:0/4088948646 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f769800eec0 con 0x7f76a80fea80 2023-06-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.933+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a819d930 con 0x7f76a80fea80 2023-06-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.933+0000 7f769effd700 1 -- 172.21.15.2:0/4088948646 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698003c10 con 0x7f76a80fea80 2023-06-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.933+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a819dec0 con 0x7f76a80fea80 2023-06-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.933+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a8105980 con 0x7f76a80fea80 2023-06-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.935+0000 7f769effd700 1 -- 172.21.15.2:0/4088948646 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f7698003d90 con 0x7f76a80fea80 2023-06-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.935+0000 7f769effd700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76940a1150 0x7f76940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.935+0000 7f76ad765700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76940a1150 0x7f76940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:04.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.935+0000 7f769effd700 1 -- 172.21.15.2:0/4088948646 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7698014070 con 0x7f76a80fea80 2023-06-23T18:50:04.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.936+0000 7f76ad765700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76940a1150 0x7f76940a3600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f76a400b2e0 tx=0x7f76a4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:04.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:04.939+0000 7f769effd700 1 -- 172.21.15.2:0/4088948646 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f7698096440 con 0x7f76a80fea80 2023-06-23T18:50:05.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.135+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76a802ca80 con 0x7f76a80fea80 2023-06-23T18:50:05.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.135+0000 7f769effd700 1 -- 172.21.15.2:0/4088948646 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7698099a60 con 0x7f76a80fea80 2023-06-23T18:50:05.140 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:50:05.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76940a1150 msgr2=0x7f76940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76940a1150 0x7f76940a3600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f76a400b2e0 tx=0x7f76a4005fb0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80fea80 msgr2=0x7f76a8068330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80fea80 0x7f76a8068330 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f769800e970 tx=0x7f769800ec80 comp rx=0 tx=0).stop 2023-06-23T18:50:05.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 shutdown_connections 2023-06-23T18:50:05.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f76940a1150 0x7f76940a3600 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a8100600 0x7f76a819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f76a80fea80 0x7f76a8068330 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 --2- 172.21.15.2:0/4088948646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76a80ffc70 0x7f76a819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 >> 172.21.15.2:0/4088948646 conn(0x7f76a80fa0e0 msgr2=0x7f76a80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:05.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 shutdown_connections 2023-06-23T18:50:05.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.137+0000 7f76b01ca700 1 -- 172.21.15.2:0/4088948646 wait complete. 2023-06-23T18:50:05.152 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:04 smithi002 ceph-mon[95947]: pgmap v1137: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-06-23T18:50:05.153 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:04 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1201366002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:50:05.153 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:50:05.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:04 smithi017 ceph-mon[97472]: pgmap v1137: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-06-23T18:50:05.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:04 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1201366002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:50:05.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 -- 172.21.15.2:0/3531434644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 msgr2=0x7f4784105300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 --2- 172.21.15.2:0/3531434644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 0x7f4784105300 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4774009990 tx=0x7f4774009ca0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 -- 172.21.15.2:0/3531434644 shutdown_connections 2023-06-23T18:50:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 --2- 172.21.15.2:0/3531434644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4784105840 0x7f4784107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 --2- 172.21.15.2:0/3531434644 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4784100600 0x7f47841029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 --2- 172.21.15.2:0/3531434644 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 0x7f4784105300 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 -- 172.21.15.2:0/3531434644 >> 172.21.15.2:0/3531434644 conn(0x7f47840fa0e0 msgr2=0x7f47840fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 -- 172.21.15.2:0/3531434644 shutdown_connections 2023-06-23T18:50:05.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.278+0000 7f478c1ed700 1 -- 172.21.15.2:0/3531434644 wait complete. 2023-06-23T18:50:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.279+0000 7f478c1ed700 1 Processor -- start 2023-06-23T18:50:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.279+0000 7f478c1ed700 1 -- start start 2023-06-23T18:50:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478c1ed700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4784100600 0x7f47841957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478c1ed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 0x7f4784195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478c1ed700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4784105840 0x7f478419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478c1ed700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478419d350 con 0x7f4784102f20 2023-06-23T18:50:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478c1ed700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478419d490 con 0x7f4784100600 2023-06-23T18:50:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478c1ed700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478419d620 con 0x7f4784105840 2023-06-23T18:50:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 0x7f4784195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 0x7f4784195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58674/0 (socket says 172.21.15.2:58674) 2023-06-23T18:50:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 -- 172.21.15.2:0/2553151626 learned_addr learned my addr 172.21.15.2:0/2553151626 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789f89700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4784100600 0x7f47841957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478a78a700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4784105840 0x7f478419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 -- 172.21.15.2:0/2553151626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4784105840 msgr2=0x7f478419cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4784105840 0x7f478419cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 -- 172.21.15.2:0/2553151626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4784100600 msgr2=0x7f47841957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4784100600 0x7f47841957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 -- 172.21.15.2:0/2553151626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47740095f0 con 0x7f4784102f20 2023-06-23T18:50:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f4789788700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 0x7f4784195d10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4774000c00 tx=0x7f4774003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f477affd700 1 -- 172.21.15.2:0/2553151626 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477401d070 con 0x7f4784102f20 2023-06-23T18:50:05.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f477affd700 1 -- 172.21.15.2:0/2553151626 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f477400bd70 con 0x7f4784102f20 2023-06-23T18:50:05.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478419d8a0 con 0x7f4784102f20 2023-06-23T18:50:05.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f477affd700 1 -- 172.21.15.2:0/2553151626 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774022c80 con 0x7f4784102f20 2023-06-23T18:50:05.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.280+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478419dd20 con 0x7f4784102f20 2023-06-23T18:50:05.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.281+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47840fcc60 con 0x7f4784102f20 2023-06-23T18:50:05.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.282+0000 7f477affd700 1 -- 172.21.15.2:0/2553151626 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4774022470 con 0x7f4784102f20 2023-06-23T18:50:05.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.283+0000 7f477affd700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47700a11d0 0x7f47700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.283+0000 7f4789f89700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47700a11d0 0x7f47700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.283+0000 7f477affd700 1 -- 172.21.15.2:0/2553151626 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f47740ce580 con 0x7f4784102f20 2023-06-23T18:50:05.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.284+0000 7f4789f89700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47700a11d0 0x7f47700a3680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4780007c60 tx=0x7f47800073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:05.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.287+0000 7f477affd700 1 -- 172.21.15.2:0/2553151626 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f4774097d60 con 0x7f4784102f20 2023-06-23T18:50:05.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:04 smithi082 ceph-mon[97889]: pgmap v1137: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2023-06-23T18:50:05.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:04 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1201366002' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:50:05.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.490+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f478402ca20 con 0x7f4784102f20 2023-06-23T18:50:05.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.490+0000 7f477affd700 1 -- 172.21.15.2:0/2553151626 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f477409b380 con 0x7f4784102f20 2023-06-23T18:50:05.494 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:50:05.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.492+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47700a11d0 msgr2=0x7f47700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.492+0000 7f478c1ed700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47700a11d0 0x7f47700a3680 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4780007c60 tx=0x7f47800073d0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.492+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 msgr2=0x7f4784195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.492+0000 7f478c1ed700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 0x7f4784195d10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4774000c00 tx=0x7f4774003a50 comp rx=0 tx=0).stop 2023-06-23T18:50:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.492+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 shutdown_connections 2023-06-23T18:50:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.493+0000 7f478c1ed700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4784105840 0x7f478419cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.493+0000 7f478c1ed700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f47700a11d0 0x7f47700a3680 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.493+0000 7f478c1ed700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4784100600 0x7f47841957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.493+0000 7f478c1ed700 1 --2- 172.21.15.2:0/2553151626 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4784102f20 0x7f4784195d10 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.493+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 >> 172.21.15.2:0/2553151626 conn(0x7f47840fa0e0 msgr2=0x7f4784101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.493+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 shutdown_connections 2023-06-23T18:50:05.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.493+0000 7f478c1ed700 1 -- 172.21.15.2:0/2553151626 wait complete. 2023-06-23T18:50:05.509 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:50:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 -- 172.21.15.2:0/1813870027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 msgr2=0x7ff7901011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 --2- 172.21.15.2:0/1813870027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 0x7ff7901011d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff780009990 tx=0x7ff780009ca0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 -- 172.21.15.2:0/1813870027 shutdown_connections 2023-06-23T18:50:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 --2- 172.21.15.2:0/1813870027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff790101710 0x7ff790107fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 --2- 172.21.15.2:0/1813870027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7900ffb90 0x7ff7900fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 --2- 172.21.15.2:0/1813870027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 0x7ff7901011d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 -- 172.21.15.2:0/1813870027 >> 172.21.15.2:0/1813870027 conn(0x7ff7900fb1b0 msgr2=0x7ff7900fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 -- 172.21.15.2:0/1813870027 shutdown_connections 2023-06-23T18:50:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 -- 172.21.15.2:0/1813870027 wait complete. 2023-06-23T18:50:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.634+0000 7ff794c52700 1 Processor -- start 2023-06-23T18:50:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff794c52700 1 -- start start 2023-06-23T18:50:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff794c52700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7900ffb90 0x7ff790074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff794c52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 0x7ff7900728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff794c52700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff790101710 0x7ff790072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff794c52700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900735a0 con 0x7ff790100d80 2023-06-23T18:50:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff794c52700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790073710 con 0x7ff7900ffb90 2023-06-23T18:50:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff794c52700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79019f7d0 con 0x7ff790101710 2023-06-23T18:50:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 0x7ff7900728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 0x7ff7900728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58688/0 (socket says 172.21.15.2:58688) 2023-06-23T18:50:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 -- 172.21.15.2:0/1942254345 learned_addr learned my addr 172.21.15.2:0/1942254345 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78e59c700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7900ffb90 0x7ff790074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78ed9d700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff790101710 0x7ff790072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 -- 172.21.15.2:0/1942254345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff790101710 msgr2=0x7ff790072e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff790101710 0x7ff790072e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 -- 172.21.15.2:0/1942254345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7900ffb90 msgr2=0x7ff790074280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7900ffb90 0x7ff790074280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 -- 172.21.15.2:0/1942254345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800095f0 con 0x7ff790100d80 2023-06-23T18:50:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78ed9d700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff790101710 0x7ff790072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-06-23T18:50:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff78dd9b700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 0x7ff7900728c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff7800095c0 tx=0x7ff780005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.635+0000 7ff77f7fe700 1 -- 172.21.15.2:0/1942254345 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78001d070 con 0x7ff790100d80 2023-06-23T18:50:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.636+0000 7ff77f7fe700 1 -- 172.21.15.2:0/1942254345 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff780004000 con 0x7ff790100d80 2023-06-23T18:50:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.636+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79019fa50 con 0x7ff790100d80 2023-06-23T18:50:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.636+0000 7ff77f7fe700 1 -- 172.21.15.2:0/1942254345 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780022cd0 con 0x7ff790100d80 2023-06-23T18:50:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.636+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79019fe70 con 0x7ff790100d80 2023-06-23T18:50:05.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.636+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790107170 con 0x7ff790100d80 2023-06-23T18:50:05.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.638+0000 7ff77f7fe700 1 -- 172.21.15.2:0/1942254345 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff780022470 con 0x7ff790100d80 2023-06-23T18:50:05.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.638+0000 7ff77f7fe700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff7780a1150 0x7ff7780a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.638+0000 7ff77f7fe700 1 -- 172.21.15.2:0/1942254345 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ff7800ce540 con 0x7ff790100d80 2023-06-23T18:50:05.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.638+0000 7ff78e59c700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff7780a1150 0x7ff7780a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.639+0000 7ff78e59c700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff7780a1150 0x7ff7780a3600 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff790100be0 tx=0x7ff78400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:05.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.642+0000 7ff77f7fe700 1 -- 172.21.15.2:0/1942254345 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7ff780097d20 con 0x7ff790100d80 2023-06-23T18:50:05.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.840+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff79002ca60 con 0x7ff790100d80 2023-06-23T18:50:05.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.841+0000 7ff77f7fe700 1 -- 172.21.15.2:0/1942254345 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff78009b340 con 0x7ff790100d80 2023-06-23T18:50:05.845 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:50:05.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff7780a1150 msgr2=0x7ff7780a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff7780a1150 0x7ff7780a3600 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff790100be0 tx=0x7ff78400b410 comp rx=0 tx=0).stop 2023-06-23T18:50:05.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 msgr2=0x7ff7900728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 0x7ff7900728c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff7800095c0 tx=0x7ff780005740 comp rx=0 tx=0).stop 2023-06-23T18:50:05.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 shutdown_connections 2023-06-23T18:50:05.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff790101710 0x7ff790072e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7ff7780a1150 0x7ff7780a3600 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7900ffb90 0x7ff790074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 --2- 172.21.15.2:0/1942254345 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff790100d80 0x7ff7900728c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 >> 172.21.15.2:0/1942254345 conn(0x7ff7900fb1b0 msgr2=0x7ff7901042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 shutdown_connections 2023-06-23T18:50:05.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.843+0000 7ff794c52700 1 -- 172.21.15.2:0/1942254345 wait complete. 2023-06-23T18:50:05.860 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:50:05.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 -- 172.21.15.2:0/1376528143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45340fea80 msgr2=0x7f45340fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 --2- 172.21.15.2:0/1376528143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45340fea80 0x7f45340fee90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4524009990 tx=0x7f4524009ca0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 -- 172.21.15.2:0/1376528143 shutdown_connections 2023-06-23T18:50:05.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 --2- 172.21.15.2:0/1376528143 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4534100600 0x7f4534106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 --2- 172.21.15.2:0/1376528143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f45340fea80 0x7f45340fee90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 --2- 172.21.15.2:0/1376528143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340ffc70 0x7f45341000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 -- 172.21.15.2:0/1376528143 >> 172.21.15.2:0/1376528143 conn(0x7f45340fa0e0 msgr2=0x7f45340fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:05.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 -- 172.21.15.2:0/1376528143 shutdown_connections 2023-06-23T18:50:05.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.985+0000 7f453a997700 1 -- 172.21.15.2:0/1376528143 wait complete. 2023-06-23T18:50:05.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f453a997700 1 Processor -- start 2023-06-23T18:50:05.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f453a997700 1 -- start start 2023-06-23T18:50:05.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f453a997700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340fea80 0x7f4534194740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f453a997700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45340ffc70 0x7f4534194c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f453a997700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4534100600 0x7f453419bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f453a997700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453419c2c0 con 0x7f4534100600 2023-06-23T18:50:05.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f453a997700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453419c400 con 0x7f45340fea80 2023-06-23T18:50:05.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f453a997700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453419c590 con 0x7f45340ffc70 2023-06-23T18:50:05.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.986+0000 7f4538f34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4534100600 0x7f453419bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f45337fe700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45340ffc70 0x7f4534194c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4538f34700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4534100600 0x7f453419bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58704/0 (socket says 172.21.15.2:58704) 2023-06-23T18:50:05.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4538f34700 1 -- 172.21.15.2:0/2676710493 learned_addr learned my addr 172.21.15.2:0/2676710493 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:05.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4533fff700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340fea80 0x7f4534194740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4538f34700 1 -- 172.21.15.2:0/2676710493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45340ffc70 msgr2=0x7f4534194c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4538f34700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45340ffc70 0x7f4534194c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4538f34700 1 -- 172.21.15.2:0/2676710493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340fea80 msgr2=0x7f4534194740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:05.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4538f34700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340fea80 0x7f4534194740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:05.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4538f34700 1 -- 172.21.15.2:0/2676710493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45240095f0 con 0x7f4534100600 2023-06-23T18:50:05.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4533fff700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340fea80 0x7f4534194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:50:05.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f4538f34700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4534100600 0x7f453419bcf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f452c007ee0 tx=0x7f452c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:05.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f45317fa700 1 -- 172.21.15.2:0/2676710493 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c00ddd0 con 0x7f4534100600 2023-06-23T18:50:05.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453419c870 con 0x7f4534100600 2023-06-23T18:50:05.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f45317fa700 1 -- 172.21.15.2:0/2676710493 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f452c00f040 con 0x7f4534100600 2023-06-23T18:50:05.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f45317fa700 1 -- 172.21.15.2:0/2676710493 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c0047d0 con 0x7f4534100600 2023-06-23T18:50:05.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.987+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453419cdd0 con 0x7f4534100600 2023-06-23T18:50:05.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.988+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453410a150 con 0x7f4534100600 2023-06-23T18:50:05.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.989+0000 7f45317fa700 1 -- 172.21.15.2:0/2676710493 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f452c003f20 con 0x7f4534100600 2023-06-23T18:50:05.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.990+0000 7f45317fa700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f451c0a1150 0x7f451c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:05.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.990+0000 7f45317fa700 1 -- 172.21.15.2:0/2676710493 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f452c0cca50 con 0x7f4534100600 2023-06-23T18:50:05.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.990+0000 7f4533fff700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f451c0a1150 0x7f451c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:05.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.991+0000 7f4533fff700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f451c0a1150 0x7f451c0a3600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f452400b5c0 tx=0x7f4524005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:05.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:05.993+0000 7f45317fa700 1 -- 172.21.15.2:0/2676710493 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f452c096160 con 0x7f4534100600 2023-06-23T18:50:06.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:05 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/4088948646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:05 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2553151626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.191 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:05 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1942254345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.210+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45341059f0 con 0x7f4534100600 2023-06-23T18:50:06.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.211+0000 7f45317fa700 1 -- 172.21.15.2:0/2676710493 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+579 (secure 0 0 0) 0x7f452c023080 con 0x7f4534100600 2023-06-23T18:50:06.216 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2916792555 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:06.216 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3628945044 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:06.216 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3628945044 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:06.217 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2835880554 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:06.217 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1541129198 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:06.217 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1477099684 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:06.217 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/595372279 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:06.217 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/647669756 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:06.217 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/647669756 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:06.217 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1251506010 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:06.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f451c0a1150 msgr2=0x7f451c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f451c0a1150 0x7f451c0a3600 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f452400b5c0 tx=0x7f4524005fb0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4534100600 msgr2=0x7f453419bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4534100600 0x7f453419bcf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f452c007ee0 tx=0x7f452c00d3b0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 shutdown_connections 2023-06-23T18:50:06.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4534100600 0x7f453419bcf0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f451c0a1150 0x7f451c0a3600 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f45340fea80 0x7f4534194740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 --2- 172.21.15.2:0/2676710493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45340ffc70 0x7f4534194c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 >> 172.21.15.2:0/2676710493 conn(0x7f45340fa0e0 msgr2=0x7f45340fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:06.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 shutdown_connections 2023-06-23T18:50:06.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.213+0000 7f453a997700 1 -- 172.21.15.2:0/2676710493 wait complete. 2023-06-23T18:50:06.220 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2023-06-23T18:50:06.230 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:05 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/4088948646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.230 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:05 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2553151626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.231 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:05 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1942254345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.231 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-06-23T18:50:06.232 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-06-23T18:50:06.250 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-06-23T18:50:06.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:05 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/4088948646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.370 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:05 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2553151626' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.371 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:05 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1942254345' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:50:06.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1803636133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a4105820 msgr2=0x7f90a4107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1803636133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a4105820 0x7f90a4107c80 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f909c00b0a0 tx=0x7f909c00b3b0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1803636133 shutdown_connections 2023-06-23T18:50:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1803636133 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a4105820 0x7f90a4107c80 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1803636133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90a41005e0 0x7f90a41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1803636133 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a4102f00 0x7f90a41052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1803636133 >> 172.21.15.2:0/1803636133 conn(0x7f90a40fa0e0 msgr2=0x7f90a40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:06.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1803636133 shutdown_connections 2023-06-23T18:50:06.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.410+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1803636133 wait complete. 2023-06-23T18:50:06.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90aa5e9700 1 Processor -- start 2023-06-23T18:50:06.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90aa5e9700 1 -- start start 2023-06-23T18:50:06.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90aa5e9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a41005e0 0x7f90a4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90aa5e9700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a4102f00 0x7f90a4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90aa5e9700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90a4105820 0x7f90a419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90aa5e9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a419f630 con 0x7f90a41005e0 2023-06-23T18:50:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90aa5e9700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a419f770 con 0x7f90a4102f00 2023-06-23T18:50:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90aa5e9700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a419f900 con 0x7f90a4105820 2023-06-23T18:50:06.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.411+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a41005e0 0x7f90a4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a41005e0 0x7f90a4197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58714/0 (socket says 172.21.15.2:58714) 2023-06-23T18:50:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a3fff700 1 -- 172.21.15.2:0/1379679830 learned_addr learned my addr 172.21.15.2:0/1379679830 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a8b86700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90a4105820 0x7f90a419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a37fe700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a4102f00 0x7f90a4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a3fff700 1 -- 172.21.15.2:0/1379679830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90a4105820 msgr2=0x7f90a419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a3fff700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90a4105820 0x7f90a419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a3fff700 1 -- 172.21.15.2:0/1379679830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a4102f00 msgr2=0x7f90a4197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a3fff700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a4102f00 0x7f90a4197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a3fff700 1 -- 172.21.15.2:0/1379679830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c009d00 con 0x7f90a41005e0 2023-06-23T18:50:06.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a3fff700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a41005e0 0x7f90a4197ab0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f909400e9c0 tx=0x7f909400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:06.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a17fa700 1 -- 172.21.15.2:0/1379679830 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940097b0 con 0x7f90a41005e0 2023-06-23T18:50:06.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a17fa700 1 -- 172.21.15.2:0/1379679830 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f909400ee60 con 0x7f90a41005e0 2023-06-23T18:50:06.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a419fbe0 con 0x7f90a41005e0 2023-06-23T18:50:06.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90a17fa700 1 -- 172.21.15.2:0/1379679830 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094005480 con 0x7f90a41005e0 2023-06-23T18:50:06.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.412+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a41a00f0 con 0x7f90a41005e0 2023-06-23T18:50:06.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.413+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a410b190 con 0x7f90a41005e0 2023-06-23T18:50:06.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.415+0000 7f90a17fa700 1 -- 172.21.15.2:0/1379679830 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f90940055e0 con 0x7f90a41005e0 2023-06-23T18:50:06.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.416+0000 7f90a17fa700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f908c0a11d0 0x7f908c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.416+0000 7f90a17fa700 1 -- 172.21.15.2:0/1379679830 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9094014070 con 0x7f90a41005e0 2023-06-23T18:50:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.416+0000 7f90a37fe700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f908c0a11d0 0x7f908c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.416+0000 7f90a37fe700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f908c0a11d0 0x7f908c0a3680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9098009c60 tx=0x7f9098009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:06.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.419+0000 7f90a17fa700 1 -- 172.21.15.2:0/1379679830 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f90940963a0 con 0x7f90a41005e0 2023-06-23T18:50:06.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.619+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90a41a06a0 con 0x7f90a41005e0 2023-06-23T18:50:06.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.621+0000 7f90a17fa700 1 -- 172.21.15.2:0/1379679830 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22421 (secure 0 0 0) 0x7f90940999c0 con 0x7f90a41005e0 2023-06-23T18:50:06.626 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-06-23T18:50:06.627 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":84,"fsid":"902733cc-11f1-11ee-9b2b-001a4aab830c","created":"2023-06-23T18:13:38.741731+0000","modified":"2023-06-23T18:21:38.066041+0000","last_up_change":"2023-06-23T18:20:02.508997+0000","last_in_change":"2023-06-23T18:19:44.943057+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-06-23T18:16:49.188592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-06-23T18:20:42.964238+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-06-23T18:20:43.842256+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9d2907d6-0f38-4387-afe6-dda12616124d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6803","nonce":2079965032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6805","nonce":2079965032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6809","nonce":2079965032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2079965032},{"type":"v1","addr":"172.21.15.2:6807","nonce":2079965032}]},"public_addr":"172.21.15.2:6803/2079965032","cluster_addr":"172.21.15.2:6805/2079965032","heartbeat_back_addr":"172.21.15.2:6809/2079965032","heartbeat_front_addr":"172.21.15.2:6807/2079965032","state":["exists","up"]},{"osd":1,"uuid":"b3d5d141-29f5-4a3e-b854-4ca05bf85d90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6801","nonce":85914245}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6803","nonce":85914245}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6807","nonce":85914245}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":85914245},{"type":"v1","addr":"172.21.15.17:6805","nonce":85914245}]},"public_addr":"172.21.15.17:6801/85914245","cluster_addr":"172.21.15.17:6803/85914245","heartbeat_back_addr":"172.21.15.17:6807/85914245","heartbeat_front_addr":"172.21.15.17:6805/85914245","state":["exists","up"]},{"osd":2,"uuid":"c62ee3f3-654e-4a1f-b47c-6dd9b9585db0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6801","nonce":625655423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6803","nonce":625655423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6807","nonce":625655423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6804","nonce":625655423},{"type":"v1","addr":"172.21.15.82:6805","nonce":625655423}]},"public_addr":"172.21.15.82:6801/625655423","cluster_addr":"172.21.15.82:6803/625655423","heartbeat_back_addr":"172.21.15.82:6807/625655423","heartbeat_front_addr":"172.21.15.82:6805/625655423","state":["exists","up"]},{"osd":3,"uuid":"d0c433bc-2983-4dac-a88c-27d6d0060480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6811","nonce":1640369824}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6813","nonce":1640369824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6817","nonce":1640369824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1640369824},{"type":"v1","addr":"172.21.15.2:6815","nonce":1640369824}]},"public_addr":"172.21.15.2:6811/1640369824","cluster_addr":"172.21.15.2:6813/1640369824","heartbeat_back_addr":"172.21.15.2:6817/1640369824","heartbeat_front_addr":"172.21.15.2:6815/1640369824","state":["exists","up"]},{"osd":4,"uuid":"820c84c1-7f7a-4c03-8380-3dffe8d9c3fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6809","nonce":2671149119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6811","nonce":2671149119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6815","nonce":2671149119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":2671149119},{"type":"v1","addr":"172.21.15.17:6813","nonce":2671149119}]},"public_addr":"172.21.15.17:6809/2671149119","cluster_addr":"172.21.15.17:6811/2671149119","heartbeat_back_addr":"172.21.15.17:6815/2671149119","heartbeat_front_addr":"172.21.15.17:6813/2671149119","state":["exists","up"]},{"osd":5,"uuid":"eac48eb4-7436-4849-bff3-ccd457ee594a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6808","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6809","nonce":102458807}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6811","nonce":102458807}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6815","nonce":102458807}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6812","nonce":102458807},{"type":"v1","addr":"172.21.15.82:6813","nonce":102458807}]},"public_addr":"172.21.15.82:6809/102458807","cluster_addr":"172.21.15.82:6811/102458807","heartbeat_back_addr":"172.21.15.82:6815/102458807","heartbeat_front_addr":"172.21.15.82:6813/102458807","state":["exists","up"]},{"osd":6,"uuid":"9e320af6-3d7b-4934-9f7a-2860876ee5a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6819","nonce":617424670}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6821","nonce":617424670}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6825","nonce":617424670}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":617424670},{"type":"v1","addr":"172.21.15.2:6823","nonce":617424670}]},"public_addr":"172.21.15.2:6819/617424670","cluster_addr":"172.21.15.2:6821/617424670","heartbeat_back_addr":"172.21.15.2:6825/617424670","heartbeat_front_addr":"172.21.15.2:6823/617424670","state":["exists","up"]},{"osd":7,"uuid":"d5b16bb7-b550-4400-846a-0d321e8af4c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6817","nonce":3002114056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6819","nonce":3002114056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6823","nonce":3002114056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":3002114056},{"type":"v1","addr":"172.21.15.17:6821","nonce":3002114056}]},"public_addr":"172.21.15.17:6817/3002114056","cluster_addr":"172.21.15.17:6819/3002114056","heartbeat_back_addr":"172.21.15.17:6823/3002114056","heartbeat_front_addr":"172.21.15.17:6821/3002114056","state":["exists","up"]},{"osd":8,"uuid":"4e7ac9f9-4c02-4639-ba72-7af18127d2eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6817","nonce":1305476760}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6819","nonce":1305476760}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6823","nonce":1305476760}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1305476760},{"type":"v1","addr":"172.21.15.82:6821","nonce":1305476760}]},"public_addr":"172.21.15.82:6817/1305476760","cluster_addr":"172.21.15.82:6819/1305476760","heartbeat_back_addr":"172.21.15.82:6823/1305476760","heartbeat_front_addr":"172.21.15.82:6821/1305476760","state":["exists","up"]},{"osd":9,"uuid":"9013ccc7-bc53-477d-8a23-2806313db552","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6827","nonce":1361624360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6829","nonce":1361624360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6833","nonce":1361624360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":1361624360},{"type":"v1","addr":"172.21.15.2:6831","nonce":1361624360}]},"public_addr":"172.21.15.2:6827/1361624360","cluster_addr":"172.21.15.2:6829/1361624360","heartbeat_back_addr":"172.21.15.2:6833/1361624360","heartbeat_front_addr":"172.21.15.2:6831/1361624360","state":["exists","up"]},{"osd":10,"uuid":"9ab48541-33f0-44f9-8d30-727fd67ecbe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6825","nonce":3783526814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6827","nonce":3783526814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6831","nonce":3783526814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3783526814},{"type":"v1","addr":"172.21.15.17:6829","nonce":3783526814}]},"public_addr":"172.21.15.17:6825/3783526814","cluster_addr":"172.21.15.17:6827/3783526814","heartbeat_back_addr":"172.21.15.17:6831/3783526814","heartbeat_front_addr":"172.21.15.17:6829/3783526814","state":["exists","up"]},{"osd":11,"uuid":"f5d7fe39-6a45-4b73-9a3b-eb7aa1af4d77","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6825","nonce":3300059850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6827","nonce":3300059850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6831","nonce":3300059850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3300059850},{"type":"v1","addr":"172.21.15.82:6829","nonce":3300059850}]},"public_addr":"172.21.15.82:6825/3300059850","cluster_addr":"172.21.15.82:6827/3300059850","heartbeat_back_addr":"172.21.15.82:6831/3300059850","heartbeat_front_addr":"172.21.15.82:6829/3300059850","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:04.510214+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:26.619305+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:16:47.184126+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:09.229683+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:31.276977+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:17:51.922995+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:13.710898+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:35.272706+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:18:56.242923+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:17.591156+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:19:39.767147+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-06-23T18:20:00.904930+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.2:0/2916792555":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6801/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:6800/3628945044":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/2835880554":"2023-06-24T18:14:36.653577+0000","172.21.15.2:0/1541129198":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1477099684":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/595372279":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6800/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:6801/647669756":"2023-06-24T18:14:09.903724+0000","172.21.15.2:0/1251506010":"2023-06-24T18:14:36.653577+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-06-23T18:50:06.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f908c0a11d0 msgr2=0x7f908c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f908c0a11d0 0x7f908c0a3680 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9098009c60 tx=0x7f9098009260 comp rx=0 tx=0).stop 2023-06-23T18:50:06.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a41005e0 msgr2=0x7f90a4197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a41005e0 0x7f90a4197ab0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f909400e9c0 tx=0x7f909400ecd0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 shutdown_connections 2023-06-23T18:50:06.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90a4105820 0x7f90a419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90a41005e0 0x7f90a4197ab0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f908c0a11d0 0x7f908c0a3680 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 --2- 172.21.15.2:0/1379679830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90a4102f00 0x7f90a4197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 >> 172.21.15.2:0/1379679830 conn(0x7f90a40fa0e0 msgr2=0x7f90a4101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:06.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 shutdown_connections 2023-06-23T18:50:06.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.623+0000 7f90aa5e9700 1 -- 172.21.15.2:0/1379679830 wait complete. 2023-06-23T18:50:06.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-06-23T18:16:49.188592+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-06-23T18:20:42.964238+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-06-23T18:20:43.842256+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-06-23T18:50:06.642 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-06-23T18:50:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.765+0000 7f88f777e700 1 -- 172.21.15.2:0/2417939203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f0100620 msgr2=0x7f88f0106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.765+0000 7f88f777e700 1 --2- 172.21.15.2:0/2417939203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f0100620 0x7f88f0106ec0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f88e800b0a0 tx=0x7f88e800b3b0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.765+0000 7f88f777e700 1 -- 172.21.15.2:0/2417939203 shutdown_connections 2023-06-23T18:50:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 --2- 172.21.15.2:0/2417939203 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f0100620 0x7f88f0106ec0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 --2- 172.21.15.2:0/2417939203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88f00fe9e0 0x7f88f00fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 --2- 172.21.15.2:0/2417939203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f00ffc70 0x7f88f01000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 -- 172.21.15.2:0/2417939203 >> 172.21.15.2:0/2417939203 conn(0x7f88f00fa000 msgr2=0x7f88f00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 -- 172.21.15.2:0/2417939203 shutdown_connections 2023-06-23T18:50:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 -- 172.21.15.2:0/2417939203 wait complete. 2023-06-23T18:50:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 Processor -- start 2023-06-23T18:50:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 -- start start 2023-06-23T18:50:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f00fe9e0 0x7f88f0068310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f00ffc70 0x7f88f019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88f0100620 0x7f88f019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f019d1f0 con 0x7f88f00fe9e0 2023-06-23T18:50:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f019d360 con 0x7f88f00ffc70 2023-06-23T18:50:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.766+0000 7f88f777e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f019d520 con 0x7f88f0100620 2023-06-23T18:50:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f00fe9e0 0x7f88f0068310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f00fe9e0 0x7f88f0068310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58726/0 (socket says 172.21.15.2:58726) 2023-06-23T18:50:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 -- 172.21.15.2:0/2045139610 learned_addr learned my addr 172.21.15.2:0/2045139610 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f4d19700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f00ffc70 0x7f88f019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f5d1b700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88f0100620 0x7f88f019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 -- 172.21.15.2:0/2045139610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88f0100620 msgr2=0x7f88f019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88f0100620 0x7f88f019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 -- 172.21.15.2:0/2045139610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f00ffc70 msgr2=0x7f88f019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f00ffc70 0x7f88f019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 -- 172.21.15.2:0/2045139610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e8009d00 con 0x7f88f00fe9e0 2023-06-23T18:50:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f551a700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f00fe9e0 0x7f88f0068310 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00c9b0 tx=0x7f88ec00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88e67fc700 1 -- 172.21.15.2:0/2045139610 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec007b00 con 0x7f88f00fe9e0 2023-06-23T18:50:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f019d800 con 0x7f88f00fe9e0 2023-06-23T18:50:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88e67fc700 1 -- 172.21.15.2:0/2045139610 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88ec00ce50 con 0x7f88f00fe9e0 2023-06-23T18:50:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88e67fc700 1 -- 172.21.15.2:0/2045139610 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec018400 con 0x7f88f00fe9e0 2023-06-23T18:50:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.767+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f019dd10 con 0x7f88f00fe9e0 2023-06-23T18:50:06.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.768+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f0105960 con 0x7f88f00fe9e0 2023-06-23T18:50:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.770+0000 7f88e67fc700 1 -- 172.21.15.2:0/2045139610 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f88ec00f450 con 0x7f88f00fe9e0 2023-06-23T18:50:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.770+0000 7f88e67fc700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88dc0a1150 0x7f88dc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.771+0000 7f88f4d19700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88dc0a1150 0x7f88dc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.771+0000 7f88e67fc700 1 -- 172.21.15.2:0/2045139610 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f88ec0ccb00 con 0x7f88f00fe9e0 2023-06-23T18:50:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.771+0000 7f88f4d19700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88dc0a1150 0x7f88dc0a3600 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f88e0007be0 tx=0x7f88e00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.774+0000 7f88e67fc700 1 -- 172.21.15.2:0/2045139610 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f88ec0962e0 con 0x7f88f00fe9e0 2023-06-23T18:50:06.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.975+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88f002c9c0 con 0x7f88f00fe9e0 2023-06-23T18:50:06.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.975+0000 7f88e67fc700 1 -- 172.21.15.2:0/2045139610 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88ec00f700 con 0x7f88f00fe9e0 2023-06-23T18:50:06.980 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-06-23T18:50:06.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88dc0a1150 msgr2=0x7f88dc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88dc0a1150 0x7f88dc0a3600 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f88e0007be0 tx=0x7f88e00073d0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f00fe9e0 msgr2=0x7f88f0068310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:06.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f00fe9e0 0x7f88f0068310 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00c9b0 tx=0x7f88ec00ccc0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 shutdown_connections 2023-06-23T18:50:06.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88f0100620 0x7f88f019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f88f00fe9e0 0x7f88f0068310 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f88dc0a1150 0x7f88dc0a3600 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 --2- 172.21.15.2:0/2045139610 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88f00ffc70 0x7f88f019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:06.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 >> 172.21.15.2:0/2045139610 conn(0x7f88f00fa000 msgr2=0x7f88f00fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:06.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 shutdown_connections 2023-06-23T18:50:06.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:06.977+0000 7f88f777e700 1 -- 172.21.15.2:0/2045139610 wait complete. 2023-06-23T18:50:06.994 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-06-23T18:50:07.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/2654656603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 msgr2=0x7fa0bc107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/2654656603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 0x7fa0bc107ca0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400b0a0 tx=0x7fa0b400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/2654656603 shutdown_connections 2023-06-23T18:50:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/2654656603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 0x7fa0bc107ca0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/2654656603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0bc100660 0x7fa0bc102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/2654656603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0bc102f80 0x7fa0bc105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/2654656603 >> 172.21.15.2:0/2654656603 conn(0x7fa0bc0fa0e0 msgr2=0x7fa0bc0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/2654656603 shutdown_connections 2023-06-23T18:50:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.118+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/2654656603 wait complete. 2023-06-23T18:50:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c3fe5700 1 Processor -- start 2023-06-23T18:50:07.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c3fe5700 1 -- start start 2023-06-23T18:50:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c3fe5700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0bc100660 0x7fa0bc1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c3fe5700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0bc102f80 0x7fa0bc195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c3fe5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 0x7fa0bc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c3fe5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19d440 con 0x7fa0bc1058a0 2023-06-23T18:50:07.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c3fe5700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19d580 con 0x7fa0bc100660 2023-06-23T18:50:07.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c3fe5700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc19d710 con 0x7fa0bc102f80 2023-06-23T18:50:07.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c2582700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 0x7fa0bc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c2582700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 0x7fa0bc19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58742/0 (socket says 172.21.15.2:58742) 2023-06-23T18:50:07.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c2582700 1 -- 172.21.15.2:0/3350442665 learned_addr learned my addr 172.21.15.2:0/3350442665 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:07.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c1d81700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0bc100660 0x7fa0bc1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.119+0000 7fa0c1580700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0bc102f80 0x7fa0bc195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0c2582700 1 -- 172.21.15.2:0/3350442665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0bc102f80 msgr2=0x7fa0bc195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0c2582700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0bc102f80 0x7fa0bc195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0c2582700 1 -- 172.21.15.2:0/3350442665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0bc100660 msgr2=0x7fa0bc1958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0c2582700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0bc100660 0x7fa0bc1958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0c2582700 1 -- 172.21.15.2:0/3350442665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4009d00 con 0x7fa0bc1058a0 2023-06-23T18:50:07.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0c2582700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 0x7fa0bc19ce70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400b780 tx=0x7fa0b400bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:07.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0b2ffd700 1 -- 172.21.15.2:0/3350442665 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b400e040 con 0x7fa0bc1058a0 2023-06-23T18:50:07.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0b2ffd700 1 -- 172.21.15.2:0/3350442665 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa0b4004e60 con 0x7fa0bc1058a0 2023-06-23T18:50:07.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0b2ffd700 1 -- 172.21.15.2:0/3350442665 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40052a0 con 0x7fa0bc1058a0 2023-06-23T18:50:07.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc19d990 con 0x7fa0bc1058a0 2023-06-23T18:50:07.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.120+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc19ddc0 con 0x7fa0bc1058a0 2023-06-23T18:50:07.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.122+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc0fcc60 con 0x7fa0bc1058a0 2023-06-23T18:50:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.122+0000 7fa0b2ffd700 1 -- 172.21.15.2:0/3350442665 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa0b401d470 con 0x7fa0bc1058a0 2023-06-23T18:50:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.123+0000 7fa0b2ffd700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa0a80a11d0 0x7fa0a80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.123+0000 7fa0b2ffd700 1 -- 172.21.15.2:0/3350442665 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa0b40cdf70 con 0x7fa0bc1058a0 2023-06-23T18:50:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.123+0000 7fa0c1d81700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa0a80a11d0 0x7fa0a80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.123+0000 7fa0c1d81700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa0a80a11d0 0x7fa0a80a3680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009c60 tx=0x7fa0b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:07.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.127+0000 7fa0b2ffd700 1 -- 172.21.15.2:0/3350442665 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fa0b4097750 con 0x7fa0bc1058a0 2023-06-23T18:50:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:06 smithi017 ceph-mon[97472]: pgmap v1138: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:50:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:06 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2676710493' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-06-23T18:50:07.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:06 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/1379679830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:50:07.300 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:06 smithi002 ceph-mon[95947]: pgmap v1138: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:50:07.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:06 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2676710493' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-06-23T18:50:07.301 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:06 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/1379679830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:50:07.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.331+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0bc02c810 con 0x7fa0bc1058a0 2023-06-23T18:50:07.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.331+0000 7fa0b2ffd700 1 -- 172.21.15.2:0/3350442665 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa0b409ad70 con 0x7fa0bc1058a0 2023-06-23T18:50:07.337 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-06-23T18:50:07.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa0a80a11d0 msgr2=0x7fa0a80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa0a80a11d0 0x7fa0a80a3680 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8009c60 tx=0x7fa0b8009260 comp rx=0 tx=0).stop 2023-06-23T18:50:07.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 msgr2=0x7fa0bc19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 0x7fa0bc19ce70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa0b400b780 tx=0x7fa0b400bd40 comp rx=0 tx=0).stop 2023-06-23T18:50:07.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 shutdown_connections 2023-06-23T18:50:07.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0bc1058a0 0x7fa0bc19ce70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fa0a80a11d0 0x7fa0a80a3680 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa0bc100660 0x7fa0bc1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 --2- 172.21.15.2:0/3350442665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0bc102f80 0x7fa0bc195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 >> 172.21.15.2:0/3350442665 conn(0x7fa0bc0fa0e0 msgr2=0x7fa0bc101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 shutdown_connections 2023-06-23T18:50:07.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.334+0000 7fa0c3fe5700 1 -- 172.21.15.2:0/3350442665 wait complete. 2023-06-23T18:50:07.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:06 smithi082 ceph-mon[97889]: pgmap v1138: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-06-23T18:50:07.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:06 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2676710493' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-06-23T18:50:07.349 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:06 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/1379679830' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-06-23T18:50:07.351 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-06-23T18:50:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.475+0000 7fcf11d7e700 1 -- 172.21.15.2:0/1113155613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c100600 msgr2=0x7fcf0c106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.475+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/1113155613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c100600 0x7fcf0c106ec0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400b0a0 tx=0x7fcf0400b3b0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.475+0000 7fcf11d7e700 1 -- 172.21.15.2:0/1113155613 shutdown_connections 2023-06-23T18:50:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.475+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/1113155613 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c100600 0x7fcf0c106ec0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.475+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/1113155613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0c0fea80 0x7fcf0c0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.475+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/1113155613 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf0c0ffc70 0x7fcf0c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.475+0000 7fcf11d7e700 1 -- 172.21.15.2:0/1113155613 >> 172.21.15.2:0/1113155613 conn(0x7fcf0c0fa0e0 msgr2=0x7fcf0c0fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:07.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 -- 172.21.15.2:0/1113155613 shutdown_connections 2023-06-23T18:50:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 -- 172.21.15.2:0/1113155613 wait complete. 2023-06-23T18:50:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 Processor -- start 2023-06-23T18:50:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 -- start start 2023-06-23T18:50:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c0fea80 0x7fcf0c102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf0c0ffc70 0x7fcf0c101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0c100600 0x7fcf0c101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c101d30 con 0x7fcf0c0fea80 2023-06-23T18:50:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c101ea0 con 0x7fcf0c0ffc70 2023-06-23T18:50:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.476+0000 7fcf11d7e700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0c19f8e0 con 0x7fcf0c100600 2023-06-23T18:50:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c0fea80 0x7fcf0c102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c0fea80 0x7fcf0c102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58752/0 (socket says 172.21.15.2:58752) 2023-06-23T18:50:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 -- 172.21.15.2:0/2272083871 learned_addr learned my addr 172.21.15.2:0/2272083871 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0affd700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf0c0ffc70 0x7fcf0c101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0bfff700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0c100600 0x7fcf0c101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 -- 172.21.15.2:0/2272083871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0c100600 msgr2=0x7fcf0c101590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0c100600 0x7fcf0c101590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 -- 172.21.15.2:0/2272083871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf0c0ffc70 msgr2=0x7fcf0c101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf0c0ffc70 0x7fcf0c101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 -- 172.21.15.2:0/2272083871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04009d00 con 0x7fcf0c0fea80 2023-06-23T18:50:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf0b7fe700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c0fea80 0x7fcf0c102a10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00e970 tx=0x7fcefc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf08ff9700 1 -- 172.21.15.2:0/2272083871 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc009790 con 0x7fcf0c0fea80 2023-06-23T18:50:07.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf0c19fbc0 con 0x7fcf0c0fea80 2023-06-23T18:50:07.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf08ff9700 1 -- 172.21.15.2:0/2272083871 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcefc00eec0 con 0x7fcf0c0fea80 2023-06-23T18:50:07.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf08ff9700 1 -- 172.21.15.2:0/2272083871 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc003c10 con 0x7fcf0c0fea80 2023-06-23T18:50:07.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.477+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0c1a00d0 con 0x7fcf0c0fea80 2023-06-23T18:50:07.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.478+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0c10a110 con 0x7fcf0c0fea80 2023-06-23T18:50:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.479+0000 7fcf08ff9700 1 -- 172.21.15.2:0/2272083871 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcefc003d90 con 0x7fcf0c0fea80 2023-06-23T18:50:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.480+0000 7fcf08ff9700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcef40a1150 0x7fcef40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.480+0000 7fcf0affd700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcef40a1150 0x7fcef40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.480+0000 7fcf08ff9700 1 -- 172.21.15.2:0/2272083871 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcefc014070 con 0x7fcf0c0fea80 2023-06-23T18:50:07.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.481+0000 7fcf0affd700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcef40a1150 0x7fcef40a3600 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009c60 tx=0x7fcf00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:07.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.484+0000 7fcf08ff9700 1 -- 172.21.15.2:0/2272083871 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7fcefc096440 con 0x7fcf0c0fea80 2023-06-23T18:50:07.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.683+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf0c10a300 con 0x7fcf0c0fea80 2023-06-23T18:50:07.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.683+0000 7fcf08ff9700 1 -- 172.21.15.2:0/2272083871 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcefc099a60 con 0x7fcf0c0fea80 2023-06-23T18:50:07.689 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-06-23T18:50:07.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcef40a1150 msgr2=0x7fcef40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcef40a1150 0x7fcef40a3600 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009c60 tx=0x7fcf00009260 comp rx=0 tx=0).stop 2023-06-23T18:50:07.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c0fea80 msgr2=0x7fcf0c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c0fea80 0x7fcf0c102a10 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00e970 tx=0x7fcefc00ec80 comp rx=0 tx=0).stop 2023-06-23T18:50:07.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 shutdown_connections 2023-06-23T18:50:07.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf0c100600 0x7fcf0c101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf0c0fea80 0x7fcf0c102a10 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7fcef40a1150 0x7fcef40a3600 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 --2- 172.21.15.2:0/2272083871 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf0c0ffc70 0x7fcf0c101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.685+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 >> 172.21.15.2:0/2272083871 conn(0x7fcf0c0fa0e0 msgr2=0x7fcf0c0678a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:07.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.686+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 shutdown_connections 2023-06-23T18:50:07.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.686+0000 7fcf11d7e700 1 -- 172.21.15.2:0/2272083871 wait complete. 2023-06-23T18:50:07.703 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-06-23T18:50:07.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 -- 172.21.15.2:0/984584635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a40fea80 msgr2=0x7f46a40fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 --2- 172.21.15.2:0/984584635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a40fea80 0x7f46a40fee90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4698009990 tx=0x7f4698009ca0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 -- 172.21.15.2:0/984584635 shutdown_connections 2023-06-23T18:50:07.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 --2- 172.21.15.2:0/984584635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a4100600 0x7f46a4106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 --2- 172.21.15.2:0/984584635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a40fea80 0x7f46a40fee90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 --2- 172.21.15.2:0/984584635 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46a40ffc70 0x7f46a41000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 -- 172.21.15.2:0/984584635 >> 172.21.15.2:0/984584635 conn(0x7f46a40fa0e0 msgr2=0x7f46a40fc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:07.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 -- 172.21.15.2:0/984584635 shutdown_connections 2023-06-23T18:50:07.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.826+0000 7f46ac328700 1 -- 172.21.15.2:0/984584635 wait complete. 2023-06-23T18:50:07.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46ac328700 1 Processor -- start 2023-06-23T18:50:07.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46ac328700 1 -- start start 2023-06-23T18:50:07.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46ac328700 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a40fea80 0x7f46a4102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46ac328700 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46a40ffc70 0x7f46a4101050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46ac328700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a4100600 0x7f46a4101590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46ac328700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4101d30 con 0x7f46a4100600 2023-06-23T18:50:07.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46ac328700 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a4101ea0 con 0x7f46a40ffc70 2023-06-23T18:50:07.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46ac328700 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a419f810 con 0x7f46a40fea80 2023-06-23T18:50:07.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a4100600 0x7f46a4101590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a4100600 0x7f46a4101590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58768/0 (socket says 172.21.15.2:58768) 2023-06-23T18:50:07.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 -- 172.21.15.2:0/3868479125 learned_addr learned my addr 172.21.15.2:0/3868479125 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-06-23T18:50:07.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46a98c3700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46a40ffc70 0x7f46a4101050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa0c4700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a40fea80 0x7f46a4102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 -- 172.21.15.2:0/3868479125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a40fea80 msgr2=0x7f46a4102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a40fea80 0x7f46a4102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 -- 172.21.15.2:0/3868479125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46a40ffc70 msgr2=0x7f46a4101050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:07.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46a40ffc70 0x7f46a4101050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:07.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 -- 172.21.15.2:0/3868479125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46980095f0 con 0x7f46a4100600 2023-06-23T18:50:07.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa0c4700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a40fea80 0x7f46a4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:50:07.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46a98c3700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46a40ffc70 0x7f46a4101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-06-23T18:50:07.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.827+0000 7f46aa8c5700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a4100600 0x7f46a4101590 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f469c00ef30 tx=0x7f469c00c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:07.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.828+0000 7f46977fe700 1 -- 172.21.15.2:0/3868479125 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c018070 con 0x7f46a4100600 2023-06-23T18:50:07.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.828+0000 7f46977fe700 1 -- 172.21.15.2:0/3868479125 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f469c00f040 con 0x7f46a4100600 2023-06-23T18:50:07.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.828+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a419fa90 con 0x7f46a4100600 2023-06-23T18:50:07.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.828+0000 7f46977fe700 1 -- 172.21.15.2:0/3868479125 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c003d00 con 0x7f46a4100600 2023-06-23T18:50:07.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.828+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a419ffa0 con 0x7f46a4100600 2023-06-23T18:50:07.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.829+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a410a100 con 0x7f46a4100600 2023-06-23T18:50:07.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.830+0000 7f46977fe700 1 -- 172.21.15.2:0/3868479125 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f469c00caa0 con 0x7f46a4100600 2023-06-23T18:50:07.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.830+0000 7f46977fe700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46900a11d0 0x7f46900a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-06-23T18:50:07.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.830+0000 7f46977fe700 1 -- 172.21.15.2:0/3868479125 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f469c0ccfa0 con 0x7f46a4100600 2023-06-23T18:50:07.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.831+0000 7f46aa0c4700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46900a11d0 0x7f46900a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-06-23T18:50:07.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.831+0000 7f46aa0c4700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46900a11d0 0x7f46900a3680 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4698006010 tx=0x7f4698005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-06-23T18:50:07.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:07.834+0000 7f46977fe700 1 -- 172.21.15.2:0/3868479125 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178008 (secure 0 0 0) 0x7f469c096780 con 0x7f46a4100600 2023-06-23T18:50:08.050 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:07 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2045139610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:07 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3350442665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.051 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:07 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/2272083871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.047+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f46a41a05b0 con 0x7f46a4100600 2023-06-23T18:50:08.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.047+0000 7f46977fe700 1 -- 172.21.15.2:0/3868479125 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+579 (secure 0 0 0) 0x7f469c099da0 con 0x7f46a4100600 2023-06-23T18:50:08.052 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2916792555 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:08.052 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/3628945044 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:08.052 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/3628945044 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:08.052 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/2835880554 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:08.053 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1541129198 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:08.053 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1477099684 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:08.053 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/595372279 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:08.053 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6800/647669756 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:08.053 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:6801/647669756 2023-06-24T18:14:09.903724+0000 2023-06-23T18:50:08.053 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.15.2:0/1251506010 2023-06-24T18:14:36.653577+0000 2023-06-23T18:50:08.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46900a11d0 msgr2=0x7f46900a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:08.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46900a11d0 0x7f46900a3680 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4698006010 tx=0x7f4698005960 comp rx=0 tx=0).stop 2023-06-23T18:50:08.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a4100600 msgr2=0x7f46a4101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-06-23T18:50:08.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a4100600 0x7f46a4101590 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f469c00ef30 tx=0x7f469c00c450 comp rx=0 tx=0).stop 2023-06-23T18:50:08.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 shutdown_connections 2023-06-23T18:50:08.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f46a4100600 0x7f46a4101590 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:08.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a40fea80 0x7f46a4102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:08.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.2:6800/3622114004,v1:172.21.15.2:6801/3622114004] conn(0x7f46900a11d0 0x7f46900a3680 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:08.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 --2- 172.21.15.2:0/3868479125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46a40ffc70 0x7f46a4101050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-06-23T18:50:08.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 >> 172.21.15.2:0/3868479125 conn(0x7f46a40fa0e0 msgr2=0x7f46a4067860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-06-23T18:50:08.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 shutdown_connections 2023-06-23T18:50:08.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-06-23T18:50:08.049+0000 7f46ac328700 1 -- 172.21.15.2:0/3868479125 wait complete. 2023-06-23T18:50:08.056 INFO:teuthology.orchestra.run.smithi002.stderr:listed 10 entries 2023-06-23T18:50:08.066 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi002.front.sepia.ceph.com... 2023-06-23T18:50:08.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:50:08.067 DEBUG:teuthology.orchestra.run.smithi002:> sudo fusermount -u /tmp/cephfs 2023-06-23T18:50:08.122 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi002.stderr:ceph-fuse[143755]: fuse finished with error 0 and tester_r 0 2023-06-23T18:50:08.189 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:07 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2045139610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:07 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3350442665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.190 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:07 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/2272083871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.362 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:07 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2045139610' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.362 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:07 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3350442665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.362 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:07 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/2272083871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-06-23T18:50:08.722 INFO:teuthology.orchestra.run:waiting for 300 2023-06-23T18:50:08.770 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2023-06-23T18:50:08.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:50:08.770 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-06-23T18:50:08.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:50:08.994 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-06-23T18:50:08.995 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:08 smithi002 ceph-mon[95947]: pgmap v1139: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:50:09.095 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-06-23T18:50:09.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:50:09.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-06-23T18:50:09.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-06-23T18:50:09.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-06-23T18:50:09.251 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:08 smithi002 ceph-mon[95947]: from='client.? 172.21.15.2:0/3868479125' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-06-23T18:50:09.252 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-06-23T18:50:09.268 INFO:tasks.cephadm:Teardown begin 2023-06-23T18:50:09.269 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/cephadm.py", line 1677, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-06-23T18:50:09.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:50:09.346 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:50:09.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:08 smithi082 ceph-mon[97889]: pgmap v1139: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:50:09.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:08 smithi082 ceph-mon[97889]: from='client.? 172.21.15.2:0/3868479125' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-06-23T18:50:09.374 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:08 smithi017 ceph-mon[97472]: pgmap v1139: 97 pgs: 97 active+clean; 8.5 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-06-23T18:50:09.374 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:08 smithi017 ceph-mon[97472]: from='client.? 172.21.15.2:0/3868479125' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-06-23T18:50:09.396 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:50:09.437 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-06-23T18:50:09.438 DEBUG:teuthology.orchestra.run.smithi002:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-23T18:50:09.472 DEBUG:teuthology.orchestra.run.smithi017:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-23T18:50:09.493 DEBUG:teuthology.orchestra.run.smithi082:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-06-23T18:50:09.511 INFO:tasks.cephadm:Stopping all daemons... 2023-06-23T18:50:09.511 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-06-23T18:50:09.512 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.a 2023-06-23T18:50:09.801 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:09 smithi002 systemd[1]: Stopping Ceph mon.a for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:10.229 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-a[95922]: 2023-06-23T18:50:09.854+0000 7f491528b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:10.229 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-a[95922]: 2023-06-23T18:50:09.854+0000 7f491528b700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-06-23T18:50:10.551 INFO:journalctl@ceph.mon.a.smithi002.stdout:Jun 23 18:50:10 smithi002 bash[168322]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-a 2023-06-23T18:50:10.653 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.a.service' 2023-06-23T18:50:10.725 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:10.726 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-06-23T18:50:10.726 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-06-23T18:50:10.727 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.b 2023-06-23T18:50:11.169 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:10 smithi017 systemd[1]: Stopping Ceph mon.b for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:11.419 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-b[97448]: 2023-06-23T18:50:11.163+0000 7f180a7bb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:11.420 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-b[97448]: 2023-06-23T18:50:11.163+0000 7f180a7bb700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-06-23T18:50:11.689 INFO:journalctl@ceph.mon.b.smithi017.stdout:Jun 23 18:50:11 smithi017 bash[132762]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-b 2023-06-23T18:50:11.861 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.b.service' 2023-06-23T18:50:11.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:11.921 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-06-23T18:50:11.922 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-06-23T18:50:11.922 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.c 2023-06-23T18:50:12.348 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:11 smithi082 systemd[1]: Stopping Ceph mon.c for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:13.098 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-c[97865]: 2023-06-23T18:50:12.773+0000 7f8e8c445700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:13.099 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-c[97865]: 2023-06-23T18:50:12.773+0000 7f8e8c445700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-06-23T18:50:13.494 INFO:journalctl@ceph.mon.c.smithi082.stdout:Jun 23 18:50:13 smithi082 bash[132554]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mon-c 2023-06-23T18:50:13.821 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mon.c.service' 2023-06-23T18:50:13.868 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:13.869 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-06-23T18:50:13.869 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-06-23T18:50:13.870 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.x 2023-06-23T18:50:14.246 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:50:13 smithi002 systemd[1]: Stopping Ceph mgr.x for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:14.550 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:50:14 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x[96190]: 2023-06-23T18:50:14.240+0000 7ff1b5010700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:14.875 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:50:14 smithi002 bash[168463]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-x 2023-06-23T18:50:14.875 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:50:14 smithi002 systemd[1]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.x.service: Succeeded. 2023-06-23T18:50:14.876 INFO:journalctl@ceph.mgr.x.smithi002.stdout:Jun 23 18:50:14 smithi002 systemd[1]: Stopped Ceph mgr.x for 902733cc-11f1-11ee-9b2b-001a4aab830c. 2023-06-23T18:50:14.878 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.x.service' 2023-06-23T18:50:14.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:14.918 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-06-23T18:50:14.919 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-06-23T18:50:14.919 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.y 2023-06-23T18:50:15.398 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:50:14 smithi017 systemd[1]: Stopping Ceph mgr.y for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:15.689 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:50:15 smithi017 bash[132920]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-y 2023-06-23T18:50:15.824 INFO:journalctl@ceph.mgr.y.smithi017.stdout:Jun 23 18:50:15 smithi017 systemd[1]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-06-23T18:50:15.827 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.y.service' 2023-06-23T18:50:15.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:15.865 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-06-23T18:50:15.866 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-06-23T18:50:15.866 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.z 2023-06-23T18:50:16.309 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:50:15 smithi082 systemd[1]: Stopping Ceph mgr.z for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:16.848 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:50:16 smithi082 bash[133187]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mgr-z 2023-06-23T18:50:16.848 INFO:journalctl@ceph.mgr.z.smithi082.stdout:Jun 23 18:50:16 smithi082 systemd[1]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-06-23T18:50:16.861 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mgr.z.service' 2023-06-23T18:50:16.899 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:16.899 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-06-23T18:50:16.900 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-06-23T18:50:16.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.0 2023-06-23T18:50:17.278 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:50:16 smithi002 systemd[1]: Stopping Ceph osd.0 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:17.550 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:50:17 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-0[108965]: 2023-06-23T18:50:17.272+0000 7f4f4582f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:17.551 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:50:17 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-0[108965]: 2023-06-23T18:50:17.272+0000 7f4f4582f700 -1 osd.0 84 *** Got signal Terminated *** 2023-06-23T18:50:17.551 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:50:17 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-0[108965]: 2023-06-23T18:50:17.272+0000 7f4f4582f700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:50:23.550 INFO:journalctl@ceph.osd.0.smithi002.stdout:Jun 23 18:50:23 smithi002 bash[168608]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-0 2023-06-23T18:50:24.932 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.0.service' 2023-06-23T18:50:24.973 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:24.974 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-06-23T18:50:24.975 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-06-23T18:50:24.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.1 2023-06-23T18:50:25.349 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:50:25 smithi017 systemd[1]: Stopping Ceph osd.1 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:25.690 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:50:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-1[102600]: 2023-06-23T18:50:25.344+0000 7f02c9160700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:25.690 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:50:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-1[102600]: 2023-06-23T18:50:25.344+0000 7f02c9160700 -1 osd.1 84 *** Got signal Terminated *** 2023-06-23T18:50:25.690 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:50:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-1[102600]: 2023-06-23T18:50:25.344+0000 7f02c9160700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:50:31.690 INFO:journalctl@ceph.osd.1.smithi017.stdout:Jun 23 18:50:31 smithi017 bash[133063]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-1 2023-06-23T18:50:32.702 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.1.service' 2023-06-23T18:50:32.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:32.745 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-06-23T18:50:32.745 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-06-23T18:50:32.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.2 2023-06-23T18:50:33.098 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:50:32 smithi082 systemd[1]: Stopping Ceph osd.2 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:33.598 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:50:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-2[103784]: 2023-06-23T18:50:33.102+0000 7f178487b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:33.598 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:50:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-2[103784]: 2023-06-23T18:50:33.102+0000 7f178487b700 -1 osd.2 84 *** Got signal Terminated *** 2023-06-23T18:50:33.598 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:50:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-2[103784]: 2023-06-23T18:50:33.102+0000 7f178487b700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:50:39.348 INFO:journalctl@ceph.osd.2.smithi082.stdout:Jun 23 18:50:39 smithi082 bash[133339]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-2 2023-06-23T18:50:40.621 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.2.service' 2023-06-23T18:50:40.663 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:40.664 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-06-23T18:50:40.664 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-06-23T18:50:40.664 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.3 2023-06-23T18:50:41.050 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:50:40 smithi002 systemd[1]: Stopping Ceph osd.3 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:41.300 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:50:41 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3[114478]: 2023-06-23T18:50:41.045+0000 7f85ac8f9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:41.301 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:50:41 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3[114478]: 2023-06-23T18:50:41.045+0000 7f85ac8f9700 -1 osd.3 84 *** Got signal Terminated *** 2023-06-23T18:50:41.301 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:50:41 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3[114478]: 2023-06-23T18:50:41.045+0000 7f85ac8f9700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:50:44.550 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:50:44 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3[114478]: 2023-06-23T18:50:44.194+0000 7f85a76ec700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:18.601270+0000 front 2023-06-23T18:50:18.601307+0000 (oldest deadline 2023-06-23T18:50:43.301495+0000) 2023-06-23T18:50:44.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:44 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:44.279+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:45.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:44.813+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:45.550 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:50:45 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3[114478]: 2023-06-23T18:50:45.163+0000 7f85a76ec700 -1 osd.3 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:18.601270+0000 front 2023-06-23T18:50:18.601307+0000 (oldest deadline 2023-06-23T18:50:43.301495+0000) 2023-06-23T18:50:45.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:45 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:45.316+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:46.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:45.782+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:46.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:46 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:46.289+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:46.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:46 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:46.193+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:47.051 INFO:journalctl@ceph.osd.3.smithi002.stdout:Jun 23 18:50:46 smithi002 bash[168882]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-3 2023-06-23T18:50:47.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:46 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:46.738+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:47.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:46.811+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:47.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:47 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:47.277+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:47.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:47 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:47.234+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:48.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:47 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:47.709+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:48.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:47.796+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:48.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:48 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:48.313+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:48.672 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.3.service' 2023-06-23T18:50:48.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:48 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:48.232+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:48.717 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:48.717 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-06-23T18:50:48.718 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-06-23T18:50:48.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.4 2023-06-23T18:50:48.837 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:48.542+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:49.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:48 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:48.745+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:49.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:48.833+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:49.109 INFO:journalctl@ceph.osd.4.smithi017.stdout:Jun 23 18:50:48 smithi017 systemd[1]: Stopping Ceph osd.4 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:49.439 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:49 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:49.215+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:49.440 INFO:journalctl@ceph.osd.4.smithi017.stdout:Jun 23 18:50:49 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-4[108080]: 2023-06-23T18:50:49.104+0000 7f64e81f0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:49.440 INFO:journalctl@ceph.osd.4.smithi017.stdout:Jun 23 18:50:49 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-4[108080]: 2023-06-23T18:50:49.104+0000 7f64e81f0700 -1 osd.4 84 *** Got signal Terminated *** 2023-06-23T18:50:49.440 INFO:journalctl@ceph.osd.4.smithi017.stdout:Jun 23 18:50:49 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-4[108080]: 2023-06-23T18:50:49.104+0000 7f64e81f0700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:50:49.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:49.288+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:49.792 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:49 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:49.420+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:49.793 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:49 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:49.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:50.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:49 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:49.713+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:50.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:49 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:49.788+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:50.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:50 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:50.244+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:50.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:50 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:50.250+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:50.801 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:50 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:50.399+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:50.801 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:50 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:50.581+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:51.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:50 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:50.673+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:51.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:50 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:50.797+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:51.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:51 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:51.287+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:51.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:51 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:51.266+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:51.840 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:51 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:51.413+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:51.840 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:51 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:51.574+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:52.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:51 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:51.690+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:52.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:51 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:51.691+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:52.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:51 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:51.836+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:52.688 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:52 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:52.312+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:52.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:52 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:52.302+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:52.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:52 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:52.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:52.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:52 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:52.446+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:53.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:52 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:52.684+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:53.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:52 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:52.684+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:53.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:52 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:52.872+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:53.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:52 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:52.872+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:50:53.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:53 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:53.337+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:53.700 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:53 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:53.314+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:53.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:53 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:53.490+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:53.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:53 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:53.601+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:53.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:53 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:53.601+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:50:54.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:53 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:53.696+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:54.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:53 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:53.696+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:54.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:53 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:53.846+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:54.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:53 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:53.846+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:50:54.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:54 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:54.322+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:54.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:54 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:54.322+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:50:54.698 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:54 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:54.309+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:54.698 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:54 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:54.309+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:50:54.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:54 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:54.594+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:54.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:54 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:54.594+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:50:54.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:54 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:54.485+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:54.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:54 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:54.485+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:50:55.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:54 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:54.694+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:55.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:54 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:54.694+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:55.065 INFO:journalctl@ceph.osd.4.smithi017.stdout:Jun 23 18:50:55 smithi017 bash[133335]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-4 2023-06-23T18:50:55.065 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:50:54 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:50:54.883+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:50:55.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:54 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:54.887+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:55.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:54 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:54.887+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:50:55.633 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:55 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:55.287+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:55.633 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:55 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:55.287+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:50:55.684 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:55 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:55.341+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:55.685 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:55 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:55.341+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:50:55.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:55 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:55.470+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:55.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:55 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:55.470+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:50:55.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:55 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:55.553+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:55.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:55 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:55.553+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:50:56.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:55 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:55.681+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:56.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:55 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:55.681+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:56.190 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:50:55 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:50:55.918+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:50:56.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:55 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:55.902+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:56.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:55 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:55.902+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:50:56.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.4.service' 2023-06-23T18:50:56.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:50:56.480 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-06-23T18:50:56.480 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-06-23T18:50:56.481 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.5 2023-06-23T18:50:56.638 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:56 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:56.343+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:56.638 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:56 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:56.343+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:50:56.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:56 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:56.293+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:56.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:56 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:56.293+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:50:56.833 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:56 smithi082 systemd[1]: Stopping Ceph osd.5 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:50:56.834 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:56.464+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:56.834 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:56.464+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:50:56.835 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:56.592+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:56.835 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:56.592+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:50:57.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:56 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:56.633+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:57.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:56 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:56.633+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:57.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:56.829+0000 7f7293893700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:50:57.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:56.829+0000 7f7293893700 -1 osd.5 84 *** Got signal Terminated *** 2023-06-23T18:50:57.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:56.829+0000 7f7293893700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:50:57.098 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:56.918+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:57.099 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:56 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:56.918+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:50:57.289 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:50:56 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:50:56.938+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:50:57.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:57 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:57.285+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:57.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:57 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:57.285+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:50:57.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:57 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:57.390+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:57.802 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:57 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:57.390+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:50:57.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:57 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:57.594+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:57.803 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:57 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:57.594+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:57.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:57 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:57.464+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:57.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:57 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:57.464+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:50:57.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:57 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:57.602+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:57.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:57 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:57.602+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:50:58.189 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:50:57 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:50:57.902+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:50:58.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:57 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:57.961+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:58.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:57 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:57.961+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:50:58.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:58 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:58.242+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:58.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:58 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:58.242+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:50:58.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:58 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:58.362+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:58.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:58 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:58.362+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:50:58.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:58 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:58.564+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:58.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:58 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:58.565+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:58.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:58 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:58.603+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:58.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:58 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:58.603+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:50:58.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:58 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:58.460+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:58.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:58 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:58.460+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:50:59.189 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:50:58 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:50:58.855+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:50:59.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:58 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:58.944+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:50:59.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:58 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:58.944+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:50:59.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:59 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:59.243+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:50:59.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:50:59 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:50:59.243+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:50:59.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:59 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:59.356+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:50:59.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:50:59 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:50:59.356+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:50:59.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:59 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:59.551+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:50:59.802 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:50:59 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:50:59.551+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:50:59.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:59 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:59.612+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:50:59.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:50:59 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:50:59.612+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:50:59.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:59 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:59.484+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:50:59.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:50:59 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:50:59.484+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:00.189 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:50:59 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:50:59.894+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:00.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:59 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:59.951+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:51:00.349 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:50:59 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:50:59.951+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:51:00.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:00 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:00.195+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:00.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:00 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:00.195+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:00.800 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:00 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:00.341+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:00.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:00 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:00.341+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:00.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:00 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:00.341+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:00.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:00 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:00.556+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:00.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:00 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:00.556+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:00.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:00 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:00.622+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:00.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:00 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:00.622+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:00.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:00 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:00.524+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:00.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:00 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:00.524+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:00.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:00 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:00.524+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:01.189 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:00 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:00.928+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:01.190 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:00 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:00.928+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:01.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:51:00 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:51:00.977+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.237761+0000 front 2023-06-23T18:50:21.237927+0000 (oldest deadline 2023-06-23T18:50:44.137940+0000) 2023-06-23T18:51:01.348 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:51:00 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5[109288]: 2023-06-23T18:51:00.977+0000 7f728de85700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:29.141233+0000 front 2023-06-23T18:50:29.141275+0000 (oldest deadline 2023-06-23T18:50:52.041334+0000) 2023-06-23T18:51:01.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:01 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:01.221+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:01.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:01 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:01.221+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:01.800 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:01 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:01.381+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:01.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:01 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:01.381+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:01.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:01 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:01.381+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:01.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:01 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:01.526+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:01.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:01 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:01.526+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:01.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:01 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:01.521+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:01.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:01 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:01.521+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:01.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:01 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:01.521+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:01.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:01 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:01.633+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:01.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:01 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:01.633+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:01.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:01 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:01.633+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:02.190 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:01 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:01.905+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:02.190 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:01 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:01.905+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:02.681 INFO:journalctl@ceph.osd.5.smithi082.stdout:Jun 23 18:51:02 smithi082 bash[133612]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-5 2023-06-23T18:51:02.681 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:02 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:02.499+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:02.681 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:02 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:02.499+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:02.682 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:02 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:02.499+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:02.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:02 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:02.241+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:02.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:02 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:02.241+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:02.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:02 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:02.241+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:02.800 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:02.343+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:02.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:02.343+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:02.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:02.343+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:02.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:02.573+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:02.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:02.573+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:02.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:02 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:02.573+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:03.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:02 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:02.677+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:03.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:02 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:02.677+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:03.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:02 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:02.677+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:03.295 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:02 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:02.949+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:03.295 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:02 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:02.949+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:03.598 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:03 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:03.514+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:03.598 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:03 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:03.514+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:03.598 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:03 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:03.514+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:03.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:03 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:03.291+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:03.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:03 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:03.291+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:03.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:03 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:03.291+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:03.800 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:03.347+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:03.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:03.347+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:03.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:03.347+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:03.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:03.589+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:03.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:03.589+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:03.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:03 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:03.589+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:03.974 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:03 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:03.708+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:03.974 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:03 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:03.708+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:03.974 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:03 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:03.708+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:03.981 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.5.service' 2023-06-23T18:51:04.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:04.022 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-06-23T18:51:04.022 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-06-23T18:51:04.023 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.6 2023-06-23T18:51:04.189 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:03 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:03.904+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:04.190 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:03 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:03.904+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:04.332 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:04 smithi002 systemd[1]: Stopping Ceph osd.6 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:04.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:04 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:04.285+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:04.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:04 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:04.285+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:04.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:04 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:04.285+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:04.800 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:04.571+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:04.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:04.571+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:04.801 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:04.571+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:04.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:04.327+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:04.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:04.327+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:04.801 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:04.327+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:04.802 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:04.407+0000 7fbb0f53d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:04.802 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:04.407+0000 7fbb0f53d700 -1 osd.6 84 *** Got signal Terminated *** 2023-06-23T18:51:04.802 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:04 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:04.407+0000 7fbb0f53d700 -1 osd.6 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:51:04.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:04 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:04.529+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:04.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:04 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:04.529+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:04.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:04 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:04.529+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:04.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:04 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:04.680+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:04.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:04 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:04.680+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:04.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:04 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:04.680+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:05.189 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:04 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:04.887+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:05.190 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:04 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:04.887+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:05.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:05 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:05.281+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:05.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:05 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:05.281+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:05.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:05 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:05.281+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:05.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:05 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:05.283+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:05.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:05 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:05.283+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:05.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:05 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:05.283+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:05.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:05 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:05.541+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:05.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:05 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:05.541+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:05.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:05 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:05.541+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:05.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:05 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:05.666+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:05.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:05 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:05.666+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:05.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:05 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:05.666+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:06.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:05 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:05.606+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:06.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:05 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:05.606+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:06.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:05 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:05.606+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:06.269 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:05 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:05.936+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:06.270 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:05 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:05.936+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:06.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:06.244+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:06.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:06.244+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:06.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:06.244+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:06.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:06 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:06.265+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:06.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:06 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:06.265+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:06.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:06 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:06.265+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:06.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:06 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:06.591+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:06.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:06 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:06.591+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:06.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:06 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:06.591+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:06.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:06 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:06.645+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:06.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:06 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:06.645+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:06.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:06 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:06.645+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:07.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:06.602+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:07.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:06.602+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:07.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:06 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:06.602+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:07.257 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:06 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:06.983+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:07.257 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:06 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:06.983+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:07.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:07 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:07.233+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:07.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:07 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:07.233+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:07.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:07 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:07.233+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:07.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:07 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:07.253+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:07.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:07 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:07.253+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:07.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:07 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:07.253+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:07.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:07 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:07.548+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:07.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:07 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:07.548+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:07.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:07 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:07.548+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:07.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:07 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:07.655+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:07.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:07 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:07.655+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:07.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:07 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:07.655+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:07.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:07 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:07.655+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:08.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:07 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:07.650+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:08.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:07 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:07.650+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:08.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:07 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:07.650+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:08.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:07 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:07.650+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:08.439 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:08 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:08.023+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:08.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:08 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:08.023+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:08.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:08 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:08.023+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:08.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:08 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:08.213+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:08.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:08 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:08.213+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:08.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:08 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:08.213+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:08.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:08.275+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:08.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:08.275+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:08.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:08.275+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:08.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:08.275+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.316765+0000 front 2023-06-23T18:50:42.316787+0000 (oldest deadline 2023-06-23T18:51:08.216891+0000) 2023-06-23T18:51:08.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:08 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:08.510+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:08.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:08 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:08.510+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:08.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:08 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:08.510+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:08.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:08 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:08.658+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:08.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:08 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:08.658+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:08.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:08 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:08.658+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:08.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:08 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:08.658+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:09.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:08.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:09.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:08.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:09.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:08.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:09.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:08 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:08.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:09.439 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:09 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:09.031+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:09.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:09 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:09.031+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:09.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:09 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:09.031+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:09.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:09 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:09.234+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:09.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:09 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:09.234+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:09.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:09 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:09.234+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:09.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:09.228+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.711673+0000 front 2023-06-23T18:50:19.711729+0000 (oldest deadline 2023-06-23T18:50:43.811846+0000) 2023-06-23T18:51:09.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:09.228+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.314155+0000 front 2023-06-23T18:50:28.314253+0000 (oldest deadline 2023-06-23T18:50:53.614235+0000) 2023-06-23T18:51:09.551 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:09.228+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.115512+0000 front 2023-06-23T18:50:34.115487+0000 (oldest deadline 2023-06-23T18:50:59.415584+0000) 2023-06-23T18:51:09.552 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6[120061]: 2023-06-23T18:51:09.228+0000 7fbb09b2f700 -1 osd.6 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.316765+0000 front 2023-06-23T18:50:42.316787+0000 (oldest deadline 2023-06-23T18:51:08.216891+0000) 2023-06-23T18:51:09.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:09 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:09.534+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:09.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:09 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:09.534+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:09.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:09 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:09.534+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:09.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:09 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:09.672+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:09.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:09 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:09.672+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:09.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:09 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:09.672+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:09.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:09 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:09.672+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:10.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:09.701+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:10.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:09.701+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:10.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:09.701+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:10.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:09 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:09.701+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:10.439 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:10 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:10.021+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:10.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:10 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:10.021+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:10.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:10 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:10.021+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:10.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:10 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:10.246+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:10.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:10 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:10.246+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:10.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:10 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:10.246+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:10.550 INFO:journalctl@ceph.osd.6.smithi002.stdout:Jun 23 18:51:10 smithi002 bash[169156]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-6 2023-06-23T18:51:10.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:10 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:10.692+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:10.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:10 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:10.692+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:10.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:10 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:10.692+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:10.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:10 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:10.692+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:10.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:10 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:10.548+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:10.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:10 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:10.548+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:10.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:10 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:10.548+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:10.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:10 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:10.548+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:10.968 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:10 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:10.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:10.969 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:10 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:10.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:10.969 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:10 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:10.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:10.969 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:10 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:10.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:11.439 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:11.050+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:11.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:11.050+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:11.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:11.050+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:11.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:11.214+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:11.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:11.214+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:11.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:11.214+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:11.834 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.6.service' 2023-06-23T18:51:11.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:11 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:11.715+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:11.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:11 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:11.715+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:11.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:11 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:11.715+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:11.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:11 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:11.715+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:11.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:11 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:11.564+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:11.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:11 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:11.564+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:11.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:11 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:11.564+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:11.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:11 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:11.564+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:11.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:11.877 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-06-23T18:51:11.878 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-06-23T18:51:11.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.7 2023-06-23T18:51:12.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:11 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:11.704+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:12.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:11 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:11.704+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:12.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:11 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:11.704+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:12.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:11 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:11.704+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:12.183 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:11 smithi017 systemd[1]: Stopping Ceph osd.7 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:12.184 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:12.081+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:12.184 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:12.081+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:12.184 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:12.081+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:12.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:12.256+0000 7eff25feb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:12.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:12.256+0000 7eff25feb700 -1 osd.7 84 *** Got signal Terminated *** 2023-06-23T18:51:12.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:12.256+0000 7eff25feb700 -1 osd.7 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:51:12.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:12.179+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:12.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:12.179+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:12.442 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:12.179+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:12.442 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:12 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:12.179+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:12.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:12.567+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:12.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:12.567+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:12.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:12.567+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:12.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:12.567+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:12.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:12.739+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:12.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:12.739+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:12.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:12.739+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:12.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:12 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:12.739+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:13.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:12 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:12.708+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:13.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:12 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:12.708+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:13.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:12 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:12.708+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:13.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:12 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:12.708+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:13.439 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:13 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:13.132+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:13.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:13 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:13.132+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:13.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:13 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:13.132+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:13.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:13 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:13.203+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:13.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:13 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:13.203+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:13.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:13 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:13.203+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:13.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:13 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:13.203+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:13.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:13 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:13.533+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:13.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:13 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:13.533+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:13.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:13 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:13.533+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:13.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:13 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:13.533+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:13.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:13 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:13.718+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:13.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:13 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:13.718+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:13.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:13 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:13.718+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:13.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:13 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:13.718+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:14.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:13 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:13.677+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:14.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:13 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:13.677+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:14.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:13 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:13.677+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:14.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:13 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:13.677+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:14.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:14 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:14.109+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:14.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:14 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:14.109+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:14.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:14 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:14.109+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:14.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:14 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:14.191+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:14.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:14 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:14.191+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:14.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:14 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:14.191+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:14.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:14 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:14.191+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:14.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:14.485+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:14.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:14.485+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:14.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:14.485+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:14.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:14.485+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:14.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:14.728+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:14.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:14.728+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:14.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:14.728+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:14.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:14 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:14.728+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:15.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:14 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:14.716+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:15.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:14 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:14.716+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:15.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:14 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:14.716+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:15.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:14 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:14.716+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:15.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:15 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:15.076+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:15.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:15 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:15.076+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:15.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:15 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:15.076+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:15.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:15 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:15.185+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:15.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:15 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:15.185+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:15.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:15 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:15.185+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:15.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:15 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:15.185+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:15.769 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:15.456+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:15.769 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:15.456+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:15.770 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:15.456+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:15.770 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:15.456+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:16.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:15.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:16.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:15.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:16.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:15.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:16.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:15.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:16.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:15 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:15.686+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:16.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:15.764+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:16.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:15.764+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:16.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:15.764+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:16.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:15.764+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:16.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:15 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:15.764+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:16.439 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:16.061+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:16.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:16.061+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:16.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:16.061+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:16.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:16.061+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.926734+0000 front 2023-06-23T18:50:52.926763+0000 (oldest deadline 2023-06-23T18:51:15.826875+0000) 2023-06-23T18:51:16.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:16.202+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:16.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:16.202+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:16.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:16.202+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:16.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:16.202+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:16.738 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:16.471+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:16.738 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:16.471+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:16.738 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:16.471+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:16.739 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:16.471+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:17.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:16 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:16.699+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:17.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:16 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:16.699+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:17.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:16 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:16.699+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:17.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:16 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:16.699+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:17.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:16 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:16.699+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:17.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:16.733+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:17.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:16.733+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:17.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:16.733+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:17.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:16.733+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:17.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:16 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:16.733+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:17.439 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:17.074+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.620293+0000 front 2023-06-23T18:50:28.620321+0000 (oldest deadline 2023-06-23T18:50:53.920560+0000) 2023-06-23T18:51:17.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:17.074+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.021828+0000 front 2023-06-23T18:50:38.021751+0000 (oldest deadline 2023-06-23T18:51:00.822588+0000) 2023-06-23T18:51:17.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:17.074+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:44.323922+0000 front 2023-06-23T18:50:44.323854+0000 (oldest deadline 2023-06-23T18:51:07.823853+0000) 2023-06-23T18:51:17.440 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7[113622]: 2023-06-23T18:51:17.074+0000 7eff20dde700 -1 osd.7 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.926734+0000 front 2023-06-23T18:50:52.926763+0000 (oldest deadline 2023-06-23T18:51:15.826875+0000) 2023-06-23T18:51:17.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:17.196+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:17.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:17.196+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:17.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:17.196+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:17.441 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:17 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:17.196+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:17.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:17.492+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:17.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:17.492+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:17.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:17.493+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:17.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:17.493+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:17.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:17.493+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:53.155630+0000 front 2023-06-23T18:50:53.155820+0000 (oldest deadline 2023-06-23T18:51:16.655519+0000) 2023-06-23T18:51:17.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:17.693+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:17.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:17.693+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:17.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:17.693+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:17.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:17.693+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:17.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:17 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:17.693+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:18.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:17 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:17.720+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:18.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:17 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:17.720+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:18.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:17 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:17.720+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:18.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:17 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:17.720+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:18.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:17 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:17.720+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:18.127 INFO:journalctl@ceph.osd.7.smithi017.stdout:Jun 23 18:51:18 smithi017 bash[133605]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-7 2023-06-23T18:51:18.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:18 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:18.245+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:18.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:18 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:18.245+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:18.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:18 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:18.245+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:18.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:18 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:18.245+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:18.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:18.535+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:18.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:18.535+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:18.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:18.535+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:18.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:18.535+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:18.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:18.535+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:53.155630+0000 front 2023-06-23T18:50:53.155820+0000 (oldest deadline 2023-06-23T18:51:16.655519+0000) 2023-06-23T18:51:18.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:18.725+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:18.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:18.725+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:18.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:18.725+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:18.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:18.725+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:18.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:18 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:18.725+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:19.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:18 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:18.688+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:19.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:18 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:18.688+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:19.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:18 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:18.688+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:19.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:18 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:18.688+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:19.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:18 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:18.688+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:19.439 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:19 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:19.286+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:19.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:19 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:19.286+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:19.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:19 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:19.286+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:19.440 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:19 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:19.286+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:19.504 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.7.service' 2023-06-23T18:51:19.546 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:19.547 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-06-23T18:51:19.547 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-06-23T18:51:19.548 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.8 2023-06-23T18:51:19.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:19.573+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:19.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:19.573+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:19.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:19.573+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:19.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:19.573+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:19.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:19.573+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:53.155630+0000 front 2023-06-23T18:50:53.155820+0000 (oldest deadline 2023-06-23T18:51:16.655519+0000) 2023-06-23T18:51:19.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 systemd[1]: Stopping Ceph osd.8 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:19.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:19.697+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:19.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:19.697+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:19.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:19.697+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:19.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:19.697+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:19.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:19.697+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:20.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:19 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:19.663+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:20.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:19 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:19.663+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:20.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:19 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:19.663+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:20.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:19 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:19.663+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:20.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:19 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:19.663+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:20.348 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:19.908+0000 7f0f0183c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:20.348 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:19.908+0000 7f0f0183c700 -1 osd.8 84 *** Got signal Terminated *** 2023-06-23T18:51:20.348 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:19 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:19.908+0000 7f0f0183c700 -1 osd.8 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:51:20.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:20 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:20.316+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:20.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:20 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:20.316+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:20.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:20 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:20.316+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:20.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:20 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:20.316+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:20.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:20 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:20.316+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:20.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:20.572+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:20.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:20.572+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:20.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:20.572+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:20.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:20.572+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:20.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:20.572+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:53.155630+0000 front 2023-06-23T18:50:53.155820+0000 (oldest deadline 2023-06-23T18:51:16.655519+0000) 2023-06-23T18:51:20.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:20.705+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:20.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:20.705+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:20.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:20.705+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:20.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:20.705+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:20.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:20 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:20.705+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:21.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:20 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:20.631+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:21.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:20 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:20.631+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:21.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:20 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:20.631+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:21.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:20 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:20.631+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:21.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:20 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:20.631+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:21.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:21 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:21.284+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:21.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:21 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:21.284+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:21.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:21 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:21.284+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:21.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:21 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:21.284+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:21.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:21 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:21.284+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:21.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:21.699+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:21.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:21.699+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:21.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:21.699+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:21.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:21.699+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:21.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:21.699+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:21.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:21.569+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:21.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:21.569+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:21.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:21.569+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:21.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:21.569+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:21.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:21 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:21.569+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:53.155630+0000 front 2023-06-23T18:50:53.155820+0000 (oldest deadline 2023-06-23T18:51:16.655519+0000) 2023-06-23T18:51:22.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:21 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:21.637+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:22.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:21 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:21.637+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:22.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:21 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:21.637+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:22.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:21 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:21.637+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:22.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:21 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:21.637+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:22.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:22 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:22.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:22.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:22 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:22.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:22.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:22 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:22.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:22.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:22 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:22.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:22.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:22 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:22.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:22.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:22.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:22.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:22.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:22.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:22.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:22.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:22.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:22.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:22.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:22.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:22.561+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:22.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:22.561+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:22.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:22.561+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:22.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:22.561+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:22.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:22.561+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:53.155630+0000 front 2023-06-23T18:50:53.155820+0000 (oldest deadline 2023-06-23T18:51:16.655519+0000) 2023-06-23T18:51:23.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:22 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:22.656+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:23.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:22 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:22.656+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:23.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:22 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:22.656+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:23.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:22 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:22.656+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:23.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:22 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:22.656+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:23.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:23 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:23.275+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:23.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:23 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:23.275+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:23.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:23 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:23.275+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:23.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:23 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:23.275+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:23.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:23 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:23.275+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:23.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:23 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:23.275+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:23.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:23.537+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:23.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:23.537+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:23.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:23.538+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:23.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:23.538+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:23.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:23.538+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:53.155630+0000 front 2023-06-23T18:50:53.155820+0000 (oldest deadline 2023-06-23T18:51:16.655519+0000) 2023-06-23T18:51:23.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:23.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:23.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:23.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:23.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:23.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:23.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:23.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:23.850 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:23 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:23.681+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:24.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:23 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:23.696+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:24.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:23 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:23.696+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:24.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:23 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:23.696+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:24.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:23 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:23.696+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:24.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:23 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:23.696+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:24.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:24 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:24.258+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:24.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:24 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:24.258+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:24.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:24 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:24.258+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:24.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:24 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:24.258+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:24.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:24 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:24.258+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:24.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:24 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:24.258+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:24.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:24.656+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:24.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:24.656+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:24.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:24.656+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:24.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:24.656+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:24.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:24.656+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:24.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:24.515+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:19.849853+0000 front 2023-06-23T18:50:19.849690+0000 (oldest deadline 2023-06-23T18:50:48.650485+0000) 2023-06-23T18:51:24.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:24.515+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.651020+0000 front 2023-06-23T18:50:28.651048+0000 (oldest deadline 2023-06-23T18:50:53.951158+0000) 2023-06-23T18:51:24.849 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:24.515+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:33.951673+0000 front 2023-06-23T18:50:33.951760+0000 (oldest deadline 2023-06-23T18:51:00.352495+0000) 2023-06-23T18:51:24.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:24.515+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:43.853730+0000 front 2023-06-23T18:50:43.853748+0000 (oldest deadline 2023-06-23T18:51:10.254562+0000) 2023-06-23T18:51:24.850 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:24 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8[114723]: 2023-06-23T18:51:24.515+0000 7f0efa62b700 -1 osd.8 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:53.155630+0000 front 2023-06-23T18:50:53.155820+0000 (oldest deadline 2023-06-23T18:51:16.655519+0000) 2023-06-23T18:51:25.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:24 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:24.653+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:25.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:24 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:24.653+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:25.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:24 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:24.653+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:25.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:24 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:24.653+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:25.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:24 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:24.653+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:25.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:24 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:24.653+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:25.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:25.254+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:25.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:25.254+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:25.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:25.254+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:25.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:25.254+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:25.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:25.254+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:25.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:25 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:25.254+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:25.848 INFO:journalctl@ceph.osd.8.smithi082.stdout:Jun 23 18:51:25 smithi082 bash[133878]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-8 2023-06-23T18:51:25.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:25 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:25.628+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:25.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:25 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:25.628+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:25.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:25 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:25.628+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:25.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:25 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:25.628+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:25.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:25 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:25.628+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:26.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:25 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:25.646+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:26.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:25 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:25.646+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:26.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:25 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:25.646+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:26.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:25 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:25.646+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:26.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:25 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:25.646+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:26.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:25 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:25.646+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:26.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:26 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:26.267+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:26.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:26 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:26.267+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:26.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:26 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:26.267+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:26.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:26 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:26.267+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:26.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:26 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:26.267+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:26.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:26 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:26.267+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:26.750 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:26.610+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:26.750 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:26.610+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:26.750 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:26.610+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:26.750 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:26.610+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:26.751 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:26.610+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:26.751 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:26.610+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:27.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:26 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:26.675+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:27.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:26 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:26.675+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:27.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:26 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:26.675+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:27.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:26 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:26.675+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:27.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:26 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:26.675+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:27.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:26 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:26.675+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:27.178 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.8.service' 2023-06-23T18:51:27.219 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:27.219 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-06-23T18:51:27.220 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-06-23T18:51:27.220 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.9 2023-06-23T18:51:27.550 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 systemd[1]: Stopping Ceph osd.9 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:27.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:27 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:27.261+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:27.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:27 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:27.261+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:27.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:27 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:27.261+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:27.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:27 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:27.261+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:27.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:27 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:27.261+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:27.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:27 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:27.261+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:28.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.596+0000 7f7e05c87700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:28.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.596+0000 7f7e05c87700 -1 osd.9 84 *** Got signal Terminated *** 2023-06-23T18:51:28.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.596+0000 7f7e05c87700 -1 osd.9 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:51:28.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.626+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:28.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.626+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:28.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.626+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:28.054 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.626+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:28.054 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.626+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:28.054 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:27 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:27.626+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:28.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:27 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:27.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:28.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:27 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:27.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:28.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:27 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:27.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:28.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:27 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:27.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:28.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:27 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:27.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:28.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:27 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:27.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:28.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:28 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:28.277+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:28.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:28 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:28.277+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:28.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:28 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:28.277+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:28.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:28 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:28.277+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:28.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:28 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:28.277+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:28.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:28 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:28.277+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:28.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:28 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:28.566+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:28.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:28 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:28.566+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:28.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:28 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:28.566+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:28.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:28 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:28.566+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:28.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:28 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:28.566+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:28.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:28 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:28.566+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:29.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:28.613+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:29.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:28.613+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:29.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:28.613+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:29.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:28.613+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:29.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:28.613+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:29.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:28 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:28.613+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:29.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:29 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:29.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:29.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:29 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:29.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:29.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:29 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:29.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:29.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:29 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:29.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:29.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:29 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:29.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:29.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:29 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:29.264+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:29.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:29 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:29.588+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:29.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:29 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:29.588+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:29.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:29 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:29.588+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:29.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:29 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:29.588+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:29.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:29 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:29.588+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:29.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:29 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:29.588+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:30.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:29.584+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:30.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:29.584+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:30.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:29.584+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:30.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:29.584+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:30.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:29.584+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:30.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:29 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:29.584+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:30.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:30 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:30.294+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:30.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:30 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:30.294+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:30.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:30 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:30.294+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:30.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:30 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:30.294+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:30.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:30 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:30.294+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:30.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:30 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:30.294+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:31.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:30.577+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:31.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:30.577+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:31.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:30.577+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:31.052 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:30.577+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:31.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:30.577+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:31.053 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:30 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:30.577+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:31.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:30 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:30.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:31.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:30 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:30.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:31.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:30 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:30.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:31.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:30 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:30.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:31.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:30 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:30.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:31.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:30 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:30.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:31.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:31.299+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:31.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:31.299+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:31.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:31.299+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:31.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:31.299+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:31.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:31.299+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:31.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:31 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:31.299+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:32.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:31 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:31.579+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:32.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:31 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:31.579+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:32.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:31 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:31.579+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:32.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:31 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:31.579+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:32.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:31 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:31.579+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:32.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:31 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:31.579+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:32.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:31 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:31.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:32.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:31 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:31.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:32.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:31 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:31.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:32.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:31 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:31.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:32.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:31 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:31.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:32.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:31 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:31.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:32.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:32 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:32.321+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:32.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:32 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:32.321+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:32.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:32 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:32.321+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:32.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:32 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:32.321+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:32.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:32 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:32.321+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:32.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:32 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:32.321+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:32.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:32 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:32.321+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:33.050 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:32.565+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.678481+0000 front 2023-06-23T18:50:21.678450+0000 (oldest deadline 2023-06-23T18:50:46.279394+0000) 2023-06-23T18:51:33.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:32.565+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:26.280034+0000 front 2023-06-23T18:50:26.280127+0000 (oldest deadline 2023-06-23T18:50:51.580167+0000) 2023-06-23T18:51:33.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:32.565+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:35.681589+0000 front 2023-06-23T18:50:35.681461+0000 (oldest deadline 2023-06-23T18:51:01.581593+0000) 2023-06-23T18:51:33.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:32.565+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.582069+0000 front 2023-06-23T18:50:41.582048+0000 (oldest deadline 2023-06-23T18:51:06.882273+0000) 2023-06-23T18:51:33.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:32.565+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:50.983437+0000 front 2023-06-23T18:50:50.983552+0000 (oldest deadline 2023-06-23T18:51:15.083260+0000) 2023-06-23T18:51:33.051 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:32 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9[125519]: 2023-06-23T18:51:32.565+0000 7f7e00279700 -1 osd.9 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.384331+0000 front 2023-06-23T18:51:00.384313+0000 (oldest deadline 2023-06-23T18:51:23.884230+0000) 2023-06-23T18:51:33.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:32 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:32.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:33.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:32 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:32.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:33.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:32 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:32.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:33.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:32 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:32.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:33.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:32 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:32.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:33.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:32 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:32.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:33.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:33 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:33.353+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:33.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:33 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:33.353+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:33.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:33 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:33.353+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:33.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:33 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:33.353+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:33.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:33 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:33.353+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:33.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:33 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:33.353+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:33.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:33 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:33.353+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:33.800 INFO:journalctl@ceph.osd.9.smithi002.stdout:Jun 23 18:51:33 smithi002 bash[169429]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-9 2023-06-23T18:51:33.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:33.590+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:33.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:33.590+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:33.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:33.590+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:33.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:33.590+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:33.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:33.590+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:33.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:33 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:33.590+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:34.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:34 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:34.390+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:34.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:34 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:34.390+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:34.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:34 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:34.390+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:34.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:34 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:34.390+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:34.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:34 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:34.390+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:34.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:34 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:34.390+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:34.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:34 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:34.390+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:35.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:34 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:34.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:35.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:34 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:34.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:35.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:34 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:34.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:35.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:34 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:34.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:35.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:34 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:34.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:35.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:34 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:34.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:35.382 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.9.service' 2023-06-23T18:51:35.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:35.424 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-06-23T18:51:35.425 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-06-23T18:51:35.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.10 2023-06-23T18:51:35.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.354+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:35.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.354+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:35.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.354+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:35.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.354+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:35.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.354+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:35.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.354+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:35.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.354+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:35.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 systemd[1]: Stopping Ceph osd.10 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:36.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:35 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:35.652+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:36.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:35 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:35.652+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:36.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:35 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:35.652+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:36.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:35 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:35.652+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:36.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:35 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:35.652+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:36.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:35 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:35.652+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:36.190 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.802+0000 7f16b8b4c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:36.190 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.802+0000 7f16b8b4c700 -1 osd.10 84 *** Got signal Terminated *** 2023-06-23T18:51:36.190 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:35 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:35.802+0000 7f16b8b4c700 -1 osd.10 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:51:36.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:36.325+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:36.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:36.325+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:36.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:36.325+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:36.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:36.325+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:36.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:36.325+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:36.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:36.325+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:36.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:36 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:36.325+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:37.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:36 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:36.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:37.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:36 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:36.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:37.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:36 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:36.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:37.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:36 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:36.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:37.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:36 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:36.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:37.100 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:36 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:36.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:37.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:37 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:37.372+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:37.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:37 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:37.373+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:37.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:37 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:37.373+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:37.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:37 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:37.373+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:37.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:37 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:37.373+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:37.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:37 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:37.373+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:37.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:37 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:37.373+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:38.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:37 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:37.613+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:38.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:37 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:37.613+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:38.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:37 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:37.613+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:38.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:37 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:37.613+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:38.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:37 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:37.613+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:38.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:37 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:37.613+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:38.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:38 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:38.386+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:38.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:38 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:38.386+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:38.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:38 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:38.386+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:38.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:38 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:38.386+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:38.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:38 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:38.386+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:38.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:38 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:38.386+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:38.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:38 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:38.386+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:38.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:38 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:38.591+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:38.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:38 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:38.591+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:38.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:38 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:38.591+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:38.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:38 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:38.591+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:38.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:38 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:38.591+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:38.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:38 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:38.591+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:39.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:39 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:39.350+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:39.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:39 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:39.350+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:39.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:39 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:39.350+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:39.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:39 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:39.350+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:39.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:39 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:39.350+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:39.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:39 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:39.350+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:39.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:39 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:39.350+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:40.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:39.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:40.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:39.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:40.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:39.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:40.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:39.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:40.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:39.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:40.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:39.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:40.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:39 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:39.595+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:40.689 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:40.309+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:20.501858+0000 front 2023-06-23T18:50:20.501884+0000 (oldest deadline 2023-06-23T18:50:45.202091+0000) 2023-06-23T18:51:40.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:40.309+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:25.202470+0000 front 2023-06-23T18:50:25.202430+0000 (oldest deadline 2023-06-23T18:50:54.003436+0000) 2023-06-23T18:51:40.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:40.309+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:34.003967+0000 front 2023-06-23T18:50:34.003854+0000 (oldest deadline 2023-06-23T18:51:02.204687+0000) 2023-06-23T18:51:40.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:40.309+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:42.205199+0000 front 2023-06-23T18:50:42.205216+0000 (oldest deadline 2023-06-23T18:51:11.605892+0000) 2023-06-23T18:51:40.690 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:40.309+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.706991+0000 front 2023-06-23T18:50:52.706972+0000 (oldest deadline 2023-06-23T18:51:20.307871+0000) 2023-06-23T18:51:40.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:40.309+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.308553+0000 front 2023-06-23T18:51:00.308628+0000 (oldest deadline 2023-06-23T18:51:23.208541+0000) 2023-06-23T18:51:40.691 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:40 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10[119169]: 2023-06-23T18:51:40.309+0000 7f16b113a700 -1 osd.10 84 heartbeat_check: no reply from 172.21.15.2:6822 osd.6 since back 2023-06-23T18:51:08.409919+0000 front 2023-06-23T18:51:08.409975+0000 (oldest deadline 2023-06-23T18:51:31.909811+0000) 2023-06-23T18:51:41.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:40.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:41.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:40.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:41.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:40.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:41.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:40.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:41.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:40.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:41.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:40.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:41.100 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:40 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:40.616+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:41.676 INFO:journalctl@ceph.osd.10.smithi017.stdout:Jun 23 18:51:41 smithi017 bash[133871]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-10 2023-06-23T18:51:42.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:41.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:42.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:41.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:42.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:41.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:42.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:41.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:42.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:41.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:42.100 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:41.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:42.100 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:41 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:41.627+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:43.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:42.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:43.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:42.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:43.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:42.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:43.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:42.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:43.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:42.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:43.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:42.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:43.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:42 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:42.605+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:43.149 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.10.service' 2023-06-23T18:51:43.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:43.192 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-06-23T18:51:43.192 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-06-23T18:51:43.192 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.11 2023-06-23T18:51:43.599 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 systemd[1]: Stopping Ceph osd.11 for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:44.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:44.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:44.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:44.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:44.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:44.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:44.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.606+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:44.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.848+0000 7fc97037d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:44.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.848+0000 7fc97037d700 -1 osd.11 84 *** Got signal Terminated *** 2023-06-23T18:51:44.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:43 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:43.848+0000 7fc97037d700 -1 osd.11 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-06-23T18:51:44.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:44.568+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:44.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:44.568+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:44.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:44.568+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:44.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:44.568+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:44.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:44.568+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:44.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:44.568+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:44.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:44 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:44.568+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:45.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:45.593+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:45.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:45.593+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:45.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:45.593+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:45.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:45.593+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:45.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:45.593+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:45.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:45.593+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:45.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:45 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:45.593+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:47.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:46.620+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:47.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:46.620+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:47.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:46.620+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:47.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:46.620+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:47.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:46.620+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:47.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:46.620+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:47.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:46.620+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:47.099 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:46 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:46.620+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6820 osd.8 since back 2023-06-23T18:51:23.600284+0000 front 2023-06-23T18:51:23.600314+0000 (oldest deadline 2023-06-23T18:51:46.500346+0000) 2023-06-23T18:51:47.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:47.573+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:47.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:47.573+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:47.848 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:47.573+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:47.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:47.573+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:47.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:47.573+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:47.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:47.573+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:47.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:47.573+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:47.849 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:47 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:47.573+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6820 osd.8 since back 2023-06-23T18:51:23.600284+0000 front 2023-06-23T18:51:23.600314+0000 (oldest deadline 2023-06-23T18:51:46.500346+0000) 2023-06-23T18:51:48.856 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:48.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6806 osd.0 since back 2023-06-23T18:50:21.189639+0000 front 2023-06-23T18:50:21.189707+0000 (oldest deadline 2023-06-23T18:50:47.590178+0000) 2023-06-23T18:51:48.857 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:48.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6804 osd.1 since back 2023-06-23T18:50:28.691437+0000 front 2023-06-23T18:50:28.691457+0000 (oldest deadline 2023-06-23T18:50:53.391383+0000) 2023-06-23T18:51:48.857 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:48.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6804 osd.2 since back 2023-06-23T18:50:38.092293+0000 front 2023-06-23T18:50:38.092263+0000 (oldest deadline 2023-06-23T18:51:01.592481+0000) 2023-06-23T18:51:48.857 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:48.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.2:6814 osd.3 since back 2023-06-23T18:50:41.593187+0000 front 2023-06-23T18:50:41.593275+0000 (oldest deadline 2023-06-23T18:51:06.893334+0000) 2023-06-23T18:51:48.857 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:48.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6812 osd.4 since back 2023-06-23T18:50:52.194817+0000 front 2023-06-23T18:50:52.194750+0000 (oldest deadline 2023-06-23T18:51:14.995291+0000) 2023-06-23T18:51:48.857 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:48.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6812 osd.5 since back 2023-06-23T18:51:00.896440+0000 front 2023-06-23T18:51:00.896386+0000 (oldest deadline 2023-06-23T18:51:26.196530+0000) 2023-06-23T18:51:48.857 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:48.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.17:6820 osd.7 since back 2023-06-23T18:51:15.399196+0000 front 2023-06-23T18:51:15.399149+0000 (oldest deadline 2023-06-23T18:51:38.899151+0000) 2023-06-23T18:51:48.858 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:48 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11[120191]: 2023-06-23T18:51:48.556+0000 7fc96916c700 -1 osd.11 84 heartbeat_check: no reply from 172.21.15.82:6820 osd.8 since back 2023-06-23T18:51:23.600284+0000 front 2023-06-23T18:51:23.600314+0000 (oldest deadline 2023-06-23T18:51:46.500346+0000) 2023-06-23T18:51:50.098 INFO:journalctl@ceph.osd.11.smithi082.stdout:Jun 23 18:51:49 smithi082 bash[134159]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-osd-11 2023-06-23T18:51:51.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@osd.11.service' 2023-06-23T18:51:51.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:51.179 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-06-23T18:51:51.180 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-06-23T18:51:51.180 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.a 2023-06-23T18:51:51.550 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:51:51 smithi002 systemd[1]: Stopping Ceph mds.a for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:52.050 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:51:51 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-a[130769]: 2023-06-23T18:51:51.558+0000 7fa5f817d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:52.051 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:51:51 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-a[130769]: 2023-06-23T18:51:51.558+0000 7fa5f817d700 -1 mds.a *** got signal Terminated *** 2023-06-23T18:51:54.803 INFO:journalctl@ceph.mds.a.smithi002.stdout:Jun 23 18:51:54 smithi002 bash[169700]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-a 2023-06-23T18:51:54.809 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.a.service' 2023-06-23T18:51:54.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:54.847 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-06-23T18:51:54.848 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-06-23T18:51:54.848 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.d 2023-06-23T18:51:55.271 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:51:54 smithi002 systemd[1]: Stopping Ceph mds.d for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:55.550 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:51:55 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-d[129796]: 2023-06-23T18:51:55.265+0000 7fdb71f4c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:55.551 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:51:55 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-d[129796]: 2023-06-23T18:51:55.265+0000 7fdb71f4c700 -1 mds.d *** got signal Terminated *** 2023-06-23T18:51:58.264 INFO:journalctl@ceph.mds.d.smithi002.stdout:Jun 23 18:51:57 smithi002 bash[169842]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-d 2023-06-23T18:51:58.269 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.d.service' 2023-06-23T18:51:58.305 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:51:58.306 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-06-23T18:51:58.306 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-06-23T18:51:58.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.g 2023-06-23T18:51:58.550 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:51:58 smithi002 systemd[1]: Stopping Ceph mds.g for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:51:59.050 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:51:58 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-g[130386]: 2023-06-23T18:51:58.698+0000 7f8958587700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:51:59.051 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:51:58 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-g[130386]: 2023-06-23T18:51:58.698+0000 7f8958587700 -1 mds.g *** got signal Terminated *** 2023-06-23T18:52:00.492 INFO:journalctl@ceph.mds.g.smithi002.stdout:Jun 23 18:52:00 smithi002 bash[169987]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-g 2023-06-23T18:52:00.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.g.service' 2023-06-23T18:52:00.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:00.538 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-06-23T18:52:00.539 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-06-23T18:52:00.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.j 2023-06-23T18:52:00.801 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:52:00 smithi002 systemd[1]: Stopping Ceph mds.j for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:01.300 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:52:00 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-j[129434]: 2023-06-23T18:52:00.923+0000 7f2070b29700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:01.301 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:52:00 smithi002 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-j[129434]: 2023-06-23T18:52:00.923+0000 7f2070b29700 -1 mds.j *** got signal Terminated *** 2023-06-23T18:52:04.012 INFO:journalctl@ceph.mds.j.smithi002.stdout:Jun 23 18:52:03 smithi002 bash[170142]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-j 2023-06-23T18:52:04.137 DEBUG:teuthology.orchestra.run.smithi002:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.j.service' 2023-06-23T18:52:04.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:04.174 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-06-23T18:52:04.174 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-06-23T18:52:04.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.b 2023-06-23T18:52:04.554 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:52:04 smithi017 systemd[1]: Stopping Ceph mds.b for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:04.940 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:52:04 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-b[122882]: 2023-06-23T18:52:04.549+0000 7f729262c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:04.940 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:52:04 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-b[122882]: 2023-06-23T18:52:04.549+0000 7f729262c700 -1 mds.b *** got signal Terminated *** 2023-06-23T18:52:11.027 INFO:journalctl@ceph.mds.b.smithi017.stdout:Jun 23 18:52:10 smithi017 bash[134129]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-b 2023-06-23T18:52:11.227 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.b.service' 2023-06-23T18:52:11.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:11.269 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-06-23T18:52:11.269 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-06-23T18:52:11.269 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.e 2023-06-23T18:52:11.670 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:52:11 smithi017 systemd[1]: Stopping Ceph mds.e for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:11.939 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:52:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-e[122201]: 2023-06-23T18:52:11.664+0000 7f10cbe9f700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:11.940 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:52:11 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-e[122201]: 2023-06-23T18:52:11.664+0000 7f10cbe9f700 -1 mds.e *** got signal Terminated *** 2023-06-23T18:52:16.375 INFO:journalctl@ceph.mds.e.smithi017.stdout:Jun 23 18:52:16 smithi017 bash[134278]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-e 2023-06-23T18:52:16.381 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.e.service' 2023-06-23T18:52:16.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:16.418 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-06-23T18:52:16.419 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-06-23T18:52:16.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.h 2023-06-23T18:52:16.689 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:52:16 smithi017 systemd[1]: Stopping Ceph mds.h for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:17.189 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:52:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-h[122542]: 2023-06-23T18:52:16.794+0000 7fb275bc8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:17.190 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:52:16 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-h[122542]: 2023-06-23T18:52:16.794+0000 7fb275bc8700 -1 mds.h *** got signal Terminated *** 2023-06-23T18:52:19.689 INFO:journalctl@ceph.mds.h.smithi017.stdout:Jun 23 18:52:19 smithi017 bash[134423]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-h 2023-06-23T18:52:19.763 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.h.service' 2023-06-23T18:52:19.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:19.799 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-06-23T18:52:19.800 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-06-23T18:52:19.801 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.k 2023-06-23T18:52:20.184 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:52:19 smithi017 systemd[1]: Stopping Ceph mds.k for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:20.440 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:52:20 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-k[121856]: 2023-06-23T18:52:20.179+0000 7f6457a71700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:20.440 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:52:20 smithi017 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-k[121856]: 2023-06-23T18:52:20.179+0000 7f6457a71700 -1 mds.k *** got signal Terminated *** 2023-06-23T18:52:21.537 INFO:journalctl@ceph.mds.k.smithi017.stdout:Jun 23 18:52:21 smithi017 bash[134562]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-k 2023-06-23T18:52:21.749 DEBUG:teuthology.orchestra.run.smithi017:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.k.service' 2023-06-23T18:52:21.784 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:21.785 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-06-23T18:52:21.785 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-06-23T18:52:21.786 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.c 2023-06-23T18:52:22.098 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:52:21 smithi082 systemd[1]: Stopping Ceph mds.c for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:22.598 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:52:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-c[123763]: 2023-06-23T18:52:22.174+0000 7fe6bc20b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:22.598 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:52:22 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-c[123763]: 2023-06-23T18:52:22.174+0000 7fe6bc20b700 -1 mds.c *** got signal Terminated *** 2023-06-23T18:52:25.612 INFO:journalctl@ceph.mds.c.smithi082.stdout:Jun 23 18:52:25 smithi082 bash[134430]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-c 2023-06-23T18:52:25.800 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.c.service' 2023-06-23T18:52:25.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:25.842 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-06-23T18:52:25.843 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-06-23T18:52:25.843 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.f 2023-06-23T18:52:26.098 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:52:25 smithi082 systemd[1]: Stopping Ceph mds.f for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:26.598 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:52:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-f[123424]: 2023-06-23T18:52:26.235+0000 7fe8ef160700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:26.598 INFO:journalctl@ceph.mds.f.smithi082.stdout:Jun 23 18:52:26 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-f[123424]: 2023-06-23T18:52:26.235+0000 7fe8ef160700 -1 mds.f *** got signal Terminated *** 2023-06-23T18:52:30.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.f.service' 2023-06-23T18:52:30.930 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:30.930 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-06-23T18:52:30.930 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-06-23T18:52:30.931 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.i 2023-06-23T18:52:31.322 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:52:30 smithi082 systemd[1]: Stopping Ceph mds.i for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:31.598 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:52:31 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-i[124104]: 2023-06-23T18:52:31.317+0000 7f5a37fb8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:31.598 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:52:31 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-i[124104]: 2023-06-23T18:52:31.317+0000 7f5a37fb8700 -1 mds.i *** got signal Terminated *** 2023-06-23T18:52:37.347 INFO:journalctl@ceph.mds.i.smithi082.stdout:Jun 23 18:52:37 smithi082 bash[134718]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-i 2023-06-23T18:52:37.464 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.i.service' 2023-06-23T18:52:37.498 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:37.498 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-06-23T18:52:37.499 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-06-23T18:52:37.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.l 2023-06-23T18:52:37.848 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:52:37 smithi082 systemd[1]: Stopping Ceph mds.l for 902733cc-11f1-11ee-9b2b-001a4aab830c... 2023-06-23T18:52:38.347 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:52:37 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-l[123077]: 2023-06-23T18:52:37.872+0000 7f49592d5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-06-23T18:52:38.348 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:52:37 smithi082 ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-l[123077]: 2023-06-23T18:52:37.872+0000 7f49592d5700 -1 mds.l *** got signal Terminated *** 2023-06-23T18:52:39.729 INFO:journalctl@ceph.mds.l.smithi082.stdout:Jun 23 18:52:39 smithi082 bash[134861]: ceph-902733cc-11f1-11ee-9b2b-001a4aab830c-mds-l 2023-06-23T18:52:39.734 DEBUG:teuthology.orchestra.run.smithi082:> sudo pkill -f 'journalctl -f -n 0 -u ceph-902733cc-11f1-11ee-9b2b-001a4aab830c@mds.l.service' 2023-06-23T18:52:39.773 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-06-23T18:52:39.774 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-06-23T18:52:39.774 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-23T18:52:41.926 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-23T18:52:43.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c --force --keep-logs 2023-06-23T18:52:45.824 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:52:45.861 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:52:45.899 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-06-23T18:52:45.929 INFO:tasks.cephadm:Archiving crash dumps... 2023-06-23T18:52:45.932 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi002/crash 2023-06-23T18:52:45.933 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash -- . 2023-06-23T18:52:46.001 INFO:teuthology.orchestra.run.smithi002.stderr:tar: /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-23T18:52:46.002 INFO:teuthology.orchestra.run.smithi002.stderr:tar: Error is not recoverable: exiting now 2023-06-23T18:52:46.003 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi017/crash 2023-06-23T18:52:46.004 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash -- . 2023-06-23T18:52:46.062 INFO:teuthology.orchestra.run.smithi017.stderr:tar: /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-23T18:52:46.063 INFO:teuthology.orchestra.run.smithi017.stderr:tar: Error is not recoverable: exiting now 2023-06-23T18:52:46.064 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi082/crash 2023-06-23T18:52:46.065 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash -- . 2023-06-23T18:52:46.128 INFO:teuthology.orchestra.run.smithi082.stderr:tar: /var/lib/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/crash: Cannot open: No such file or directory 2023-06-23T18:52:46.128 INFO:teuthology.orchestra.run.smithi082.stderr:tar: Error is not recoverable: exiting now 2023-06-23T18:52:46.129 INFO:tasks.cephadm:Checking cluster log for badness... 2023-06-23T18:52:46.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-06-23T18:52:46.167 INFO:teuthology.orchestra.run.smithi002.stderr:grep: /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph.log: No such file or directory 2023-06-23T18:52:46.169 INFO:tasks.cephadm:Compressing logs... 2023-06-23T18:52:46.170 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-06-23T18:52:46.211 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-06-23T18:52:46.214 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-06-23T18:52:46.252 INFO:teuthology.orchestra.run.smithi017.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-06-23T18:52:46.252 INFO:teuthology.orchestra.run.smithi017.stderr:: No such file or directory 2023-06-23T18:52:46.255 INFO:teuthology.orchestra.run.smithi002.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-23T18:52:46.255 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-06-23T18:52:46.256 INFO:teuthology.orchestra.run.smithi082.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-06-23T18:52:46.256 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-06-23T18:52:46.258 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.10.log 2023-06-23T18:52:46.259 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.h.log 2023-06-23T18:52:46.259 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.e.log 2023-06-23T18:52:46.260 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.4.log 2023-06-23T18:52:46.260 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.1.log 2023-06-23T18:52:46.260 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.k.log 2023-06-23T18:52:46.262 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log 2023-06-23T18:52:46.263 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.7.log 2023-06-23T18:52:46.263 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.b.log 2023-06-23T18:52:46.263 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.b.log 2023-06-23T18:52:46.263 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.y.log 2023-06-23T18:52:46.264 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.0.log 2023-06-23T18:52:46.264 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.g.log 2023-06-23T18:52:46.264 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.a.log 2023-06-23T18:52:46.264 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.6.log 2023-06-23T18:52:46.265 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.3.log 2023-06-23T18:52:46.265 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.d.log 2023-06-23T18:52:46.265 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.x.log 2023-06-23T18:52:46.265 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.3.log: /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.9.log 2023-06-23T18:52:46.265 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log 2023-06-23T18:52:46.265 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.j.log 2023-06-23T18:52:46.265 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.9.log: /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.a.log 2023-06-23T18:52:46.266 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2023-06-23T18:52:46.266 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.b.log: /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.b.log: /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.y.log: 94.5% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.k.log.gz 2023-06-23T18:52:46.267 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.5.log 2023-06-23T18:52:46.267 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.2.log 2023-06-23T18:52:46.267 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.11.log 2023-06-23T18:52:46.268 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.z.log 2023-06-23T18:52:46.268 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.8.log 2023-06-23T18:52:46.269 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.a.log: /var/log/ceph/ceph-client.0.log: 94.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.g.log.gz 2023-06-23T18:52:46.269 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.a.log.gz 2023-06-23T18:52:46.269 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.d.log.gz 2023-06-23T18:52:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.f.log 2023-06-23T18:52:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.c.log 2023-06-23T18:52:46.270 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.i.log 2023-06-23T18:52:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.c.log 2023-06-23T18:52:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.c.log: gzip /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.i.log: -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log 2023-06-23T18:52:46.271 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.l.log 2023-06-23T18:52:46.271 INFO:teuthology.orchestra.run.smithi017.stderr: 93.4% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.y.log.gz 2023-06-23T18:52:46.272 INFO:teuthology.orchestra.run.smithi002.stderr: 94.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.j.log.gz 2023-06-23T18:52:46.273 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log: /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.l.log: 93.5% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.z.log.gz 2023-06-23T18:52:46.274 INFO:teuthology.orchestra.run.smithi082.stderr: 94.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.l.log.gz 2023-06-23T18:52:46.277 INFO:teuthology.orchestra.run.smithi082.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2023-06-23T18:52:46.517 INFO:teuthology.orchestra.run.smithi082.stderr: 91.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log.gz 2023-06-23T18:52:46.720 INFO:teuthology.orchestra.run.smithi017.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-06-23T18:52:46.915 INFO:teuthology.orchestra.run.smithi017.stderr: 91.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log.gz 2023-06-23T18:52:46.959 INFO:teuthology.orchestra.run.smithi002.stderr: 90.3% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mgr.x.log.gz 2023-06-23T18:52:47.256 INFO:teuthology.orchestra.run.smithi002.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-06-23T18:52:47.308 INFO:teuthology.orchestra.run.smithi002.stderr: 91.7% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-volume.log.gz 2023-06-23T18:52:48.475 INFO:teuthology.orchestra.run.smithi082.stderr: 92.0% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.c.log.gz 2023-06-23T18:52:48.609 INFO:teuthology.orchestra.run.smithi017.stderr: 92.0% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.b.log.gz 2023-06-23T18:52:49.158 INFO:teuthology.orchestra.run.smithi017.stderr: 95.0% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.h.log.gz 2023-06-23T18:52:51.770 INFO:teuthology.orchestra.run.smithi017.stderr: 96.3% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.e.log.gz 2023-06-23T18:52:51.906 INFO:teuthology.orchestra.run.smithi002.stderr: 90.3% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mon.a.log.gz 2023-06-23T18:52:53.724 INFO:teuthology.orchestra.run.smithi082.stderr: 96.5% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.f.log.gz 2023-06-23T18:54:12.245 INFO:teuthology.orchestra.run.smithi002.stderr: 90.8% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.9.log.gz 2023-06-23T18:54:25.631 INFO:teuthology.orchestra.run.smithi017.stderr: 91.3% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.4.log.gz 2023-06-23T18:54:31.566 INFO:teuthology.orchestra.run.smithi017.stderr: 90.9% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.10.log.gz 2023-06-23T18:54:32.872 INFO:teuthology.orchestra.run.smithi002.stderr: 90.7% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.6.log.gz 2023-06-23T18:54:40.813 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.3.log.gz 2023-06-23T18:54:42.411 INFO:teuthology.orchestra.run.smithi017.stderr: 90.8% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.1.log.gz 2023-06-23T18:54:52.744 INFO:teuthology.orchestra.run.smithi017.stderr: 90.7% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.7.log.gz 2023-06-23T18:54:59.486 INFO:teuthology.orchestra.run.smithi002.stderr: 90.6% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.0.log.gz 2023-06-23T18:55:09.686 INFO:teuthology.orchestra.run.smithi082.stderr: 90.8% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.8.log.gz 2023-06-23T18:55:13.354 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.2.log.gz 2023-06-23T18:55:21.663 INFO:teuthology.orchestra.run.smithi082.stderr: 90.7% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.11.log.gz 2023-06-23T18:55:32.299 INFO:teuthology.orchestra.run.smithi082.stderr: 90.8% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-osd.5.log.gz 2023-06-23T18:55:41.192 INFO:teuthology.orchestra.run.smithi082.stderr: 96.3% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.i.log.gz 2023-06-23T18:58:26.493 INFO:teuthology.orchestra.run.smithi002.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2023-06-23T18:58:26.496 INFO:teuthology.orchestra.run.smithi002.stderr: 2023-06-23T18:58:26.497 INFO:teuthology.orchestra.run.smithi002.stderr:real 5m40.269s 2023-06-23T18:58:26.497 INFO:teuthology.orchestra.run.smithi002.stderr:user 7m14.672s 2023-06-23T18:58:26.497 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m17.246s 2023-06-23T18:58:35.475 INFO:teuthology.orchestra.run.smithi017.stderr: 95.8% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.b.log.gz 2023-06-23T18:58:35.484 INFO:teuthology.orchestra.run.smithi017.stderr: 2023-06-23T18:58:35.485 INFO:teuthology.orchestra.run.smithi017.stderr:real 5m49.253s 2023-06-23T18:58:35.485 INFO:teuthology.orchestra.run.smithi017.stderr:user 7m18.241s 2023-06-23T18:58:35.485 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m16.775s 2023-06-23T19:00:44.712 INFO:teuthology.orchestra.run.smithi082.stderr: 94.9% -- replaced with /var/log/ceph/902733cc-11f1-11ee-9b2b-001a4aab830c/ceph-mds.c.log.gz 2023-06-23T19:00:44.714 INFO:teuthology.orchestra.run.smithi082.stderr: 2023-06-23T19:00:44.714 INFO:teuthology.orchestra.run.smithi082.stderr:real 7m58.485s 2023-06-23T19:00:44.714 INFO:teuthology.orchestra.run.smithi082.stderr:user 8m49.330s 2023-06-23T19:00:44.715 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m22.548s 2023-06-23T19:00:44.715 INFO:tasks.cephadm:Archiving logs... 2023-06-23T19:00:44.716 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi002/log 2023-06-23T19:00:44.717 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T19:01:38.533 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi017/log 2023-06-23T19:01:38.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T19:02:28.033 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi082/log 2023-06-23T19:02:28.034 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2023-06-23T19:03:33.099 INFO:tasks.cephadm:Removing cluster... 2023-06-23T19:03:33.100 DEBUG:teuthology.orchestra.run.smithi002:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c --force 2023-06-23T19:03:34.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c --force 2023-06-23T19:03:36.035 DEBUG:teuthology.orchestra.run.smithi082:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 902733cc-11f1-11ee-9b2b-001a4aab830c --force 2023-06-23T19:03:37.715 INFO:tasks.cephadm:Removing cephadm ... 2023-06-23T19:03:37.715 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-23T19:03:37.737 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-23T19:03:37.771 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf /home/ubuntu/cephtest/cephadm 2023-06-23T19:03:37.801 INFO:tasks.cephadm:Teardown complete 2023-06-23T19:03:37.802 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-06-23T19:03:37.814 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_076bbebc42a14f7d568aaa78eabb0038327bcb23/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_8627af3e0adcb765a3c249fcc209cba9f4873e1b/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-06-23T19:03:37.815 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-06-23T19:03:37.815 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T19:03:37.818 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T19:03:37.820 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-06-23T19:03:37.982 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-06-23T19:03:37.993 INFO:teuthology.task.clock:Checking final clock skew... 2023-06-23T19:03:37.993 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T19:03:37.997 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T19:03:37.999 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-06-23T19:03:38.013 INFO:teuthology.orchestra.run.smithi002.stderr:bash: ntpq: command not found 2023-06-23T19:03:38.015 INFO:teuthology.orchestra.run.smithi082.stderr:bash: ntpq: command not found 2023-06-23T19:03:38.016 INFO:teuthology.orchestra.run.smithi017.stderr:bash: ntpq: command not found 2023-06-23T19:03:38.036 INFO:teuthology.orchestra.run.smithi082.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T19:03:38.037 INFO:teuthology.orchestra.run.smithi082.stdout:=============================================================================== 2023-06-23T19:03:38.037 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 43 +611us[ +562us] +/- 98ms 2023-06-23T19:03:38.037 INFO:teuthology.orchestra.run.smithi082.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -3972us[-4020us] +/- 65ms 2023-06-23T19:03:38.037 INFO:teuthology.orchestra.run.smithi082.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +5847us[+5798us] +/- 92ms 2023-06-23T19:03:38.037 INFO:teuthology.orchestra.run.smithi082.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T19:03:38.039 INFO:teuthology.orchestra.run.smithi017.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T19:03:38.039 INFO:teuthology.orchestra.run.smithi017.stdout:=============================================================================== 2023-06-23T19:03:38.039 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +478us[ +478us] +/- 98ms 2023-06-23T19:03:38.039 INFO:teuthology.orchestra.run.smithi017.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -4121us[-4010us] +/- 65ms 2023-06-23T19:03:38.040 INFO:teuthology.orchestra.run.smithi017.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 +5684us[+5795us] +/- 92ms 2023-06-23T19:03:38.040 INFO:teuthology.orchestra.run.smithi017.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T19:03:38.041 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-06-23T19:03:38.041 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2023-06-23T19:03:38.041 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +417us[ +603us] +/- 98ms 2023-06-23T19:03:38.042 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -4163us[-3977us] +/- 65ms 2023-06-23T19:03:38.042 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +5667us[+5667us] +/- 92ms 2023-06-23T19:03:38.042 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-06-23T19:03:38.043 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-06-23T19:03:38.056 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-06-23T19:03:38.058 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-06-23T19:03:38.073 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T19:03:38.278 DEBUG:teuthology.orchestra.run.smithi017:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T19:03:38.453 DEBUG:teuthology.orchestra.run.smithi082:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-06-23T19:03:38.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T19:03:38.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T19:03:38.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T19:03:38.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T19:03:38.710 DEBUG:teuthology.orchestra.run.smithi082:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-06-23T19:03:38.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T19:03:38.742 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-06-23T19:03:38.756 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-06-23T19:03:38.772 INFO:teuthology.task.internal:Duration was 3731.901470 seconds 2023-06-23T19:03:38.772 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-06-23T19:03:38.788 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-06-23T19:03:38.789 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T19:03:38.792 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T19:03:38.794 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-06-23T19:03:38.852 INFO:teuthology.orchestra.run.smithi017.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T19:03:38.861 INFO:teuthology.orchestra.run.smithi082.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T19:03:38.883 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-06-23T19:03:39.230 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-06-23T19:03:39.231 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2023-06-23T19:03:39.231 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T19:03:39.266 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2023-06-23T19:03:39.267 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T19:03:39.300 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2023-06-23T19:03:39.301 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-06-23T19:03:39.324 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-06-23T19:03:39.325 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T19:03:39.328 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T19:03:39.343 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-06-23T19:03:39.426 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-06-23T19:03:39.427 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T19:03:39.429 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T19:03:39.432 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-06-23T19:03:39.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-06-23T19:03:39.560 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-06-23T19:03:39.560 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T19:03:39.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T19:03:39.647 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-06-23T19:03:39.703 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-06-23T19:03:39.718 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T19:03:39.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T19:03:39.723 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-06-23T19:03:39.757 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2023-06-23T19:03:39.758 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2023-06-23T19:03:39.758 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2023-06-23T19:03:39.780 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T19:03:39.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T19:03:39.837 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T19:03:39.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T19:03:39.854 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2023-06-23T19:03:39.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-06-23T19:03:39.870 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-06-23T19:03:39.884 INFO:teuthology.task.internal:Transferring archived files... 2023-06-23T19:03:39.885 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi002 2023-06-23T19:03:39.886 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T19:03:39.935 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi017 2023-06-23T19:03:39.936 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T19:03:39.987 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873/remote/smithi082 2023-06-23T19:03:39.987 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-06-23T19:03:40.031 INFO:teuthology.task.internal:Removing archive directory... 2023-06-23T19:03:40.032 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T19:03:40.035 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T19:03:40.037 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2023-06-23T19:03:40.076 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-06-23T19:03:40.092 INFO:teuthology.task.internal:Not uploading archives. 2023-06-23T19:03:40.092 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-06-23T19:03:40.108 INFO:teuthology.task.internal:Tidying up after the test... 2023-06-23T19:03:40.109 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T19:03:40.111 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T19:03:40.113 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-06-23T19:03:40.126 INFO:teuthology.orchestra.run.smithi002.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 19:03 /home/ubuntu/cephtest 2023-06-23T19:03:40.128 INFO:teuthology.orchestra.run.smithi017.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 19:03 /home/ubuntu/cephtest 2023-06-23T19:03:40.133 INFO:teuthology.orchestra.run.smithi082.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 23 19:03 /home/ubuntu/cephtest 2023-06-23T19:03:40.137 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-06-23T19:03:40.150 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-06-23T19:03:40.169 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-06-23T19:03:40.169 INFO:teuthology.nuke:Checking targets against current locks 2023-06-23T19:03:40.199 DEBUG:teuthology.nuke:shortname: smithi002 2023-06-23T19:03:40.199 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T19:03:40.227 DEBUG:teuthology.nuke:shortname: smithi017 2023-06-23T19:03:40.227 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T19:03:40.233 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 17:55:52.388548', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-23T19:03:40.256 DEBUG:teuthology.nuke:shortname: smithi082 2023-06-23T19:03:40.257 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-06-23T19:03:40.265 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 17:55:52.390669', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-23T19:03:40.294 INFO:teuthology.orchestra.console:Power off smithi002 2023-06-23T19:03:40.295 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T19:03:40.311 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-06-23_17:37:30-fs-wip-rishabh-improvements-authmon-distro-default-smithi/7313873', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-06-23 17:55:52.389731', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-06-23T19:03:40.321 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:40.348 INFO:teuthology.orchestra.console:Power off smithi017 2023-06-23T19:03:40.349 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T19:03:40.374 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:40.393 INFO:teuthology.orchestra.console:Power off smithi082 2023-06-23T19:03:40.394 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-06-23T19:03:40.418 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:44.362 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:44.406 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:44.454 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:48.478 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:48.518 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:48.567 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:48.692 INFO:teuthology.orchestra.console:Power off for smithi002 completed 2023-06-23T19:03:48.729 INFO:teuthology.orchestra.console:Power off for smithi017 completed 2023-06-23T19:03:52.684 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi082.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-06-23T19:03:52.897 INFO:teuthology.orchestra.console:Power off for smithi082 completed 2023-06-23T19:03:53.003 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-workunit/postgres}} duration: 3731.9014704227448 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2023-06-23T19:03:53.003 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-06-23T19:03:53.098 INFO:teuthology.run:FAIL